DEVICE INITIALIZATION AND AUTHENTICATION FOR COMPUTING SYSTEMS AND APPLICATIONS

Abstract

Embodiments refer to applications, platforms, architectures, etc., for authenticating and initializing a peripheral device. For example, one or more initialization operations may be performed on the peripheral device before it is authenticated. In these and other embodiments, authentication may be based on information maintained during initialization, such as error detection code values ​​and / or message counter values. Performing the authentication and initialization operations in the disclosed manner can reduce the time required for authenticating and initializing peripheral devices compared to the typical practice of authenticating first and then initializing.

Description

BACKGROUND

[0001] Computer systems can perform one or more operations (e.g., processing operations, presentation operations, application operations, etc.) based on sensor data obtained using one or more sensors. For example, computer systems can display information based on sensor data (e.g., displaying a camera or video feed), process the sensor data for decision-making, or determine specific system statuses, etc.

[0002] In these and other embodiments, certain computing systems that use sensor data can be configured to initialize one or more sensors used to obtain sensor data that is then used by the computing systems. For example, computing systems can be configured to perform one or more initialization operations with respect to one or more sensors in order to initialize the sensors so that the sensors are configured in a specific way.

[0003] The computing systems that use sensor data can also be configured to perform authentication operations related to authenticating the sensors used to obtain such sensor data. These authentication operations can help verify that the sensor data received from the sensors is trustworthy.

[0004] In some cases, the authentication operations may include authenticating the sensors themselves. Additionally or alternatively, the authentication operations may include verifying that the sensors are initialized in the manner prescribed by the computing systems.

[0005] In some traditional implementations, authentication operations can take place before and / or during sensor initialization. For example, before performing one or more initialization operations, a specific sensor can be authenticated by a suitable computing system to verify that the sensor is the expected type and possesses capabilities that satisfy certain criteria. In these and other cases, after the sensor itself has been authenticated, the sensor and the computing system can communicate with each other based on the sensor authentication, using secure communication (e.g., via encrypted communication) for the initialization operations.The operations corresponding to securing communications can be part of the authentication operations, so securing communications can help ensure that the initialization takes place as ordered by the computing system.

[0006] However, such a process can cause delays in the initialization process. For example, initialization operations may not begin until the sensor itself has been authenticated. Sensor authentication may require other hardware and / or software (e.g., cryptographic engines, key storage, etc.) to be running and initialized, which can further delay the initialization operations. Additionally, the use of secure communications related to the initialization operations can incur additional computational resource overhead, which can further delay the initialization process.

[0007] Such initialization delays can make it difficult for some systems to meet certain criteria. For example, in automotive contexts, a specific key performance indicator (KPI) related to backup cameras might be that the images captured by the backup cameras are displayed on a screen within a certain timeframe. Initialization delays in backup cameras can make it difficult to meet a specific criterion associated with such a KPI. OVERVIEW

[0008] According to one or more embodiments of the present disclosure, systems and methods can allow for faster initialization of peripheral devices of a computing system—e.g., sensors such as camera sensors—while simultaneously authenticating such devices and verifying that the device configurations—as obtained through initialization—are correct. For example, according to one or more embodiments, one or more initialization operations with respect to a peripheral device can be performed before the peripheral device is authenticated. In these and other embodiments, the peripheral device can be fully initialized before any authentication operations with respect to verifying the peripheral device itself and / or verifying the initialization are performed.Performing the authentication and initialization operations in the disclosed manner can reduce the time spent authenticating and initializing peripheral devices compared to authenticating first and then initializing, as is typically done.

[0009] The invention is defined by the claims. To illustrate the invention, aspects and embodiments that may or may not be within the scope of the claims are described herein.

[0010] Embodiments relating to applications, platforms, architecture, etc., for authenticating and initializing a peripheral device are disclosed. For example, one or more initialization operations can be performed on the peripheral device before it is authenticated. In these and other embodiments, authentication can be based on information maintained during initialization, such as error detection code values ​​and / or message counter values. Performing the authentication and initialization operations in the disclosed manner can reduce the time required for authenticating and initializing peripheral devices compared to the typical practice of authenticating first and then initializing.

[0011] The revelation extends to any novel aspects or features described and / or illustrated herein.

[0012] Further features of the disclosure are characterized by the independent and dependent claims.

[0013] Any feature in one aspect of the disclosure can be applied to other aspects of the disclosure in any suitable combination. In particular, procedural aspects can be applied to device or system aspects and vice versa.

[0014] Furthermore, features implemented in hardware can also be implemented in software, and vice versa. Any reference to software and hardware features herein should be interpreted accordingly.

[0015] Each system or device feature as described herein can also be provided as a process feature, and vice versa. Functionally described system and / or device aspects (including means plus functional features) can alternatively be expressed in terms of their corresponding structure, such as a suitably programmed processor and associated memory.

[0016] It should also be understandable that certain combinations of the various features described and defined in any aspect of the revelation can be implemented and / or provided and / or used independently of one another.

[0017] This disclosure also provides computer programs and computer program products comprising software code adapted, when executed on a data processing device, to perform any of the procedures described herein and / or to embody any of the setup and system features described herein, including any or all of the partial steps of a procedure.

[0018] The disclosure also provides a computer or computing system (including networked or distributed systems) that includes an operating system supporting a computer program for performing any of the procedures described herein and / or embodying any of the device or system features described herein.

[0019] The revelation also provides a computer-readable medium on which one or more of the aforementioned computer programs are stored.

[0020] The revelation also provides a signal that carries one or more of the aforementioned computer programs.

[0021] The disclosure extends to processes and / or devices and / or systems as described herein with reference to the accompanying drawings.

[0022] Aspects and embodiments of the disclosure will now be described exclusively by way of example with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The present systems and methods for device initialization and authentication for autonomous and semi-autonomous systems and applications are described in detail below with reference to the accompanying drawings, wherein: Fig. 1A illustrates an exemplary system relating to authentication according to a peripheral device in accordance with one or more embodiments of the present disclosure; Fig. 1B an exemplary process that takes place between the device and the computing system of Fig. 1A relating to the initialization and authentication of the device, illustrated in accordance with one or more embodiments of the present disclosure; Fig. 2 a flowchart illustrating a method for performing device authentication in accordance with one or more embodiments of the present disclosure; Fig. 3A is an illustration of an exemplary autonomous vehicle in accordance with one or more embodiments of the present disclosure; Fig. 3B is an example of camera locations and fields of view for the exemplary autonomous vehicle of Fig. 3A in accordance with one or more embodiments of the present disclosure; Fig. 3C presents a block diagram of an exemplary system architecture for the exemplary autonomous vehicle of Fig. 3A in accordance with one or more embodiments of the present disclosure; Fig. 3D system representation for communication between one or more cloud-based servers and the exemplary autonomous vehicle by Fig. 3A in accordance with one or more embodiments of the present disclosure; Fig. 4 is a block representation of an exemplary computing device suitable for use in implementing one or more embodiments of the present disclosure; and Fig. 5 is a block representation of an exemplary data center suitable for use in implementing one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0024] The systems and methods disclosed herein relate to the authentication of peripheral devices according to a computing system. The peripheral devices (e.g., sensors) can provide data (referred to in this disclosure as "device data") to the computing system, and the computing system can perform one or more operations based on the received device data. For example, the computing system can display information based on device data (e.g., display a camera or video feed) and / or process the device data for decision-making or to determine the specific status of another system (e.g., a vehicle) to which the computing system can correspond (e.g., be a part, control, monitor, etc.).

[0025] In these and other embodiments, the computing system can be configured to configure one or more of the peripheral devices. For example, the computing system can be configured to perform one or more initialization operations (e.g., during power-up or startup) with respect to the peripheral devices, such that the peripheral devices are configured to operate in a certain way during the operation—e.g., providing device data to the computing system in a specific manner (e.g., format, time frame, etc.).

[0026] The computer system can, for example, as part of its initialization operations, perform one or more write operations with respect to the peripheral devices (e.g., writing data to one or more registers of a peripheral processing system), which can configure the peripheral devices in a specific way. Additionally or alternatively, the computer system can perform one or more read operations as part of its initialization operations, whereby information can be read from the peripheral devices (e.g., from one or more registers of the peripheral device processing system).

[0027] In these and other embodiments, the computing system can itself authenticate the peripheral devices. For example, in some embodiments, authentication can include verification by the computing system that the peripheral devices from which device data can be obtained are the devices expected by the computing system to provide such device data (e.g., a certain type of device, a certain model of device, etc.). For example, in embodiments relating to machines (e.g., vehicles), the peripheral devices can include one or more sensors that produce sensor data that can be used by a related computing system. Such sensors can have certain associated performance requirements (e.g., resolution, data collection frequency, range, etc.).Performance requirements can help ensure that information provided by such sensor data meets certain criteria associated with the use of such sensor data. Verifying the sensors themselves can therefore allow the computing system to verify that the sensors are indeed sensors that meet specified performance requirements and are thus capable of producing sensor data that satisfies the relevant criteria.

[0028] In these and other embodiments, and as another example, authentication may include verifying that the configuration of the peripheral devices has taken place in the prescribed manner during the corresponding initialization of such devices. For example, it may be verified that the data written to the peripheral devices during initialization is the same data that the computing system ordered to be written to the peripheral devices. In the present disclosure, reference to "authentication of a peripheral device" may generally refer to the authentication or verification of the device itself and / or the authentication or verification of a configuration of the device via initialization operations.

[0029] As detailed in the present disclosure, according to one or more embodiments of the present disclosure, one or more initialization operations can be performed with respect to one or more peripheral devices (e.g., sensors) before the peripheral devices are authenticated. In these and other embodiments, the peripheral devices can be fully initialized before any authentication operations are performed with respect to verifying the peripheral devices themselves and / or verifying the initialization. Such a process can be realized by utilizing certain operations that take place during initialization, as detailed in the disclosure. In contrast, traditional approaches typically exclude the authentication of peripheral devices prior to initialization and the performance of certain operations during initialization (e.g.,(using secure communication protocols) to help authenticate the initialization.

[0030] Performing authentication operations in the disclosed manner can reduce the time required for authentication and initialization of peripheral devices compared to authenticating first and then initializing. For example, the peripheral devices may be capable of performing initialization operations while other hardware and / or software used to perform one or more authentication operations are being initialized. In contrast, such operations are typically performed sequentially in cases where authentication is performed first.Additionally or alternatively, the time required to perform the actual initialization operations can be reduced by reducing or eliminating the processing overhead traditionally required to secure the initialization data communications, which can involve up to hundreds of communications. In contrast, as discussed in more detail in the present disclosure, the communication of post-initialization information used to verify that the initialization is correct may involve only one or a few secure communications.

[0031] One or more embodiments of the present disclosure may relate to device authentication (e.g., sensor authentication) associated with ego-machines and / or components of one or more ego-machines, which may include any applicable machine or system capable of performing one or more autonomous or semi-autonomous operations. Exemplary ego-machines may include, but are not limited to, other vehicles (land, sea, space, and / or air), robots, robotic platforms, etc. For example, the ego-machine computing applications may include one or more applications that can be executed by an autonomous or semi-autonomous vehicle, such as an exemplary autonomous vehicle 300 (alternatively referred to herein as "vehicle 300" or "ego-machines 300"), which, with reference to Fig. 3A-3D is described. In the present disclosure, a reference to “autonomous vehicle” or “semi-autonomous vehicle” may include any vehicle that can be configured to perform one or more autonomous or semi-autonomous navigation or driving operations. As such, such vehicles may include those that require a driver or in which a driver can also perform such operations.

[0032] Additionally or alternatively, the systems and procedures described herein may, without limitation, be used by non-autonomous vehicles or machines, semi-autonomous vehicles or machines (e.g., in one or more adaptive driver assistance systems (ADAS)), autonomous vehicles or machines, guided and unguided robots or robotic platforms, warehouse vehicles, all-terrain vehicles, vehicles coupled to one or more trailers, flying vehicles, boats, shuttle vehicles, emergency vehicles, motorcycles, electric or motorized bicycles, aircraft, construction vehicles, underwater vehicles, drones, and / or other types of vehicles.Furthermore, the systems and methods described herein can be used for a number of purposes, for example, but not limited to, machine control, machine locomotion, machine propulsion, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and monitoring, simulation and digital twinning, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actuator simulation and / or digital twinning, generative AI, data center processing, conversational AI (such as by employing one or more language models, such as one or more large language models (LLMs), vision language models (VLMs), multimodal language models, etc.), light transport simulation (e.g., ray tracing, path tracing, etc.).), collaborative content creation for 3D assets, cloud computing and / or other suitable applications.

[0033] Disclosed embodiments may be included in a range of different systems, such as automotive systems (e.g., a control system for an autonomous or semi-autonomous machine, a perception system for an autonomous or semi-autonomous machine), systems implemented using a robot, antenna systems, media systems, boat systems, intelligent area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twinning operations, systems implemented using an edge device, systems involving one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least partially in a data center, systems for performing conversational AI operations (e.g.,Systems implementing one or more LLMs, systems implementing one or more Vision Language Models (VLMs), systems implementing one or more multimodal language models, systems performing one or more generative AI operations, systems hosting real-time streaming applications, systems presenting one or more virtual reality content, augmented reality content or mixed reality content, systems performing light transport simulation, systems performing collaborative content creation for 3D assets, systems implemented at least partially using cloud computing resources, and / or other types of systems.

[0034] The embodiments of the present disclosure are explained with reference to the accompanying figures. It is understood that the figures are diagrammatic and schematic representations of such exemplary embodiments and are not limiting, but are also not necessarily drawn to scale. In the figures, features with the same reference numerals denote the same structures and functions, unless otherwise described.

[0035] Fig. Figure 1A illustrates an exemplary system 100 relating to authentication according to a peripheral device 102 (“device 102”) according to one or more embodiments of the present disclosure. In general, the system 100 can include the device 102 and a computing system 104. It should be understood that this and other arrangements described herein are presented only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, groupings of functions, etc.) can be used in addition to or instead of those shown, and other elements can be omitted entirely.

[0036] For example, System 100 can include any number of peripheral devices with respect to which authentication operations can be performed. Additionally or alternatively, System 100 can include multiple controllers configured to perform authentication operations, individually and / or collectively, such as those described herein. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in conjunction with other components, in any suitable combination and position. Various functions described herein as being performed by entities can be executed using hardware, firmware, and / or software. For example, various functions can be executed using a processor that executes instructions stored in memory.

[0037] Furthermore, System 100 can be implemented in any applicable system, device, or apparatus where device authentication can be performed. As an example, and not limited to, System 100 can be implemented in vehicle-related situations, such as with regard to Vehicle 300. Fig. 3A-3D.

[0038] In some embodiments, the device 102 may include any suitable device that may be configured to provide data (referred to as "device data") to the computer system 104 for use by the computer system 104. For example, the device 102 may include a separate computer system configured to acquire data and / or provide it to the computer system 104.

[0039] As another example, the device 102 can include a sensor system (hereinafter generally referred to as a "sensor") configured to acquire data (hereinafter referred to as "sensor data") and to provide such sensor data to the computing system 104. The sensors can include any type of sensor configured to acquire sensor data. Additionally or alternatively, the device 102 can correspond to a machine, such as an ego machine. For example, the device 102 can be any of those described with reference to the vehicle 300, as described with reference to Fig. 3A-3D, include the sensors described. For example, in some embodiments, the device 102 may include a camera (e.g., a backup camera) configured to provide a video feed to the computing system 104.

[0040] In these and other embodiments, the device 102 can include one or more processing systems configured to perform one or more operations with respect to the device 102. For example, the processing systems can be configured to format data in a certain way, generate data (e.g., generate data based on sensor detections), control the receipt and / or communication of data, etc. In these and other embodiments, the processing system of the device 102 can be implemented by and / or as one or more computing devices, such as those related to Fig. 4 will be described in more detail.

[0041] The computing system 104 can include any suitable system configured to perform operations based on device data received from the device 102. In some embodiments, the computing system 104 can be a localized system in which its components may be located in a relatively similar location. Additionally or alternatively, the computing system 104 can be a distributed system in which one or more components of the computing system 104 may be located remotely.

[0042] In some embodiments, the computing system 104 may include code and routines configured to initiate the execution of the operations described with respect to the computing system 104. Additionally or alternatively, the computing system 104 may be implemented using hardware, including one or more processors, CPUs, graphics processing units (GPUs), data processing units (DPUs), parallel processing units (PPUs), microprocessors (e.g., for performing or controlling the execution of one or more operations), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), accelerators (e.g.,Deep learning accelerators (DLAs), one or more programmable vision accelerators (PVAs), which may include one or more vector processing units (VPUs), one or more direct memory access (DMA) systems, one or more pixel processing engines (PPEs), etc., and / or other processor types. In these and other embodiments, the computing system 104 can be implemented using a combination of hardware and software. In the present disclosure, operations described as being performed by the computing system 104 may include operations that the computing system 104 can perform itself or that it can cause to be performed by another device.

[0043] In these or other embodiments, the computing system 104 can be implemented by and / or as one or more computing devices, such as those relating to Fig. 4 will be described in more detail. Additionally or alternatively, the computing system 104 of a machine, such as the one with reference to Fig. The vehicle described in 3A-3D corresponds to (e.g., being enclosed within it, controlling it, etc.). In these and other embodiments, the computing system 104 can be a data center, such as the one described in relation to Fig. 5 described data center 500, correspond (e.g. being included in it, controlling it, etc.).

[0044] In some embodiments, the computing system 104 can be configured to perform one or more initialization operations with respect to the device 102. Additionally or alternatively, the computing system 104 can be configured to perform one or more authentication operations with respect to authenticating the device 102. In these and other embodiments, the computing system 104 can be configured to perform the authentication operations after one or more of the initialization operations have been carried out (e.g., after initializing the device 102). Such a process can be implemented by utilizing certain operations that take place during sensor initialization.

[0045] For example, during sensor initialization, one or more error-checking operations can be performed on initialization data communicated between the initializing computer system and the sensor. These error-checking operations can be used to detect whether changes have occurred in the initialization data and are typically used to determine whether random changes that could corrupt the initialization data have occurred (e.g., during communication).

[0046] An exemplary mechanism that can be used for error checking can include the generation of one or more error detection codes—such as a cyclic redundancy check (CRC) code—based on initialization data communicated between Device 102 and Computer 104. For example, both Device 102 and Computer 104 can determine CRC codes and corresponding values ​​based on initialization data communicated during Device 102's initialization. Cases where the CRC codes do not match indicate that the initialization data that should have been stored on a page differs from the initialization data that was actually stored, thus indicating data corruption.In contrast, cases where the CRC codes match indicate that the initialization data that should have been stored on a page is the same as the initialization data that was actually stored, thus verifying the integrity of the data.

[0047] Another verification mechanism related to initialization can include maintaining message counters. Generally, message counters can indicate a running number of messages communicated during initialization. For example, a write message counter, which counts the number of write messages, can be maintained. Additionally or alternatively, a read message counter, which counts the number of read messages, can be maintained. In these and other embodiments, a read / write message counter, which counts the number of both read and write messages, can be maintained. In some cases, such message counters can be used to identify whether or not messages were lost between the device 102 and the computer system 104 during initialization.Additionally or alternatively, the message counters can be used to identify whether the device 102 has received a message that did not originate from the computing system 104, or vice versa.

[0048] According to one or more embodiments of the present disclosure, such fault detection codes and / or message counters (generally referred to as "tracking codes") can also be used as mechanisms that allow authentication operations to be performed after sensor initialization. For example, in some embodiments, authentication operations can be performed following device initialization to authenticate the device 102 itself. In some embodiments, the authentication of the device itself can be performed in the same or a similar manner as in cases where the device 102 can be authenticated prior to initialization.

[0049] Based on device authentication, secure communication can be established between the device 102 and the computer system 104. Additionally or alternatively, the device 102 can communicate one or more tracking code values ​​to the computer system 104, which are determined by the device 102 based on initialization data communicated during device initialization. In some embodiments, for example, the device 102 can communicate a write CRC value to the computer system 104 via secure communication. This write CRC value is based on data written to the device 102 as a result of write operations performed during device initialization.Additionally or alternatively, the device 102 can communicate a read CRC value to the computing system 104 via secure communication. This CRC value is based on data read from the sensor during sensor initialization. In these and other embodiments, the device 102 can communicate one or more message counter values ​​to the computing system 104.

[0050] The computing system 104 can compare the received tracking code values ​​with expected values ​​for the received tracking code values—for example, where the expected values ​​can be determined at the computing system 104. In response to the received tracking code values ​​and expected tracking code values ​​matching, the computing system 104 can verify that the device 102 was initialized as expected and can accordingly verify the initialization. Conversely, in response to the received tracking code values ​​and expected tracking code values ​​not matching, the computing system 104 cannot verify that the device 102 was initialized as expected and can accordingly not verify the initialization. In some embodiments, one or more operations can be performed according to the authentication and verification of the device 102 as with reference to Fig. 1B will be carried out as described.

[0051] Fig. Figure 1B illustrates an exemplary process 150 that can be performed between the device 102 and the computing system 104 with respect to the initialization and authentication of the device, according to one or more embodiments of the present disclosure. Process 150 describes several operations that can be performed; however, it is understood that individual implementations may include additional operations and / or omit one or more operations.

[0052] In some embodiments, the process 150 may include one or more initialization operations 110. In some embodiments, the initialization operations 110 may include the communication of data between the computer system 104 and the device 102 for configuring the device 102 in some way.

[0053] For example, in some embodiments, the initialization operations 110 may include one or more write operations. In some embodiments, the write operations may include instructions from the computing system 104 to the device 102 to write certain information to the device 102. In these and other embodiments, the write instructions may include the data to be written to the processing system of the device 102 (e.g., to the memory of the processing system of the device 102, such as registers). The data to be written may configure the device 102 in a specific way. In these and other embodiments, the device 102 may write such information in response to receiving the write instructions.

[0054] Additionally or alternatively, the initialization operations 110 can include one or more read operations. The read operations can relate to data read by the device 102 (e.g., from the memory of the device 102). In these and other embodiments, read operations can include instructions to the device 102 from the computer system 104 regarding information to be read by the device 102. In these and other embodiments, the read operations can include the device 102 reading the instruction data. Additionally or alternatively, the read operations can include the device 102 communicating the read data to the computer system 104.

[0055] In these and other embodiments, the device 102 can perform an operation 112 with respect to the initialization operations 110. The operation 112 can include maintaining one or more first tracking codes. Additionally or alternatively, the computing system 104 can perform an operation 114 with respect to the initialization operations 110. The operation 114 can include maintaining one or more second tracking codes.

[0056] In some embodiments, the initial tracking codes can correspond to one or more error checking operations that can be performed with respect to the initialization data (e.g., write data and / or read data) communicated and / or accessed during initialization. In some embodiments, different error detection codes can be maintained with respect to the initialization operations. For example, in some embodiments, a write error detection code (e.g., write CRC code) and a corresponding value can be maintained based on write operations performed during the initialization operations. In these and other embodiments, a read error detection code (e.g., read CRC code) and a corresponding value can be maintained based on read operations performed during the initialization operations.Additionally or alternatively, a read / write error detection code (e.g., read / write CRC code) and a corresponding value can be maintained based on both read and write operations performed during initialization operations 110. In some embodiments, the first and second tracking codes can include the error detection codes (e.g., CRC codes).

[0057] In these and other embodiments, one or more message counters and their respective values ​​can be maintained by the device 102 and the system 104 with reference to the initialization operations 110. In some embodiments, the first tracking codes and the second tracking codes can include the message counters in addition to the fault detection codes.

[0058] In some embodiments, the process 150, following the initialization operations 110, may include one or more authentication operations. In some embodiments, the authentication operations may include device verification operations 116. The verification operations 116 can be used to verify the device 102 itself. For example, the verification operations 116 can be used to verify that the device 102 is the expected type of device 102, possessing capabilities that satisfy certain criteria. For example, the verification operations 116 can be used to verify that the device 102 meets certain performance requirements by verifying that the device 102 is a certain type of device capable of fulfilling such requirements.

[0059] In some embodiments, the verification operations 116 include any suitable operations or process that can be performed to verify the device 102 itself. For example, in some embodiments, cryptographic session keys can be generated by the device 102 and the computing system 104 according to any suitable process—e.g., prior to performing the verification operations 116 or as part of the verification operations 116. In these and other embodiments, the verification operations 116 can include exchanging the cryptographic session keys between the device 102 and the computing system 104 to authenticate the device 102 and the computing system 104 to each other using any suitable mechanism.

[0060] For example, in some embodiments, the device 102 can be verified using public key certification. For example, the device 102 can provide access to its public key certificate via an unsecured link, and the computing system 104 can use a known public key with signing authority (which is not a secret) to cryptographically verify the certificate.

[0061] As another example, the device 102 can be verified based on a secret key mechanism. For instance, the same secret key can be provided to the device 102 and the computer system 104 in a secure (trusted) environment, so that they can verify each other with the known secret key when operating in an insecure environment.

[0062] In some embodiments, process 150 may include operation 118. Operation 118 establishes a secure communication session between the device 102 and the computing system 104. In these and other embodiments, the secure communication session can be established using the cryptographic session keys used as part of verification operations 116. The secure communication session can be established using any suitable protocol that allows secure communications (e.g., encrypted communications) to be exchanged between the device 102 and the computing system 104.

[0063] In some embodiments, process 150 may also include operation 120. Operation 120 may include one or more device authentication operations associated with verifying the initialization of device 102—for example, verifying that device 102 has been configured in the manner prescribed by the computer system 104. In some embodiments, the computer system 104 may request that device 102 communicate one or more values ​​corresponding to one or more tracking codes to the computer system 104. In these and other embodiments, the request may be made via secure communication in accordance with the secure communication session.

[0064] For example, in some embodiments, the computing system 104 can request that the device 102 communicate a first write CRC code value, a first read CRC code value, a first read / write CRC code value, a first write counter value, a first read counter value and / or a first read / write counter value.

[0065] In Operation 122, the device 102 can communicate the requested initial tracking code values ​​to the computer system 104. In this and other embodiments, the device 102 can communicate the requested initial tracking code values ​​via secure communication. Secure communication can help ensure that the information received at the computer system 104 is the same as the information communicated by the device 102. Accordingly, secure communication can maintain the integrity of the information communicated to verify initialization (e.g., maintain the integrity of the initial tracking code values).

[0066] In operation 124, the computing system 104 can compare the received first tracking code values ​​with the corresponding second tracking code values. The second tracking code values ​​can be values ​​that the computing system 104 has determined based on the write and / or read operations, and can be the expected values ​​of the first tracking code values ​​received by the device 102.

[0067] The computing system 104 can, for example, perform the following actions: compare a second write CRC value maintained by the computing system 104 with a first read CRC value received by the device 102; compare a second read CRC value maintained by the computing system 104 with a first read CRC value received by the device 102; compare a second read / write CRC value maintained by the computing system 104 with a first read / write CRC value received by the device 102; compare a second write message counter value maintained by the computing system 104 with a first write message counter value received by the device 102; compare a second read message counter value maintained by the computing system 104 with a first read message counter value received by the device 102.and / or comparing a second read / write message counter value maintained by the computing system 104 with a first read / write message counter value received by the device 102.

[0068] In Operation 126, the computing system can verify the initialization based on comparisons. For example, cases where a specific first track code value does not match its corresponding second track code value indicate that initialization data corresponding to such a value, which should have been stored and / or read on a page, differs from the initialization data that was actually stored and / or read, thus indicating data corruption. As such, in cases where one or more of the first track code values ​​do not match their corresponding second track code values, it can be determined that the initialization was not performed as expected—for example, it was corrupted and / or affected in some way—and the initialization cannot be verified or authenticated.

[0069] In contrast, cases where the specific first tracker code values ​​match their corresponding second tracker code values ​​indicate that the corresponding initialization data that should have been stored and / or read on a page is the same as the initialization data that was actually stored and / or read, thus verifying data integrity. As such, in cases where one or more of the first tracker code values ​​match their corresponding second tracker code values, it can be determined that the initialization was performed as expected and the initialization can be verified or authenticated. In some embodiments, the initialization may not be verified unless all of the first tracker code values ​​match their corresponding second tracker code values.

[0070] Accordingly, performing process 150 can be used to authenticate the device 102 (e.g., the device itself and its initialization) after the initialization has been performed, while also maintaining the integrity of the authentication. Furthermore, performing the authentication operations in relation to Fig. 1A and Fig. In the manner described in 1B, compared to authenticating first and then initializing as is typically done, the time required for authenticating and initializing the device 102 is reduced.

[0071] For example, the device 102 may be capable of performing initialization operations while other hardware and / or software used to perform one or more authentication operations are being initialized. In contrast, such operations are typically performed serially in cases where authentication is performed first. Additionally or alternatively, the time required to perform the actual initialization operations can be reduced by reducing or eliminating the processing overhead traditionally required to secure the initialization data communications, which can involve up to hundreds of communications. In contrast, the communication of the tracking codes after initialization may involve only one or a few secure communications.

[0072] Modifications, additions, or omissions may be made to Fig. 1A and Fig. 1B and the associated descriptions may be made without deviating from the scope of this disclosure. For example, in some embodiments, the system 100 may include any number of devices that can be initialized and authenticated, and / or any number of computing systems that can perform the initialization and / or authentication. Alternatively or additionally, the sequence of operations described with reference to the process 150 may differ. In these and other embodiments, one or more operations of the process 150 may be omitted or modified. For example, in some embodiments, the device 102 may communicate the initial tracking code values ​​without receiving a request from the computing system 104 to do so.As another example, although the description is such that the authentication operations are performed after the device 102 has been fully initialized, some embodiments may include performing a partial initialization and then performing the authentication operations after the partial initialization but before a full initialization.

[0073] Fig. Figure 2 is a flowchart illustrating a method 200 for performing device authentication in accordance with one or more embodiments of the present disclosure. One or more operations of the method 200 may be performed by a suitable system, device, or apparatus, such as one or more components of the system 100. Fig. 1A, the autonomous vehicle system(s) relating to Fig. 3A-3D describes the computing device(s) that are related to Fig. 4 is / are described, and / or the data system(s) that refers to Fig. 5 is / will be described in the present revelation.

[0074] Method 200 can include a block B202. Block B202 can include one or more initialization operations with respect to a peripheral device, such as device 102. Fig. 1A. In these and other embodiments, the initialization operations between the peripheral device and a computer system, such as computer system 104 from Fig. 1A, are performed. In some embodiments, one or more of the initialization operations can be one or more of those described in the present disclosure with reference to Fig. 1A and Fig. Include the initialization operations described in 1B - e.g., one or more of those relating to Fig. 1A initialization operations described 110.

[0075] In block B204, one or more authentication operations can be performed with respect to the peripheral device. The authentication operations can be performed after the peripheral device has been initialized, or at least after the peripheral device has been at least partially initialized. The authentication operations can include verifying the device itself, as described in the present disclosure—e.g., with respect to Fig. 1A and Fig. 1B. In these and other embodiments, the authentication operations may include verifying the device as described in the present disclosure – e.g., with reference to Fig. 1A and Fig. 1B.

[0076] Additionally or alternatively, the authentication operations can be based on tracking codes maintained during initialization, as described in the present disclosure – e.g., with reference to Fig. 1A and Fig. 1B. In some embodiments, one or more of the authentication operations may be one or more of those described in the present disclosure with reference to Fig. 1A and Fig. Include the authentication operations described in 1B - e.g., one or more of those relating to Fig. Operations 116, 118, 120, 122, 124 or 126 as described in 1B.

[0077] Modifications, additions, or omissions may be made to Method 200 without deviating from the scope of this disclosure. For example, although illustrated as discrete blocks, various blocks of Method 200 may be subdivided into additional blocks, combined into fewer blocks, or eliminated, depending on the specific implementations. Furthermore, in some embodiments, Method 200 may be used to perform multiple different authentications of several different peripheral devices. EXEMPLARY AUTONOMOUS VEHICLE

[0078] Fig. Figure 3A is an illustration of an exemplary autonomous vehicle 300 in accordance with some embodiments of the present disclosure. The autonomous vehicle 300 (alternatively referred to herein as the “vehicle 300”) may, without limitation, include a passenger vehicle such as a car, truck, bus, rescue vehicle, shuttle, electric or motorized bicycle, motorcycle, fire engine, police vehicle, ambulance, boat, construction vehicle, underwater vehicle, drone and / or another type of vehicle (e.g., one that is unmanned and / or that carries one or more passengers).Autonomous vehicles are generally described in terms of automation levels, as defined by the National Highway Traffic Safety Administration (NHTSA), a division of the U.S. Department of Transportation, and the Society of Automotive Engineers (SAE) in their "Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles" (Standard No. J3016-201806, published June 15, 2018; Standard No. J3016-201609, published September 30, 2016; and earlier and future versions of this standard). The Vehicle 300 may be capable of functionality corresponding to one or more of Levels 3 through 5 of autonomous driving levels. The Vehicle 300 may be capable of functionality corresponding to one or more of Levels 1 through 5 of autonomous driving levels.The vehicle 300 can, for example, depending on its configuration, be capable of driver assistance (Level 1), partial automation (Level 2), conditional automation (Level 3), high automation (Level 4), and / or full automation (Level 5). The term "autonomous," as used herein, may include any and / or all types of autonomy for the vehicle 300 or any other machine, such as fully autonomous, highly autonomous, conditionally autonomous, partially autonomous, providing assistive autonomy, semi-autonomous, mainly autonomous, or any other designation.

[0079] The vehicle 300 can include components such as a chassis, a vehicle body, wheels (e.g., 2, 4, 6, 8, 18, etc.), tires, axles, and other vehicle components. The vehicle 300 can include a drive system 350, such as an internal combustion engine, a hybrid-electric drive unit, a fully electric motor, and / or another type of drive system. The drive system 350 can be connected to a powertrain of the vehicle 300, which may include a transmission to enable the propulsion of the vehicle 300. The drive system 350 can be controlled in response to receiving signals from the throttle / accelerator device 352.

[0080] A steering system 354, which may include a steering wheel, can be used to steer the vehicle 300 (e.g., along a desired path or route) when the drive system 350 is in operation (e.g., when the vehicle is in motion). The steering system 354 can receive signals from a steering actuator 356. The steering wheel may be optional for full automation functionality (Level 5).

[0081] The brake sensor system 346 can be used to operate the vehicle brakes in response to receiving signals from the brake actuators 348 and / or brake sensors.

[0082] One or more controllers 336, which control one or more CPU(s), system-on-chips (“SoCs”) 304 ( Fig. The controller(s) 336, which may include one or more GPUs (e.g., 3C), can provide signals (e.g., representing commands) for one or more components and / or systems of the vehicle 300. For example, the controller(s) can send signals to operate the vehicle brakes via one or more brake actuators 348, to operate the steering system 354 via one or more steering actuators 356, and / or to operate the propulsion system 350 via throttle / accelerator device 352. The controller(s) 336 can include one or more (e.g., integrated) onboard computing devices (e.g., supercomputers) that process sensor signals and issue operational commands (e.g., signals representing commands) to enable autonomous driving and / or to assist a human driver in driving the vehicle 300.The controller(s) 336 may include a first controller 336 for autonomous driving functions, a second controller 336 for functional safety functions, a third controller 336 for artificial intelligence functionality (e.g., computer vision), a fourth controller 336 for infotainment functionality, a fifth controller 336 for redundancy in emergency situations, and / or other controllers. In some embodiments, a single controller 336 may handle two or more of the above functionalities, two or more controllers 336 may handle a single functionality, and / or any combination thereof.

[0083] The controller(s) 336 can provide signals to control one or more components and / or one or more systems of the vehicle 300 in response to sensor data received from one or more sensors (e.g. sensor inputs). The sensor data can be received, for example, without restriction, from the following: sensors for global navigation satellite systems 358 (e.g., global positioning system sensors), radar sensors 360, ultrasonic sensors 362, lidar sensors 364, inertial measurement unit (IMU) sensors 366 (e.g., accelerometers, gyroscopes, magnetic compasses, magnetometers, etc.), microphones 396, stereo cameras 368, wide-angle cameras 370 (e.g., fisheye cameras), infrared cameras 372, surround-view cameras 374 (e.g., 360-degree cameras), long-range and / or medium-range cameras 398, velocity sensors 344 (e.g.,for measuring the speed of the vehicle 300), vibration sensors 342, steering sensors 340, brake sensors 346 (e.g. as part of the brake sensor system 346) and / or other sensor types.

[0084] One or more of the controllers 336 can receive inputs (e.g., represented by input data) from an instrument cluster 332 of the vehicle 300 and provide outputs (e.g., represented by output data, display data, etc.) via a human-machine interface (HMI) display 334, an acoustic alarm, a loudspeaker, and / or via other components of the vehicle 300. The outputs can include information such as vehicle speed, velocity, time, map data (e.g., the HD map 322 of Fig. 3C), location data (e.g., the location data of vehicle 300, as shown on a map), direction, location of other vehicles (e.g., an occupancy grid), information about objects and the status of objects as perceived by the controller(s) 336, etc. The HMI display 334 can, for example, show information about the presence of one or more objects (e.g., a road sign, caution sign, traffic light sequence, etc.) and / or information about driving maneuvers that the vehicle has performed, is performing, or will perform (e.g., changing lanes now, exit 34B in two miles, etc.).

[0085] The vehicle 300 can further include a network interface 324, which can use one or more wireless antenna(s) 326 and / or one or more modem(s) to communicate over one or more networks. For example, the network interface 324 can be capable of communication over LTE, WCDMA, UMTS, GSM, CDMA2000, etc. The wireless antenna(s) 326 can also enable communication between objects in the environment (e.g., vehicles, mobile devices, etc.) using local area network(s), such as Bluetooth, Bluetooth LE, Z-Wave, ZigBee, etc., and / or for a low-power wide-area network (LPWAN), such as LoRaWAN, SigFox, etc.

[0086] Fig. 3B is an example of camera locations and fields of view for the exemplary autonomous vehicle 300 from Fig. 3A in accordance with embodiments of the present disclosure. The cameras and their respective fields of view are an exemplary embodiment and are not intended to be limiting. For example, additional and / or alternative cameras may be included and / or the cameras may be located at different locations on the vehicle 300.

[0087] The camera types may include, but are not limited to, digital cameras adapted for use with the components and / or systems of the Vehicle 300. The camera(s) may operate at Automotive Safety Integrity Level (ASIL) B and / or another ASIL. Depending on the configuration, the camera types may be capable of any frame rate, such as 60 frames per second (fps), 120 fps, 240 fps, etc. The cameras may be capable of using rolling shutters, global shutters, another type of shutter, or a combination thereof.In some embodiments, the color filter array may include a red-clear-clear-clear (RCCC) color filter array, a red-clear-clear-blue (RCCB) color filter array, a red-blue-green clear (RBGC) color filter array, a Foveon X3 color filter array, a Bayer sensor (RGGB) color filter array, a monochrome sensor color filter array, and / or another type of color filter array. In some embodiments, clear-pixel cameras, such as cameras with an RCCC, RCCB, and / or RBGC color filter array, may be used in an effort to increase light sensitivity.

[0088] In some examples, one or more of the cameras can be used to perform advanced driver assistance system (ADAS) functions (e.g., as part of a redundant or fail-safe system). For example, a multi-function monocular camera can be installed to provide functions such as lane departure warning, traffic sign recognition, and intelligent headlight control. One or more of the cameras (e.g., all of the cameras) can simultaneously record and provide image data (e.g., video).

[0089] One or more of the cameras can be mounted in a mounting assembly, such as a custom (3D-printed) assembly, to eliminate stray light and reflections from inside the car (e.g., reflections off the dashboard in the windshield mirrors) that can impair the camera's image capture capabilities. With regard to side mirror mounting assemblies, the side mirror assemblies can be custom 3D-printed so that the camera mounting plate matches the shape of the side mirror. In some examples, the camera(s) can be integrated into the side mirror. For side-view cameras, the camera(s) can also be integrated within four pillars at each corner of the cabin.

[0090] Cameras with a field of view that includes parts of the environment in front of the vehicle (e.g., forward-facing cameras) can be used for surround view to identify ahead routes and obstacles and, with the aid of one or more controllers and / or control SoCs, to assist in providing information critical for generating an occupancy grid and / or determining preferred vehicle routes. Forward-facing cameras can be used to perform many of the same ADAS functions as LiDAR, including emergency braking, pedestrian detection, and collision avoidance. Forward-facing cameras can also be used for ADAS functions and systems, including lane departure warnings (LDW), autonomous cruise control (ACC), and / or other functions, such as traffic sign recognition.

[0091] Various cameras can be used in a forward-facing configuration, for example, including a monocular camera platform that incorporates a CMOS (Complementary Metal Oxide Semiconductor) color imager. Another example could be one or more 370mm wide-angle cameras that can be used to detect objects entering the field of view from the periphery (e.g., pedestrians, crossing traffic, or bicycles). Although in Fig. While Figure 3B illustrates only one wide-angle camera, there can be any number of wide-angle cameras 370 on the vehicle 300. Furthermore, one or more remote camera(s) 398 (e.g., a pair of long-range stereo cameras) can be used for depth-based object detection, especially for objects for which a neural network has not yet been trained. The remote camera(s) 398 can also be used for object detection and classification, as well as simple object tracking.

[0092] One or more stereo cameras 368 can also be included in a forward-facing configuration. The stereo camera(s) 368 can include an integrated control unit comprising a scalable processing unit that can provide programmable logic (FPGA) and a multi-core microprocessor with an integrated CAN or Ethernet interface on a single chip. Such a unit can be used to generate a 3D map of the vehicle's surroundings, including a distance estimate for all points in the image. An alternative stereo camera(s) 368 can include a compact stereo vision sensor, which can include two camera lenses (one each on the left and right) and an image processing chip that measures the distance from the vehicle to the target object and processes the generated information (e.g.,Metadata) can be used to activate the autonomous emergency braking and lane departure warning functions. Other types of stereo camera(s) 368 can be used in addition to or as an alternative to those described herein.

[0093] Cameras with a field of view that includes parts of the surroundings at the side of the vehicle 300 (e.g., side-view cameras) can be used for surround view, which provides information used to generate and update the occupancy grid and to generate side-impact collision warnings. For example, one or more surround camera(s) 374 (e.g., four surround cameras 374 as in Fig. (Figure 3B illustrates) is positioned on the vehicle 300. The surround view camera(s) 374 can include one or more wide-angle cameras 370, fisheye cameras, 360-degree cameras, and / or the like. For example, four fisheye cameras can be positioned at the front, rear, and sides of the vehicle. In an alternative arrangement, the vehicle can use three surround view cameras 374 (e.g., left, right, and rear) and can use one or more other cameras (e.g., a forward-facing camera) as a fourth surround view camera.

[0094] Cameras with a field of view that includes parts of the surroundings at the rear of the vehicle 300 (e.g., rear-view cameras) can be used for parking assistance, surround view, rear collision warnings, and generating and updating the occupancy grid. A wide range of cameras can be used, including, but not limited to, cameras that are also suitable as forward-facing cameras (e.g., long-range and / or medium-range cameras 398, stereo cameras 368, infrared cameras 372, etc.), as described herein.

[0095] Fig. 3C is a block diagram of an exemplary system architecture for the exemplary autonomous vehicle 300 from Fig. 3A in accordance with some embodiments of the present disclosure. It should be understood that these and other arrangements described herein are presented only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, groupings of functions, etc.) may be used in addition to or instead of those shown, and other elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that may be implemented as discrete or distributed components, or in conjunction with other components, and in any suitable combination and position. Various functions described herein as being performed by entities may be executed by hardware, firmware, and / or software. For example, various functions may be performed by a processor executing instructions stored in memory.

[0096] The components, features and systems of the 300 vehicle in Fig. 3C are each illustrated as connected via bus 302. Bus 302 may include a Controller Area Network (CAN) data interface (alternatively referred to herein as the CAN bus). A CAN can be a network within the vehicle 300 that is used to support the control of various features and functionality of the vehicle 300, such as actuation of the brakes, acceleration, braking, steering, windshield wipers, etc. A CAN bus may be configured to have dozens or even hundreds of nodes, each with its own unique identifier (e.g., a CAN ID). The CAN bus can be read to obtain steering wheel angle, ground speed, engine revolutions per minute (RPM), button positions, and / or other vehicle status indicators. The CAN bus may be ASIL-B compliant.

[0097] Although described herein as a CAN bus, this is not intended to be restrictive. For example, FlexRay and / or Ethernet may be used in addition to or as an alternative to the CAN bus. Furthermore, although a single line is used to represent the 302 bus, this is not intended to be restrictive. For example, there may be any number of 302 buses, which may include one or more CAN buses, one or more FlexRay buses, one or more Ethernet buses, and / or one or more other types of buses using a different protocol. In some examples, two or more 302 buses may be used to perform different functions and / or for redundancy. For example, a first 302 bus may be used for collision avoidance functionality, and a second 302 bus may be used for actuation control.In any given example, the bus 302 can communicate with any of the vehicle 300's components, and two or more buses 302 can communicate with the same components. In some examples, each SoC 304, each controller 336, and / or each computer within the vehicle can have access to the same input data (e.g., inputs from vehicle 300 sensors) and be connected to a common bus, such as the CAN bus.

[0098] The vehicle 300 can include one or more controllers 336, such as those mentioned herein in relation to Fig. 3A. The controller(s) 336 can be used for a number of functions. The controller(s) 336 can be coupled with any of the various other components and systems of the vehicle 300 and used for controlling the vehicle 300, the vehicle 300's artificial intelligence, the vehicle 300's infotainment system, and / or the like.

[0099] The vehicle 300 can include one or more system-on-a-chip (SoC) 304. The SoC 304 can include one or more CPUs 306, GPUs 308, one or more processors 310, caches 312, accelerators 314, data storage 316, and / or other components and features not illustrated. The SoC(s) 304 can be used to control the vehicle 300 in a variety of platforms and systems. For example, the SoC(s) 304 in a system (e.g., the system of the vehicle 300) can be combined with an HD card 322, enabling map refreshes and / or updates via a network interface 324 from one or more servers (e.g., the server(s) 378). Fig. 3D) can be obtained.

[0100] The CPU(s) 306 may include a CPU cluster or CPU complex (alternatively referred to herein as "CCPLEX"). The CPU(s) 306 may include multiple cores and / or L2 caches. For example, in some embodiments, the CPU(s) 306 may include eight cores in a coherent multiprocessor configuration. In some embodiments, the CPU(s) 306 may include four dual-core clusters, each cluster having a dedicated L2 cache (e.g., a 2 MB L2 cache). The CPU(s) 306 (e.g., the CCPLEX) may be configured to support concurrent cluster operation, allowing a combination of CPU(s) 306 clusters to be active at any given time.

[0101] The CPU(s) 306 can implement power management capabilities that include one or more of the following features: individual hardware blocks can be automatically clock-gated (automatically disconnected from the clock signal in a gate) when idle to save dynamic power; each core clock can be disconnected when the core is not actively executing instructions due to the execution of WFI / WFE instructions; each core can be independently power-gated; each core cluster can be independently clock-gated when all cores are clock-gated or power-gated; and / or each core cluster can be independently power-gated when all cores are power-gated.The CPU(s) 306 can further implement an improved power state management algorithm, specifying the permissible power states and expected wake-up times, and the hardware / microcode for the core, cluster, and CCPLEX determining the best power state to enter. The processing cores can support simplified software sequences for entering power states, offloading the work to the microcode.

[0102] The GPU(s) 308 may include an integrated GPU (alternatively referred to herein as the "iGPU"). The GPU(s) 308 may be programmable and efficient for parallel workloads. The GPU(s) 308 may, in some examples, use an enhanced tensor instruction set. The GPU(s) 308 may include one or more streaming microprocessors, each of which may include an L1 cache (e.g., an L1 cache with a minimum storage capacity of 96 KB), and two or more of the streaming microprocessors may share an L2 cache (e.g., an L2 cache with a storage capacity of 512 KB). In some embodiments, the GPU(s) 308 may include at least eight streaming microprocessors. The GPU(s) 308 can use Compute Application Programming Interface(s) (API(s)). Furthermore, the GPU(s) 308 can utilize one or more parallel computing platforms and / or programming models (e.g.,NVIDIA's CUDA).

[0103] The GPU(s) 308 can be performance-optimized to achieve the best performance in automotive and embedded applications. For example, the GPU(s) 308 can be manufactured using a FinFET field-effect transistor. However, this is not intended to be a limitation, and the GPU(s) 308 can be manufactured using other semiconductor manufacturing processes. Each streaming microprocessor can contain a number of mixed-precision processing cores divided into multiple blocks. For example, and without limitation, 64 PF32 cores and 32 PF64 cores can be divided into four processing blocks. In such an example, each processing block could be allocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, two NVIDIA TENSOR CORES for mixed-precision deep learning matrix arithmetic, an L0 instruction cache, a warp scheduler, a dispatch unit and / or a 64 KB register file.Furthermore, streaming microprocessors can include independent parallel integer and floating-point data paths to enable efficient execution of workloads with a mix of computational and addressing operations. Streaming microprocessors can include independent thread scheduling functions to allow for finer-grained synchronization and collaboration between parallel threads. Streaming microprocessors can also include a combined L1 data cache and shared memory to improve performance while simplifying programming.

[0104] The GPU(s) 308 can include high-bandwidth memory (HBM) and / or a 16 GB HBM2 memory subsystem to provide a peak memory bandwidth of approximately 900 GB / second in some examples. In some examples, synchronous graphics random access memory (SGRAM), such as double-data-rate type five synchronous graphics random access memory (GDDR5), can be used in addition to or as an alternative to the HBM memory.

[0105] The GPU(s) 308 can include unified memory technology, including access counters, to allow more accurate migration of memory pages to the processor that accesses them most frequently, thereby improving efficiency for memory areas shared by processors. In some examples, support for address translation services (ATS) can be used to allow the GPU(s) 308 to directly access the page tables of the CPU(s) 306. In such examples, an address translation request can be sent to the CPU(s) 306 if the memory management unit (MMU) of the GPU(s) 308 experiences an erroneous access. In response, the CPU(s) 306 can search its page tables for the virtual-to-physical mapping for the address and send the translation back to the GPU(s) 308.As such, Unified Memory technology can allow a single unified virtual address space for the memory of both the CPU(s) 306 and the GPU(s) 308, thereby simplifying the programming of the GPU(s) 308 and the porting of applications to the GPU(s) 308.

[0106] Furthermore, GPU(s) 308 can include an access counter that tracks the frequency of GPU(s) 308 access to the memory of other processors. The access counter can help ensure that memory pages are moved to the physical memory of the processor that accesses them most frequently.

[0107] The SoC(s) 304 can include any number of Cache(s) 312, including those described herein. For example, the Cache(s) 312 can include an L3 cache that is available to both the CPU(s) 306 and the GPU(s) 308 (e.g., connected to both the CPU(s) 306 and the GPU(s) 308). The Cache(s) 312 can include a write-back cache capable of tracking row states, e.g., using a cache coherence protocol (e.g., MEI, MESI, MSI, etc.). The L3 cache can include 4 MB or more, although smaller cache sizes can be used.

[0108] The SoC(s) 304 may include one or more arithmetic logic units (ALUs) that can be used to perform processing with respect to any of the tasks or operations of the vehicle 300, such as processing DNNs. Furthermore, the SoC(s) 304 may include one or more floating-point units (FPUs) – or other mathematical or numerical coprocessor types – for performing mathematical operations within the system. For example, the SoC(s) 304 may include one or more FPUs that are integrated as execution units within one or more CPU(s) 306 and / or GPU(s) 308.

[0109] The SoC(s) 304 can include one or more accelerators 314 (e.g., hardware accelerators, software accelerators, or a combination thereof). For example, the SoC(s) 304 can include a hardware acceleration cluster, which may include optimized hardware accelerators and / or a large amount of on-chip memory. The large on-chip memory (e.g., 4 MB SRAM) can enable the hardware acceleration cluster to accelerate neural networks and other computations. The hardware acceleration cluster can be used to complement the GPU(s) 308 and offload some tasks from the GPU(s) 308 (e.g., to free up more GPU cycles for other tasks). As an example, the accelerator(s) 314 can be used for targeted workloads (e.g. perception, convolutional neural networks (CNNs) etc.).), which are stable enough to be amenable to acceleration. The term “CNN”, as used herein, can include all types of CNNs, including region-based or regional convolutional neural networks (RCNNs) and fast RCNNs (e.g., such as those used for object detection).

[0110] The Accelerator 314 (e.g., the hardware acceleration cluster) can include one or more deep learning accelerators (DLAs). The DLA(s) can include one or more tensor processing units (TPUs), which can be configured to provide an additional ten trillion operations per second for deep learning applications and inference. The TPUs can be accelerators configured and optimized to perform image processing functions (e.g., for CNNs, RCNNs, etc.). The DLA(s) can further be optimized for a specific set of neural network types and floating-point operations, as well as for inference. The design of the DLA(s) can provide more performance per millimeter than a general-purpose GPU and far surpasses the performance of a CPU.The TPU(s) can perform several functions, including a single-instance folding function that supports, for example, INT8, INT16 and FP16 data types for both features and weights, as well as post-processor functions.

[0111] The DLA(s) can quickly and efficiently execute neural networks, especially CNNs, on processed or unprocessed data for any variety of functions, including, but not limited to: a CNN for object identification and detection using camera sensor data; a CNN for distance estimation using camera sensor data; a CNN for emergency vehicle detection and identification using microphone data; a CNN for facial recognition and vehicle owner identification using camera sensor data; and / or a CNN for security-related events.

[0112] The DLA(s) can perform any function of the GPU(s) 308, and using an inference accelerator, a designer can, for example, use either the DLA(s) or the GPU(s) 308 for any given function. A designer can, for instance, focus the processing of CNNs and floating-point operations on the DLA(s) and leave other functions to the GPU(s) 308 and / or another accelerator 314.

[0113] The Accelerator 314 (e.g., the hardware acceleration cluster) can include one or more programmable vision accelerators (PVAs), which may alternatively be referred to herein as computer vision accelerators. The PVA(s) can be designed and configured to accelerate computer vision algorithms for advanced driver assistance systems (ADAS), autonomous driving applications, augmented reality (AR), and / or virtual reality (VR). The PVA(s) can provide a balance between performance and flexibility. For example, and without limitation, each PVA can include any number of reduced instruction set compute cores (RISC), direct memory access (DMA), and / or any number of vector processors.

[0114] The RISC cores can interact with image sensors (e.g., the image sensors of any of the cameras described herein), image signal processor(s), and / or the like. Each RISC core can include any memory location. The RISC cores can use any of a number of protocols, depending on the implementation. In some examples, the RISC cores can run a real-time operating system (RTOS). The RISC cores can be implemented using one or more integrated circuit devices, application-specific integrated circuits (ASICs), and / or memory devices. For example, the RISC cores can include an instruction cache and / or tightly coupled RAM.

[0115] The DMA can enable components of the PVA(s) to access system memory independently of the CPU(s). The DMA can support any number of features provided to optimize the PVA, including, but not limited to, support for multidimensional addressing and / or circular addressing. In some examples, the DMA can support up to six or more addressing dimensions, which may include block width, block height, block depth, horizontal block gradation, vertical block gradation, and / or depth gradation.

[0116] Vector processors can be programmable processors designed to efficiently and flexibly execute computer vision algorithms and provide signal processing capabilities. In some examples, the PVA may include a PVA core and two vector processing subsystem partitions. The PVA core may include a processor subsystem, DMA engine(s) (e.g., two DMA engines), and / or other peripherals. The vector processing subsystem may act as the primary processing engine of the PVA and may include a vector processing unit (VPU), an instruction cache, and / or vector memory (e.g., VMEM). A VPU core may include a digital signal processor, such as a single-instruction multiple data (SIMD) or very-long instruction word (VLIW) signal processor. The combination of SIMD and VLIW can improve throughput and speed.

[0117] Each vector processor can include an instruction cache and be coupled to dedicated memory. Consequently, in some examples, each vector processor can be configured to operate independently of the others. In other examples, vector processors enclosed in a given PVA can be configured to utilize data parallelism. For example, in some embodiments, the multitude of vector processors enclosed in a single PVA can execute the same computer vision algorithm, but in different regions of an image. In other examples, the vector processors enclosed in a given PVA can simultaneously execute different computer vision algorithms on the same image, or even different algorithms on sequential images or portions of an image.Among other things, any number of PVAs can be included in a hardware acceleration cluster, and any number of vector processors can be included in each of the PVAs. Furthermore, the PVA(s) can include additional ECC (Error Correction Code) memory to improve overall system security.

[0118] The Accelerator 314 (e.g., the hardware acceleration cluster) can include a computer vision network on a single chip and SRAM to provide high-bandwidth, low-latency SRAM for the Accelerator 314. In some examples, the on-chip memory can include at least 4 MB of SRAM, consisting, for example, and without limitation, of eight field-configurable memory blocks accessible by both the PVA and the DLA. Each pair of memory blocks can include an Advanced Peripheral Bus (APB) interface, a configuration circuit, a controller, and a multiplexer. Any type of memory can be used. The PVA and DLA can access the memory via a backbone, providing high-speed memory access for both the PVA and DLA.The backbone can include a computer vision network on a chip that connects the PVA and DLA to the memory (e.g., using the APB).

[0119] The computer vision network can include an interface on a single chip that, prior to the transmission of control signals / addresses / data, ensures that both the PVA and the DLA provide ready-to-use and valid signals. Such an interface can provide separate phases and channels for the transmission of control signals / addresses / data, as well as burst communication for continuous data transfer. This type of interface can conform to ISO 26262 or IEC 61508 standards, although other standards and protocols can also be used.

[0120] In some examples, the SoC(s) 304 may include a real-time ray-tracing hardware accelerator as described in US Patent Application No. 16 / 101,232, filed on August 10, 2018. The real-time ray-tracing hardware accelerator can be used for fast and efficient determination of the positions and extents of objects (e.g., within a world model), for generating real-time visualization simulations, for interpreting radar signals, for synthesizing and / or analyzing sound propagation, for simulating SONAR systems, for general wave propagation simulation, for comparison with lidar data for localization purposes, and / or for other functions and / or other purposes. In some embodiments, one or more tree traversal units (TTUs) may be used to perform one or more ray-tracing-related operations.

[0121] The Accelerator 314 (e.g., the hardware accelerator cluster) has a wide range of applications in autonomous driving. The PVA can be a programmable vision accelerator used for key processing stages in ADAS and autonomous vehicles. The PVA's capabilities are well-suited for algorithmic domains requiring predictable processing with low power consumption and low latency. In other words, the PVA performs well with semi-dense or dense regular computations, even with small datasets, that require predictable runtimes with low latency and low power consumption. In the context of autonomous vehicle platforms, PVAs are therefore designed to execute classical computer vision algorithms, as they are efficient at object detection and operate with integer mathematics.

[0122] For example, according to one embodiment of the technology, the PVA is used to perform computer stereo vision. A semi-global matching-based algorithm can be used in some examples, although this is not intended to be a limiting factor. Many Level 3-5 autonomous driving applications require motion estimation / stereo matching during operation (on-the-fly) (e.g., structure from motion, pedestrian detection, lane detection, etc.). The PVA can perform computer stereo vision functions with input from two monocular cameras.

[0123] In some examples, the PVA can be used to perform dense optical flow processing, such as processing raw radar data (e.g., using 4D Fast Fourier Transform) to produce processed radar data. In other examples, the PVA is used for time-of-flight depth processing, by processing the raw time-of-flight data to provide, for example, processed time-of-flight data.

[0124] The DLA can be used to operate any type of network to improve steering and driving safety, for example, a neural network that outputs a confidence score for each object detection. Such a confidence score can be interpreted as a probability or as providing a relative "weight" to each detection compared to other detections. This confidence score allows the system to make further decisions about which detections should be considered true positives and not false positives. For example, the system can set a confidence threshold and only consider detections that exceed this threshold as true positives.In an automatic emergency braking (AEB) system, false positive detections would cause the vehicle to automatically initiate emergency braking, which is obviously undesirable. Therefore, only the most confident detections should be considered as triggers for AEB. The DLA can operate a neural network to regress the confidence level. The neural network can use as input at least a subset of parameters, such as the bounding frame dimensions, the ground plane estimate obtained (e.g. from another subsystem), the output of an inertial measurement unit (IMU) sensor 366 correlated with the vehicle's orientation 300, the distance, 3D position estimates of the object obtained from the neural network and / or other sensors (e.g., LIDAR sensor(s) 364 or RADAR sensor(s) 360).

[0125] The SoC(s) 304 may include one or more data stores 316 (e.g., memory). The data store 316 may be on-chip memory of the SoC(s) 304 capable of storing neural networks to be executed on the GPU and / or the DLA. In some examples, the data store 316 may be large enough to store multiple instances of neural networks for redundancy and safety. The data store 316 may include L2 or L3 cache(s) 312. Reference to the data store 316 may include reference to the memory associated with the PVA, DLA, and / or other accelerator(s) 314, as described herein.

[0126] The SoC(s) 304 can include one or more Processor(s) 310 (e.g., embedded processors). The Processor(s) 310 can include a Boot and Power Management Processor, which may be a dedicated processor and subsystem that handles boot power and management functions and associated security enforcement. The Boot and Power Management Processor can be part of the SoC(s) 304's boot sequence and provide runtime power management services. The Boot and Power Management Processor can provide clock and voltage programming, support for low-power system state transitions, management of the SoC(s) 304's thermals and temperature sensors, and / or management of the SoC(s) 304's power supply states.Each temperature sensor can be implemented as a ring oscillator whose output frequency is proportional to the temperature, and the SoC(s) 304 can use ring oscillators to detect temperatures of the CPU(s) 306, GPU(s) 308, and / or accelerator(s) 314. If it is determined that the temperatures exceed a threshold, the boot and power management processor can enter a temperature fault routine and put the SoC(s) 304 into a lower power consumption state and / or put the vehicle 300 into a chauffeur-to-safe-stop mode (e.g., bring the vehicle 300 to a safe stop).

[0127] The 310 processor(s) can further include a set of embedded processors that can serve as an audio processing engine. The audio processing engine can be an audio subsystem that provides full hardware support for multi-channel audio across multiple interfaces and a wide and flexible range of audio I / O interfaces. In some examples, the audio processing engine is a dedicated processor core with a digital signal processor and dedicated RAM.

[0128] The 310 processor(s) may further include an always-on processor engine that can provide the necessary hardware features to support the management of low-power sensors and wake-up use cases. The always-on processor engine may include a processor core, tightly coupled RAM, supporting peripherals (e.g., timers and interrupt controllers), various I / O controller peripherals, and routing logic.

[0129] The 310 processor(s) can further include a security cluster engine, which comprises a dedicated processor subsystem for handling security management for automotive applications. The security cluster engine can include two or more processor cores, tightly coupled RAM, supporting peripherals (e.g., timers, an interrupt controller, etc.), and / or routing logic. In a security mode, the two or more cores can operate in lockstep mode, functioning as a single core with comparison logic to detect differences between their operations.

[0130] The 310 processor(s) may also include a real-time camera engine, which may include a dedicated processor subsystem for handling real-time camera management.

[0131] The 310 processor(s) may also include a high dynamic range signal processor, which may include an image signal processor, which is a hardware engine that is part of the camera processing pipeline.

[0132] The 310 processor(s) can include a video image compositor, which may be a processing block (e.g., implemented on a microprocessor) that implements video post-processing functions required by a video playback application to produce the final image for the playback window. The video image compositor can perform lens distortion correction on the 370 wide-angle camera(s), the 374 surround-view camera(s), and / or the in-cabin surveillance camera sensors. An in-cabin surveillance camera sensor is preferably monitored by a neural network running on a separate instance of the Advanced SoC and configured to identify and respond to in-cabin events.An in-cabin system can perform lip reading to activate cellular service and make phone calls, dictate emails, change the vehicle's destination, activate or change the infotainment system and vehicle settings, or provide voice-activated web browsing. Certain functions are only available to the driver when the vehicle is operating in autonomous mode and are otherwise disabled.

[0133] The video image compositor can include enhanced temporal noise reduction for both spatial and temporal noise reduction. For example, if motion occurs in a video, the noise reduction can weight spatial information accordingly, thereby reducing the impact of information provided by adjacent frames. If an image or part of an image does not contain motion, the temporal noise reduction performed by the video image compositor can use information from the previous image to reduce noise in the current image.

[0134] The video image compositor can also be configured to perform stereo equalization on the input stereo lens frames. Furthermore, the video image compositor can be used for user interface composition when the operating system desktop is in use and the GPU(s) 308 are not required to continuously render new surfaces. The video image compositor can also be used to offload the GPU(s) 308, even when the GPU(s) 308 are powered on and actively performing 3D rendering, to improve performance and responsiveness.

[0135] The SoC(s) 304 may further include a Mobile Industry Processor Interface (MIPI) camera serial interface for receiving video and camera input, a high-speed interface, and / or a video input block that can be used for camera and associated pixel input functions. The SoC(s) 304 may also include one or more input / output controllers that can be software-controlled and used to receive I / O signals not designated for a specific role.

[0136] The SoC(s) 304 can also include a wide selection of peripheral interfaces to enable communication with peripheral devices, audio codecs, power management, and / or other devices. The SoC(s) 304 can be used to process data from cameras (e.g., connected via Gigabit Multimedia Serial Link and Ethernet), sensors (e.g., LiDAR sensor(s) 364, RADAR sensor(s) 360, etc., which may be connected via Ethernet), data from bus 302 (e.g., vehicle speed 300, steering wheel position, etc.), data from GNSS sensor(s) 358 (e.g., connected via an Ethernet bus or a CAN bus), etc. The SoC(s) 304 can also include dedicated high-performance mass storage controllers, which can include their own DMA engines and can be used to offload routine data management tasks from the CPU(s) 306.

[0137] The SoC(s) 304 can be an end-to-end platform with a flexible architecture encompassing automation levels 3-5, providing a comprehensive functional safety architecture that leverages and efficiently utilizes computer vision and ADAS techniques for diversity and redundancy, and offers a platform for a flexible, reliable powertrain software stack, along with deep learning tools. The SoC(s) 304 can be faster, more reliable, and even more energy- and space-efficient than conventional systems. For example, the accelerator(s) 314, in combination with the CPU(s) 306, GPU(s) 308, and data storage(s) 316, can provide a fast, efficient platform for autonomous vehicles with levels 3-5.

[0138] This technology thus provides capabilities and functionality that cannot be achieved by conventional systems. For example, computer vision algorithms can be run on CPUs that can be configured using a higher-level programming language, such as C, to execute a wide range of processing algorithms on a wide range of visual data. However, CPUs are often unable to meet the performance requirements of many computer vision applications, such as those related to execution time and power consumption. In particular, many CPUs are unable to execute complex object detection algorithms in real time, which is a requirement for in-vehicle ADAS applications and for practical Level 3-5 autonomous vehicles.

[0139] Unlike conventional systems, the technology described herein, by providing a CPU complex, a GPU complex, and a hardware acceleration cluster, allows multiple neural networks to run simultaneously and / or sequentially and to combine the results to enable Level 3-5 autonomous driving functionality. For example, a CNN running on the DLA or the dGPU (e.g., the GPU(s) 320) can include text and word recognition, allowing the supercomputer to read and understand traffic signs, including those for which the neural network has not been specifically trained. The DLA can further include a neural network capable of identifying and interpreting a sign, providing a semantic understanding of the sign, and passing this semantic understanding to the route planning modules running on the CPU complex.

[0140] As another example, multiple networks can operate simultaneously, as required for Level 3, 4, or 5 driving. For instance, a warning sign consisting of "Caution: Flashing lights indicate black ice" along with an electric light can be interpreted independently or jointly by several neural networks. The sign itself can be identified as a traffic sign by a first neural network (e.g., a trained neural network), while the text "Flashing lights indicate black ice" can be interpreted by a second neural network, which informs the vehicle's route planning software (preferably running on the CPU complex) that black ice is present when flashing lights are detected.The flashing light can be identified by operating a third, deployed neural network over several frames, thus informing the vehicle's route planning software of the presence (or absence) of flashing lights. All three neural networks can operate simultaneously, for example, within the DLA and / or on the GPU(s) 308.

[0141] In some examples, a CNN for facial recognition and vehicle owner identification can use data from camera sensors to identify the presence of an authorized driver and / or vehicle owner of the vehicle 300. The always-on sensor processing engine can be used to unlock the vehicle when the owner approaches the driver's door and turns on the lights, and to disable the vehicle in security mode when the owner leaves the vehicle. In this way, the SoC(s) 304 provides security against theft and / or forced vehicle removal.

[0142] In another example, a CNN for emergency vehicle detection and identification can use data from microphones 396 to detect and identify emergency vehicle sirens. Unlike conventional systems that use general classifiers to detect sirens and manually extract features, the SoC(s) 304 can use the CNN to classify ambient and urban noise as well as visual data. In a preferred embodiment, the CNN, operating on the DLA, is trained to identify the relative approach speed of the emergency vehicle (e.g., using the Doppler effect). The CNN can also be trained to identify emergency vehicles specific to the local area in which the vehicle operates, as identified by the GNSS sensor(s) 358.For example, when operating in Europe, the CNN will attempt to detect European sirens, and when operating in the United States, the CNN will attempt to identify only North American sirens. A control program, supported by 362 ultrasonic sensors, can be used, once an emergency vehicle is detected, to execute a safety routine for the emergency vehicle, such as slowing down, pulling over to the side of the road, parking, and / or idling until the emergency vehicle(s) have passed.

[0143] The vehicle can include one or more CPU(s) 318 (e.g., discrete CPU(s) or dCPU(s)) that can be coupled to the SoC(s) 304 via a high-speed connection (e.g., PCIe). The CPU(s) 318 can, for example, include an x86 processor. The CPU(s) 318 can be used to perform any of a number of functions, including, for example, arbitrating potentially inconsistent results between ADAS sensors and the SoC(s) 304 and / or monitoring the status and state of the Controller(s) 336 and / or Infotainment SoC 330.

[0144] The Vehicle 300 can include one or more GPU(s) 320 (e.g., discrete GPU(s) or dGPU(s)) that can be coupled to the SoC(s) 304 via a high-speed connection (e.g., NVIDIA's NVLINK). The GPU(s) 320 can provide additional artificial intelligence functionality, such as by running redundant and / or different neural networks, and can be used to train and / or update neural networks based on input (e.g., sensor data) from sensors of the Vehicle 300.

[0145] The vehicle 300 can further include the network interface 324, which can include one or more wireless antennas 326 (e.g., one or more wireless antennas for different communication protocols, such as a cellular antenna, a Bluetooth antenna, etc.). The network interface 324 can be used to enable wireless connectivity via the internet to the cloud (e.g., to the server(s) 378 and / or other network devices), to other vehicles, and / or to computing devices (e.g., passenger client devices). Communication with other vehicles can be established via a direct connection between the two vehicles and / or an indirect connection (e.g., via networks and the internet). Direct connections can be provided using a vehicle-to-vehicle communication link.The vehicle-to-vehicle communication link can provide the vehicle 300 with information about vehicles in its vicinity (e.g., vehicles in front of, to the side of, and / or behind the vehicle 300). This functionality can be part of a cooperative adaptive cruise control functionality of the vehicle 300.

[0146] The network interface 324 can include a SoC that provides modulation and demodulation functionality, enabling the controller(s) 336 to communicate over wireless networks. The network interface 324 can include a radio frequency front end for upconverting baseband to radio frequency and downconverting radio frequency to baseband. The frequency conversions can be performed using known processes and / or superheterodyne processes. In some examples, the radio frequency front-end functionality can be provided by a separate chip. The network interface can include wireless functionality for communication over LTE, WCDMA, UMTS, GSM, CDMA2000, Bluetooth, Bluetooth LE, Wi-Fi, Z-Wave, ZigBee, LoRaWAN, and / or other wireless protocols.

[0147] The vehicle 300 may further include one or more data storage devices 328, which may include off-chip (e.g., off-SoC(s) 504) storage. The data storage device(s) 328 may include one or more storage elements, including RAM, SRAM, DRAM, VRAM, flash, hard disks, and / or other components and / or devices capable of storing at least one data bit.

[0148] The vehicle 300 can also include one or more GNSS sensors 358. The GNSS sensor(s) 358 (e.g., GPS, supported GPS sensors, differential GPS sensors (DGPS), etc.) serve to support mapping, perception, occupancy grid generation, and / or route planning functions. Any number of GNSS sensors 358 can be used, including, for example, and without limitation, one GPS using a USB port with an Ethernet-to-serial (e.g., RS-232) bridge.

[0149] The vehicle 300 can further include one or more RADAR sensor(s) 360. The RADAR sensor(s) 360 can be used by the vehicle 300 for long-range vehicle detection, even in darkness and / or adverse weather conditions. The functional RADAR safety levels can be ASIL B. In some examples, the RADAR sensor(s) 360 can use CAN and / or bus 302 (e.g., to transmit data generated by the RADAR sensor(s) 360) for control and access to object tracking data, with access to Ethernet for raw data. A wide range of RADAR sensor types can be used. For example, and without limitation, the RADAR sensor(s) 360 can be suitable for front, rear, and side RADAR applications. In some examples, a pulse Doppler radar sensor(s) is / are used.

[0150] The 360° radar sensor(s) can include various configurations, such as long-range with a narrow field of view, short-range with a wide field of view, side coverage with short-range, etc. In some examples, long-range radar can be used for adaptive cruise control functionality. Long-range radar systems can provide a wide field of view, achieved through two or more independent scans, for example, within a range of 250 m. The 360° radar sensor(s) can assist in distinguishing between stationary and moving objects and can be used by ADAS systems for emergency braking assistance and forward collision warning. Long-range radar sensors can include monostatic multimodal radar with multiple (e.g., six or more) fixed radar antennas and a high-speed CAN and FlexRay interface.In an example with six antennas, the central four antennas can generate a focused beam pattern designed to record the area around vehicle 300 at higher speeds with minimal interference from traffic in adjacent lanes. The other two antennas can expand the field of view, allowing vehicles entering or leaving vehicle 300's lane to be detected quickly.

[0151] Mid-range radar systems, for example, can include a range of up to 160 m (front) or 80 m (rear) and a field of view of up to 42 degrees (front) or 150 degrees (rear). Short-range radar systems can, without limitation, include radar sensors designed for installation at both ends of the rear bumper. When such radar sensor systems are installed at both ends of the rear bumper, they can generate two beams that constantly monitor the blind spot in one direction, rearward and close to the vehicle.

[0152] Short-range radar systems can be used in the ADAS system to detect a blind spot and / or to assist with a lane change.

[0153] The vehicle 300 can further include one or more ultrasonic sensor(s) 362. The ultrasonic sensor(s) 362 can be positioned on the front, rear, and / or sides of the vehicle 300 and can be used for parking assistance and / or for generating and updating an occupancy grid. A wide range of ultrasonic sensor(s) 362 can be used, and different ultrasonic sensor(s) 362 can be used for different detection ranges (e.g., 2.5 m, 4 m). The ultrasonic sensor(s) 362 can operate according to the functional safety levels of ASIL B.

[0154] The vehicle 300 can include one or more LiDAR sensors 364. The LiDAR sensor(s) 364 can be used for object and pedestrian detection, emergency braking, collision avoidance, and / or other functions. The LiDAR sensor(s) 364 can operate at functional safety level ASIL B. In some examples, the vehicle 300 can include multiple LiDAR sensors 364 (e.g., two, four, six, etc.) that can use Ethernet (e.g., to provide data to a Gigabit Ethernet switch).

[0155] In some examples, the LiDAR sensor(s) 364 can provide a list of objects and their distances for a 360-degree field of view. A commercially available LiDAR sensor(s) 364, for example, may have a specified range of approximately 100 m with an accuracy of 2 cm to 3 cm and support for a 100 Mbps Ethernet connection. In some examples, one or more non-protruding LiDAR sensors 364 may be used. In such examples, the LiDAR sensor(s) 364 may be implemented as a small device that can be embedded in the front, rear, sides, and / or corners of the vehicle 300. The LIDAR sensor(s) 364 can provide a horizontal field of view of up to 120 degrees and a vertical field of view of up to 35 degrees in such examples, with a range of 200 m, even for objects with low reflectivity.One or more front-mounted LIDAR sensor(s) 364 can be configured for a horizontal field of view between 45 degrees and 135 degrees.

[0156] In some examples, LiDAR technologies, such as 3D flash LiDAR, can also be used. 3D flash LiDAR uses a laser flash as a transmission source to illuminate the vehicle's surroundings up to approximately 200 m. A flash LiDAR unit includes a sensor that records the laser pulse travel time and the reflected light at each pixel, which corresponds to the vehicle's range to objects. With flash LiDAR, highly accurate and distortion-free images of the surroundings can be generated with each laser flash. In some examples, four flash LiDAR sensors can be used, one on each side of the vehicle. Available 3D flash LiDAR systems include a solid-state 3D LiDAR camera with a rigid array and no moving parts except for a fan (e.g., a non-scanning LiDAR device).The Blitz LIDAR device can use a 5-nanosecond Class I (eye-safe) laser pulse per frame and capture reflected laser light in the form of 3D distance point clouds and co-recorded intensity data. By using Blitz LIDAR, and because Blitz LIDAR is a solid-state device with no moving parts, the LIDAR sensor(s) may be less susceptible to motion blur, vibration, and / or shock.

[0157] The vehicle may further include one or more IMU sensor(s) 366. The IMU sensor(s) 366 may be located in the center of the rear axle of the vehicle 300 in some examples. The IMU sensor(s) 366 may include, for example, without limitation, one or more accelerometers, one or more magnetometers, one or more gyroscopes, one or more magnetic compasses, and / or other sensor types. In some examples, such as in six-axis applications, the IMU sensor(s) 366 may include accelerometers and gyroscopes, while in nine-axis applications, the IMU sensor(s) 366 may include accelerometers, gyroscopes, and magnetometers.

[0158] In some embodiments, the IMU sensor(s) 366 can be implemented as a miniature, high-performance GPS-aided inertial navigation system (GPS / INS) that combines microelectromechanical system (MEMS) inertial sensors, a high-sensitivity GPS receiver, and advanced Kalman filter algorithms to provide estimates of position, velocity, and orientation. As such, in some examples, the IMU sensor(s) 366 can enable the vehicle 300 to estimate its course without requiring input from a magnetic sensor by directly observing and correlating velocity changes from a GPS to the IMU sensor(s) 366. In some examples, the IMU sensor(s) 366 and the GNSS sensor(s) 358 can be combined in a single integrated unit.

[0159] The vehicle can include one or more microphones 396, which are placed in and / or around the vehicle 300. The microphone(s) 396 can be used, among other things, for emergency vehicle detection and identification.

[0160] The vehicle may also include any number of camera types, including one or more stereo cameras 368, wide-angle cameras 370, infrared cameras 372, surround-view cameras 374, long-range cameras and / or medium-range cameras 398, and / or other camera types. The cameras may be used to capture image data around the entire periphery of the vehicle 300. The types of cameras used depend on the embodiment and requirements for the vehicle 300, and any combination of camera types may be used to provide the necessary coverage around the vehicle 300. Furthermore, the number of cameras may vary depending on the embodiment. For example, the vehicle may include six cameras, seven cameras, ten cameras, twelve cameras, and / or any other number of cameras.The cameras can, for example, support Gigabit Multimedia Serial Link (GMSL) and / or Gigabit Ethernet communication without restriction. Each camera is described in more detail below. Fig. 3A and Fig. 3B described.

[0161] The vehicle 300 can further include one or more vibration sensors 342. The vibration sensor(s) 342 can measure vibrations of vehicle components, such as the axle(s). For example, changes in vibrations can indicate a change in the road surface. In another example, if two or more vibration sensors 342 are used, the differences between the vibrations can be used to determine the friction or slipperiness of the road surface (e.g., if the difference in vibration is between a driven axle and a freely rotating axle).

[0162] The vehicle 300 may include an ADAS system 338. The ADAS system 338 may include a SoC in some examples. The ADAS system 338 may include systems for autonomous / adaptive / automatic cruise control (ACC), cooperative adaptive cruise control (CACC), forward crash warning (FCW), automatic emergency braking (AEB), lane departure warnings (LDW), lane keep assist (LKA), blind spot warning (BSW), rear cross-traffic warning (RCTW), collision warning systems (CWS), lane centering (LC), and / or other features and functions.

[0163] The ACC systems can use one or more radar sensors (360°), lidar sensors (364°), and / or one or more cameras. The ACC systems can include longitudinal ACC and / or lateral ACC. Longitudinal ACC monitors and controls the distance to the vehicle immediately in front of the vehicle (300°) and automatically adjusts the vehicle's speed to maintain a safe distance from vehicles ahead. Lateral ACC maintains the distance and advises the vehicle (300°) to change lanes if necessary. Lateral ACC is related to other ADAS applications, such as LCA and CWS.

[0164] CACC uses information from other vehicles, which can be received via the network interface 324 and / or the wireless antenna(s) 326 from other vehicles either wirelessly or indirectly via a network connection (e.g., via the internet). Direct connections can be provided through a vehicle-to-vehicle (V2V) communication link, while indirect connections can be an infrastructure-to-vehicle (I2V) communication link. Generally, the V2V communication concept provides information about vehicles immediately ahead (e.g., vehicles directly in front of and in the same lane as vehicle 300), while the I2V communication concept provides information about traffic at a greater distance. CACC systems can incorporate one or both of the I2V and V2V information sources.Thanks to the information about the vehicles in front of the vehicle 300, the CACC can be more reliable and has the potential to improve traffic flow and reduce congestion on the road.

[0165] FCW systems are designed to alert the driver to a hazard, allowing the driver to take corrective action. FCW systems utilize a forward-facing camera and / or 360° radar sensors, coupled with a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically connected to driver feedback, such as a display, speaker, and / or vibrating component. FCW systems can provide a warning, for example, in the form of an audible signal, a visual warning, a vibration, and / or a rapid braking pulse.

[0166] AEB systems detect an impending forward collision with another vehicle or object and can automatically apply the brakes if the driver does not take corrective action within a specified time or distance parameter. AEB systems use one or more forward-facing cameras and / or one or more 360° radar sensors coupled to a dedicated processor, DSP, FPGA, and / or ASIC. When the AEB system detects a hazard, it will typically first warn the driver to take corrective action to avoid the collision. If the driver fails to take corrective action, the AEB system will automatically apply the brakes in an effort to avoid or at least mitigate the impact of the predicted collision. AEB systems may include techniques such as dynamic brake assist and / or anticipatory braking.

[0167] Lane Departure Warning (LDW) systems provide visual, audible, and / or tactile warnings, such as steering wheel or seat vibrations, to alert the driver if the vehicle crosses lane markings. An LDW system will not activate if the driver indicates an intention to leave the lane, for example, by using a turn signal. LDW systems may use forward-facing cameras coupled to a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically coupled to driver feedback, such as a display, speaker, and / or vibration component.

[0168] Lane Keeping Assist (LKA) systems are a variant of Lane Departure Warning (LDW) systems. LKA systems provide steering or braking input to correct the vehicle if it begins to leave its lane. Blind Spot Warning (BSW) systems detect vehicles in a vehicle's blind spot and warn the driver. BSW systems can provide a visual, audible, and / or tactile alert to indicate that merging or changing lanes is unsafe. The system can provide an additional warning if the driver uses a turn signal. BSW systems may use rear-facing camera(s) and / or radar sensor(s).

[0169] RCTW systems can provide visual, audible, and / or tactile alerts when an object outside the reversing camera's field of view is detected while the vehicle is reversing. Some RCTW systems include AEB to ensure the vehicle's brakes are applied to avoid a collision. RCTW systems can use one or more rear-facing radar sensors coupled to a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically coupled to driver feedback, such as a display, speaker, and / or vibrating component.

[0170] Conventional ADAS systems can be prone to false positives, which can be annoying and distracting for a driver, but are generally not catastrophic because the ADAS systems warn the driver and allow the driver to decide whether a safety condition actually exists and to act accordingly. In an autonomous vehicle 300, however, the vehicle 300 itself must decide, in the case of conflicting results, whether the result should be considered by a primary computer or a secondary computer (e.g., a first controller 336 or a second controller 336). For example, the ADAS system 338 in some embodiments may be a backup and / or secondary computer that provides perceptual information to a rationality module of the backup computer. The rationality monitor of the backup computer may run redundant software on hardware components to detect perceptual errors and dynamic driving tasks.Output from the ADAS system 338 can be provided to a higher-level MCU. If output from the primary and secondary computers conflict, the higher-level MCU must decide how to resolve the conflict to ensure safe operation.

[0171] In some examples, the primary computer can be configured to provide the higher-level MCU with a confidence score indicating its level of trust in the chosen result. If the confidence score exceeds a certain threshold, the higher-level MCU can follow the primary computer's guidance, regardless of whether the secondary computer provides a conflicting or inconsistent result. If the confidence score does not reach a certain threshold, and if the primary and secondary computers display different results (e.g., a conflict), the higher-level MCU can arbitrate between the computers to determine the appropriate result.

[0172] The higher-level MCU can be configured to run a neural network(s) trained and configured to determine, based on output from the primary and secondary computers, the conditions under which the secondary computer will generate false alarms. Thus, the neural network(s) in the higher-level MCU can learn when the secondary computer's output can be trusted and when it cannot. For example, if the secondary computer is a radar-based FCW system, the neural network(s) in the higher-level MCU can learn when the FCW system identifies metallic objects that are not actually hazards, such as a drainage grate or a manhole cover, triggering an alarm.Similarly, if the secondary computer is a camera-based lane departure warning (LDW) system, a neural network in the higher-level MCU can learn to override the LDW when cyclists or pedestrians are present and leaving the lane is indeed the safest maneuver. In embodiments that include one or more neural networks running on the higher-level MCU, the higher-level MCU can include at least one DLA or GPU suitable for running the neural network(s) with associated memory. In preferred embodiments, the higher-level MCU can include a component of the SoC(s) 304 and / or be included as such.

[0173] In other examples, the ADAS system 338 can include a secondary computer that performs ADAS functionality using traditional computer vision rules. As such, the secondary computer can use classic computer vision rules (if-then), and the presence of one or more neural networks in the higher-level MCU can improve reliability, safety, and performance. For example, the overall system becomes more fault-tolerant due to the different implementation and intentional non-identity, particularly with regard to errors caused by software functionality (or the software-hardware interface).For example, if there is a software bug or error in the software running on the primary computer, and the non-identical software code running on the secondary computer provides the same overall result, the higher-level MCU can have greater confidence that the overall result is correct and that the bug in the software or hardware on the primary computer does not cause a significant error.

[0174] In some examples, the output of the ADAS system 338 can be fed into the perception block of the primary computer and / or into the dynamic driving task block of the primary computer. For example, if the ADAS system 338 displays a forward collision warning due to an object immediately in front of it, the perception block can use this information in object identification. In other examples, the secondary computer may have its own neural network that is trained, thus reducing the risk of false positives, as described herein.

[0175] The Vehicle 300 may further include the Infotainment SoC 330 (e.g., an In-Vehicle Infotainment System (IVI)). Although illustrated and described as an SoC, the Infotainment System may not be an SoC and may include two or more discrete components. The Infotainment SoC 330 may include a combination of hardware and software that can be used to provide the Vehicle 300 with audio (e.g., music, a personal digital assistant, navigation instructions, news, radio, etc.), video (e.g., television, movies, streaming, etc.), telephone (e.g., hands-free calling), network connectivity (e.g., LTE, Wi-Fi, etc.), and / or information services (e.g., navigation systems, rear parking assist, a radio data system, vehicle-related information such as fuel level, total distance traveled, brake fluid level, oil level, door open / closed status, air filter information, etc.).For example, the Infotainment SoC 330 can include radios, turntables, navigation systems, video players, USB and Bluetooth connectivity, car computers, in-car entertainment, WiFi, steering wheel audio controls, hands-free voice control, a heads-up display (HUD), HMI display 334, a telematics device, a control panel (e.g., for controlling and / or interacting with various components, functions, and / or systems), and / or other components. The Infotainment SoC 330 can also be used to provide information (e.g., visual and / or audible) to the vehicle's user(s), such as information from the ADAS system 338, autonomous driving information like planned vehicle maneuvers, trajectories, environmental information (e.g., intersection information, vehicle information, road information, etc.), and / or other information.

[0176] The Infotainment SoC 330 can include GPU functionality. The Infotainment SoC 330 can communicate with other devices, systems, and / or components of the vehicle 300 via bus 302 (e.g., CAN bus, Ethernet, etc.). In some examples, the Infotainment SoC 330 can be coupled with a higher-level MCU so that the infotainment system's GPU can perform some self-regulating functions if the primary controller(s) 336 (e.g., the primary and / or backup computer of the vehicle 300) fails. In such an example, the Infotainment SoC 330 can put the vehicle 300 into a chauffeur-to-safe-stop mode, as described herein.

[0177] The vehicle 300 may further include an instrument cluster 332 (e.g., a digital instrument panel, an electronic instrument cluster, a digital instrument panel, etc.). The instrument cluster 332 may include a controller and / or a supercomputer (e.g., a discrete controller or a discrete supercomputer). The instrument cluster 332 may include a set of instruments such as a speedometer, fuel gauge, oil pressure gauge, tachometer, odometer, turn signals, gear shift indicator, seatbelt warning light(s), parking brake warning light(s), engine malfunction light(s), airbag (SRS) system information, lighting controls, safety system controls, navigation information, etc. In some examples, information can be displayed and / or shared between the infotainment SoC 330 and the instrument cluster 332.In other words, the Instrument Cluster 332 can be included as part of the Infotainment SoC 330, or vice versa.

[0178] Fig. 3D is a system representation according to some embodiments of the present disclosure for communication between one or more cloud-based server(s) and the exemplary autonomous vehicle 300 of Fig. 3A. The System 376 can include one or more Servers 378, one or more Network(s) 390, and vehicles, including the Vehicle 300. The Server(s) 378 can include a variety of GPUs 384(A)-384(H) (collectively referred to hereafter as GPUs 384), PCIe Switches 382(A)-382(H) (collectively referred to hereafter as PCIe Switches 382), and / or CPUs 380(A)-380(B) (collectively referred to hereafter as CPUs 380). The GPUs 384, CPUs 380, and PCIe Switches can be interconnected using high-speed links, such as, but not limited to, NVIDIA-developed NVLink interfaces 388 and / or PCIe links 386. In some examples, the GPUs 384 are connected via an NVLink and / or NVSwitch SoC, and the GPUs 384 and PCIe switches 382 are connected via PCIe links. Although eight GPUs 384, two CPUs 380, and two PCIe switches are illustrated, this is not intended to be limiting.Depending on the configuration, each Server 378 can include any number of GPUs 384, CPUs 380, and / or PCIe switches. For example, the Server 378 can include eight, sixteen, thirty-two, and / or more GPUs 384.

[0179] The server(s) 378 can receive image data from the network(s) 390 and the vehicles, representing images that show unexpected or changed road conditions, such as recently started roadworks. The server(s) 378 can transmit neural networks 392, updated neural networks 392, and / or map information 394, including information regarding traffic and road conditions, to the vehicles via the network(s) 390 and the vehicles. The map information updates 394 can include updates to the HD map 322, such as information about construction sites, potholes, detours, floods, and / or other obstacles.In some examples, the neural networks 392, the updated neural networks 392 and / or the map information 394 may result from new training and / or new experiences represented in data received from any number of vehicles in the vicinity, and / or based on training performed at a data center (e.g. using the server(s) 378 and / or other servers).

[0180] Server 378 can be used to train machine learning models (e.g., neural networks) based on training data. The training data can be generated by the vehicles and / or in a simulation (e.g., using a game engine). In some examples, the training data is tagged (e.g., if the neural network benefits from supervised learning) and / or preprocessed, while in other examples, the training data is untagged and / or preprocessed (e.g., if the neural network does not require supervised learning).Training can be performed according to one or more classes of machine learning techniques, including, but not limited to, classes such as: supervised training, semi-supervised training, unsupervised training, self-learning, reinforcement learning, federated learning, transfer learning, feature learning (including principal component and cluster analysis), multilinear subspace learning, manifold learning, representational learning (including sparse dictionary learning), rule-based machine learning, anomaly detection, and any other variants or combinations thereof. After the machine learning models have been trained, they can be used by the vehicles (e.g., transmitted to the vehicles via network(s) 390) and / or the machine learning models can be used by server(s) 378 for remote monitoring of the vehicles.

[0181] In other examples, the Server 378 can receive data from the vehicles and apply the data to current real-time neural networks for intelligent real-time inference. The Server 378 can include supercomputers for deep learning and / or dedicated AI computers powered by GPU(s) 384, such as NVIDIA's DGX and DGX Station machines. However, in some examples, the Server 378 can also include deep learning infrastructure that uses only CPU-powered data centers.

[0182] The deep learning infrastructure of server(s) 378 can be capable of fast real-time inference and use this capability to assess and verify the state of the processors, software, and / or associated hardware in vehicle 300. For example, the deep learning infrastructure can receive periodic updates from vehicle 300, such as a sequence of images and / or objects that vehicle 300 has located within that sequence of images (e.g., through computer vision and / or other machine learning object classification techniques).The deep learning infrastructure can operate its own neural network to identify the objects and compare them with the objects identified by the vehicle 300, and if the results do not match and the infrastructure concludes that the AI ​​in the vehicle 300 is faulty, the server(s) 378 can send a signal to the vehicle 300 and instruct a fail-safe computer of the vehicle 300 to take over control, notify the passengers and perform a safe parking maneuver.

[0183] The Server 378 can include GPU(s) 384 and one or more programmable inference accelerators (e.g., NVIDIA's TensorRT). The combination of GPU-powered servers and inference acceleration can enable real-time responsiveness. In other examples, such as when performance is less critical, servers powered by CPUs, FPGAs, and other processors can be used for inference purposes. EXAMPLE CALCULATION DEVICE

[0184] Fig. Figure 4 is a block diagram of an exemplary computing device(s) 400 suitable for use in implementing some embodiments of the present disclosure. The computing device 400 may include a connection system 402 that directly or indirectly couples the following devices: memory 404, one or more central processing units (CPUs) 406, one or more graphics processing units (GPUs) 408, a communication interface 410, input / output (I / O) ports 412, input / output components 414, a power supply 416, one or more presentation components 418 (e.g., display(s)), and one or more logic units 420. In at least one embodiment, the computing device(s) 400 may include one or more virtual machines (VMs) and / or any of its components may include virtual components (e.g., virtual hardware components).For non-restrictive examples, one or more of the GPUs 408 may comprise one or more vGPUs, one or more of the CPUs 406 may comprise one or more vCPUs, and / or one or more of the logic units 420 may comprise one or more virtual logic units. As such, a computing device(s) 400 may include discrete components (e.g., a complete GPU dedicated to the computing device 400), virtual components (e.g., a portion of a GPU dedicated to the computing device 400), or a combination thereof.

[0185] Although the various blocks of Fig. Where components 4 are shown to be connected via lines through the connection system 402, this is not intended to be restrictive and is merely for clarity. For example, in some embodiments, a presentation component 418, such as a display device, may be considered an I / O component 414 (e.g., if the display is a touchscreen). As another example, the CPUs 406 and / or GPUs 408 may include memory (e.g., the memory 404 may be representative of a storage device, in addition to the memory of the GPUs 408, the CPUs 406, and / or other components). In other words, the computing device of Fig. Figure 4 is for illustrative purposes only. No distinction is made between categories such as "workstation", "server", "laptop", "desktop", "tablet", "client device", "mobile device", "handheld device", "game console", "electronic control unit (ECU)", "virtual reality system" and / or other device or system types, as all fall under the category of computing device. Fig. 4 are considered.

[0186] The 402 interconnect system can represent one or more links or buses, such as an address bus, a data bus, a control bus, or a combination thereof. The 402 interconnect system can include one or more bus or link types, such as an Industry Standard Architecture (ISA) bus, an Extended Industry Standard Architecture (EISA) bus, a Video Electronics Standard Association (VESA) bus, a Peripheral Component Connection (PCI) bus, a Peripheral Component Connection Express (PCIe) bus, and / or other bus or link types. In some embodiments, there are direct connections between components. For example, the CPU 406 can be directly connected to the memory 404. Similarly, the CPU 406 can be directly connected to the GPU 408. In the case of a direct or point-to-point connection between components, the 402 interconnect system can include a PCIe link to facilitate the connection.In these examples, the computing device 400 does not need to include a PCI bus.

[0187] Memory 404 can include any of a number of computer-readable media. The computer-readable media can be any available media accessible to the computing device 400. The computer-readable media can include both volatile and non-volatile media, and both removable and non-removable media. As an example, and not as a limitation, the computer-readable media can include computer storage media and communication media.

[0188] Computer storage media can include both volatile and non-volatile media and / or removable and non-removable media, implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 404 can store computer-readable instructions (e.g., representing a program and / or program element, such as an operating system).Computer storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technology, CD-ROM, Digital Versatile Discs (DVDs) or other optical disk storage, magnetic cartridges, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the Computing Device 400. As used herein, computer storage media do not, per se, include signals.

[0189] Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other data types in a modulated data signal, such as a carrier wave or other transport mechanism, and include information delivery media. The term "modulated data signal" can refer to a signal for which one or more of its characteristics are set or modified in a way that encodes information in the signal. By way of example, and not limited to this, computer storage media can include wired media, such as a wired network or a directly wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media. Combinations of any of the foregoing should also be included in the scope of computer-readable media.

[0190] The CPU(s) 406 can be configured to execute at least some of the computer-readable instructions to control one or more components of the Computing Device 400 to perform one or more of the procedures and / or processes described herein. The CPU(s) 406 can include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of handling a plurality of software threads concurrently. The CPU(s) 406 can include any type of processor and may include different types of processors depending on the type of Computing Device 400 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers).For example, depending on the type of Computing Device 400, the processor can be an Advanced RISC Machines (ARM) processor implemented using Reduced Instruction Set Computing (RISC), or an x86 processor implemented using Complex Instruction Set Computing (CISC). The Computing Device 400 can include one or more CPUs 406 in addition to one or more microprocessors or complementary coprocessors, such as math coprocessors.

[0191] In addition to or as an alternative to the CPU(s) 406, the GPU(s) 408 may be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 400 to perform one or more of the methods and / or processes described herein. One or more of the GPU(s) 408 may be an integrated GPU (e.g., in one or more of the CPU(s) 406) and / or one or more of the GPU(s) 408 may be a discrete GPU. In embodiments, one or more of the GPU(s) 408 may be a coprocessor of one or more of the CPU(s) 406. The GPU(s) 408 may be used by the computing device 400 to render graphics (e.g., 3D graphics) or to perform general-purpose calculations. For example, the GPU(s) 408 can be used for general-purpose computing on GPUs (GPGPU).The GPU(s) 408 can include hundreds or thousands of cores capable of handling hundreds or thousands of software threads simultaneously. The GPU(s) 408 can generate pixel data for output images in response to rendering commands (e.g., rendering commands from the CPU(s) 406 received via a host interface). The GPU(s) 408 can include graphics memory, such as display memory, for storing pixel data or any other suitable data, such as GPGPU data. The display memory can be included as part of the memory 404. The GPU(s) 408 can include two or more GPUs operating in parallel (e.g., via a link). The link can connect the GPUs directly (e.g., using an NVLink) or connect them via a switch (e.g., using an NVSwitch).When combined, each GPU can generate 408 pixel data or GPGPU data for different dividers, or output for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU can include its own memory or share memory with other GPUs.

[0192] In addition to or as an alternative to the CPU(s) 406 and / or the GPU(s) 408, the logic unit(s) 420 may be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 400 to perform one or more of the methods and / or processes described herein. In embodiments, the CPU(s) 406, the GPU(s) 408, and / or the logic unit(s) 420 may discretely or jointly perform any combination of the methods, processes, and / or parts thereof. One or more of the logic units 420 may be part of and / or integrated into one or more of the CPU(s) 406 and / or the GPU(s) 408, and / or one or more of the logic units 420 may be discrete components or otherwise external to the CPU(s) 406 and / or the GPU(s) 408.In embodiments, one or more of the logic units 420 can be a coprocessor of one or more of the CPU(s) 406 and / or one of the GPU(s) 408.

[0193] Examples of logic unit(s) 420 include: one or more processing cores and / or components thereof, such as data processing units (DPUs), tensor cores (TCs), tensor processing units (TPUs), pixel visual cores (PVCs), vision processing units (VPUs), graphics processing clusters (GPCs), texture processing clusters (TPCs), streaming multiprocessors (SMs), tree traversal units (TTUs), artificial intelligence accelerators (AIAs), deep learning accelerators (DLAs), arithmetic logic units (ALUs), application-specific integrated circuits (ASICs), floating-point units (FPUs), input / output (I / O) elements, peripheral component interconnect (PCI) or peripheral component connection express (PCIe) elements and / or the like.

[0194] The communication interface 410 can include one or more receivers, transmitters, and / or transmit-receivers that enable the computing device 400 to communicate with other computing devices over an electronic communication network, including wired and / or wireless communication. The communication interface 410 can include components and functionality to enable communication over any number of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communicating over Ethernet or InfiniBand), low-power wide-area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet.In one or more embodiments, the logic unit(s) 420 and / or communication interface 410 may include one or more data processing units (DPUs) to directly transfer data received via a network and / or the connection system 402 to (e.g., a memory of) one or more GPU(s) 408.

[0195] The I / O ports 412 enable the computing device 400 to be logically coupled with other devices, including the I / O components 414, the presentation component(s) 418, and / or other components, some of which may be built into (e.g., integrated with) the computing device 400. Illustrative I / O components 414 include a microphone, mouse, keyboard, joystick, gamepad, game controller, satellite table, scanner, printer, wireless device, etc. The I / O components 414 can provide a natural user interface (NUI) that processes air gestures, speech, or other physical input generated by a user. In some cases, input can be transmitted to an appropriate network element for further processing.A NUI can implement any combination of speech recognition, pen recognition, facial recognition, biometric recognition, gesture recognition both on and adjacent to the screen, air gestures, head and eye tracking, and touch recognition (as described in more detail in this disclosure), associated with a display of the computing device 400. The computing device 400 can include depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof for gesture detection and recognition. Furthermore, the computing device 400 can include accelerometers or gyroscopes (e.g., as part of an inertial measurement unit (IMU)) that enable motion detection.In some examples, the output from the accelerometers or gyroscopes can be used by the Computing Device 400 to render immersive augmented reality or virtual reality.

[0196] The power supply 416 can include a hardwired power supply, a battery power supply, or a combination of both. The power supply 416 can provide power to the computing device 400 to enable the components of the computing device 400 to operate.

[0197] The presentation component(s) 418 can include a display (e.g., a monitor, a touchscreen, a television screen, a heads-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. The presentation component(s) 418 can receive data from other components (e.g., the GPU(s) 408, the CPU(s) 406, DPUs, etc.) and output the data (e.g., as an image, video, sound, etc.). EXEMPLARY DATA CENTER

[0198] Fig. Figure 5 illustrates an exemplary data center 500 that can be used in at least one embodiment of the present disclosure. The data center 500 can include a data center infrastructure layer 510, a framework layer 520, a software layer 530, and / or an application layer 540.

[0199] As in Fig. As shown in Figure 5, the data center infrastructure layer 510 can include a resource orchestrator 512, clustered compute resources 514, and node compute resources (“node CRs”) 516(1)-516(N), where “N” is a positive integer. In at least one embodiment, node CRs 516(1)-516(N) can include, but are not limited to, any number of central processing units (CPUs) or other processors (including DPUs, accelerators, field-programmable gate arrays (FPGAs), graphics processing units or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memories), storage devices (e.g., solid-state or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power modules, and / or cooling modules, etc. In some embodiments, one or more node CRs can be controlled by the node CRs.s 516(1)-516(N) correspond to a server that has one or more of the computing resources mentioned above. Furthermore, in some embodiments, the node CRs 516(1)-516(N) may include one or more virtual components, such as vGPUs, vCPUs and / or the like, and / or may correspond to one or more of the node CRs 516(1)-516(N) of a virtual machine (VM).

[0200] In at least one embodiment, grouped compute resources 514 can include separate groupings of node CRs 516 located in one or more racks (not shown), or many racks located in data centers at different geographic locations (also not shown). Separate groupings of node CRs 516 within grouped compute resources 514 can include grouped compute, network, storage, or memory resources that can be configured or allocated to support one or more workloads. In at least one embodiment, multiple node CRs 516, including CPUs, GPUs, DPUs, and / or other processors, can be grouped in one or more racks to provide compute resources to support one or more workloads.The one or more racks can also include any number of power modules, cooling modules and / or network switches in any combination.

[0201] The resource orchestrator 512 can configure or otherwise control one or more node CRs 516(1)-516(N) and / or grouped compute resources 514. In at least one embodiment, the resource orchestrator 512 can include a software design infrastructure (SDI) management entity for the data center 500. The resource orchestrator 512 can include hardware, software, or a combination thereof.

[0202] In at least one embodiment, as in Fig. As shown in Figure 5, a framework layer 520 can include a job scheduler 532, a configuration manager 534, a resource manager 536, and / or a distributed file system 538. The framework layer 520 can include a framework to support software 532 of software layer 530 and / or one or more applications 542 of application layer 540. The software 532 or application 542 can each include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. The framework layer 520 can be, but is not limited to, a type of free and open-source software web application framework, such as Apache Spark™ (hereinafter "Spark"), which can use the distributed file system 538 for large-scale data processing (e.g., "big data").In at least one embodiment, the job scheduler 532 can include a Spark driver to facilitate the scheduling of workloads supported by various layers of the data center 500. The configuration manager 534 can be capable of configuring various layers, such as the software layer 530 and the framework layer 520, including Spark and the distributed file system 538, to support high-volume data processing. The resource manager 536 can be capable of managing clustered or grouped compute resources allocated or assigned to support the distributed file system 538 and the job scheduler 532. In at least one embodiment, the clustered or grouped compute resources can include a grouped compute resource 514 at the data center infrastructure layer 510.The Resource Manager 536 can coordinate with the Resource Orchestrator 512 to manage these allocated or assigned computing resources.

[0203] In at least one embodiment, software 532, which is enclosed in software layer 530, can include software used by at least parts of the node CRs 516(1)-516(N), grouped computing resources 514, and / or distributed file system 538 of framework layer 520. One or more types of software can include, but are not limited to, internet web page search software, email virus scanning software, database software, and streaming video content software.

[0204] In at least one embodiment, one or more application(s) 542 enclosed in the application layer 540 may include one or more types of applications used by at least parts of the node CRs 516(1)-516(N), grouped compute resources 514, and / or distributed file system 538 of the framework layer 520. One or more types of applications may include, but are not limited to, any number of a genomics application, a cognitive computation application, and a machine learning application, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0205] In at least one embodiment, any configuration manager 534, resource manager 536, and resource orchestrator 512 can implement any number and any type of self-modifying operations based on any set and any type of data captured in any technically feasible manner. Self-modifying operations can relieve a data center operator of the data center 500 of the burden of potentially making poor configuration decisions and potentially avoiding underutilized and / or poorly performing parts of a data center.

[0206] The Data Center 500 may include tools, services, software, or other resources for training one or more machine learning models or for predicting or inferring information using one or more machine learning models according to one or more embodiments described herein. For example, one or more machine learning models may be trained by calculating weight parameters according to a neural network architecture using software and / or computing resources as described in the present disclosure relating to the Data Center 500.In at least one embodiment, trained or deployed machine learning models corresponding to one or more neural networks can be used to infer and predict information using the resources described in the present disclosure with respect to the data center 500 by using weight parameters calculated via one or more training techniques, such as, but not limited to, those described herein.

[0207] In at least one embodiment, the data center can use 500 CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described in this disclosure can be configured as a service to allow users to train or perform information inference, such as image recognition, speech recognition, or other artificial intelligence services. EXEMPLARY NETWORK ENVIRONMENTS

[0208] Network environments suitable for use in implementing embodiments of the disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may be connected to one or more instances of the computing device(s) 400 of Fig. 4. Implemented – for example, each device can include similar components, features, and / or functionality to the computing device(s) 400. Furthermore, if backend devices (e.g., servers, NAS, etc.) are implemented, the backend devices can be included as part of a data center 500; an example of this is described in more detail below with reference to… Fig. 5 described.

[0209] Components in a network environment can communicate with each other over a network, which can be wired, wireless, or both. The network can include multiple networks or a network of networks. For example, the network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks, such as the internet, and / or a public switched telephone network (PSTN), and / or one or more private networks. If the network includes a wireless telecommunications network, components such as a base station, a communications tower, or even access points (as well as other components) can provide wireless connectivity.

[0210] Compatible network environments can include one or more peer-to-peer network environments—in which case no server may be included in a network environment—and one or more client-server network environments—in which case one or more servers may be included in a network environment. In peer-to-peer network environments, functionality described herein with respect to one or more servers can be implemented on any number of client devices.

[0211] In at least one embodiment, a network environment can include one or more cloud-based network environments, a distributed computing environment, a combination thereof, etc. A cloud-based network environment can include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. A framework layer can include a framework to support software of a software layer and / or one or more application(s) of an application layer. The software or application(s) can each include web-based service software or applications. In embodiments, one or more of the client devices can use the web-based service software or applications (e.g.,by accessing the service software and / or applications via one or more application programming interfaces (APIs). The framework layer can be a type of free and open-source software web application framework, but is not limited to those that can use a distributed file system for large-volume data processing (e.g., "big data").

[0212] A cloud-based network environment can provide cloud computing and / or cloud storage, performing any combination of computing and / or data storage functions as described herein (or one or more parts thereof). Any of these various functions can be distributed across multiple locations of central or core servers (e.g., one or more data centers that may be distributed across a state, region, country, the Earth, etc.). If a connection to a user (e.g., a client device) is relatively close to an edge server, a core server may designate at least some of the functionality for that edge server(s). A cloud-based network environment can be private (e.g., restricted to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0213] The client device(s) may include at least some of the components, features, and functionality described herein with reference to Fig.The 400 described exemplary computing device(s) include 400. By way of example, and not limited to, a client device may be a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or global positioning device, video player, video camera, surveillance device or system, vehicle, boat, flying vehicle, virtual machine, drone, robot, handheld communication device, hospital device, gaming device or system, entertainment system, vehicle computer system, embedded system controller, remote control, device, consumer electronics device, workstation, edge device, any combination of these described devices, or any other suitable device.

[0214] Disclosure can generally be described in the context of computer code or machine-usable instructions, including computer-executable instructions such as program modules that are executed by a computer or other machine, such as a personal data assistant or a handheld device. Generally, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs specific tasks or implements concrete abstract data types. Disclosure can be exercised in various system configurations, including handheld devices, consumer electronics, general-purpose computers, other specialized computing devices, etc. Disclosure can also be exercised in distributed computing environments where tasks are performed by remote processing devices linked via a communication network.

[0215] As used herein, any mention of "and / or" with reference to two or more elements shall be interpreted as referring to only one element or a combination of elements. For example, "element A, element B, and / or element C" may include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or elements A, B, and C. Furthermore, "at least one of element A or element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Additionally, the use of the term "based on" shall not be interpreted as "based only on" or "based only on."Rather, a first element “based on” a second element includes cases where the first element is based on the second element, but may also be based on one or more additional elements.

[0216] The subject matter of this disclosure is described herein with specificity to satisfy legal requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the inventors have provided that the claimed subject matter may also be embodied in other ways to include different steps or combinations of steps similar to those described in this document, in conjunction with other current or future technologies. Furthermore, although the terms "step" and / or "block" may be used herein to denote different elements of methods employed, they should not be interpreted as implying a specific sequence of or between different steps disclosed herein, except where the sequence of individual steps is explicitly described.

[0217] The technology of this disclosure is illustrated, for example, by various aspects described below. Several examples of aspects of this disclosure are described as numbered examples (1, 2, 3, etc.) for convenience. These are provided as examples and do not limit the present disclosure. The aspects of the various implementations described herein may be omitted, substituted for aspects of other implementations, or combined with aspects of other implementations, unless the context otherwise requires. For example, one or more aspects of Example 1 below may be omitted, substituted for one or more aspects of an example (e.g., Example 2) or examples, or combined with aspects of another example.

[0218] The following is a non-limiting summary of some exemplary implementations presented herein.

[0219] Example 1: A procedure that includes the following: Performing, through a computing system, a multitude of initialization operations related to initializing a sensor; and Performing, by the computing system, one or more authentication operations with regard to authenticating the sensor after the sensor has been initialized.

[0220] Example 2: Procedure according to Example 1, wherein the one or more authentication operations include one or more of the following: Verification, by the computing system, of the actual sensor; or Verify, through the computing system, that the sensor has been initialized according to the multitude of initialization operations.

[0221] Example 3: Method according to Example 2, wherein the verification of the sensor initialization is based on at least one or more initial tracking codes maintained by the sensor during initialization.

[0222] Example 4: Method according to Example 3, wherein the verification of the sensor initialization is further based at least on a comparison between a first tracking code value corresponding to a specific first tracking code of one or more first tracking codes and a second tracking code value that is an expected tracking code value of the specific first tracking code.

[0223] Example 5: Procedure according to Example 4, wherein the second tracking code value is determined by the computing system at least on the basis of data communicated with reference to one or more initialization operations of the plurality of initialization operations.

[0224] Example 6: Procedure according to Example 5, wherein the first tracking code value is communicated by the sensor to the computer system via secure communication.

[0225] Example 7: Method according to Example 6, wherein the secure communication is based at least on the sensor itself being verified via one or more authentication operations.

[0226] Example 8: Procedure according to Examples 3 to 7, wherein the first tracking code includes at least one or more of the following: a cyclic redundancy check code (CRC code); or a message counter.

[0227] Example 9: Procedure according to a previous example, wherein the plurality of initialization operations includes one or more of the following: one or more write operations in which the computing system orders data to be written to the sensor; or one or more read operations in which the computing system orders data to be read from the sensor.

[0228] Example 10: Method according to a previous example, wherein the sensor includes a camera.

[0229] Example 11: A computing system that includes the following: one or more processors for performing operations, including: Performing a variety of initialization operations related to initializing a peripheral device associated with the computing system; and Performing one or more authentication operations related to authenticating the peripheral device after the peripheral device has been initialized.

[0230] Example 12: Computing system according to Example 11, wherein the one or more authentication operations include one or more of the following: Verifying the actual peripheral device; or Verify that the peripheral device has been initialized according to the multitude of initialization operations.

[0231] Example 13: Computing system according to Example 12, wherein the verification of the initialization of the peripheral device is based on at least one or more first tracking codes maintained by the peripheral device during initialization.

[0232] Example 14: Computing system according to Example 13, wherein the verification of the initialization of the peripheral device is further based at least on a comparison between a first tracking code value corresponding to a specific first tracking code of one or more first tracking codes and a second tracking code value that is an expected tracking code value of the specific first tracking code.

[0233] Example 15: Computing system according to Example 14, wherein the second tracking code value is determined by the computing system at least on the basis of data communicated with reference to one or more initialization operations of the plurality of initialization operations.

[0234] Example 16: Computing system according to one of Examples 11 to 15, wherein the peripheral device includes a sensor corresponding to an ego machine.

[0235] Example 17: Computing system according to one of Examples 11 to 16, wherein the system includes at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for conducting digital twinning operations; a system for performing a light transport simulation; a system for conducting collaborative content creation for 3D assets; a system for performing deep learning operations; a system for presenting at least one type of augmented reality content, virtual reality content, or mixed reality content; a system for hosting one or more real-time streaming applications; a system implemented using an edge device; a system that is implemented using a robot; a system for performing conversational AI operations; a system for performing one or more generative AI operations; a system that implements one or more large language models (LLMs); a system that implements one or more Visual Language Models (VLMs); a system that implements one or more multimodal language models; a system for generating synthetic data; a system that includes one or more virtual machines (VMs); a system that is at least partially implemented in a data center; or a system that is implemented at least partially using cloud computing resources.

[0236] Example 18: One or more processors comprising the following: Processor switching technology for performing operations, including: Performing, through a computing system, a multitude of initialization operations related to initializing a sensor; and Performing, by the computing system, one or more authentication operations with respect to authenticating the sensor after the sensor has been at least partially initialized, wherein the one or more authentication operations are based on at least one or more initial tracking codes maintained during the initialization of the sensor.

[0237] Example 19: The one or more processors according to Example 18, wherein the one or more authentication operations are based at least on a comparison between a first tracking code value corresponding to a specific first tracking code of the one or more first tracking codes and a second tracking code value that is an expected tracking code value of the specific first tracking code.

[0238] Example 20: The one or more processors according to Example 19, wherein the second tracking code value is determined by the computing system at least on the basis of data communicated with reference to one or more initialization operations of the plurality of initialization operations.

[0239] It is understood that the aspects and embodiments described above are merely examples and that modifications in detail may be made within the scope of the claims.

[0240] Each device, method and feature disclosed in the description and (if applicable) in the claims and drawings can be provided independently or in any suitable combination.

[0241] Reference numerals appearing in the claims serve only for illustration and are not intended to have any limiting effect on the scope of the claims. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 16 / 101,232

[0120] Cited non-patent literature

[0000] Society of Automotive Engineers (SAE) “Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles” (Standard No. J3016-201806, published on June 15, 2018, Standard No. J3016-201609, published on September 30, 2016

[0078]

Claims

[1] A procedure comprising the following: Performing, through a computing system, a multitude of initialization operations related to initializing a sensor; and Performing, by the computing system, one or more authentication operations with regard to authenticating the sensor after the sensor has been initialized. [2] Method according to claim 1, wherein the one or more authentication operations include one or more of the following: Verification, by the computing system, of the actual sensor; or Verify, through the computing system, that the sensor has been initialized according to the multitude of initialization operations. [3] Method according to claim 2, wherein the verification of the sensor initialization is based on at least one or more first tracking codes maintained by the sensor during initialization. [4] Method according to claim 3, wherein the verification of the sensor initialization is further based at least on a comparison between a first tracking code value corresponding to a specific first tracking code of one or more first tracking codes and a second tracking code value that is an expected tracking code value of the specific first tracking code. [5] Method according to claim 4, wherein the second tracking code value is determined by the computing system at least on the basis of data communicated with reference to one or more initialization operations of the plurality of initialization operations. [6] Method according to claim 5, wherein the first tracking code value is communicated by the sensor to the computing system via secure communication. [7] Method according to claim 6, wherein the secure communication is based at least on the sensor itself being verified via one or more authentication operations. [8] Method according to any one of claims 3 to 7, wherein the first tracking code includes at least one or more of the following: a cyclic redundancy check code (CRC code); or a message counter. [9] Method according to any preceding claim, wherein the plurality of initialization operations includes one or more of the following: one or more write operations in which the computing system orders data to be written to the sensor; or one or more read operations in which the computing system orders data to be read from the sensor. [10] Method according to a preceding claim, wherein the sensor includes a camera. [11] A computing system comprising the following: one or more processors for performing operations, including: Performing a variety of initialization operations related to initializing a peripheral device associated with the computing system; and Performing one or more authentication operations related to authenticating the peripheral device after the peripheral device has been initialized. [12] Computing system according to claim 11, wherein the one or more authentication operations include one or more of the following: Verifying the actual peripheral device; or Verify that the peripheral device has been initialized according to the multitude of initialization operations. [13] Computing system according to claim 12, wherein the verification of the initialization of the peripheral device is based on at least one or more first tracking codes maintained by the peripheral device during initialization. [14] Computing system according to claim 13, wherein the verification of the initialization of the peripheral device is further based at least on a comparison between a first tracking code value corresponding to a specific first tracking code of one or more first tracking codes and a second tracking code value that is an expected tracking code value of the specific first tracking code. [15] Computing system according to claim 14, wherein the second tracking code value is determined by the computing system at least on the basis of data communicated with reference to one or more initialization operations of the plurality of initialization operations. [16] Computing system according to one of claims 11 to 15, wherein the peripheral device includes a sensor corresponding to an ego machine. [17] Computing system according to any one of claims 11 to 16, wherein the system comprises at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for conducting digital twinning operations; a system for performing a light transport simulation; a system for conducting collaborative content creation for 3D assets; a system for performing deep learning operations; a system for presenting at least one type of augmented reality content, virtual reality content, or mixed reality content; a system for hosting one or more real-time streaming applications; a system implemented using an edge device; a system that is implemented using a robot; a system for performing conversational AI operations; a system for performing one or more generative AI operations; a system that implements one or more large language models (LLMs); a system that implements one or more Visual Language Models (VLMs); a system that implements one or more multimodal language models; a system for generating synthetic data; a system that includes one or more virtual machines (VMs); a system that is at least partially implemented in a data center; or a system that is implemented at least partially using cloud computing resources. [18] One or more processors comprising the following: Processor switching technology for performing operations, including: Performing, through a computing system, a multitude of initialization operations related to initializing a sensor; and Performing, by the computing system, one or more authentication operations with respect to authenticating the sensor after the sensor has been at least partially initialized, wherein the one or more authentication operations are based on at least one or more initial tracking codes maintained during the initialization of the sensor. [19] The one or more processors according to claim 18, wherein the one or more authentication operations are based at least on a comparison between a first tracking code value corresponding to a specific first tracking code of the one or more first tracking codes and a second tracking code value that is an expected tracking code value of the specific first tracking code. [20] The one or more processors according to claim 19, wherein the second tracking code value is determined by the computing system at least on the basis of data communicated with reference to one or more initialization operations of the plurality of initialization operations.

Images