Neural network generation

Abstract

Devices, systems, and technologies for generating one or more neural networks. In at least one embodiment, the processor comprises one or more circuits that use one or more first neural networks to generate one or more second versions of one or more second neural networks—based at least partially on one or more hardware resources to be used to perform one or more first versions of one or more second neural networks and one or more second versions of one or more second neural networks.

Description

Technology Field

[0001] At least one embodiment relates to processing resources used to perform and facilitate artificial intelligence. For example, at least one embodiment relates to processors or computing systems used to generate a compressed neural network according to various novel techniques described herein. Background Technology

[0002] Generating multiple versions of neural networks to be executed on different platforms, computer systems, or any other hardware resources can consume significant time and computing resources because it requires knowledge of different types of platforms, computer systems, or any other hardware resources. The amount of time and computing resources used to generate new or updated neural networks can be improved. Brief explanation of the drawing

[0003] FIG. 1 illustrates an exemplary system for generating one or more neural networks performed by hardware resources according to at least one embodiment. FIG. 2 illustrates an exemplary system for training one or more generator neural networks according to at least one embodiment. FIG. 3 illustrates an exemplary process for generating one or more neural networks performed by hardware resources according to at least one embodiment. FIG. 4 illustrates an exemplary process for training one or more generator neural networks according to at least one embodiment. FIG. 5 illustrates an exemplary process for fine-tuning one or more generator neural networks according to at least one embodiment. FIG. 6 illustrates an exemplary system for generating one or more neural networks performed by hardware resources according to at least one embodiment. FIG. 7 illustrates an exemplary system that uses one or more application programming interfaces (APIs) to generate one or more neural networks performed by hardware resources according to at least one embodiment. FIG. 8a illustrates logic according to at least one embodiment. FIG. 8b illustrates logic according to at least one embodiment. FIG. 9 illustrates the training and deployment of a neural network according to at least one embodiment. FIG. 10 illustrates an exemplary data center system according to at least one embodiment. FIG. 11a illustrates an example of an autonomous vehicle according to at least one embodiment. FIG. 11b illustrates examples of camera positions and fields of view for the autonomous vehicle of FIG. 11a according to at least one embodiment. FIG. 11c is a block diagram illustrating an exemplary system architecture for an autonomous vehicle of FIG. 11a according to at least one embodiment. FIG. 11d is a diagram illustrating a system for communication between the autonomous vehicle of FIG. 11a and cloud-based server(s) according to at least one embodiment. FIG. 12 is a block diagram illustrating a computer system according to at least one embodiment. FIG. 13 is a block diagram illustrating a computer system according to at least one embodiment. FIG. 14 illustrates a computer system according to at least one embodiment. FIG. 15 illustrates a computer system according to at least one embodiment. FIG. 16a illustrates a computer system according to at least one embodiment. FIG. 16b illustrates a computer system according to at least one embodiment. FIG. 16c illustrates a computer system according to at least one embodiment. FIG. 16d illustrates a computer system according to at least one embodiment. FIGS. 16e and FIGS. 16f illustrate a shared programming model according to at least one embodiment. FIG. 17 illustrates exemplary integrated circuits and associated graphics processors according to at least one embodiment. FIGS. 18a and FIGS. 18b illustrate exemplary integrated circuits and associated graphics processors according to at least one embodiment. FIGS. 19a and FIGS. 19b illustrate additional exemplary graphics processor logic according to at least one embodiment. FIG. 20 illustrates a computer system according to at least one embodiment. FIG. 21a illustrates a parallel processor according to at least one embodiment. FIG. 21b illustrates a partition unit according to at least one embodiment. FIG. 21c illustrates a processing cluster according to at least one embodiment. FIG. 21d illustrates a graphics multiprocessor according to at least one embodiment. FIG. 22 illustrates a multi-GPU (graphics processing unit) system according to at least one embodiment. FIG. 23 illustrates a graphics processor according to at least one embodiment. FIG. 24 is a block diagram illustrating a processor microarchitecture for a processor according to at least one embodiment. FIG. 25 illustrates a deep learning application processor according to at least one embodiment. FIG. 26 is a block diagram illustrating an exemplary neuromorphic processor according to at least one embodiment. FIG. 27 illustrates at least a portion of a graphics processor according to one or more embodiments. FIG. 28 illustrates at least a portion of a graphics processor according to one or more embodiments. FIG. 29 illustrates at least a portion of a graphics processor according to one or more embodiments. FIG. 30 is a block diagram of a graphics processing engine of a graphics processor according to at least one embodiment. FIG. 31 is a block diagram of at least a portion of a graphics processor core according to at least one embodiment. FIGS. 32a and FIGS. 32b illustrate thread execution logic including an array of processing elements of a graphics processor core according to at least one embodiment. FIG. 33 illustrates a "PPU (parallel processing unit)" according to at least one embodiment. FIG. 34 illustrates a "general processing cluster (GPC)" according to at least one embodiment. FIG. 35 illustrates a memory partition unit of a "PPU (parallel processing unit)" according to at least one embodiment. FIG. 36 illustrates a streaming multi-processor according to at least one embodiment. FIG. 37 is an exemplary data flow diagram for an advanced computing pipeline according to at least one embodiment. FIG. 38 is a system diagram of an exemplary system for training, adapting, instantiating, and deploying a machine learning model in an advanced computing pipeline according to at least one embodiment. FIG. 39 includes an exemplary example of an advanced computing pipeline (3810A) for processing image data according to at least one embodiment. FIG. 40a includes an exemplary data flow diagram of a virtual device supporting an ultrasonic device according to at least one embodiment. FIG. 40b includes an exemplary data flow diagram of a virtual device supporting a CT scanner according to at least one embodiment. FIG. 41a illustrates a data flow diagram for a process of training a machine learning model according to at least one embodiment. FIG. 41b is an exemplary example of a client-server architecture that enhances annotation tools with pre-trained annotation models according to at least one embodiment. FIG. 42 illustrates components of a system accessing a large language model according to at least one embodiment. Specific details for implementing the invention

[0004] In the following description, various technologies and systems are described. For the purposes of explanation, specific configurations and details are presented to provide a thorough understanding of possible ways to implement these technologies. However, it will also be apparent that the technologies described below may be implemented in different configurations without specific details. Furthermore, well-known modes may be omitted or simplified to avoid obscuring the technologies being described.

[0005] FIG. 1 illustrates an exemplary system (100) for generating one or more neural networks to be performed by one or more hardware resources according to at least one embodiment. In at least one embodiment, the system (100) is a combination of hardware and software, both of which are described herein. In at least one embodiment, the system (100) includes a neural network training module (110) and a neural network generation module (120).

[0006] In at least one embodiment, as used in any implementation described herein, terms such as “module” and nominalized verbs described throughout FIGS. 1 through 42 (e.g., neural network training module (110), neural network generation module (120), training module (230), training module (610), compressed neural network generation module (612), training data collection module (614), etc.) each refer to any combination of software logic, hardware logic and / or circuits configured to provide the functionality described herein.

[0007] In at least one embodiment, the software described throughout FIGS. 1 to 42 includes, for example, operating systems, device drivers, application software, database software, graphics software (e.g., Radeon, Intel Graphics), web browsers, development software (e.g., integrated development environments, code editors, compilers, interpreters), network software (e.g., Intel PROset, Intel Advanced Network Services), simulation software, real-time operating systems (RTOS), artificial intelligence software (e.g., Scikit-learn, TensorFlow, PyTorch, Accord.NET, Apache Machout), robot software (ROBEL, MS AirSi, Apollo Baidu, AWS RoboMaker, ROSbot 2.0, Poppy Project), firmware (e.g., BIOS / UEFI, routers, smartphones, consumer electronics, embedded systems, printers, solid-state drives (SSDs)), application programming interfaces (APIs), and containerized software (e.g., Nginx, Apache HTTP Server, MySQL, PostgreSQL, Redis, It includes Memcached, Node.js, Elasticsearch, Gitlab, Jenkins, WordPress), container orchestration platforms (e.g., Kubernetes, Docker Swarm, Apache Mesos, Nomad, Amazon ECS, Microsoft Azure Kubernetes Service, Google Kubernetes Engine, Red Hat OpenShift, Rancher), or any other implementation implemented as a software package, code, and / or instruction set.

[0008] In at least one embodiment, the hardware described throughout FIGS. 1 to 42 comprises, for example, hardwired circuits, programmable circuits, state machine circuits, fixed-function circuits, execution unit circuits, and / or firmware storing instructions executed by the programmable circuits, either alone or in any combination. In at least one embodiment, the circuit may be part of a larger system (e.g., integrated circuit (IC), system on-chip (SoC), central processing unit (CPU), graphics processing unit (GPU), data processing unit (DPU), digital signal processor (DSP), tensor processing unit (TPU), accelerated processing unit (APU), application-specific integrated circuits (ASIC), intelligent processing unit (IPU), neural processing unit (NPU), smart network interface controller (SmartNIC), vision processing unit (VPU), field-programmable gate array (FPGA), etc.).

[0009] In at least one embodiment, unless explicitly stated otherwise, the neural networks described throughout FIGS. 1 to 42 are one of a feedforward neural network, a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM) network, a generative adversarial network (GAN), a restricted Boltzmann machine (RBM), deep belief networks (DBN), a radial basis function network (RBFN), a Hopfield network, self-organizing maps, a perceptron having one or more layers, modular neural networks, spiking neural networks, deep reinforcement learning networks, echo state networks, time-delay neural networks, support vector machines, attention-based neural networks, autoencoders, graph neural networks (e.g., graph convolution networks), transformation autoencoder and / or transformer neural networks (e.g., BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer)), or large language models (LLM). The above refers to the above. In at least one embodiment, the transducer neural networks refer to a type of architecture that uses self-attention mechanisms (e.g., multi-head attention) to process sequential data.

[0010] In at least one embodiment, the neural networks (e.g., neural networks (114), neural networks (232), neural networks (234), neural networks (236)) are trained using various neural network training techniques (e.g., supervised learning, unsupervised learning, reinforcement learning, transmission learning, online learning, batch learning, federated learning).

[0011] In at least one embodiment, in addition to performing the various operations described in conjunction with FIGS. 1 through 7, the neural networks or parts of the neural networks may perform various computer vision and natural language processing (NLP) operations, unless explicitly otherwise stated. In at least one embodiment, the various computer vision operations include object recognition, face recognition, image segmentation, object tracking, gesture recognition and optical character recognition, and augmented reality, either alone or in any combination. In at least one embodiment, the various NLP operations include sentiment analysis, chatbot creation, language translation, text detection, text recognition, text summarization, named entity recognition, text classification, speech recognition, computer program (e.g., set of codes) creation, and text generation, either alone or in any combination.

[0012] In at least one embodiment, the neural network training module (110) is a module for training one or more neural networks (e.g., neural network (114), neural network (232), neural network (236), and generator neural network described together with FIGS. 3 to 5). In at least one embodiment, the one or more neural networks are further described together with at least FIG. 2.

[0013] In at least one embodiment, a neural network training module (110) receives training data (112) for training one or more neural networks (114). In at least one embodiment, the training data (112) includes hardware features of different computer systems (e.g., NVIDIA H100, AMD MI300 APU, Intel Habana's Gaudi Chip). In at least one embodiment, the hardware features are further described with at least FIG. 2. In at least one embodiment, the training data (112) includes software features of the different computer systems. In at least one embodiment, the software features are further described with at least FIG. 2. In at least one embodiment, the training data (112) includes first software programs (e.g., code examples) that utilize the software features and the hardware features. In at least one embodiment, hardware resources execute the first software programs. In at least one embodiment, the first software programs are further described with at least FIG. 2.

[0014] In at least one embodiment, the training data (112) includes second software programs (e.g., code examples) for running different versions of compressed neural networks on different platforms, computer systems, or any other hardware resources. In at least one embodiment, the second software programs are further described together with at least FIG. 2. In at least one embodiment, the training data (112) includes manual feedback (e.g., heuristic dataset) containing multiple configuration information selected by one or more engineers based on their experience.

[0015] In at least one embodiment, the neural network training module (110) trains one or more neural networks (114) to generate one or more trained neural networks (116). In at least one embodiment, the neural network training module (110) uses the hardware features, the software features, and the first software programs to initially train one or more neural networks (114) to generate one or more trained neural networks (116).

[0016] In at least one embodiment, the neural network training module (110) further trains one or more neural networks (116) using the second software programs (e.g., performs first fine-tuning, which is further described with FIG. 2). In at least one embodiment, after performing the first fine-tuning, one or more trained neural networks (116) may be prepared for deployment. In at least one embodiment, the neural network training module (110) further trains one or more neural networks (116) using the manual feedback (e.g., performs second fine-tuning, which is further described with FIG. 2). In at least one embodiment, after performing the second fine-tuning, one or more trained neural networks (116) are prepared for deployment. In at least one embodiment, the neural network training module (110) uses a training framework (904), one or more neural networks (114) include an untrained neural network (906), and one or more trained neural networks (116) include a trained neural network (908).

[0017] In at least one embodiment, the neural network training module (110) includes a model training system (3704), and one or more neural networks (114) include pre-training models (3806) and / or an initial model (4104) and perform model training (3714) to generate a refined model (4112). In at least one embodiment, one or more trained neural networks (116) include the refined model (4112).

[0018] In at least one embodiment, the neural network generation module (120) is a module that generates one or more versions of one or more compressed neural networks, and the one or more versions correspond to different platforms, computer systems, or any other hardware resources. In at least one embodiment, the neural network generation module (120) uses one or more trained neural networks (e.g., neural networks (124), neural networks (234), neural networks (236)) to generate the one or more neural networks (e.g., compressed neural networks). In at least one embodiment, the neural network generation module (120) deploys one or more trained neural networks (116) in a computer system so that the one or more deployed neural networks (124) generate the one or more neural networks defined or implemented by one or more software programs (122).

[0019] In at least one embodiment, the compressed neural networks described in conjunction with FIGS. 1 through 7 refer to neural networks that are not only compressed using general model compression techniques but are also further optimized on any other hardware resource, such as a specific platform, computer systems, or NVIDIA H100 GPU. In at least one embodiment, the optimization utilizes the inherent features and capabilities of the hardware resource to maximize the efficiency, speed, and effectiveness of the neural network while it is executed on the said platform.

[0020] In at least one embodiment, neural networks are compressed using software tools (e.g., TensorRT, openVINO, Intel Math Kernel Library for Deep Neural Networks (NKL-DNN), oneAPI toolkit) designed to optimize said neural networks for deployment on specific hardware resources (e.g., NVIDIA GPUs). In at least one embodiment, said compression includes combining two or more layers of one or more neural networks into a single layer. In at least one embodiment, said compression includes adjusting the numerical precision of the computations (e.g., from 32-bit floating-point to 16-bit or 8-bit) to reduce memory usage and increase throughput with minimal impact on accuracy. In at least one embodiment, generating different versions involves dynamically compressing the neural networks based on data types that only the hardware resources can support. In at least one embodiment, neural networks optimized for the INT8 format must be converted into different versions to be executed by a set of hardware resources that support only FP32 or FP16.

[0021] In at least one embodiment, the compression includes selecting the most efficient computation kernels for specific operations in one or more neural networks based on the target GPU architecture. In at least one embodiment, the compression includes performing dynamic tensor memory allocation. In at least one embodiment, the compression includes utilizing the sparsity in the weights of the model (results of pruning) to skip zero-value weights during computation.

[0022] In at least one embodiment, the compression includes utilizing Intel Threading Building Blocks (TBB) and OpenMP to efficiently distribute computations across multiple CPU cores. In at least one embodiment, the compression fully utilizes Intel’s CPU vectorized instructions, such as AVX-512, including executing Intel’s optimized mathematical routines from libraries such as oneDNN to ensure that each operation in one or more neural networks is executed as efficiently as possible. In at least one embodiment, the compression includes optimizing lightweight neural networks for tasks such as fault detection or predictive maintenance using Movidius VPU accelerators or Intel Atom processors using OpenVINO.

[0023] In at least one embodiment, the compression includes using mobile-specific libraries to ensure that one or more neural networks are aligned with the capabilities of mobile GPUs. In at least one embodiment, the compression includes using NVIDIA's Jetson to compress one or more neural networks (e.g., recurrent neural networks, transducer neural networks) that perform speech recognition.

[0024] In at least one embodiment, the compression includes optimizing model compression techniques on a destination platform, a computer system, or any other hardware resource. In at least one embodiment, the model compression techniques include pruning, quantization, knowledge classification, weight sharing, and low-order factorization.

[0025] In at least one embodiment, the neural network generation module (120) receives one or more indications (118) as inputs for one or more deployed neural networks (124). In at least one embodiment, one or more indications (118) include a previous version of the neural networks being compressed for a previous hardware resource, the name of the previous hardware resource, and / or the name of the new hardware resource, for performing the new version of the compressed neural networks to be generated by the neural network generation module (120). In at least one embodiment, generating the neural networks (e.g., compressed neural networks) includes generating one or more software programs (122) that define or implement the new version of the compressed neural networks. In at least one embodiment, one or more software programs (122) include code or any other instructions that cause the one or more compressed neural networks to be deployed on a destination platform, a computer system, or any other hardware resource. In at least one embodiment, the code may be written using Python with TensorFlow, PyTorch, Keras, Scikit-learn, C++ with Caffe, OpenCV's DNN module, Java with Deeplearning4j, DL4J, Javascript with TensorFlow.js, Brain.Js, R with KerasR and MXNet, Julia with Flux,jil and Knet, MATLAB, ONNX, TensorRT, TVM, OpenVINO, etc. In at least one embodiment, one or more software programs (122) are used to deploy a compressed model (e.g., a neural network) based on a deployment system (3706).

[0026] In at least one embodiment, one or more software programs (122) include software code written in programming languages, and said software code includes import libraries, definitions of neural network architecture (e.g., input layers, hidden layers, output layers), loss function, optimizer, data preprocessing, training, evaluation and prediction.

[0027] In at least one embodiment, one or more deployed neural networks (124) include a large language model (4242). In at least one embodiment, one or more deployed neural networks (124) receive input data (5210) to generate output data (4220) including a software program (122) for implementing the one or more target neural networks. In at least one embodiment, a large language model API (4204) is used to receive input data (5210) and to generate output data (4220) including a software program (122) for implementing the one or more target neural networks.

[0028] In at least one embodiment, the system (100) includes one or more circuits that use one or more first neural networks to generate one or more second versions of one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of one or more second neural networks and one or more second versions of one or more second neural networks. In at least one embodiment, the one or more first neural networks refer to one or more deployed neural networks (124). In at least one embodiment, the one or more second neural networks refer to one or more compressed neural networks defined by a software program (122). In at least one embodiment, the hardware resources include computer systems, platforms, or any other combination of hardware and software described herein capable of executing compressed neural networks.

[0029] In at least one embodiment, different versions of neural networks are generated by a neural network generation module (120), and the neural networks are compressed or optimized for each set of hardware resources. In at least one embodiment, a first version of the neural networks is compressed (e.g., converted to a low preview format) to fit hardware resources that support INT8 operations, and if a new set of hardware resources does not support it, a second version of the neural networks is generated to fit other data types (e.g., FP32) while minimizing loss of accuracy and using other optimization techniques specific to the new set of hardware resources. In at least one embodiment, the neural network generation module (120) generates compressed neural networks to be performed on various hardware resources supported by their own software stacks. In at least one embodiment, there may be a set of APIs used only for NVIDIA hardware, and a separate set of APIs used only for Intel hardware. FIG. 2 illustrates an exemplary system (200) for training one or more generator neural networks according to at least one embodiment. In at least one embodiment, the system (200) is a combination of hardware and software described together with FIG. 1. In at least one embodiment, the system (200) includes a training module (230).

[0030] In at least one embodiment, the training module (230) is a module for training one or more neural networks (e.g., neural network (114), neural network (116), neural network (124), neural network (232), neural network (234), neural network (236)) that generate one or more compressed neural networks, and said one or more compressed neural networks are deployed on a platform, computer system, or any other hardware resource different from where said one or more compressed neural networks were previously deployed. In at least one embodiment, said one or more compressed neural networks can perform various tasks (e.g., computer vision, NLP) described together with FIG. 1. In at least one embodiment, the training module (230) includes a model training system (3704).

[0031] In at least one embodiment, the training module (230) receives information of first computer systems, platforms, or any other hardware resources (e.g., NVIDIA) (212), information of second computer systems, platforms, or any other hardware resources (e.g., Intel) (214), and information of third computer systems, platforms, or any other hardware resources (e.g., AMD) (216). In at least one embodiment, additional information of additional computer systems, platforms, or any other hardware resources not illustrated in FIG. 2 exists.

[0032] In at least one embodiment, there exists a separate module (not illustrated in FIG. 2) configured to receive information from the first computer systems (212), information from the second computer systems (214), and information from the third computer systems (216) for the training module (230). In at least one embodiment, the training module (230) and / or said separate module receive said information from different sources (e.g., websites (NVIDIA, Intel, AMD), white papers, analysis articles, release notes, tutorials, toolkits, etc.) using different techniques (e.g., APIs, manual copy-paste, automated web scraping (e.g., BeautifulSoup, Scrapy, Octoparse, ParseHub), HTML parsing, DOM (document object model) parsing, web crawling, text pattern matching (Regex), headless browsers, subscriptions (e.g., RSS feeds), etc.).

[0033] In at least one embodiment, the training module (230) or the separate module (e.g., the data collection module (616)) generates hardware features of the computer systems (222) using information of the first computer systems (212), information of the second computer systems (214), information of the third computer systems (216), and / or additional information not illustrated in FIG. 2. In at least one embodiment, the hardware features include core count, clock speed, data type (e.g., floating-point 32, integer 8), memory capacity and type, memory bandwidth, router cores, ray tracing cores, tensor cores, thermal design power, API support, connectivity and ports, form factor and dimensions, manufacturing process, electrical characteristics, data propagation delays, power consumption, integrated peripherals, application circuits, or any other features obtainable from datasheets for the chip.

[0034] In at least one embodiment, the hardware features for NVIDIA (obtained from information of the first computer systems (212)) include, without limitation, chip information such as Tesla (e.g., Tesla K80, P100, V100, T4), Ampere (e.g., A100), Volta (e.g., V100), Turing (e.g., RTX 2080), Pascal (e.g., P100), DGX systems (e.g., DGX-1, DGX A100), Jetson (e.g., Jetson Nano, Jetson TX1 / TX2, Jetson Xavier NX, and Jetson AGX Xavier), BlueField for DPUs, Mellanox for networking products, DRIVE (e.g., DRIVE Xavier, DRIVE Pegasus, DRIVE Orin), and RTX series. In at least one embodiment, the different chips provide one or more features, without limitation, such as CUDA cores, precision support (e.g., INT8, FP16, FP32, FP64), tensor cores, sparsity acceleration, high memory bandwidth and capacity, multi-instance GPU, RT cores, NVLink, NVSwitch, etc.

[0035] In at least one embodiment, the hardware features for NVIDIA (obtained from information of the second computer systems (214)) include, without limitation, chip information such as Xeon Scalable processors (e.g., Xeon Phi), Nervana (e.g., NNP-I, NNP-T), Movidius (e.g., Movidius Myriad X VPU), Agilex, Stratix, Arria, EyeQ, Loihi, Habana Labs Gaudi, and Goya Processors (e.g., Gaudi AI Training Processor, Goya AI Inference Processor). In at least one embodiment, the different chips provide, without limitation, one or more features such as Intel Deep learning Boost, Neural Compute engine, Neuromorphic Computing Architecture, RDMA over Converged Ethernet, etc.

[0036] In at least one embodiment, the hardware features for NVIDIA (obtained from information of the third computer systems (216)) include, without limitation, chip information such as the Ryzen Treaddripper Series, Ryzen 9, 7, 5 Series, EPYC Server Processors, Radeon Instinct Series, Radeon Pro Series, Radeon RX 5000 and 6000 Series. In at least one embodiment, the different chips provide, without limitation, one or more features such as Zen microarchitecture, PCIe 4.0 / 5.0 support, Ray tracing capabilities, etc.

[0037] In at least one embodiment, a training module (230) or a separate module (e.g., a data collection module (616)) generates software features of computer systems (224) using information of the first computer systems (212), information of the second computer systems (214), information of the third computer systems (216), and / or additional information. In at least one embodiment, software features of platforms, computer systems, or any other hardware resources refer to software stacks used to perform one or more computational operations using said platforms, computer systems, or any other hardware resources. In at least one embodiment, a software stack includes software programs, operating systems, runtime environments, development frameworks, programming languages, libraries, APIs, etc.

[0038] In at least one embodiment, the software features for NVIDIA (e.g., tools, functions, libraries, APIs) (obtained from information of the first computer systems (212)) include, without limitation, CUDA, cuDNN, TensorRT, NVIDIA Collective Communication Library, NVIDIA RAPIDS, NVIDIA Deep Learning AI and HPC Software, NVIDIA DALI, NVIDIA Nsight Tools, NVIDIA JetPack SDK, NVIDIA Deep Learning GPU Training System, NVIDIA Clara, NVIDIA Omniverse, etc.

[0039] In at least one embodiment, the software features for Intel (e.g., tools, functions, libraries, APIs) (obtained from information of the second computer systems (214)) include, without limitation, OpenVINO, oneAPI, Intel Data Analytics Acceleration Library, Intel Math Kernel Library, Intel Distribution for Python, Intel Machine Learning Scaling Library, Intel Optimization for TensorFlow, Intel Optimization for PyTorch, Intel FPGA SDK for OpenCl, Intel Deep Learning Boost, Intel Movidius Neural Compute SDK, Intel RealSense SDK, Intel Integrated Performance Primitives, etc.

[0040] In at least one embodiment, the software features for AMD (e.g., tools, functions, libraries, APIs) (obtained from information of the third computer systems (216)) include, without limitation, ROCm, AMD MIOpen, AMD Infinity Hub, AMD Blis and libFLAME, AMD Math Kernel Library, Heterogeneous-Compute Interface for Portability, AMD Data Parallel C++, AMD Optimized Libraries, Radeon ProRender, etc.

[0041] In at least one embodiment, the training module (230) or the separate module (e.g., the data collection module (616)) generates information (e.g., code examples) associated with the connection of hardware and software features of the computer systems (226) using information of the first computer systems (212), information of the second computer systems (214), information of the third computer systems (216), and / or additional information. In at least one embodiment, the connection includes using CUDA APIs to control one or more chips (e.g., Tesla P100) for neural network inference / training. In another example, in at least one embodiment, the connection includes using oneAPI APIs to control one or more chips (e.g., Gaudi / Goya) for neural network inference / training. In another example, in at least one embodiment, the connection includes using ROCm functions to control one or more chips (e.g., Radeon) for neural network inference / training. In at least one embodiment, the APIs and functions are defined in one or more software programs.

[0042] In at least one embodiment, the training module (230) trains the initial neural networks (232) using information associated with the hardware features of the computer systems (222), the software features of the computer systems (224), and the connection of the hardware and software features of the computer systems (226) to generate the first fine-tuned neural networks (234). In at least one embodiment, the initial neural networks (232) include a pre-trained neural network (e.g., GPT, GPT-2, GPT-3, GPT-4, BERT, RoBERTa, DistilBERT, T5 (Text-To-Text Transfer Transformer), XLNet, Codex, BART, BlendorBot, Turing-NLG, LaMDA, CTRL, Gopher, RETRO, Chinchilla, Megatron-LM, LLaMA, ERNIE, ALBERT, ELECTRA, BLOOM, OpenAI Codex, Transformer-XL, AlexNet, VGGNet, RestNet, Inception, EfficientNet, YOLO, R-CNN, Mask R-CNN, StyleGAN, StyleGAN2, OpenPose, ULMFIT, TextBlob, spaCy's models, DeepSpeech, Wavenet, Tacotron, WaveGlow, ROS's SLAM, U-Net, DeepVariant, AlphaFold, etc.).

[0043] In at least one embodiment, the training module (230) trains the initial neural network (232) using the training framework (904), the initial neural networks (232) include an untrained neural network (906), and the first fine-tuned neural networks (234) include a trained neural network (908). In at least one embodiment, one or more initial neural networks (232) include pre-trained models (3806), pre-trained models (4142), and / or an initial model (4104). In at least one embodiment, the training is for the initial neural network (232) to learn basic knowledge about source and target platforms, computer systems, or any other hardware resources to solve model transformation tasks (e.g., creation of different versions to be placed on different hardware resources).

[0044] In at least one embodiment, the training module (230) trains the first fine-tuned neural networks (234) using the optimization dataset (228) to generate the second fine-tuned neural networks (236). In at least one embodiment, the optimization dataset (228) includes one-to-one correspondences (e.g., code examples) between source computer systems and target computer systems. In at least one embodiment, the one-to-one correspondences include groups of codes that deploy compressed neural networks (e.g., GPT-3) on the source system (e.g., Intel Gaudi) and on the target system (e.g., NVIDIA H100), and the relationships show a tight mapping between separate codes for different computer systems. In at least one embodiment, the one-to-one correspondences include compressed neural networks to be executed by different computer systems, platforms, or any other hardware resources.

[0045] In at least one embodiment, the training module (230) trains the first fine-tuned neural networks (234) training framework (904), and the first fine-tuned neural networks (234) include an untrained neural network (906) and second fine-tuned neural networks (236) and a trained neural network (908). In at least one embodiment, the second fine-tuned neural networks (236) include a refined model (4112) trained using model training (3714). In at least one embodiment, the training module (230) compares the output of the first fine-tuned neural networks (234) with an optimization dataset (228) during training. In at least one embodiment, the training module (230) uses the compared difference as the input and output of a contrast learning algorithm to prompt the tuning of the first fine-tuned neural networks (234).

[0046] In at least one embodiment, the training module (230) trains second fine-tuned neural networks (236) using a heuristic dataset (242) to generate trained neural networks. In at least one embodiment, the heuristic dataset (242) includes a user's prior experience of deploying neural networks on different computer systems to determine a preferred platform. In at least one embodiment, the prior experience includes using a specific computer system (e.g., NVIDIA H100) over another computer system (e.g., Gaudi chip in Intel Habana) due to precision support (e.g., FP8 support). In at least one embodiment, the prior experience includes using a specific computer system (e.g., NVIDIA H100) over another computer system (e.g., AMD MI300 APU) due to the efficiency of the trained neural network. In at least one embodiment, the prior experience includes preferences for software (e.g., CUDA over oneAPI or ROCm) among various users.

[0047] In at least one embodiment, the training module (230) trains second fine-tuned neural networks (236) using a training framework (904), the second fine-tuned neural networks (236) include an untrained neural network (906), and the trained neural networks include a trained neural network (908). In at least one embodiment, the trained neural networks include a refined model (4112) trained using model training (3714). In at least one embodiment, one or more neural networks after the second fine-tuning include a refined model (4112) trained using model training (3714).

[0048] In at least one embodiment, one or more trained neural networks (238) are prepared to be deployed on one or more computer systems to generate one or more computer programs that allow one or more compressed neural networks (e.g., different neural networks described together with FIG. 1) to be optimally deployed to allow the other neural networks to perform various computer vision, natural language processing tasks, etc. In at least one embodiment, one or more trained neural networks (238) generate different versions of one or more compressed neural networks depending on a target platform, computer system, or any other hardware resources to execute the one or more compressed neural networks.

[0049] In at least one embodiment, the trained neural networks include a neural network (124), one or more generator neural networks described together with FIG. 3, and / or one or more neural networks used in block 510. In at least one embodiment, the trained neural networks include a large language model (4212) available to generate output data (4220) using input data (5210). In at least one embodiment, a large language model API (4204) may be used to generate output data (4220).

[0050] In at least one embodiment, the system (200) includes one or more circuits that use one or more first neural networks to generate one or more second versions of one or more second neural networks based, at least partially, on one or more hardware resources to be used to perform one or more first versions of one or more second neural networks and one or more second versions of one or more second neural networks. In at least one embodiment, the one or more first neural networks refer to any of the first fine-tuned neural networks (234), the second fine-tuned neural networks (236), or the trained neural networks. In at least one embodiment, the one or more second neural networks refer to one or more compressed neural networks described herein.

[0051] In at least one embodiment, the one or more trained neural networks generate different versions of neural networks (e.g., the first version, the second version) such that the neural networks are compressed on various hardware resources having different features. In at least one embodiment, the first version may be compressed to be executed by a first hardware resource that supports only FP32, whereas the second version may be compressed to be executed by a second hardware resource that also supports INT8.

[0052] FIG. 3 illustrates an exemplary process (300) for generating one or more compressed neural networks performed by hardware resources according to at least one embodiment. Although the exemplary process (300) is described as a series of steps or operations, it will be understood that at least one embodiment of the process (300) includes modified or rearranged steps or operations, or omits certain steps or operations, except where explicitly mentioned or logically required, such as when the output of one step or operation is used as an input to another step. In at least one embodiment, each block of the process (300) is performed by one or more entities described in conjunction with FIG. 1, FIG. 2, FIG. 6, and FIG. 7, either alone or in any combination. In at least one embodiment, the one or more entities include a neural network training module (110), a neural network generation module (120), a training module (230), a training module (610), a compressed neural network generation module (612), and a training data collection module (614).

[0053] In at least one embodiment, the one or more entities further include a combination of hardware and software described together with FIG. 1. In at least one embodiment, various functions or blocks are performed by a processor that executes instructions stored in memory (e.g., computer-readable, machine-readable) to perform the process (300). In at least one embodiment, the process (300) may also be implemented as computer-readable instructions (e.g., macro instructions, micro-instructions) stored on a computer storage medium or provided by a standalone application, service, or hosted service (in combination with a standalone or other hosted service). In at least one embodiment, the computer-readable instructions performed by at least one processor (e.g., processor(s) (602)) are provided by one or more programming models (e.g., CUDA oneAPI, ROCm). In at least one embodiment, the processor(s) (602) perform one or more blocks of the process (300). In at least one embodiment, one or more APIs (710) or software programs (702) perform one or more blocks of the process (300), either individually or in combination.

[0054] In block 302, the one or more entities receive one or more inputs representing a source platform, and one or more compressed neural networks for the source platform and the target platform, in order to perform a new version of the one or more compressed neural networks according to at least one embodiment. In at least one embodiment, the representation includes the names of the source platform and the target platform. In at least one embodiment, the one or more inputs are received via an API or a web interface.

[0055] In block 304, according to at least one embodiment, the one or more entities use one or more generator neural networks (e.g., neural network (116), neural network (124), neural network (232), neural network (234), neural network (236)) to generate a new version of one or more compressed neural networks for the target platform using the one or more inputs.

[0056] In block 306, the one or more entities deploy the one or more compressed neural networks on the target platform so that the target platform can perform the new version of the one or more compressed neural networks. In at least one embodiment, the one or more entities include a deployment system (3706).

[0057] In at least one embodiment, at least one of blocks 302, 304, and / or 306 uses one or more first neural networks to generate one or more second versions of one or more second neural networks based, at least partially, on one or more hardware resources to be used to perform one or more first versions of one or more second neural networks and one or more second versions of one or more second neural networks. In at least one embodiment, one or more first neural networks refer to one or more generator neural networks. In at least one embodiment, one or more second versions refer to the new versions of one or more compressed neural networks. In at least one embodiment, one or more first versions refer to the compressed neural networks for a source platform.

[0058] FIG. 4 illustrates an exemplary process (400) for training one or more generator neural networks according to at least one embodiment. Although the exemplary process (400) is described as a series of steps or operations, it will be understood that at least one embodiment of the process (400) includes modified or rearranged steps or operations, or omits certain steps or operations, except where explicitly mentioned or logically required, such as when the output of one step or operation is used as an input to another step. In at least one embodiment, each block of the process (400) is performed by one or more entities described in conjunction with FIG. 1, FIG. 2, FIG. 6, and FIG. 7, either alone or in any combination. In at least one embodiment, the one or more entities include a neural network training module (110), a neural network generation module (120), a training module (230), a training module (610), a compressed neural network generation module (612), and a training data collection module (614).

[0059] In at least one embodiment, the one or more entities further include a combination of hardware and software described together with FIG. 1. In at least one embodiment, various functions are performed by a processor that executes instructions stored in memory (e.g., computer-readable, machine-readable) to perform the process (400). In at least one embodiment, the process (400) may also be implemented as computer-readable instructions (e.g., macro instructions, micro-instructions) stored on a computer storage medium or provided by a standalone application, service, or hosted service (in combination with a standalone or other hosted service). In at least one embodiment, the computer-readable instructions performed by at least one processor (e.g., processor(s) (602)) are provided by one or more programming models (e.g., CUDA oneAPI, ROCm). In at least one embodiment, the processor(s) (602) perform one or more blocks of the process (400). In at least one embodiment, one or more APIs (710) or software programs (702) perform one or more blocks of the process (400), either individually or in combination.

[0060] In block 404, according to at least one embodiment, the one or more entities receive information of different computer systems. In at least one embodiment, the information is obtained from websites (e.g., NVIDIA, Intel, AMD), white papers, analysis articles, release notes, tutorials, toolkits, etc. In at least one embodiment, the one or more entities use different techniques to obtain the information, which is described together with FIG. 2. In at least one embodiment, the one or more entities collect hardware features of the different computer systems based on the received information. In at least one embodiment, the hardware features include hardware features of computer systems (222).

[0061] In block 404, according to at least one embodiment, the one or more entities collect software features of the different computer systems based on received information. In at least one embodiment, the software features include software features of the computer systems (224).

[0062] In block 406, according to at least one embodiment, the one or more entities collect software programs based on received information. In at least one embodiment, the software programs include codes that provide examples of methods for utilizing hardware features of a platform, a computer system, or any other hardware resources based on software features. In at least one embodiment, the software programs include a connection of software features (226) and hardware that can be represented as one or more software programs. In at least one embodiment, the one or more software programs are executed by a plurality of platforms, computer systems, or any other hardware resources.

[0063] In block 408, the one or more entities train one or more generator neural networks (e.g., neural networks (116), neural networks (124), neural networks (232)) based on collected hardware features, software features, and software programs. In at least one embodiment, the one or more generator neural networks include pre-trained large language models capable of generating one or more software programs and / or one or more compressed neural networks.

[0064] In at least one embodiment, one or more entities train the one or more neural networks using a training framework (904). In at least one embodiment, one or more generator neural networks include an untrained neural network (906), and one or more trained neural networks include a trained neural network (908). In at least one embodiment, the one or more trained neural networks include neural networks (234) or neural networks (116). In at least one embodiment, the one or more trained neural networks include a large language model (4212). In at least one embodiment, the one or more entities include a model training system (3704), and the one or more trained neural networks include a refined model (4112) trained using model training (3714).

[0065] In at least one embodiment, at least one of blocks 402, 404, 406, and / or 408 uses one or more first neural networks to generate one or more second versions of one or more second neural networks based, at least partially, on one or more hardware resources to be used to perform one or more first versions of one or more second neural networks and one or more second versions of one or more second neural networks. In at least one embodiment, one or more first neural networks refer to the generator neural networks.

[0066] FIG. 5 illustrates an exemplary process (500) for fine-tuning one or more generator neural networks according to at least one embodiment. Although the exemplary process (500) is described as a series of steps or operations, it will be understood that at least one embodiment of the process (500) includes modified or rearranged steps or operations, or omits certain steps or operations, except where explicitly mentioned or logically required, such as when the output of one step or operation is used as an input to another step. In at least one embodiment, each block of the process (500) is performed by one or more entities described in conjunction with FIG. 1, FIG. 2, FIG. 6, and FIG. 7, either alone or in any combination. In at least one embodiment, the one or more entities include a neural network training module (110), a neural network generation module (120), a training module (230), a training module (610), a compressed neural network generation module (612), and a training data collection module (614).

[0067] In at least one embodiment, the one or more entities further include a combination of hardware and software described together with FIG. 1. In at least one embodiment, various functions are performed by a processor that executes instructions stored in memory (e.g., computer-readable, machine-readable) to perform the process (500). In at least one embodiment, the process (500) may also be implemented as computer-readable instructions (e.g., macro instructions, micro-instructions) stored on a computer storage medium or provided by a standalone application, service, or hosted service (in combination with a standalone or other hosted service). In at least one embodiment, the computer-readable instructions performed by at least one processor (e.g., processor(s) (602)) are provided by one or more programming models (e.g., CUDA oneAPI, ROCm). In at least one embodiment, the processor(s) (602) perform one or more blocks of the process (500). In at least one embodiment, one or more APIs (710) or software programs (702) perform one or more blocks of the process (500), either individually or in combination.

[0068] In block 502, according to at least one embodiment, the one or more entities receive an optimization dataset comprising pairs of software programs having one-to-one correspondences on different platforms, computer systems, or any other hardware resources. In at least one embodiment, the optimization dataset includes pairs of compressed neural networks (e.g., defined by software programs) deployed for each platform, computer system, or other hardware resource. In at least one embodiment, one or more compressed neural networks include a first version executed by a first hardware resource and a second version executed by a second hardware resource. In at least one embodiment, the optimization dataset includes an optimization dataset (228). In at least one embodiment, the one or more entities receive the optimization dataset from websites (e.g., NVIDIA, Intel, AMD), white papers, analysis articles, release notes, tutorials, toolkits, etc. In at least one embodiment, the one or more entities use different techniques to obtain the information, which are described together with FIG. 2.

[0069] In block 504, according to at least one embodiment, the one or more entities train one or more neural networks (e.g., neural networks (114), neural networks (234), and the one or more generator neural networks trained as a result of performing block 410) based on the optimized dataset. In at least one embodiment, the one or more entities train the one or more neural networks using a training framework (904). In at least one embodiment, the one or more neural networks include an untrained neural network (906), and the one or more trained neural networks include a trained neural network (908). In at least one embodiment, the one or more trained neural networks include neural networks (236) or neural networks (116). In at least one embodiment, the one or more entities include a model training system (3704), and the one or more trained neural networks include a refined model (4112) trained using model training (3714).

[0070] In block 506, according to at least one embodiment, the one or more entities receive a heuristic dataset containing configuration information received from a user (e.g., AI engineers). In at least one embodiment, the configuration information includes a heuristic dataset (242).

[0071] In block 508, according to at least one embodiment, the one or more entities train the one or more neural networks based on the heuristic dataset. In at least one embodiment, the one or more neural networks include a network (114), neural networks (236), and one or more neural networks trained as a result of performing block 504. In at least one embodiment, the one or more entities train the one or more neural networks using a training framework (904). In at least one embodiment, the one or more neural networks include an untrained neural network (906), and the one or more trained neural networks include a trained neural network (908). In at least one embodiment, the one or more trained neural networks include neural networks (238) or neural networks (116). In at least one embodiment, the one or more entities include a model training system (3704), and the one or more trained neural networks include a refined model (4112) trained using model training (3714). In at least one embodiment, the one or more neural networks may generate compressed neural networks (in the form of software programs) in different versions depending on the type of platform, computer system, or any other hardware resource.

[0072] In at least one embodiment, at least one of blocks 502, 504, 506, and / or 508 uses one or more first neural networks to generate one or more second versions of one or more second neural networks based, at least partially, on one or more hardware resources to be used to perform one or more first versions of one or more second neural networks and one or more second versions of one or more second neural networks. In at least one embodiment, one or more first neural networks refer to one or more neural networks that are trained as part of performing blocks 502, 504, 506, and / or 508. In at least one embodiment, one or more second versions refer to the compressed neural networks (in the form of software programs) in different versions depending on the type of platform, computer system, or any other hardware resource.

[0073] FIG. 6 illustrates an exemplary system (600) for generating one or more compressed neural networks performed by hardware resources according to at least one embodiment. In at least one embodiment, the system (600) uses one or more non-transient machine-readable media that store on the media a set of instructions that, when performed by one or more processors (e.g., processor(s) (602)), cause said one or more processors to perform an optimized arrangement of one or more neural networks. In at least one embodiment, the system (600) includes one or more processor(s) (602) that use one or more first neural networks to generate one or more second versions of said second neural networks based at least partially on one or more hardware resources to be used to perform one or more first versions of said second neural networks. In at least one embodiment, the processor(s) (602) include a training module (610), a compressed neural network generation module (612), and a training data collection module (614).

[0074] In at least one embodiment, the processor(s) (602) implement at least a part of the system (100) and / or system (200). In at least one embodiment, the processor (602) performs blocks 302, 304, 306, 402, 404, 406, 408, 410, 502, 504, 506, 508 and / or 510.

[0075] In at least one embodiment, the processor(s) (602) perform logic (815). In at least one embodiment, the processor(s) (602) are part of a data center (1000). In at least one embodiment, the processor(s) (602) is part of a CPU (e.g., CPU (1106), CPU (1118)) and / or is a processor (1110). In at least one embodiment, the processor(s) (602) are part of a CPU (1180(A)) or a CPU (1180(B)). In at least one embodiment, the processor(s) (602) are a processor (1202). In at least one embodiment, the processor(s) (602) are a processor (1310). In at least one embodiment, the processor(s) (602) are part of a CPU (1402). In at least one embodiment, the processor(s) (602) are part of a computer (1510). In at least one embodiment, the processor(s) (602) are any of the multi-core processors (1605(1) ... 1605(M)). In at least one embodiment, the processor(s) (602) is the processor (1607). In at least one embodiment, the processor(s) (602) is the application processor (1705). In at least one embodiment, the processor(s) (602) is the processor (2002). In at least one embodiment, the processor(s) (602) is the processor (2202). In at least one embodiment, the processor(s) (602) is the processor (2400). In at least one embodiment, the processor(s) (602) is the processor (2702). In at least one embodiment, the processor(s) (602) is the processor (2800) described herein.

[0076] In at least one embodiment, the processor(s) (602) are part of a GPU (1118) or a GPU (1120). In at least one embodiment, the processor(s) (602) are part of any of the GPUs (1184(A) ... 1184(H)). In at least one embodiment, the processor(s) (602) are part of a graphics card (1212). In at least one embodiment, the processor(s) (602) are part of a parallel processing unit (1414). In at least one embodiment, the processor(s) (602) are part of any of the GPUs (1610(1) ... (N)). In at least one embodiment, the processor(s) (602) are part of a graphics acceleration module (1646). In at least one embodiment, the processor(s) (602) are a graphics processor (1710). In at least one embodiment, the processor(s) (602) are a graphics processor (1810). In at least one embodiment, the processor(s) (602) are a graphics processor (1840). In at least one embodiment, the processor(s) (602) include at least one graphics core (1900). In at least one embodiment, the processor(s) (602) are part of a general-purpose graphics processor unit (GPGPU) (1930). In at least one embodiment, the processor(s) (602) are a parallel processor (2012). In at least one embodiment, the processor(s) (602) are a parallel processor (2100). In at least one embodiment, the processor(s) (602) are a graphics multiprocessor (2132). In at least one embodiment, the processor(s) (602) are part of any of the GPGPUs (2206A ... 2206D). In at least one embodiment, the processor(s) (602) are a graphics processor (2300). In at least one embodiment, the processor(s) (602) are part of a deep learning application processor (2500). In at least one embodiment, the processor(s) (602) are part of a neuromorphic processor (2600).In at least one embodiment, the processor(s) (602) are a graphics processor (2708). In at least one embodiment, the processor(s) (602) are an integrated graphics processor (2808). In at least one embodiment, the processor(s) (602) are a graphics processor (2900). In at least one embodiment, the processor(s) (602) are part of the graphics processing engine (3010) described herein. In at least one embodiment, the processor(s) (602) are any of the shader processors (3107A ... 3107F) described herein. In at least one embodiment, the processor(s) (602) are a shader processor (3202). In at least one embodiment, the processor(s) (602) are connected to a graphics execution unit (3208). In at least one embodiment, the processor(s) (602) are part of a parallel processing unit (PPU) (3300). In at least one embodiment, the processor(s) (602) are part of a general processor cluster (GPC) (3400). In at least one embodiment, the processor(s) (602) are a streaming multiprocessor (3600). In at least one embodiment, the processor(s) (602) are part of hardware (3722). In at least one embodiment, the processor(s) (602) perform a model training system (3704). In at least one embodiment, the processor(s) (602) are a processor (4206).

[0077] In at least one embodiment, the training module (610) is a module that trains one or more neural networks to generate a software program that defines the architecture of a compressed model (e.g., an optimized neural network) on a target platform, a computer system, or any other hardware resource.

[0078] In at least one embodiment, the training module (610) performs fine-tuning of a general pre-trained LLM with the approximate hardware characteristics dataset, the approximate software stacks dataset, and the approximate hardware-software connection dataset collected by the data collection module (614), and the fine-tuning aims to utilize the semantic understanding and analysis capabilities of the pre-trained LLM to learn basic knowledge about source and target platforms, computer systems, or any other hardware resources. In at least one embodiment, the fine-tuned pre-trained LLM refers to the approximate-fine-tuned LLM.

[0079] In at least one embodiment, the training module (610) performs additional training of the pre-trained LLM by generating some standard model compression tasks for some conventional neural networks as prompts and compares it with what the pre-trained LLM generates (e.g., corresponding code examples for deploying these compressed models on the source and target platforms). In at least one embodiment, the training is based, at least partially, on a comparison of the output of the approximate fine-tuned LLM with the fine-grained model compression dataset collected by the training data collection module (614), and the trained LLM refers to the fine-tuned LLM. In at least one embodiment, additional training is performed on the fine-tuned LLM by performing manual feedback (described together with FIGS. 1 through 5) to add the user's prior experience.

[0080] In at least one embodiment, the compressed neural network generation module (612) is a module that uses one or more neural networks to generate a compressed model (e.g., an optimized neural network) on a target computer system. In at least one embodiment, the compressed neural network generation module (612) generates computer programs that deploy the compressed model on a target platform using an LLM (e.g., a pre-trained LLM, a roughly fine-tuned LLM, a fine-tuned LLM) trained by the training module (610). In at least one embodiment, the LLM receives the name of the original platform, computer system, or any other hardware resource, the compressed model for the original platform, computer system, or any other hardware resource, or any other hardware resource, and / or the name of the target platform, computer system, or any other hardware resource as inputs for generating the compressed model for the target platform, computer system, or any other hardware resource.

[0081] In at least one embodiment, the training data collection module (614) is a module that collects different information (e.g., configurations, features) of different computer systems (e.g., platforms provided by NVIDIA, Intel, AMD).

[0082] In at least one embodiment, the training data collection module (614) collects introductions of hardware configurations and features from websites (e.g., datasheets of chips, white papers of computer systems, or analysis articles between different hardware) to generate a rough hardware characteristic dataset (e.g., Dataset_h).

[0083] In at least one embodiment, the training data collection module (614) collects introductions of software stacks and libraries (e.g., release notes and tutorials of software libraries, white papers of software toolkits, or analysis articles between different software stacks) from websites to generate a dataset of approximate software stacks (e.g., Dataset_s).

[0084] In at least one embodiment, the training data collection module (614) collects introductions and code examples of methods using software methods to call hardware features on a special platform to generate an approximate hardware-software connected dataset (e.g., Dataset_hs) (e.g., methods to accelerate convolution operations with a standard CUDA toolkit).

[0085] In at least one embodiment, the training data collection module (614) collects pairs of code examples having one-to-one correspondences on source and target platforms, computer systems, or any other hardware resources for model compression operations (e.g., a method using the quantization tool provided by TensorRT to train-post-quantize an FP32 model into an INT8 format compatible with an NVIDIA A100 GPU, and the exact same semantic example using the quantization tool provided by Intel Neural Compressor to train-post-quantize an FP32 model into an INT8 format compatible with an Intel Gaudi v2 chip) to generate a fine-grained model compression dataset (e.g., Dataset_c).

[0086] In at least one embodiment, the training data collection module (614) collects introductions of software stacks and libraries (e.g., release notes and tutorials of software libraries, white papers of software toolkits, or analysis articles between different software stacks) from websites to generate a dataset of approximate software stacks.

[0087] In at least one embodiment, generating a dataset of approximate software stacks; in at least one embodiment, compressing the models as part of deploying the models on the target platform so that the target platform can perform the models, tensor operations (e.g., addition, subtraction, multiplication, division, dot product, matrix multiplication, reshape, squeeze, expand, slice, concatenate, stack, transpose, inverse, determinant, eigenvalue, eigenvector, singular value decomposition, exponential, logarithm, square root, abs, power, convolution, pooling, ReLu, softmax, batch normalization, gradient computation, backpropagation, broadcasting, It includes accelerating indexing, one-hot encoding, etc.

[0088] In at least one embodiment, one or more platforms / frameworks / APIs, such as CUDA, oneAPI, OpenCL, ROCM, etc., are intended to accelerate one or more operations performed by the model by performing parallel execution, batch processing, kernel optimization, and memory utilization, either individually or together with one or more hardware acceleration libraries, such as TensorFlow, PyTorch, Keras, Theano, Microsoft Cognitive Toolkit, MXnet, Caffe, Chainer, DeepLearning4J, ONNX, JAX. In at least one embodiment, one or more operations include, without limitation, the tensor operations, training (e.g., forward pass, back pass, weight updates), and inference.

[0089] FIG. 7 illustrates an exemplary system (700) that uses one or more application programming interfaces (APIs) to generate one or more compressed neural networks performed by hardware resources according to at least one embodiment. In at least one embodiment, the system (700) performs an optimized deployment of one or more neural networks on a plurality of platforms, computer systems, or any other hardware resources. In at least one embodiment, the system (700) uses one or more first neural networks to generate one or more second versions of one or more second neural networks based at least partially on one or more hardware resources to be used to perform one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks.

[0090] In at least one embodiment, the software program (702) is a module described together with FIG. 1. In at least one embodiment, the software program (702) includes one or more modules. In at least one embodiment, one or more APIs (710) include the API described together with FIG. 1. In at least one embodiment, one or more APIs (710) are a set of software instructions that, when executed, cause one or more processors to perform one or more computational operations. In at least one embodiment, one or more APIs (710) refer to a block of reusable code capable of performing a specific task, accepting data, processing it, and returning a result. In at least one embodiment, the reusable block of code refers to one or more APIs (710) allowing them to be called multiple times within the software program (702) to perform their designated actions without the need to repeatedly write the same code using potentially different input values. In at least one embodiment, one or more APIs (710) are distributed or otherwise provided as part of one or more libraries (706), runtimes (704), drivers (704), and / or any other grouping of software and / or executable code further described herein. In at least one embodiment, one or more APIs (710) perform one or more computational operations in response to a call by a software program (702).

[0091] In at least one embodiment, a software program (702) is a set of text of software code, commands, instructions, or other sequences for instructing a computing device (including a processor such as the processor (702)) to initiate one or more different sets of instructions, such as APIs (710) or API functions (712), to perform one or more computational operations and / or to execute. In at least one embodiment, the functionality provided by one or more APIs (710) includes software functions (712), such as those available to accelerate one or more parts of the software programs (702) using one or more parallel processing units (PPUs), such as graphics processing units (GPUs). In at least one embodiment, the software program (702) includes a compiler.

[0092] In at least one embodiment, one or more APIs (710) include a hardware interface to one or more circuits that perform one or more computational operations. In at least one embodiment, one or more software APIs (710) described herein are implemented as one or more circuits that perform one or more techniques described below in conjunction with FIGS. 1 through 6. In at least one embodiment, one or more software programs (702) include instructions that, when executed, cause one or more hardware devices and / or circuits to perform one or more techniques described above in conjunction with FIGS. 1 through 6.

[0093] In at least one embodiment, a software program (702), such as software programs implemented by a user, utilizes one or more APIs (710) to perform various computing operations, such as memory reservation, matrix multiplication, arithmetic operations, or any computing operations performed by PPUs (parallel processing units), such as GPUs (graphics processing units), as further described herein. In at least one embodiment, one or more APIs (710) provide a set of callable functions (712), referred to herein as APIs, API functions, and / or functions, which individually perform one or more computing operations, such as computing operations related to parallel computing. In at least one embodiment, one or more APIs (710) provide a function (712) that causes one or more first neural networks to generate one or more second versions of one or more second neural networks based at least partially on one or more hardware resources to be used to perform one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, one or more APIs (710) provide functions (712) that cause the neural network to perform one or more operations, such as by the processor returning the called function to the processor that calls the neural network. In at least one embodiment, the processor includes processor(s) (602). In at least one embodiment, the neural network is one of the neural networks described together with FIG. 1.

[0094] In at least one embodiment, one or more software programs (702) interact with or otherwise communicate with one or more APIs (710) to perform one or more computing operations using one or more PPUs, such as GPUs. In at least one embodiment, one or more computing operations using one or more PPUs include at least one or more groups of computing operations that are at least partially accelerated by execution by said one or more PPUs. In at least one embodiment, one or more software programs (702) interact with one or more APIs (710) using a remote or local interface to facilitate parallel computing.

[0095] In at least one embodiment, the interface includes software instructions that, when executed, provide access to one or more functions (712) provided by one or more APIs (710). In at least one embodiment, the software program (702) uses a local interface when a software developer compiles the software program (702) with one or more libraries (706) that include or otherwise provide access to one or more APIs (710). In at least one embodiment, the software program (702) is statically compiled with uncompiled source code or pre-compiled libraries (706) that include instructions for performing one or more APIs (710). In at least one embodiment, the software program (702) is dynamically compiled, and the one or more software program uses a linker to link to one or more pre-compiled libraries (706) that include one or more APIs (710).

[0096] In at least one embodiment, a software program (702) uses a remote interface when a software developer uses a library (706) containing one or more APIs (710) through a network or other remote communication medium, or executes a software program that communicates with it in a different way. In at least one embodiment, one or more libraries (706) contain one or more APIs (710) to be executed by a remote computing service, such as a computing resource service provider. In another embodiment, one or more libraries (706) containing one or more APIs (710) will be executed by any other computing host that provides said one or more APIs (710) to one or more software programs (702). In at least one embodiment, one or more libraries (706) include Intel Math Kernel Library (MKL), Intel Data Analytics Acceleration Library (DAAL), Intel Integrated Performance Primitives (IPP), Intel Threading Building Blocks (TBB), Intel oneAPI DPC++ / C++ Compiler Zen Software Studio, ROCm Hub, Vitis Software Platform, and Vitis AI.

[0097] In at least one embodiment, one or more software programs (702) use one or more APIs (710) to allocate and otherwise manage memory to be used by said software programs (702). In at least one embodiment, one or more software programs (702) use one or more APIs (710) to allocate and otherwise manage memory to be used by one or more parts of said software programs (702) to be accelerated using one or more PPUs, such as GPUs or any other accelerator or processor further described herein. These software programs (702) request a neural network to generate one or more parts of an image based on one or more parts, at least partially.

[0098] In at least one embodiment, one or more APIs (710) include APIs that facilitate parallel computing. In at least one embodiment, the API (710) is provided by a driver and / or runtime (704). In at least one embodiment, the API (710) is provided by a CUDA user-mode driver. In at least one embodiment, the API (710) is provided by a CUDA runtime. In at least one embodiment, the driver (704) is software instructions and data values ​​that, when executed, perform or otherwise facilitate the operation of one or more functions (712) of the API (710) during the loading and execution of one or more parts of a software program (702). In at least one embodiment, the driver (704) includes Intel Graphics Drivers, Intel Chipset Drivers, Intel Network Adapter Drivers, Intel Audio Drivers, drivers for Intel Movidius VPUs and / or Intel Nervana neural network processors, and drivers that work with AMD Software: PRO Edition, AMD Radeon ProRender, AMD Software: Adrenalin Edition, AMD Ryzen Master Utility, and AMD StoreMI Technology, and AMD ROCm.

[0099] In at least one embodiment, the runtime (704) includes software instructions and data values ​​that, when executed, perform or otherwise facilitate the operation of one or more functions (712) of the API (710) during the execution of the software program (702). In at least one embodiment, the runtime (704) includes the Intel Graphics Runtime, the Intel oneAPI runtime, and the AMD Radeon Open Compute Platform. In at least one embodiment, one or more software programs (702) utilize one or more APIs (710) implemented or otherwise provided by the driver and / or runtime (704) to perform arithmetic operations combined by said one or more software programs (702) during execution by said one or more PPUs, such as GPUs.

[0100] In at least one embodiment, one or more software programs (702) use one or more APIs (710) provided by a driver and / or runtime (704) to perform combination arithmetic operations of one or more PPUs, such as GPUs. In at least one embodiment, one or more APIs (710) provide combination arithmetic operations through the driver and / or runtime (704) as described above. In at least one embodiment, one or more software programs (702) use one or more APIs (710) provided by a driver and / or runtime (704) to allocate or otherwise reserve one or more blocks of memory (714) of one or more PPUs, such as GPUs. In at least one embodiment, one or more software programs (702) use one or more APIs (710) provided by a driver and / or runtime (704) to allocate or otherwise reserve blocks of memory.

[0101] To improve the availability of software programs (702) to be accelerated by one or more PPUs, such as GPUs, and / or the optimization of one or more parts of said software programs (702), in an embodiment, one or more APIs (710) provide one or more API functions (712) that use one or more first neural networks to generate one or more second versions of said second neural networks based at least partially on one or more hardware resources to be used to execute one or more first versions of said second neural networks and one or more second versions of said second neural networks using the systems and processes described in conjunction with FIGS. 1 through 6. In at least one embodiment, the system (700) includes a processor comprising one or more circuits that execute one or more software programs that combine two or more APIs into a single API.

[0102] In at least one embodiment described in FIGS. 1 through 7, one or more technical improvements are provided, either alone or in combination with at least one embodiment in FIGS. 8 through 42. In at least one embodiment, the at least one embodiment described in FIGS. 1 through 7 results in the efficient deployment of a compressed model (e.g., optimized neural networks) on different platforms, computer systems, or any other hardware resources (e.g., NVIDIA, AMD, Intel, etc.) without the need to convert or transmit different hardware and / or software. In one example, the at least one embodiment described in FIGS. 1 through 7 results in the efficient deployment of a compressed model (e.g., optimized neural networks) on the different platforms, computer systems, or any other hardware resources without excessive iteration of training of the compressed model to maintain the accuracy of the compressed model on the platforms, computer systems, or any other hardware resources that are different from the originally deployed platforms, computer systems, or any other hardware resources. In another example, the at least one embodiment described in FIGS. 1 through 7 performs the deployment of a compressed model while saving time and computing resources. In another example, the at least one embodiment described in FIGS. 1 through 7 performs the deployment and generation of a compressed model without requiring the user of the compressed model to learn information associated with the different computer systems and to manually redesign it to adapt to new platforms, computer systems, or any other hardware resources (e.g., using an intermediate open-source format). In another example, the at least one embodiment described in FIGS. 1 through 7 performs the accurate and efficient deployment of compression by performing prompt tuning and semantic inference.

[0103] logic

[0104] FIG. 8a illustrates logic (815) that may be used in one or more devices to perform operations such as those discussed herein according to at least one embodiment, as described elsewhere in this specification. In at least one embodiment, the logic (815) is used to perform inference and / or training operations associated with one or more embodiments. In at least one embodiment, the logic (815) is inference and / or training logic. Details regarding the logic (815) are provided below together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic refers to any combination of software logic, hardware logic, and / or firmware logic that provides the functionality or operations described herein, and the logic may be implemented, collectively or individually, as a circuit forming part of a larger system, e.g., an integrated circuit (IC), a system-on-chip (SoC), or one or more processors (e.g., a CPU, a GPU).

[0105] In at least one embodiment, the logic (815) may, without limitation, include code and / or data storage (801) that stores forward and / or output weights and / or input / output data and / or other parameters to configure neurons or layers of a neural network used for training and / or inference in one or more embodiments. In at least one embodiment, the training logic (815) may include, or be connected to, code and / or data storage (801) that stores graph code or other software that controls the timing and / or order in which weight and / or other parameter information is loaded to configure logic including integer and / or floating-point units (collectively, arithmetic logic units). In at least one embodiment, code such as graph code loads weight or other parameter information to processor ALUs based on the architecture of the neural network to which such code corresponds. In at least one embodiment, the code and / or data storage (801) stores the weight parameters and / or input / output data of each layer of a neural network that is trained or used with one or more embodiments during the forward propagation of the input / output data and / or weight parameters during training and / or inference using an aspect of one or more embodiments. In at least one embodiment, any part of the code and / or data storage (801) may be included with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory.

[0106] In at least one embodiment, any part of the code and / or data storage (801) may be inside or outside of one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or code and / or data storage (801) may be cache memory, "DRAM" (dynamic randomly addressable memory), "SRAM" (static randomly addressable memory), non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the choice of whether the code and / or code and / or data storage (801) is, for example, inside or outside of a processor, or includes DRAM, SRAM, flash, or some other storage type may depend on the available storage on-chip versus off-chip, the latency requirements of the training and / or inference functions being performed, the batch size of the data used for the inference and / or training of the neural network, or some combination of these factors.

[0107] In at least one embodiment, the logic (815) may, without limitation, include a code and / or data storage (805) that stores reverse and / or output weights and / or input / output data corresponding to neurons or layers of a neural network used for training and / or inference in one or more embodiments. In at least one embodiment, the code and / or data storage (805) stores weight parameters and / or input / output data of each layer of a neural network that is trained or used with one or more embodiments during the reverse propagation of input / output data and / or weight parameters during training and / or inference using one or more embodiments. In at least one embodiment, the training logic (815) may include, or be connected to, a code and / or data storage (805) that stores graph code or other software that controls the timing and / or order in which weight and / or other parameter information is loaded to construct logic including integer and / or floating-point units (collectively, arithmetic logic units).

[0108] In at least one embodiment, a code such as a graph code causes the loading of weights or other parameter information into processor ALUs based on the architecture of the neural network corresponding to such code. In at least one embodiment, any part of the code and / or data storage (805) may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. In at least one embodiment, any part of the code and / or data storage (805) may be inside or outside of one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage (805) may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the choice of whether the code and / or data storage (805) is, for example, inside or outside the processor, or includes DRAM, SRAM, flash memory, or some other storage type may depend on the available storage on-chip versus off-chip, the latency requirements of the training and / or inference functions being performed, the batch size of the data used for the inference and / or training of the neural network, or some combination of these factors.

[0109] In at least one embodiment, the code and / or data storage (801) and the code and / or data storage (805) may be separate storage structures. In at least one embodiment, the code and / or data storage (801) and the code and / or data storage (805) may be combined storage structures. In at least one embodiment, the code and / or data storage (801) and the code and / or data storage (805) may be partially combined and partially separated. In at least one embodiment, any part of the code and / or data storage (801) and the code and / or data storage (805) may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory.

[0110] In at least one embodiment, the logic (815) may, without limitation, include one or more "ALU(s)" (arithmetic logic unit(s)) (810) comprising integer and / or floating-point units that perform logical and / or mathematical operations based at least partially on, or represented by, a training and / or inference code (e.g., graph code), and the result may produce activations (e.g., output values ​​from layers or neurons in a neural network) stored in an activation storage (820) which are functions of input / output and / or weight parameter data stored in the code and / or data storage (801) and / or the code and / or data storage (805). In at least one embodiment, activations stored in the activation storage (820) are generated according to linear algebra and / or matrix-based mathematics performed by the ALU(s) (810) in response to the execution of instructions or other code, wherein weight values ​​stored in the code and / or data storage (805) and / or data storage (801) are used as operands together with other values ​​such as bias values, gradient information, momentum values, or other parameters or hyperparameters, any or all of which may be stored in the code and / or data storage (805) or the code and / or data storage (801) or other storage on or off-chip.

[0111] In at least one embodiment, the ALU(s) (810) are contained within one or more processors or other hardware logic devices or circuits, whereas in other embodiments, the ALU(s) (810) may be outside the processor or other hardware logic devices or circuits using them (e.g., co-processors). In at least one embodiment, the ALUs (810) may be contained within the execution units of the processor or in a bank of ALUs accessible by the execution units of the processor distributed within the same processor or among different types of different processors (e.g., central processing units, graphics processing units, fixed function units, etc.). In at least one embodiment, the code and / or data storage (801), code and / or data storage (805), and activation storage (820) may share the processor or other hardware logic devices or circuits, whereas in other embodiments, they may be in different processors or other hardware logic devices or circuits, or some combination of the same and different processors or other hardware logic devices or circuits. In at least one embodiment, any part of the activation storage (820) may be included in other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. Additionally, the inference and / or training code may be stored together with other code accessible to the processor or other hardware logic or circuits, and may be fetched and / or processed using the processor's fetch, decoding, scheduling, execution, retirement and / or other logic circuits.

[0112] In at least one embodiment, the activation storage (820) may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the activation storage (820) may be wholly or partially located within or outside one or more processors or other logic circuits. In at least one embodiment, the choice of whether the activation storage (820) is, for example, inside or outside of a processor, or includes DRAM, SRAM, flash memory, or some other storage type may depend on the available storage on-chip versus off-chip, the latency requirements of the training and / or inference functions being performed, the batch size of the data used for the inference and / or training of the neural network, or some combination of these factors.

[0113] In at least one embodiment, the logic (815) illustrated in FIG. 8a is Google's TensorFlow® Processing Unit, Graphcore TM It can be used with an IPU (inference processing unit) or an ASIC (application-specific integrated circuit) such as Intel Corp’s Nervana® (e.g., "Lake Crest") processor. In at least one embodiment, the logic (815) illustrated in FIG. 8a can be used with other hardware such as CPU (central processing unit) hardware, GPU (graphics processing unit) hardware, or FPGAs (field programmable gate arrays).

[0114] FIG. 8b illustrates logic (815) according to at least one embodiment. In at least one embodiment, the logic (815) is inference and / or training logic. In at least one embodiment, the logic (815) may, without limitation, include hardware logic in which computational resources are dedicated or otherwise exclusively used, along with weight values ​​or other information corresponding to one or more layers of neurons in a neural network. In at least one embodiment, the logic (815) illustrated in FIG. 8b is Google’s TensorFlow® Processing Unit, Graphcore TMIt may be used with an inference processing unit (IPU) or an application-specific integrated circuit (ASIC), such as Intel Corp’s Nervana® (e.g., "Lake Crest") processor. In at least one embodiment, the logic (815) illustrated in FIG. 8b may be used with other hardware, such as central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or field programmable gate arrays (FPGAs). In at least one embodiment, the logic (815) includes, without limitation, code and / or data storage (801) and code and / or data storage (805), which may be used to store code (e.g., graph code), weight values ​​and / or bias values, gradient information, momentum values, and / or other information including other parameters or hyperparameter information. In at least one embodiment illustrated in FIG. 8b, the code and / or data storage (801) and the code and / or data storage (805) are each associated with a dedicated computational resource, such as computational hardware (802) and computational hardware (806). In at least one embodiment, the computational hardware (802) and the computational hardware (806) each include one or more ALUs that perform mathematical functions, such as linear algebraic functions, only on the information stored in the code and / or data storage (801) and the code and / or data storage (805), respectively, and the result is stored in the activation storage (820).

[0115] In at least one embodiment, the code and / or data storage (801 and 805) and the corresponding computation hardware (802 and 806) each correspond to different layers of the neural network, and the resulting activation from one storage / computation pair (801 / 802) of the code and / or data storage (801) and the computation hardware (802) is provided as an input to the next storage / computation pair (805 / 806) of the code and / or data storage (805) and the computation hardware (806) to mirror the conceptual organization of the neural network. In at least one embodiment, each of the storage / computation pairs (801 / 802 and 805 / 806) may correspond to more than one neural network layer. In at least one embodiment, additional storage / computation pairs (not shown) following or parallel to the storage / computation pairs (801 / 802 and 805 / 806) may be included in the logic (815).

[0116] Neural network training and batching

[0117] FIG. 9 illustrates the training and deployment of a deep neural network according to at least one embodiment. In at least one embodiment, an untrained neural network (906) is trained using a training dataset (902). In at least one embodiment, the training framework (904) is the PyTorch framework, whereas in other embodiments, the training framework (904) is TensorFlow, Boost, Caffe, Microsoft Cognitive Toolkit / CNTK, MXNet, Chainer, Keras, Deeplearning4j, or another training framework. In at least one embodiment, the training framework (904) enables the untrained neural network (906) to be trained using the processing resources described herein to generate a trained neural network (908). In at least one embodiment, weights may be selected by pre-training using a deep belief network or randomly. In at least one embodiment, training may be performed in a supervised, partially supervised, or unsupervised manner.

[0118] In at least one embodiment, the untrained neural network (906) is trained using supervised learning, wherein the training dataset (902) includes inputs paired with desired outputs for inputs, or the training dataset (902) includes inputs having known outputs and the outputs of the neural network (906) are manually graded. In at least one embodiment, the untrained neural network (906) is trained in a supervised manner, processes inputs from the training dataset (902), and compares the resulting outputs with a set of expected or desired outputs. In at least one embodiment, errors are then propagated back through the untrained neural network (906). In at least one embodiment, the training framework (904) adjusts the weights controlling the untrained neural network (906). In at least one embodiment, the training framework (904) includes tools for monitoring how well an untrained neural network (906) is converging toward a model, such as a trained neural network (908), which is suitable for generating accurate answers as in the result (914), based on input data such as a new dataset (912). In at least one embodiment, the training framework (904) refines the output of the untrained neural network (906) using a loss function and adjustment algorithm, such as stochastic gradient descent, by iteratively training the untrained neural network (906) while adjusting the weights. In at least one embodiment, the training framework (904) trains the untrained neural network (906) until the untrained neural network (906) achieves the desired accuracy. In at least one embodiment, the trained neural network (908) may then be deployed to implement any number of machine learning operations.

[0119] In at least one embodiment, an untrained neural network (906) is trained using unsupervised learning, wherein the untrained neural network (906) attempts to train itself using unlabeled data. In at least one embodiment, the unsupervised learning training dataset (902) will include input data without any associated output data or "real-world data" data. In at least one embodiment, the untrained neural network (906) can learn groupings within the training dataset (902) and determine how individual inputs relate to the untrained dataset (902). In at least one embodiment, unsupervised training may be used to generate a self-organizing map in a trained neural network (908) capable of performing operations useful for reducing the dimensionality of the new dataset (912). In at least one embodiment, unsupervised training may also be used to perform anomaly detection, which allows for the identification of data points in the new dataset (912) that deviate from the normal patterns of the new dataset (912).

[0120] In at least one embodiment, semi-supervised learning may be used, wherein the training dataset (902) is a technique comprising a mixture of labeled data and unlabeled data. In at least one embodiment, the training framework (904) may be used to perform incremental learning, as with transmitted learning techniques. In at least one embodiment, incremental learning enables the trained neural network (908) to adapt to a new dataset (912) without forgetting the knowledge injected into the trained neural network (908) during initial training.

[0121] In at least one embodiment, the training framework (904) is a framework processed in relation to a software development toolkit, such as the OpenVINO (Open Visual Inference and Neural Network Optimization) toolkit. In at least one embodiment, the OpenVINO toolkit is a toolkit such as those developed by Intel Corporation in Santa Clara, California. In at least one embodiment, OpenVINO includes logic (815) or uses logic (815) to perform the operations described herein. In at least one embodiment, an SoC, integrated circuit, or processor uses OpenVINO to perform the operations described herein.

[0122] In at least one embodiment, OpenVINO is a toolkit for facilitating the development of applications for various tasks and operations, specifically neural network applications, such as human vision emulation, speech recognition, natural language processing, recommendation systems, and / or variations thereof. In at least one embodiment, OpenVINO supports neural networks such as convolutional neural networks (CNNs), recurrent and / or attention-based neural networks, and / or various other neural network models. In at least one embodiment, OpenVINO supports various software libraries such as OpenCV, OpenCL, and / or variations thereof.

[0123] In at least one embodiment, OpenVINO supports neural network models for various tasks and operations such as classification, segmentation, object detection, face recognition, speech recognition, pose estimation (e.g., humans and / or objects), monocular depth estimation, image inpainting, style transfer, action recognition, colorization, and / or variations thereof.

[0124] In at least one embodiment, OpenVINO includes one or more software tools and / or modules for model optimization, also referred to as a model optimizer. In at least one embodiment, the model optimizer is a command-line tool that facilitates transitions between training and deployment of neural network models. In at least one embodiment, the model optimizer optimizes neural network models for execution on various devices and / or processing units, such as GPUs, CPUs, PPUs, GPGPUs, and / or variations thereof. In at least one embodiment, the model optimizer generates an internal representation of the model and optimizes the model to generate an intermediate representation. In at least one embodiment, the model optimizer reduces the number of layers in the model. In at least one embodiment, the model optimizer removes layers of the model used for training. In at least one embodiment, the model optimizer performs various neural network operations such as modifying inputs to the model (e.g., resizing inputs to the model), modifying the size of inputs to the model (e.g., modifying the batch size of the model), modifying the model structure (e.g., modifying layers of the model), normalization, standardization, quantization (e.g., converting weights of the model from a first representation such as floating-point to a second representation such as integers), and / or variations thereof.

[0125] In at least one embodiment, OpenVINO includes one or more software libraries for inference, also referred to as an inference engine. In at least one embodiment, the inference engine is a C++ library or any suitable programming language library. In at least one embodiment, the inference engine is used to infer input data. In at least one embodiment, the inference engine implements various classes to infer input data and generate one or more results. In at least one embodiment, the inference engine implements one or more API functions to process intermediate representations, set input and / or output formats, and / or run a model on one or more devices.

[0126] In at least one embodiment, OpenVINO provides various capabilities for heterogeneous execution of one or more neural network models. In at least one embodiment, heterogeneous execution, or heterogeneous computing, refers to one or more computing processes and / or systems utilizing one or more types of processors and / or cores. In at least one embodiment, OpenVINO provides various software functions for executing a program on one or more devices. In at least one embodiment, OpenVINO provides various software functions for executing a program and / or parts of a program on different devices. In at least one embodiment, OpenVINO provides various software functions for executing, for example, a first part of code on a CPU and a second part of code on a GPU and / or FPGA. In at least one embodiment, OpenVINO provides various software functions for executing one or more layers of a neural network on one or more devices (for example, a first set of layers on a first device, such as a GPU, and a second set of layers on a second device, such as a CPU).

[0127] In at least one embodiment, OpenVINO includes various functionalities similar to functionalities associated with CUDA programming models, such as various neural network model operations associated with frameworks such as TensorFlow, PyTorch, and / or variations thereof. In at least one embodiment, one or more CUDA programming model operations are performed using OpenVINO. In at least one embodiment, the various systems, methods, and / or techniques described herein are implemented using OpenVINO.

[0128] data center

[0129] FIG. 10 illustrates an exemplary data center (1000) in which at least one embodiment may be used. In at least one embodiment, the data center (1000) includes a data center infrastructure layer (1010), a framework layer (1020), a software layer (1030), and an application layer (1040).

[0130] In at least one embodiment, as illustrated in FIG. 10, the data center infrastructure layer (1010) may include a resource orchestrator (1012), grouped computing resources (1014), and node computing resources ("node CRs") (1016(1) to 1016(N)), where "N" represents any positive integer (which may be an integer "N" different from that used in other drawings). In at least one embodiment, the node CRs (1016(1) to 1016(N)) may include, but are not limited to, any number of "CPUs" (central processing units) or other processors (including accelerators, FPGAs (field programmable gate arrays), graphics processors, etc.), memory storage devices (1018(1) to 1018(N)) (e.g., dynamic read-only memory, solid-state storage, or disk drives), "NW I / O (network input / output) devices, network switches, "VMs" (virtual machines), power modules, and cooling modules, etc. In at least one embodiment, one or more node CRs from the node CRs (1016(1) to 1016(N)) may be servers having one or more of the computing resources mentioned above.

[0131] In at least one embodiment, the grouped computing resources (1014) may include node CRs housed within one or more racks (not shown), or separate groups of many racks (also not shown) housed in data centers at various geographic locations. In at least one embodiment, the separate groups of node CRs within the grouped computing resources (1014) may include grouped computing, network, memory, or storage resources that may be configured or allocated to support one or more workloads. In at least one embodiment, some node CRs, including CPUs or processors, may be grouped within one or more racks to provide computing resources that support one or more workloads. In at least one embodiment, one or more racks may also include any number of power modules, cooling modules, and network switches in any combination.

[0132] In at least one embodiment, the resource orchestrator (1012) may organize or otherwise control one or more node CRs (1016(1) to 1016(N)) and / or grouped computing resources (1014). In at least one embodiment, the resource orchestrator (1012) may include a "software design infrastructure" (SDI) management entity for the data center (1000). In at least one embodiment, the resource orchestrator (812) may include hardware, software, or a combination of some of these.

[0133] In at least one embodiment, as illustrated in FIG. 10, the framework layer (1020) includes a task scheduler (1022), a configuration manager (1024), a resource manager (1026), and a distributed file system (1028). In at least one embodiment, the framework layer (1020) may include a framework that supports software (1032) of the software layer (1030) and / or one or more application(s) (1042) of the application layer (1040). In at least one embodiment, the software (1032) or application(s) (1042) may each include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer (1020) may include, but is not limited to, Apache Spark, which can utilize the distributed file system (1028) for large-scale data processing (e.g., "big data"). TMIt may be a type of free and open-source software web application framework such as (hereinafter, "Spark"). In at least one embodiment, the job scheduler (1022) may include a Spark driver that facilitates the scheduling of workloads supported by various layers of the data center (1000). In at least one embodiment, the configuration manager (1024) may configure different layers such as a framework layer (1020) and a software layer (1030) that include Spark and a distributed file system (1028) to support large-scale data processing. In at least one embodiment, the resource manager (1026) may be able to manage clustered or grouped computing resources allocated or mapped thereto for the support of the distributed file system (1028) and the job scheduler (1022). In at least one embodiment, the clustered or grouped computing resources may include computing resources (1014) grouped in the data center infrastructure layer (1010). In at least one embodiment, the resource manager (1026) can coordinate with the resource orchestrator (1012) to manage these mapped or allocated computing resources.

[0134] In at least one embodiment, the software (1032) included in the software layer (1030) may include software used by at least some of the node CRs (1016(1) to 1016(N)), grouped computing resources (1014), and / or the distributed file system (1028) of the framework layer (1020). In at least one embodiment, one or more types of software may include, but are not limited to, Internet web page search software, email virus scan software, database software, and streaming video content software.

[0135] In at least one embodiment, the application(s) (1042) included in the application layer (1040) may include at least some of the node CRs (1016(1) to 1016(N)), grouped computing resources (1014), and / or one or more types of applications used by the distributed file system (1028) of the framework layer (1020). In at least one embodiment, one or more types of applications may include any number of genomics applications, cognitive computing applications, and machine learning applications, including, but not limited to, training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used with one or more embodiments.

[0136] In at least one embodiment, any of the configuration manager (1024), resource manager (1026), and resource orchestrator (1012) may implement any number and type of self-correcting actions based on any amount and type of data acquired in any technically feasible manner. In at least one embodiment, the self-correcting actions may mitigate the data center operator of the data center (1000) from making poor configuration decisions and avoiding underutilized and / or poorly performing parts of the data center.

[0137] In at least one embodiment, the data center (1000) may include tools, services, software, or other resources for training one or more machine learning models according to one or more embodiments described herein, or for using one or more machine learning models to predict or infer information. For example, in at least one embodiment, the machine learning model may be trained by calculating weight parameters according to a neural network architecture using the software and computing resources described above with respect to the data center (1000). In at least one embodiment, the trained machine learning model corresponding to one or more neural networks may be used to infer or predict information using the resources described above with respect to the data center (1000) by using weight parameters calculated through one or more training techniques described herein.

[0138] In at least one embodiment, a data center may use CPUs, ASICs (application-specific integrated circuits), GPUs, FPGAs, or other hardware to perform training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described above may be configured as a service that allows users to train or perform inference of information, such as image recognition, speech recognition, or other artificial intelligence services.

[0139] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a data center (1000) to infer or predict operations based, at least partially, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0140] At least one embodiment of FIGS. 1 through 10 performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0141] autonomous vehicles

[0142] FIG. 11a illustrates an example of an autonomous vehicle (1100) according to at least one embodiment. In at least one embodiment, the autonomous vehicle (1100) (alternatively referred to herein as “vehicle (1100)”) may be a passenger vehicle, such as a passenger car, truck, bus, and / or other type of vehicle that accommodates one or more passengers, without limitation. In at least one embodiment, the vehicle (1100) may be a semi-tractor-trailer truck used to transport cargo. In at least one embodiment, the vehicle (1100) may be an airplane, a robotic vehicle, or another type of vehicle.

[0143] Autonomous vehicles may be described in terms of levels of automation defined by the NHTSA (National Highway Traffic Safety Administration), a division of the US Department of Transportation, and the SAE (Society of Automotive Engineers) Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (e.g., Standard No. J3016-201806 published on June 15, 2018; Standard No. J3016-201609 published on September 30, 2016; and previous and future versions of these standards). In at least one embodiment, the vehicle (1100) may be capable of functionality according to one or more of Level 1 through Level 5 of autonomous driving levels. For example, in at least one embodiment, the vehicle (1100) may be capable of conditional automation (Level 3), high level of automation (Level 4), and / or full automation (Level 5), depending on the embodiment.

[0144] In at least one embodiment, the vehicle (1100) may include, without limitation, components such as a chassis, a body, wheels (e.g., 2, 4, 6, 8, 18, etc.), tires, axles, and other components of the vehicle. In at least one embodiment, the vehicle (1100) may include, without limitation, a propulsion system (1150), such as an internal combustion engine, a hybrid power plant, an all-electric engine, and / or another type of propulsion system. In at least one embodiment, the propulsion system (1150) may be connected to a drive train of the vehicle (1100), which may, without limitation, include a transmission that enables propulsion of the vehicle (1100). In at least one embodiment, the propulsion system (1150) may be controlled in response to receiving signals from a throttle / accelerator(s) (1152).

[0145] In at least one embodiment, a steering system (1154), which may include, without limitation, a steering wheel, is used to steer the vehicle (1100) (e.g., along a desired path or route) when the propulsion system (1150) is operating (e.g., when the vehicle (1100) is moving). In at least one embodiment, the steering system (1154) may receive signals from steering actuator(s) (1156). In at least one embodiment, the steering wheel may be optional for full automation (Level 5) functionality. In at least one embodiment, a brake sensor system (1146) may be used to actuate the vehicle brakes in response to receiving signals from brake actuator(s) (1148) and / or brake sensors.

[0146] In at least one embodiment, a controller(s) (1136), which may include, without limitation, one or more "SoCs" (system on chips) (not shown in FIG. 11a) and / or "GPU(s)" (graphics processing unit(s)), provides signals (e.g., commands) to one or more components and / or systems of a vehicle (1100). For example, in at least one embodiment, the controller(s) (1136) may transmit signals to operate vehicle brakes via brake actuator(s) (1148), to operate a steering system (1154) via steering actuator(s) (1156), and to operate a propulsion system (1150) via throttle / accelerator(s) (1152). In at least one embodiment, the controller(s) (1136) may include one or more onboard (e.g., integrated) computing devices for processing sensor signals, and output action commands (e.g., signals representing commands) that enable autonomous driving and / or assist a human driver in driving the vehicle (1100). In at least one embodiment, the controller(s) (1136) may include a first controller for autonomous driving functions, a second controller for functional safety functions, a third controller for artificial intelligence functions (e.g., computer vision), a fourth controller for infotainment functions, a fifth controller for redundancy in emergencies, and / or other controllers. In at least one embodiment, a single controller may handle two or more of the above functions, two or more controllers may handle a single function, and / or any combination thereof.

[0147] In at least one embodiment, the controller(s) (1136) provide signals for controlling one or more components and / or systems of the vehicle (1100) in response to sensor data (e.g., sensor inputs) received from one or more sensors. In at least one embodiment, sensor data are, for example, and without limitation, "GNSS" (global navigation satellite systems) sensor(s) (1158) (e.g., Global Positioning System sensor(s)), RADAR sensor(s) (1160), ultrasonic sensor(s) (1162), LIDAR sensor(s) (1164), "IMU" (inertial measurement unit) sensor(s) (1166) (e.g., accelerometer(s), gyroscope(s), magnetic compass or magnetic compasses, magnetometer(s), etc.), microphone(s) (1196), stereo camera(s) (1168), wide-view camera(s) (1170) (e.g., fisheye camera), infrared camera(s) (1172), surround camera(s) (1174) (e.g., 360-degree camera), long-range camera (not shown in FIG. 11a), medium-range camera(s) (not shown in FIG. 11a) It may be received from speed sensor(s) (1144) (e.g., for measuring the speed of a vehicle (1100)), vibration sensor(s) (1142), steering sensor(s) (1140), brake sensor(s) (e.g., as part of a brake sensor system (1146)), and / or other sensor types.

[0148] In at least one embodiment, one or more of the controller(s) (1136) receive inputs (e.g., represented by input data) from the device cluster (1132) of the vehicle (1100) and may provide outputs (e.g., represented by output data, display data, etc.) through an HMI (human-machine interface) display (1134), an audible annunciator, a loudspeaker, and / or other components of the vehicle (1100). In at least one embodiment, the outputs may include information such as vehicle speed, velocity, time, map data (e.g., a high-definition map (not shown in FIG. 11a)), location data (e.g., the location of the vehicle (1100), such as on a map), direction, the location of other vehicles (e.g., an occupancy grid), information regarding objects and the state of objects as perceived by the controller(s) (1136). For example, in at least one embodiment, the HMI display (1134) displays information regarding the presence of one or more objects (e.g., street signs, warning signs, traffic light changes, etc.) and / or information regarding driving operations that have been, are being, or will be done (e.g., changing lanes now, taking Exit 34B within 2 miles, etc.).

[0149] In at least one embodiment, the vehicle (1100) further comprises a network interface (1124) capable of using wireless antenna(s) (1126) and / or modem(s) to communicate through one or more networks. For example, in at least one embodiment, the network interface (1124) can communicate through "LTE" (Long-Term Evolution), "WCDMA" (Wideband Code Division Multiple Access), "UMTS" (Universal Mobile Telecommunications System), "GSM" (Global System for Mobile communication), IMT-CDMA Multi-Carrier ("CDMA2000") networks, etc. In at least one embodiment, the wireless antenna(s) (1126) can also enable communication between objects in an environment (e.g., vehicles, mobile devices, etc.) using local area network(s) such as Bluetooth, Bluetooth "LE" (Low Energy), Z-Wave, ZigBee, etc., and / or "LPWANs" (low power wide-area network(s)) protocols such as LoRaWAN, SigFox, etc.

[0150] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a vehicle (1100) to infer or predict operations based, at least partially, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0151] At least one embodiment of FIGS. 1 through 11a performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0152] FIG. 11b illustrates examples of camera positions and views for the autonomous vehicle (1100) of FIG. 11a according to at least one embodiment. In at least one embodiment, the cameras and their respective views are exemplary embodiments and are not intended to be limiting. For example, in at least one embodiment, additional and / or alternative cameras may be included and / or the cameras may be located at different positions on the vehicle (1100).

[0153] In at least one embodiment, camera types for the cameras may include digital cameras that are adaptable for use with components and / or systems of the vehicle (1100), but are not limited thereto. In at least one embodiment, the camera(s) may operate at "ASIL (automotive safety integrity level) B and / or other ASILs. In at least one embodiment, the camera types may be capable of any image capture rate, such as 60 fps (frames per second), 1220 fps, 240 fps, etc., depending on the embodiment. In at least one embodiment, the cameras may be capable of using rolling shutters, global shutters, other types of shutters, or a combination thereof. In at least one embodiment, the color filter array may include an "RCCC (red clear clear clear) color filter array, an "RCCB (red clear clear blue) color filter array, an "RBGC (red blue green clear) color filter array," a Foveon X3 color filter array, a Bayer sensor ("RGGB") color filter array, a monochromatic sensor color filter array, and / or other types of color filter arrays. In at least one embodiment, transparent pixel cameras, such as cameras with RCCC, RCCB, and / or RBGC color filter arrays, may be used in an effort to increase light sensitivity.

[0154] In at least one embodiment, one or more of the camera(s) may be used to perform "ADAS (advanced driver assistance systems)" functions (e.g., as part of a redundant or safety-assist design). For example, in at least one embodiment, a Multi-Function Mono Camera may be installed to provide functions including lane departure warning, traffic sign assistance, and intelligent headlamp control. In at least one embodiment, one or more of the camera(s) (e.g., all cameras) may simultaneously record and provide image data (e.g., video).

[0155] In at least one embodiment, one or more cameras may be mounted in a mounting assembly, such as a custom-designed ("3D" three-dimensional) printed assembly, to block stray light and reflections from within the vehicle (1100) that may interfere with camera image data capture capabilities (e.g., reflections from the dashboard reflected in the windshield mirrors). Referring to wing-mirror mounting assemblies, in at least one embodiment, wing-mirror assemblies may be custom 3D printed so that the camera mounting plate matches the shape of the wing-mirror. In at least one embodiment, the camera(s) may be integrated into the wing-mirrors. In at least one embodiment, for side-view cameras, the camera(s) may also be integrated within four pillars located at each corner of the cabin.

[0156] In at least one embodiment, cameras having a field of view including parts of the environment in front of the vehicle (1100) (e.g., front cameras) may be used for a surround view to help identify forward-facing paths and obstacles, as well as to help provide important information for generating an occupancy grid and / or determining desirable vehicle paths with the help of one or more of controller(s) (1136) and / or control SoCs. In at least one embodiment, front cameras may be used to perform many ADAS functions similar to LIDAR, including, without limitation, emergency braking, pedestrian detection, and collision avoidance. In at least one embodiment, front cameras may also be used for ADAS functions and systems, including, without limitation, other functions such as Lane Departure Warnings (LDW), Autonomous Cruise Control (ACC), and / or traffic sign recognition.

[0157] In at least one embodiment, various cameras including a monocular camera platform comprising a CMOS ("complementary metal oxide semiconductor") color imager may be used in the forward configuration. In at least one embodiment, a wide-view camera (1170) may be used to recognize objects entering the view from the surroundings (e.g., pedestrians, crossing traffic, or bicycles). Although only one wide-view camera (1170) is illustrated in FIG. 11b, in other embodiments, any number (including zero) of wide-view cameras may be present on the vehicle (1100). In at least one embodiment, any number of long-view camera(s) (1198) (e.g., a pair of long-view stereo cameras) may be used for depth-based object detection, particularly for objects for which the neural network has not yet been trained. In at least one embodiment, the long-view camera(s) (1198) may also be used for object detection and classification as well as basic object tracking.

[0158] In at least one embodiment, any number of stereo camera(s) (1168) may also be included in the forward configuration. In at least one embodiment, one or more of the stereo camera(s) (1168) may include an integrated control unit comprising a scalable processing unit capable of providing a multi-core microprocessor and programmable logic ("FPGA") with an integrated "CAN (Controller Area Network)" or Ethernet interface on a single chip. In at least one embodiment, such a unit may be used to generate a 3D map of the environment of the vehicle (1100) including distance estimates for all points in the image. In at least one embodiment, one or more of the stereo camera(s) (1168) may include, without limitation, a compact stereo vision sensor(s) that may include two camera lenses (one on the left and one on the right), and an image processing chip that may measure the distance from the vehicle (1100) to a target object and use generated information (e.g., metadata) to enable autonomous emergency braking and lane departure warning functions. In at least one embodiment, in addition to those described herein, or alternatively, other types of stereo camera(s) (1168) may be used.

[0159] In at least one embodiment, cameras having a field of view that includes parts of the environment for the sides of the vehicle (1100) (e.g., side-view cameras) may be used for a surround view to provide information used to generate side impact collision warnings as well as to generate and update an occupancy grid. For example, in at least one embodiment, surround camera(s) (1174) (e.g., four surround cameras as exemplified in FIG. 11b) may be positioned on the vehicle (1100). In at least one embodiment, the surround camera(s) (1174) may include, without limitation, any number and combination of wide-view cameras, fisheye camera(s), 360-degree camera(s), and / or similar cameras. For example, in at least one embodiment, four fisheye cameras may be positioned on the front, rear, and sides of the vehicle (1100). In at least one embodiment, the vehicle (1100) may use three surround camera(s) (1174) (e.g., left, right, and rear) and may utilize one or more other camera(s) (e.g., front camera) as a fourth surround-view camera.

[0160] In at least one embodiment, cameras having a field of view including parts of the environment behind the vehicle (1100) (e.g., rear-view cameras) may be used for parking assistance, surround view, rear collision warning, and the generation and updating of an occupancy grid. In at least one embodiment, as described herein, but not limited thereto, a wide variety of cameras may be used, including cameras also suitable as front camera(s) (e.g., long-range cameras (1198) and / or mid-range camera(s) (1176), stereo camera(s) (1168), infrared camera(s) (1172), etc.).

[0161] At least one embodiment of FIGS. 1 through 11b performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0162] FIG. 11c is a block diagram illustrating an exemplary system architecture for the autonomous vehicle (1100) of FIG. 11a according to at least one embodiment. In at least one embodiment, each of the components, features, and systems of the vehicle (1100) of FIG. 11c is illustrated as being connected via a bus (1102). In at least one embodiment, the bus (1102) may, without limitation, include a CAN data interface (alternatively referred to herein as a “CAN bus”). In at least one embodiment, the CAN may be a network within the vehicle (1100) used to assist in the control of various features and functions of the vehicle (1100), such as the operation of brakes, acceleration, braking, steering, windshield wipers, etc. In at least one embodiment, the bus (1102) may be configured to have tens or even hundreds of nodes, each having its own unique identifier (e.g., CAN ID). In at least one embodiment, the bus (1102) can be read to detect steering wheel angle, ground speed, engine RPMs (revolutions per minute), button positions, and / or other vehicle status indicators. In at least one embodiment, the bus (1102) may be an ASIL B compliant CAN bus.

[0163] In at least one embodiment, in addition to CAN or alternatively, FlexRay and / or Ethernet protocols may be used. In at least one embodiment, any number of buses forming the bus (1102) may exist, which may include, without limitation, zero or more CAN buses, zero or more FlexRay buses, zero or more Ethernet buses, and / or zero or more other types of buses using different protocols. In at least one embodiment, two or more buses may be used to perform different functions and / or may be used for redundancy. For example, the first bus may be used for collision avoidance functionality, and the second bus may be used for operation control. In at least one embodiment, each bus of the bus (1102) may communicate with any of the components of the vehicle (1100), and two or more buses of the bus (1102) may communicate with corresponding components. In at least one embodiment, any number of “SoC(s)” (system(s) on chip(s)) (1104) (such as SoC (1104(A)) and SoC (1104(B))), each controller(s) (1136), and / or each computer in the vehicle may have access to the same input data (e.g., inputs from sensors of the vehicle (1100)) and may be connected to a common bus, such as a CAN bus.

[0164] In at least one embodiment, the vehicle (1100) may include one or more controller(s) (1136), such as those described herein in connection with FIG. 11a. In at least one embodiment, the controller(s) (1136) may be used for various functions. In at least one embodiment, the controller(s) (1136) may be connected to any of the various other components and systems of the vehicle (1100) and may be used for the control of the vehicle (1100), artificial intelligence of the vehicle (1100), infotainment for the vehicle (1100), and / or other functions.

[0165] In at least one embodiment, the vehicle (1100) may include any number of SoCs (1104). In at least one embodiment, each of the SoCs (1104) may include, without limitation, "CPU(s)" (central processing units) (1106), "GPU(s)" (graphics processing units) (1108), processor(s) (1110), cache(s) (1112), accelerator(s) (1114), data storage(s) (1116), and / or other components and features not exemplified. In at least one embodiment, the SoC(s) (1104) may be used to control the vehicle (1100) on various platforms and systems. For example, in at least one embodiment, the SoC(s) (1104) may be combined with a “HD” (High Definition) map (1122) and a system (e.g., a system of a vehicle (1100)) that can obtain map refreshes and / or updates from one or more servers (not shown in FIG. 11c) via a network interface (1124).

[0166] In at least one embodiment, the CPU(s) (1106) may include a CPU cluster or CPU complex (alternatively referred to herein as "CCPLEX"). In at least one embodiment, the CPU(s) (1106) may include multiple cores and / or "L2" (level two) caches. For example, in at least one embodiment, the CPU(s) (1106) may include eight cores in a coherent multi-processor configuration. In at least one embodiment, the CPU(s) (1106) may include four dual-core clusters, each having a dedicated L2 cache (e.g., 2 MB (megabyte) L2 cache). In at least one embodiment, the CPU(s) (1106) (e.g., CCPLEX) may be configured to support concurrent cluster operations that enable any combination of the clusters of the CPU(s) (1106) to be active at any given time.

[0167] In at least one embodiment, one or more of the CPU(s) (1106) may implement power management capabilities including, without limitation, one or more of the following features: individual hardware blocks may be automatically clock-gated when idle to save dynamic power; each core clock may be gated when such core is not actively executing instructions due to the execution of "WFI (Wait for Interrupt) / "WFE (Wait for Event) instructions; each core may be power-gated independently; each core cluster may be clock-gated independently when all cores are clock-gated or power-gated; and / or each core cluster may be power-gated independently when all cores are power-gated. In at least one embodiment, the CPU(s) (1106) may further implement an enhanced algorithm for managing power states, wherein allowed power states and expected wake-up times are specified, and hardware / microcode determines which best power state to enter for the core, cluster, and CCPLEX. In at least one embodiment, the processing cores may support simplified power state entry sequences in software as tasks offloaded to the microcode.

[0168] In at least one embodiment, the GPU(s) (1108) may include an integrated GPU (alternatively referred to herein as "iGPU"). In at least one embodiment, the GPU(s) (1108) may be programmable and efficient for parallel workloads. In at least one embodiment, the GPU(s) (1108) may use an enhanced tensor instruction set. In at least one embodiment, the GPU(s) (1108) may include one or more streaming microprocessors, each of which may include an L1 (level one) cache (e.g., an L1 cache having at least 96 KB of storage capacity), and two or more streaming microprocessors may share an L2 cache (e.g., an L2 cache having 512 KB of storage capacity). In at least one embodiment, the GPU(s) (1108) may include at least eight streaming microprocessors. In at least one embodiment, the GPU(s) (1108) may use computing API(s) (application programming interface(s)). In at least one embodiment, the GPU(s) (1108) may use one or more parallel computing platforms and / or programming models (e.g., NVIDIA's CUDA model).

[0169] In at least one embodiment, one or more of the GPU(s) (1108) may be power-optimized for best performance in automotive and embedded use cases. For example, in at least one embodiment, the GPU(s) (1108) may be manufactured on FinFET (Fin field-effect transistor) circuits. In at least one embodiment, each streaming microprocessor may incorporate a number of mixed-precision processing cores partitioned into a number of blocks. For example, and without limitation, 64 PF32 cores and 32 FP64 cores may be partitioned into 4 processing blocks. In at least one embodiment, each processing block may be allocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, 2 mixed-precision NVIDIA Tensor cores for deep learning matrix arithmetic, an L0 (level zero) instruction cache, a scheduler (e.g., a warp scheduler) or sequencer, a dispatch unit, and / or a 64 KB register file. In at least one embodiment, streaming microprocessors may include independent parallel integer and floating-point data paths to provide efficient execution of workloads with a mix of computation and addressing computations. In at least one embodiment, streaming microprocessors may include independent thread scheduling capabilities to enable finer-grained synchronization and cooperation between parallel threads. In at least one embodiment, streaming microprocessors may include a combinational L1 data cache and a shared memory unit to improve performance while simplifying programming.

[0170] In at least one embodiment, one or more of the GPU(s) (1108) may include a “HBM” (high bandwidth memory) and / or 16 GB HBM2 memory subsystem to provide a peak memory bandwidth of about 900 GB / sec in some examples. In at least one embodiment, in addition to HBM memory or alternatively, “SGRAM” (synchronous graphics random-access memory), such as “GDDR5” (graphics double data rate type five synchronous random-access memory), may be used.

[0171] In at least one embodiment, the GPU(s) (1108) may include integrated memory technology. In at least one embodiment, "ATS" (address translation services) support may be used to allow the GPU(s) (1108) to directly access the page tables of the CPU(s) (1106). In at least one embodiment, when the GPU of the embodiment, GPU(s) (1108) "MMU" (memory management unit) experiences a miss, an address translation request may be sent to the CPU(s) (1106). In response, in at least one embodiment, two of the CPU(s) (1106) may look up their page tables for virtual-to-physical mapping of the address and send the translation back to the GPU(s) (1108). In at least one embodiment, the integrated memory technology may allow a single integrated virtual address space for the memory of both the CPU(s) (1106) and the GPU(s) (1108), thereby simplifying GPU(s) (1108) programming and porting applications to the GPU(s) (1108).

[0172] In at least one embodiment, the GPU(s) (1108) may include any number of access counters capable of keeping track of the frequency of access by the GPU(s) (1108) to the memory of other processors. In at least one embodiment, the access counter(s) may help ensure that memory pages are moved to the physical memory of the processor that accesses the pages most frequently, thereby improving efficiency for memory ranges shared between processors.

[0173] In at least one embodiment, one or more of the SoC(s) (1104) may include any number of cache(s) (1112) including those described herein. For example, in at least one embodiment, the cache(s) (1112) may include an "L3" (level three) cache available to both the CPU(s) (1106) and the GPU(s) (1108) (e.g., connected to the CPU(s) (1106) and the GPU(s) (1108). In at least one embodiment, the cache(s) (1112) may include a write-back cache capable of keeping track of the states of lines, such as by using a cache coherence protocol (e.g., MEI, MESI, MSI, etc.). In at least one embodiment, the L3 cache may include 4 MB or more of memory, depending on the embodiment, but smaller cache sizes may be used.

[0174] In at least one embodiment, one or more of the SoC(s) (1104) may include one or more accelerator(s) (1114) (e.g., hardware accelerators, software accelerators, or a combination thereof). In at least one embodiment, the SoC(s) (1104) may include a hardware acceleration cluster that may include optimized hardware accelerators and / or large on-chip memory. In at least one embodiment, large on-chip memory (e.g., 4 MB of SRAM) may enable the hardware acceleration cluster to accelerate neural networks and other computations. In at least one embodiment, the hardware acceleration cluster may be used to complement the GPU(s) (1108) and to offload some of the tasks of the GPU(s) (1108) (e.g., to free up more cycles of the GPU(s) (1108) to perform other tasks). In at least one embodiment, the accelerator(s) (1114) may be used for targeted workloads that are stable enough to adapt to acceleration (e.g., cognition, "CNNs" (convolutional neural networks), "RNNs" (recurrent neural networks), etc.). In at least one embodiment, the CNN may include region-based or "RCNNs" (regional convolutional neural networks) and Fast RCNNs (e.g., as used for object detection) or other types of CNNs.

[0175] In at least one embodiment, the accelerator(s) (1114) (e.g., a hardware acceleration cluster) may include one or more "DLAs" (deep learning accelerators). In at least one embodiment, the DLA(s) may include, without limitation, one or more "TPUs" (Tensor processing units) which may be configured to provide an additional 10 trillion operations per second for deep learning applications and inference. In at least one embodiment, the TPUs may be accelerators configured to perform image processing functions (e.g., for CNNs, RCNNs, etc.) and optimized for this purpose. In at least one embodiment, the DLA(s) may be further optimized for inference as well as for a specific set of neural network types and floating-point operations. In at least one embodiment, the design of the DLA(s) may provide more performance per millimeter than a conventional general-purpose GPU and typically significantly exceeds the performance of a CPU. In at least one embodiment, the TPU(s) may perform some functions, including single-instance convolution functions that support, for example, INT8, INT16, and FP16 data types for both features and weights, as well as post-processor functions. In at least one embodiment, the DLA(s) may rapidly and efficiently execute neural networks, particularly CNNs, on processed or unprocessed data for any of various functions, for example, and without limitation, including: a CNN for object identification and detection using data from camera sensors; a CNN for distance estimation using data from camera sensors; a CNN for emergency vehicle detection and identification using data from microphones; a CNN for face recognition and vehicle owner identification using data from camera sensors; and / or a CNN for security and / or safety-related events.

[0176] In at least one embodiment, the DLA(s) can perform any function of the GPU(s) (1108), and by using an inference accelerator, for example, the designer can target the DLA(s) or the GPU(s) (1108) for any function. For example, in at least one embodiment, the designer can concentrate the processing of CNNs and floating-point operations on the DLA(s) and leave other functions to the GPU(s) (1108) and / or accelerator(s) (1114).

[0177] In at least one embodiment, the accelerator(s) (1114) may include a "programmable vision accelerator" (PVA), which may alternatively be referred to herein as a computer vision accelerator. In at least one embodiment, the PVA may be designed and configured to accelerate computer vision algorithms for "advanced driver assistance systems" (ADAS) (1138), autonomous driving, "augmented reality" (AR) applications, and / or "virtual reality" (VR) applications. In at least one embodiment, the PVA may provide a balance between performance and flexibility. For example, in at least one embodiment, each PVA may include, for example, and without limitation, any number of "RISC" (reduced instruction set computer) cores, "DMA" (direct memory access), and / or any number of vector processors.

[0178] In at least one embodiment, RISC cores may interact with image sensors (e.g., image sensors of any cameras described herein), image signal processor(s), etc. In at least one embodiment, each RISC core may include any amount of memory. In at least one embodiment, RISC cores may use any of a number of protocols, depending on the embodiment. In at least one embodiment, RISC cores may run a "RTOS" (real-time operating system). In at least one embodiment, RISC cores may be implemented using one or more integrated circuit devices, "ASICs" (application-specific integrated circuits), and / or memory devices. For example, in at least one embodiment, RISC cores may include an instruction cache and / or tight-fitting RAM.

[0179] In at least one embodiment, the DMA may enable the components of the PVA to access system memory independently of the CPU(s) (1106). In at least one embodiment, the DMA may support any number of features used to provide optimization to the PVA, including, but not limited to, supporting multidimensional addressing and / or circular addressing. In at least one embodiment, the DMA may support addressing of up to six or more dimensions, including, without limitation, block width, block height, block depth, horizontal block stepping, vertical block stepping, and / or depth stepping.

[0180] In at least one embodiment, vector processors may be programmable processors designed to efficiently and flexibly execute programming for computer vision algorithms and provide signal processing capabilities. In at least one embodiment, the PVA may include a PVA core and two vector processing subsystem partitions. In at least one embodiment, the PVA core may include a processor subsystem, DMA engine(s) (e.g., two DMA engines), and / or other peripherals. In at least one embodiment, the vector processing subsystem may operate as the main processing engine of the PVA and may include a "VPU (vector processing unit)," an instruction cache, and / or vector memory (e.g., "VMEM"). In at least one embodiment, the VPU core may include a digital signal processor, such as, for example, a "SIMD (single instruction, multiple data)" or "VLIW (very long instruction word)" digital signal processor. In at least one embodiment, a combination of SIMD and VLIW may enhance throughput and speed.

[0181] In at least one embodiment, each of the vector processors may include an instruction cache and may be connected to dedicated memory. As a result, in at least one embodiment, each of the vector processors may be configured to run independently of other vector processors. In at least one embodiment, the vector processors included in a specific PVA may be configured to utilize data parallelization. For example, in at least one embodiment, multiple vector processors included in a single PVA may execute a common computer vision algorithm on different regions of an image. In at least one embodiment, the vector processors included in a specific PVA may execute different computer vision algorithms simultaneously on a single image, or even execute different algorithms on sequential images or parts of an image. In at least one embodiment, among other things, any number of PVAs may be included in a hardware acceleration cluster, and any number of vector processors may be included in each PVA. In at least one embodiment, the PVA may include additional "ECC (error correcting code) memory" to enhance overall system safety.

[0182] In at least one embodiment, the accelerator(s) (1114) may include a computer vision network on-chip and static random-access memory (SRAM) to provide high-bandwidth, low-latency SRAM for the accelerator(s) (1114). In at least one embodiment, the on-chip memory may include at least 4 MB SRAM, for example and without limitation, containing eight field-configurable memory blocks that can be accessed by both the PVA and the DLA. In at least one embodiment, each pair of memory blocks may include an advanced peripheral bus (APB) interface, a configuration circuit, a controller, and a multiplexer. In at least one embodiment, any type of memory may be used. In at least one embodiment, the PVA and the DLA may access the memory through a backbone that provides high-speed access to the memory to the PVA and the DLA. In at least one embodiment, the backbone may include a computer vision network on-chip that interconnects the PVA and the DLA to the memory (for example, using APB).

[0183] In at least one embodiment, the computer vision network on-chip may include an interface that determines whether both the PVA and DLA provide ready and valid signals prior to the transmission of any control signal / address / data. In at least one embodiment, the interface may provide separate phases and separate channels for transmitting control signals / addresses / data, as well as burst-type communications for continuous data transmission. In at least one embodiment, the interface may comply with ISO (International Organization for Standardization) 26262 or IEC (International Electrotechnical Commission) 61508 standards, but other standards and protocols may be used.

[0184] In at least one embodiment, one or more of the SoC(s) (1104) may include a real-time ray-tracing hardware accelerator. In at least one embodiment, the real-time ray-tracing hardware accelerator may be used to rapidly and efficiently determine the locations and scales of objects (e.g., within a world model), to generate real-time visualization simulations, to interpret RADAR signals, to synthesize and / or analyze sound propagation, to simulate SONAR systems, to simulate general wave propagation, to compare with LIDAR data for the purposes of localization and / or other functions, and / or for other uses.

[0185] In at least one embodiment, the accelerator(s) (1114) may have a wide range of applications for autonomous driving. In at least one embodiment, the PVA may be used for major processing stages in ADAS and autonomous vehicles. In at least one embodiment, the capabilities of the PVA are a good match for algorithmic domains requiring predictable processing at low power and low latency. In other words, the PVA performs well for semi-dense or dense normal computations, even for small data sets, which may require predictable runtimes at low latency and low power. In at least one embodiment, as in the vehicle (1100), the PVAs may be designed to execute classical computer vision algorithms, as they may be efficient in object detection and operate in integer mathematics.

[0186] For example, according to at least one embodiment of the technology, the PVA is used to perform computer stereo vision. In at least one embodiment, a semi-global matching-based algorithm may be used in some examples, but this is not intended to be limiting. In at least one embodiment, applications for Level 3 to 5 autonomous driving use motion estimation / stereo matching (e.g., structure from motion, pedestrian recognition, lane detection, etc.) on-the-fly. In at least one embodiment, the PVA can perform computer stereo vision functions on inputs from two monocular cameras.

[0187] In at least one embodiment, the PVA may be used to perform dense optical flow. For example, in at least one embodiment, the PVA may process raw RADAR data (e.g., using 4D Fast Fourier Transform) to provide processed RADAR data. In at least one embodiment, the PVA is used for time-of-flight depth processing, for example, by processing raw time-of-flight data to provide processed time-of-flight data.

[0188] In at least one embodiment, the DLA may be used to execute any type of network to enhance control and driving safety, including, for example and without limitation, a neural network that outputs a measure of confidence for each object detection. In at least one embodiment, confidence may be expressed or interpreted as a probability or as providing a relative "weight" for each detection compared to other detections. In at least one embodiment, the confidence measure enables the system to make further decisions regarding which detections should be considered as true positive detections rather than false positive detections. In at least one embodiment, the system may set a threshold for confidence and consider only detections that exceed the threshold as true positive detections. In an embodiment where an "AEB (automatic emergency braking) system" is used, false positive detections would cause the vehicle to automatically perform emergency braking, which is obviously undesirable. In at least one embodiment, highly reliable detections may be considered as triggers for AEB. In at least one embodiment, the DLA may execute a neural network to regress the confidence value. In at least one embodiment, the neural network may take, as its input, at least a subset of parameters such as, among others, bounding box dimensions, ground plane estimates obtained (e.g., from another subsystem), vehicle (1100) orientation, distance, and output from an IMU sensor(s) (1166) corresponding to 3D position estimates of an object obtained from the neural network and / or other sensors (e.g., LIDAR sensor(s) (1164) or RADAR sensor(s) (1160)).

[0189] In at least one embodiment, one or more of the SoC(s) (1104) may include data storage(s) (1116) (e.g., memory). In at least one embodiment, the data storage(s) (1116) may be on-chip memory of the SoC(s) (1104) capable of storing neural networks to be executed on the GPU(s) (1108) and / or DLA. In at least one embodiment, the data storage(s) (1116) may have a capacity large enough to store multiple instances of neural networks for redundancy and safety. In at least one embodiment, the data storage(s) (1116) may include L2 or L3 cache(s).

[0190] In at least one embodiment, one or more of the SoC(s) (1104) may include any number of processor(s) (1110) (e.g., embedded processors). In at least one embodiment, the processor(s) (1110) may include a boot and power management processor, which may be a dedicated processor and subsystem for handling boot power and management functions and related security enforcement. In at least one embodiment, the boot and power management processor may be part of the boot sequence of the SoC(s) (1104) and may provide runtime power management services. In at least one embodiment, the boot power and management processor may provide clock and voltage programming, assistance in system low-power state transitions, management of SoC(s) (1104) thermals and temperature sensors, and / or management of SoC(s) (1104) power states. In at least one embodiment, each temperature sensor may be implemented as a ring oscillator whose output frequency is proportional to the temperature, and the SoC(s) (1104) may use the ring oscillators to detect the temperatures of the CPU(s) (1106), GPU(s) (1108), and / or accelerator(s) (1114). In at least one embodiment, when it is determined that the temperatures exceed a threshold, the boot and power management processor may then enter a temperature failure routine to put the SoC(s) (1104) into a lower power state and / or put the vehicle (1100) into a safe stop mode (e.g., bring the vehicle (1100) to a safe stop).

[0191] In at least one embodiment, the processor(s) (1110) may further include a set of embedded processors capable of serving as an audio processing engine that may be an audio subsystem enabling full hardware support for multi-channel audio through a plurality of interfaces and a wide and flexible range of audio I / O interfaces. In at least one embodiment, the audio processing engine is a dedicated processor core having a digital signal processor with dedicated RAM.

[0192] In at least one embodiment, the processor(s) (1110) may further include an always-on processor engine capable of providing hardware features necessary to support low-power sensor management and to enable use cases. In at least one embodiment, the always-on processor engine may include, without limitation, a processor core, tight-fitting RAM, supporting peripherals (e.g., timers and interrupt controllers), various I / O controller peripherals, and routing logic.

[0193] In at least one embodiment, the processor(s) (1110) may further include a safety cluster engine, without limitation, comprising a dedicated processor subsystem for handling safety management for automotive applications. In at least one embodiment, the safety cluster engine may, without limitation, include two or more processor cores, tightly coupled RAM, supporting peripherals (e.g., timers, interrupt controllers, etc.), and / or routing logic. In safety mode, in at least one embodiment, the two or more cores may operate in lockstep mode and function as a single core having comparison logic for detecting any differences between their operations. In at least one embodiment, the processor(s) (1110) may further include a real-time camera engine, without limitation, comprising a dedicated processor subsystem for handling real-time camera management. In at least one embodiment, the processor(s) (1110) may further include a high-dynamic-range signal processor, without limitation, comprising an image signal processor which is a hardware engine that is part of the camera processing pipeline.

[0194] In at least one embodiment, the processor(s) (1110) may include a video image synthesizer which may be a processing block (e.g., implemented on a microprocessor) that implements video post-processing functions required by a video playback application to produce a final image for a player window. In at least one embodiment, the video image synthesizer may perform lens distortion correction for wide-view camera(s) (1170), surround camera(s) (1174), and / or cabin-in-cabin monitoring camera sensor(s). In at least one embodiment, the cabin-in-cabin monitoring camera sensor(s) are preferably monitored by a neural network running on another instance of the SoC (1104) configured to identify cabin events and respond accordingly. In at least one embodiment, the cabin-in-cabin system may perform lip reading, without limitation, to enable cellular services and make phone calls, dictate emails, change the vehicle's destination, enable or change the vehicle's infotainment system and settings, or provide voice-enabled web surfing. In at least one embodiment, certain functions are available to the driver when the vehicle is operating in autonomous mode and otherwise disabled.

[0195] In at least one embodiment, the video image synthesizer may include enhanced temporal noise reduction for both spatial and temporal noise reduction. For example, in at least one embodiment where motion occurs in the video, the noise reduction appropriately weights spatial information to reduce the weights of information provided by adjacent frames. In at least one embodiment where the image or part of the image does not contain motion, the temporal noise reduction performed by the video image synthesizer may use information from a previous image to reduce noise in the current image.

[0196] In at least one embodiment, the video image synthesizer may also be configured to perform stereo rectification on input stereo lens frames. In at least one embodiment, the video image synthesizer may be additionally used for user interface synthesis when the operating system desktop is in use, and the GPU(s) (1108) are not required to continuously render new surfaces. In at least one embodiment, when the GPU(s) (1108) are powered on and 3D rendering is active, the video image synthesizer may be used to offload the GPU(s) (1108) to improve performance and responsiveness.

[0197] In at least one embodiment, one or more of the SoC(s) (1104) may further include a video input block that can be used for a MIPI (mobile industry processor interface) camera serial interface, a high-speed interface, and / or camera and associated pixel input functions for receiving video and input from a camera. In at least one embodiment, one or more of the SoC(s) (1104) may further include an input / output controller(s) that can be controlled by software and can be used to receive I / O signals that are not committed to a specific role.

[0198] In at least one embodiment, one or more of the SoC(s) (1104) may additionally include a wide range of peripheral interfaces that enable communication with peripherals, audio encoders / decoders ("codecs"), power management, and / or other devices. In at least one embodiment, the SoC(s) (1104) may be used to process data from cameras (e.g., connected via Gigabit Multimedia Serial Link and Ethernet channels), sensors (e.g., LIDAR sensor(s) (1164), RADAR sensor(s) (1160), etc., which may be connected via Ethernet channels), data from a bus (1102) (e.g., speed of the vehicle (1100), steering wheel position, etc.), data from GNSS sensor(s) (1158) (e.g., connected via an Ethernet bus or CAN bus), etc. In at least one embodiment, one or more of the SoC(s) (1104) may additionally include dedicated high-performance mass storage controllers that may include their own DMA engines and may be used to free the CPU(s) (1106) from routine data management tasks.

[0199] In at least one embodiment, the SoC(s) (1104) may be an end-to-end platform with a flexible architecture spanning automation levels 3-5, thereby providing a comprehensive functional safety architecture that provides a platform for a flexible, reliable driving software stack, along with deep learning tools, which utilizes and efficiently employs computer vision and ADAS technologies for diversity and redundancy. In at least one embodiment, the SoC(s) (1104) may be faster, more reliable, and much more energy-efficient and space-efficient than conventional systems. For example, in at least one embodiment, the accelerator(s) (1114), when combined with CPU(s) (1106), GPU(s) (1108), and data storage(s) (1116), may provide a fast and efficient platform for Level 3-5 autonomous vehicles.

[0200] In at least one embodiment, computer vision algorithms may be executed on CPUs that can be configured to execute a wide variety of processing algorithms across a wide variety of visual data using a high-level programming language such as C. However, in at least one embodiment, CPUs often cannot meet the performance requirements of many computer vision applications, such as those related to execution time and power consumption, for example. In at least one embodiment, many CPUs cannot execute complex object detection algorithms in real time, which are used in in-vehicle ADAS applications and actual Level 3 to 5 autonomous vehicles.

[0201] The embodiments described herein allow multiple neural networks to be executed simultaneously and / or sequentially, and allow the results to be combined together to enable Level 3 to 5 autonomous driving capabilities. For example, in at least one embodiment, a CNN running on a DLA or individual GPU (e.g., GPU(s) (1120)) may include text and word recognition, thereby enabling the reading and understanding of traffic signs, including signs that the neural network has not specifically trained on. In at least one embodiment, the DLA may further include a neural network capable of identifying, interpreting, and providing a semantic understanding of the sign, and capable of transmitting that semantic understanding to path planning modules running on a CPU complex.

[0202] In at least one embodiment, for Level 3, 4, or 5 driving, multiple neural networks may be executed simultaneously. For example, in at least one embodiment, a warning sign mentioning "Caution: flashing lights indicate icy conditions" along with electric lights may be interpreted independently or collectively by some neural networks. In at least one embodiment, this warning sign itself may be identified as a traffic sign by a first deployed neural network (e.g., a trained neural network), and the text "flashing lights indicate icy conditions" may be interpreted by a second deployed neural network, which notifies the vehicle's path planning software (preferably executed on a CPU complex) that icy conditions exist when flashing lights are detected. In at least one embodiment, flashing lights may be identified by operating a third deployed neural network over multiple frames, which notifies the vehicle's path planning software of the presence (or absence) of flashing lights. In at least one embodiment, all three neural networks can be executed simultaneously, such as within the DLA and / or on the GPU(s) (1108).

[0203] In at least one embodiment, a CNN for face recognition and vehicle owner identification can use data from camera sensors to identify the presence of an authorized driver and / or owner of the vehicle (1100). In at least one embodiment, an always-on sensor processing engine can be used to unlock the vehicle when the owner approaches the driver's door and turns on the lights, and, in security mode, to disable the vehicle when the owner leaves the vehicle. In this way, the SoC(s) (1104) provide security against theft and / or vehicle hijacking.

[0204] In at least one embodiment, the CNN for emergency vehicle detection and identification may use data from microphones (1196) to detect and identify emergency vehicle sirens. In at least one embodiment, the SoC(s) (1104) use the CNN to classify visual data as well as environmental and urban sounds. In at least one embodiment, the CNN running on the DLA is trained to identify the relative closing speed of emergency vehicles (e.g., by using Doppler effects). In at least one embodiment, the CNN may also be trained to identify specific emergency vehicles in the local area where the vehicle is operating, as identified by the GNSS sensor(s) (1158). In at least one embodiment, when operating in Europe, the CNN will attempt to detect European sirens, and when operating in North America, the CNN will attempt to identify only North American sirens. In at least one embodiment, once an emergency vehicle is detected, a control program may be used to execute an emergency vehicle safety routine to slow down, stop, park, and / or idle the vehicle with the assistance of ultrasonic sensor(s) (1162) until the emergency vehicles pass.

[0205] In at least one embodiment, the vehicle (1100) may include CPU(s) (1118) (e.g., individual CPU(s), or dCPU(s)) that can be connected to the SoC(s) (1104) via a high-speed interconnect (e.g., PCIe). In at least one embodiment, the CPU(s) (1118) may include, for example, an X86 processor. The CPU(s) (1118) may be used to perform any of various functions, for example, mediating potentially inconsistent results between ADAS sensors and the SoC(s) (1104), and / or monitoring the status and health of the controller(s) (1136) and / or the infotainment "infotainment SoC" (infotainment system on a chip) (1130). In at least one embodiment, the SoC(s) (1104) includes one or more interconnects, and the interconnects may include PCIe (peripheral component interconnect express).

[0206] In at least one embodiment, the vehicle (1100) may include GPU(s) (1120) (e.g., discrete GPU(s) or dGPU(s)) that can be connected to the SoC(s) (1104) via a high-speed interconnect (e.g., NVIDIA's NVLINK channel). In at least one embodiment, the GPU(s) (1120) may provide additional artificial intelligence capabilities, such as by running redundant and / or different neural networks, and may be used to train and / or update neural networks based, at least partially, on inputs (e.g., sensor data) from the sensors of the vehicle (1100).

[0207] In at least one embodiment, the vehicle (1100) may further include a network interface (1124) which may include, without limitation, wireless antenna(s) (1126) (one or more wireless antennas for different communication protocols, such as cellular antennas, Bluetooth antennas, etc.). In at least one embodiment, the network interface (1124) may be used to enable wireless connectivity with other vehicles and / or computing devices (e.g., passenger client devices) to Internet cloud services (e.g., by server(s) and / or other network devices). In at least one embodiment, to communicate with other vehicles, a direct link may be established between the vehicle (1100) and other vehicles and / or an indirect link may be established (e.g., across networks and via the Internet). In at least one embodiment, direct links may be provided using vehicle-to-vehicle communication links. In at least one embodiment, a vehicle-to-vehicle communication link may provide vehicle (1100) information regarding vehicles in close proximity to the vehicle (1100) (e.g., vehicles in front, to the side, and / or behind the vehicle (1100)). In at least one embodiment, this aforementioned functionality may be part of the cooperative adaptive cruise control functionality of the vehicle (1100).

[0208] In at least one embodiment, the network interface (1124) may include an SoC that provides modulation and demodulation functionality and enables controller(s) (1136) to communicate through wireless networks. In at least one embodiment, the network interface (1124) may include a radio frequency front-end for up-conversion from baseband to radio frequency and down-conversion from radio frequency to baseband. In at least one embodiment, frequency conversions may be performed in any technically feasible manner. For example, frequency conversions may be performed through well-known processes and / or using super-heterodyne processes. In at least one embodiment, the radio frequency front-end functionality may be provided by a separate chip. In at least one embodiment, the network interfaces may include radio functionality for communicating via LTE, WCDMA, UMTS, GSM, CDMA2000, Bluetooth, Bluetooth LE, Wi-Fi, Z-Wave, ZigBee, LoRaWAN, and / or other wireless protocols.

[0209] In at least one embodiment, the vehicle (1100) may further include data storage(s) (1128) which may include, without limitation, off-chip (e.g., off SoC(s) (1104)) storage. In at least one embodiment, the data storage(s) (1128) may include, without limitation, one or more storage elements including RAM, SRAM, DRAM (dynamic random-access memory), VRAM (video random-access memory), flash memory, hard disks, and / or other components and / or devices capable of storing at least one bit of data.

[0210] In at least one embodiment, the vehicle (1100) may further include GNSS sensor(s) (1158) (e.g., GPS and / or auxiliary GPS sensors) that assist in mapping, perception, occupancy grid generation, and / or path planning functions. In at least one embodiment, any number of GNSS sensor(s) (1158) may be used, for example and without limitation, including a GPS using a USB connector with an Ethernet-serial (e.g., RS-232) bridge.

[0211] In at least one embodiment, the vehicle (1100) may additionally include RADAR sensor(s) (1160). In at least one embodiment, the RADAR sensor(s) (1160) may be used by the vehicle (1100) for long-range vehicle detection, even in darkness and / or severe weather conditions. In at least one embodiment, the RADAR functional safety levels may be ASIL B. In at least one embodiment, the RADAR sensor(s) (1160) may use a CAN bus and / or bus (1102) for control and to access object tracking data (e.g., to transmit data generated by the RADAR sensor(s) (1160), while accessing Ethernet channels to access raw data in some examples. In at least one embodiment, a wide variety of RADAR sensor types may be used. For example, and without limitation, the RADAR sensor(s) (1160) may be suitable for front, rear, and side RADAR applications. In at least one embodiment, one or more of the RADAR sensor(s) (1160) are pulse Doppler RADAR sensors.

[0212] In at least one embodiment, the RADAR sensor(s) (1160) may include different configurations, such as long range with a narrow field of view, short range with a wide field of view, short range side coverage, etc. In at least one embodiment, long range RADAR may be used for adaptive cruise control functionality. In at least one embodiment, long range RADAR systems may provide a wide field of view realized by two or more independent scans, such as within a range of 250 m (meter). In at least one embodiment, the RADAR sensor(s) (1160) may help distinguish between static and moving objects and may be used by an ADAS system (1138) for emergency braking assistance and forward collision warning. In at least one embodiment, the sensors (1160(s)) included in the long range RADAR system may, without limitation, include a number (e.g., six or more) of fixed RADAR antennas and a monostatic multimode RADAR with high-speed CAN and FlexRay interfaces. In at least one embodiment, by means of six antennas, the central four antennas can generate a focused beam pattern designed to record the surroundings of the vehicle (1100) at higher speeds with minimal interference from traffic in adjacent lanes. In at least one embodiment, the other two antennas can extend the field of view, enabling rapid detection of vehicles entering or leaving the lane of the vehicle (1100).

[0213] In at least one embodiment, medium-range RADAR systems may include a range of up to 160 m (forward) or 80 m (rear), and a field of view of up to 42 degrees (forward) or 150 degrees (rear), for example. In at least one embodiment, short-range RADAR systems may include, without limitation, any number of RADAR sensor(s) (1160) designed to be installed at both ends of the rear bumper. When installed at both ends of the rear bumper, in at least one embodiment, the RADAR sensor system may generate two beams that continuously monitor blind spots in the rearward direction and at the side of the vehicle. In at least one embodiment, short-range RADAR systems may be used in an ADAS system (1138) for blind spot detection and / or lane change assistance.

[0214] In at least one embodiment, the vehicle (1100) may further include ultrasonic sensor(s) (1162). In at least one embodiment, the ultrasonic sensor(s) (1162), which may be located at the front, rear, and / or side positions of the vehicle (1100), may be used for parking assistance and / or to generate and update an occupancy grid. In at least one embodiment, a wide variety of ultrasonic sensor(s) (1162) may be used, and different ultrasonic sensor(s) (1162) may be used for different detection ranges (e.g., 2.5 m, 4 m). In at least one embodiment, the ultrasonic sensor(s) (1162) may operate at ASIL B functional safety levels.

[0215] In at least one embodiment, the vehicle (1100) may include LIDAR sensor(s) (1164). In at least one embodiment, the LIDAR sensor(s) (1164) may be used for object and pedestrian detection, emergency braking, collision avoidance, and / or other functions. In at least one embodiment, the LIDAR sensor(s) (1164) may operate at a functional safety level ASIL B. In at least one embodiment, the vehicle (1100) may include a plurality of LIDAR sensors (1164) (e.g., two, four, six, etc.) capable of using an Ethernet channel (e.g., to provide data to a Gigabit Ethernet switch).

[0216] In at least one embodiment, the LIDAR sensor(s) (1164) may provide a list of objects and their distances for a 360-degree field of view. In at least one embodiment, a commercially available LIDAR sensor(s) (1164) may have an advertised range of approximately 100 m, for example, with an accuracy of 2 cm to 3 cm and support for 100 Mbps Ethernet connectivity. In at least one embodiment, one or more non-protruding LIDAR sensors may be used. In this embodiment, the LIDAR sensor(s) (1164) may include a small device that can be embedded in the front, rear, side, and / or corner locations of the vehicle (1100). In at least one embodiment, the LIDAR sensor(s) (1164) may provide a field of view of up to 120 degrees horizontal and 35 degrees vertical, with a range of 200 m, even for low-reflectivity objects. In at least one embodiment, the front-mounted LIDAR sensor(s) (1164) may be configured for a horizontal field of view between 45 and 135 degrees.

[0217] In at least one embodiment, LIDAR technologies, such as 3D flash LIDAR, may also be used. In at least one embodiment, 3D flash LIDAR uses a laser flash as a transmission source to illuminate the surroundings of the vehicle (1100) up to approximately 200 m. In at least one embodiment, the flash LIDAR unit includes, without limitation, a receiver that records the laser pulse passage time and reflected light on each pixel, which eventually corresponds to the range from the vehicle (1100) to the objects. In at least one embodiment, the flash LIDAR may allow very accurate and distortion-free images of the surroundings to be generated with all laser flashes. In at least one embodiment, four flash LIDAR sensors, one on each side of the vehicle (1100), may be placed. In at least one embodiment, 3D flash LIDAR systems include, without limitation, a solid-state 3D gaze array LIDAR camera (e.g., a non-scanning LIDAR device) that has no moving parts other than a fan. In at least one embodiment, the flash LIDAR device may use a 5 nanosecond class I (eye-safe) laser pulse per frame and may capture the reflected laser light as a 3D range point cloud and co-registered intensity data.

[0218] In at least one embodiment, the vehicle (1100) may additionally include IMU sensor(s) (1166). In at least one embodiment, the IMU sensor(s) (1166) may be located at the center of the rear axle of the vehicle (1100). In at least one embodiment, the IMU sensor(s) (1166) may include accelerometer(s), magnetometer(s), gyroscope(s), magnetic compass, magnetic compass, and / or other sensor types, for example, and without limitation. In at least one embodiment, such as in 6-axis applications, the IMU sensor(s) (1166) may include accelerometers and gyroscopes, without limitation. In at least one embodiment, such as in 9-axis applications, the IMU sensor(s) (1166) may include accelerometers, gyroscopes, and magnetometers, without limitation.

[0219] In at least one embodiment, the IMU sensor(s) (1166) may be implemented as a miniature, high-performance GPS / INS (GPS-Aided Inertial Navigation System) that combines MEMS (micro-electro-mechanical systems) inertial sensors, a high-sensitivity GPS receiver, and an advanced Kalman filtering algorithm to provide estimates of position, speed, and attitude. In at least one embodiment, the IMU sensor(s) (1166) may enable the vehicle (1100) to estimate its direction of travel without requiring input from a magnetic sensor by directly observing and correlating changes in speed from the GPS to the IMU sensor(s) (1166). In at least one embodiment, the IMU sensor(s) (1166) and the GNSS sensor(s) (1158) may be combined into a single integrated unit.

[0220] In at least one embodiment, the vehicle (1100) may include microphone(s) (1196) placed within and / or around the vehicle (1100). In at least one embodiment, the microphone(s) (1196) may be used, among other things, for emergency vehicle detection and identification.

[0221] In at least one embodiment, the vehicle (1100) may additionally include any number of camera types, including stereo camera(s) (1168), wide-view camera(s) (1170), infrared camera(s) (1172), surround camera(s) (1174), long-range camera(s) (1198), medium-range camera(s) (1176), and / or other camera types. In at least one embodiment, cameras may be used to capture image data around the entire periphery of the vehicle (1100). In at least one embodiment, the types of cameras used depend on the vehicle (1100). In at least one embodiment, any combination of camera types may be used to provide the necessary coverage around the vehicle (1100). In at least one embodiment, the number of cameras deployed may vary depending on the embodiment. For example, in at least one embodiment, the vehicle (1100) may include six cameras, seven cameras, ten cameras, twelve cameras, or any other number of cameras. In at least one embodiment, the cameras may support GMSL (Gigabit Multimedia Serial Link) and / or Gigabit Ethernet communication, for example and without limitation. In at least one embodiment, each camera may be as previously described in more detail in connection with FIGS. 11a and FIGS. 11b.

[0222] In at least one embodiment, the vehicle (1100) may further include vibration sensor(s) (1142). In at least one embodiment, the vibration sensor(s) (1142) may measure vibrations of components of the vehicle (1100), such as axle(s). For example, in at least one embodiment, changes in vibrations may indicate changes in road surfaces. In at least one embodiment, when two or more vibration sensors (1142) are used, differences between vibrations may be used to determine friction or slippage of the road surface (e.g., when there is a difference in vibration between a power-driven axle and a freely rotating axle).

[0223] In at least one embodiment, the vehicle (1100) may include an ADAS system (1138). In at least one embodiment, the ADAS system (1138) may include, in some examples, an SoC, without limitation. In at least one embodiment, the ADAS system (1138) may include any number and combination of "ACC (autonomous / adaptive / automatic cruise control) system, "CACC (cooperative adaptive cruise control) system, "FCW (forward crash warning) system, "AEB (automatic emergency braking) system, "LDW (lane departure warning) system, "LKA (lane keep assist) system, "BSW (blind spot warning) system, "RCTW (rear cross-traffic warning) system, "CW (collision warning) system, "LC (lane centering) system, and / or other systems, features, and / or functionalities, without limitation.

[0224] In at least one embodiment, the ACC system may use RADAR sensor(s) (1160), LIDAR sensor(s) (1164), and / or any number of camera(s). In at least one embodiment, the ACC system may include a longitudinal ACC system and / or a lateral ACC system. In at least one embodiment, the longitudinal ACC system monitors and controls the distance to another vehicle immediately ahead of the vehicle (1100) and automatically adjusts the speed of the vehicle (1100) to maintain a safe distance from the vehicles ahead. In at least one embodiment, the lateral ACC system performs distance maintenance and advises the vehicle (1100) to change lanes when necessary. In at least one embodiment, the lateral ACC is related to other ADAS applications, such as LC and CW.

[0225] In at least one embodiment, the CACC system uses information from other vehicles that can be received from other vehicles via a wireless antenna(s) (1126) and / or a network interface (1124), either indirectly or via a wireless link, through a network connection (e.g., via the Internet). In at least one embodiment, direct links may be provided by a "V2V" (vehicle-to-vehicle) communication link, while indirect links may be provided by an "I2V" (infrastructure-to-vehicle) communication link. Generally, V2V communication provides information regarding vehicles immediately preceding (e.g., vehicles immediately ahead of vehicle (1100) and in the same lane), whereas I2V communication provides information regarding traffic further ahead. In at least one embodiment, the CACC system may include either or both of I2V and V2V information sources. In at least one embodiment, if information about the vehicles ahead of the vehicle (1100) is provided, the CACC system may be more reliable and has the potential to improve traffic flow smoothness and reduce congestion on the road.

[0226] In at least one embodiment, the FCW system is designed to alert the driver to danger so that the driver can take corrective action. In at least one embodiment, the FCW system uses a front camera and / or RADAR sensor(s) (1160) connected to a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically connected to provide driver feedback, such as a display, speaker, and / or vibration component. In at least one embodiment, the FCW system may provide a warning, such as in the form of sound, visual warning, vibration, and / or a rapid brake pulse.

[0227] In at least one embodiment, the AEB system detects an imminent forward collision with another vehicle or other object and may automatically apply the brakes if the driver does not take corrective action within specified time or distance parameters. In at least one embodiment, the AEB system may use front camera(s) and / or RADAR sensor(s) (1160) connected to a dedicated processor, DSP, FPGA, and / or ASIC. In at least one embodiment, when the AEB system detects a hazard, it will typically first alert the driver to take corrective action to avoid the collision, and if the driver does not take corrective action, the AEB system may automatically apply the brakes in an effort to prevent or at least mitigate the effects of the predicted collision. In at least one embodiment, the AEB system may include technologies such as dynamic brake assist and / or impending accident braking.

[0228] In at least one embodiment, the LDW system provides visual, auditory, and / or tactile warnings, such as steering wheel or seat vibrations, to alert the driver when the vehicle (1100) crosses lane markings. In at least one embodiment, the LDW system is not activated when the driver indicates an intentional lane departure, such as by activating a turn signal. In at least one embodiment, the LDW system may use front-facing cameras connected to a dedicated processor, DSP, FPGA, and / or ASIC, which are electrically connected to provide driver feedback, such as a display, speaker, and / or vibration component. In at least one embodiment, the LKA system is a variation of the LDW system. In at least one embodiment, the LKA system provides steering input or braking to correct the vehicle (1100) when the vehicle (1100) begins to deviate from its lane.

[0229] In at least one embodiment, the BSW system detects and warns drivers of vehicles in the blind spot of the vehicle. In at least one embodiment, the BSW system may provide visual, auditory, and / or tactile warnings to indicate that it is unsafe to merge into or change lanes. In at least one embodiment, the BSW system may provide additional warnings when the driver uses turn signals. In at least one embodiment, the BSW system may use rear-facing camera(s) and / or RADAR sensor(s) (1160) connected to a dedicated processor, DSP, FPGA, and / or ASIC, which is electrically connected to driver feedback, such as a display, speaker, and / or vibration component.

[0230] In at least one embodiment, the RCTW system may provide visual, auditory, and / or tactile notification when an object is detected outside the rear-camera range while the vehicle (1100) is reversing. In at least one embodiment, the RCTW system includes an AEB system that ensures the vehicle brakes are applied to avoid an accident. In at least one embodiment, the RCTW system may use one or more rear RADAR sensor(s) (1160) connected to a dedicated processor, DSP, FPGA, and / or ASIC, which are electrically connected to provide driver feedback, such as a display, speaker, and / or vibration component.

[0231] In at least one embodiment, conventional ADAS systems may be susceptible to false positive results that can be annoying and distracting to the driver, but are not typically fatal because conventional ADAS systems alert the driver and allow the driver to determine whether a safety condition truly exists and to act accordingly. In at least one embodiment, the vehicle (1100) itself determines whether to pay attention to results from the main computer or the subcomputer (e.g., the first controller or the second controller among the controllers (1136)) in the event of conflicting results. For example, in at least one embodiment, the ADAS system (1138) may be a backup and / or subcomputer for providing cognitive information to a backup computer rationality module. In at least one embodiment, the backup computer rationality monitor may run various redundant software on hardware components to detect failures in cognitive and dynamic driving tasks. In at least one embodiment, outputs from the ADAS system (1138) may be provided to a supervisory MCU. In at least one embodiment, if outputs from the main computer and outputs from the subcomputer conflict, the supervisory MCU determines a method to coordinate the conflict to ensure safe operation.

[0232] In at least one embodiment, the main computer may be configured to provide a reliability score to the supervisory MCU, indicating the reliability of the main computer in the selected result. In at least one embodiment, if the reliability score exceeds a threshold, the supervisory MCU may follow the direction of the main computer regardless of whether the sub-computer provides conflicting or inconsistent results. In at least one embodiment, if the reliability score does not meet the threshold and the main and sub-computers display different results (e.g., conflicting), the supervisory MCU may mediate between the computers to determine an appropriate result.

[0233] In at least one embodiment, the supervisory MCU may be configured to run a neural network(s) trained and configured to determine conditions under which the subcomputer provides false alarms, based at least partially on outputs from the main computer and outputs from the subcomputer. In at least one embodiment, the neural network(s) in the supervisory MCU may learn when the outputs of the subcomputer can be trusted and when they cannot be trusted. For example, in at least one embodiment, when the subcomputer is a RADAR-based FCW system, the neural network(s) in the supervisory MCU may learn when the FCW system identifies metal objects that are actually harmless, such as drainage plates or manhole covers, which trigger an alarm. In at least one embodiment, when the subcomputer is a camera-based LDW system, the neural network in the supervisory MCU may learn to override the LDW when cyclists or pedestrians are present and lane departure is actually the safest operation. In at least one embodiment, the supervising MCU may include at least one of a DLA or a GPU suitable for running neural network(s) with associated memory. In at least one embodiment, the supervising MCU may include a component of an SoC(s) (1104) and / or may be included as such a component.

[0234] In at least one embodiment, the ADAS system (1138) may include a subcomputer that performs ADAS functionality using traditional computer vision rules. In at least one embodiment, the subcomputer may use classical computer vision rules (if-then), and the presence of neural network(s) in the supervisory MCU may improve reliability, safety, and performance. For example, in at least one embodiment, various implementations and intentional non-identity make the entire system more fault-tolerant, particularly against failures caused by software (or software-hardware interface) functionality. For example, in at least one embodiment, if there is a software bug or error in the software running on the main computer and non-identical software code running on the subcomputer provides a consistent overall result, then the supervisory MCU may have greater confidence that the overall result is accurate and that the bug in the software or hardware on the main computer does not cause data errors.

[0235] In at least one embodiment, the output of the ADAS system (1138) may be supplied to the perception block of the main computer and / or the dynamic driving task block of the main computer. For example, in at least one embodiment, if the ADAS system (1138) indicates a forward accident warning due to an object immediately ahead, the perception block may use this information when identifying objects. In at least one embodiment, the subcomputer may have its own neural network that is trained as described herein and thus reduces the risk of false positives.

[0236] In at least one embodiment, the vehicle (1100) may additionally include an infotainment SoC (1130) (e.g., an in-vehicle infotainment system (IVI)). Although illustrated and described as an SoC, the infotainment system SoC (1130) may not be an SoC in at least one embodiment and may include two or more individual components, without limitation. In at least one embodiment, the infotainment SoC (1130) may include, without limitation, a combination of hardware and software that can be used to provide audio (e.g., music, personal information terminal, navigation commands, news, radio, etc.), video (e.g., TV, movies, streaming, etc.), telephone (e.g., hands-free calling), network connectivity (e.g., LTE, WiFi, etc.), and / or information services (e.g., navigation systems, rear-parking assist, wireless data systems, vehicle-related information such as fuel level, total distance covered, brake fuel level, oil level, door open / closed, air filter information, etc.) to the vehicle (1100). For example, the infotainment SoC (1130) may include radios, disc players, navigation systems, video players, USB and Bluetooth connectivity, vehicle computers, in-car entertainment, WiFi, steering wheel audio controls, hands-free voice control, HUD (heads-up display), HMI display (1134), telematics devices, control panels (e.g., for controlling and / or interacting with various components, features, and / or systems), and / or other components.In at least one embodiment, the infotainment SoC (1130) may be further used to provide information (e.g., visual and / or auditory) to the user(s) of the vehicle (1100), such as information from the ADAS system (1138), autonomous driving information such as planned vehicle operations, trajectories, surrounding environment information (e.g., intersection information, vehicle information, road information, etc.), and / or other information.

[0237] In at least one embodiment, the infotainment SoC (1130) may include any amount and type of GPU functionality. In at least one embodiment, the infotainment SoC (1130) may communicate with other devices, systems, and / or components of the vehicle (1100) via the bus (1102). In at least one embodiment, the infotainment SoC (1130) may be connected to a supervisory MCU so that the GPU of the infotainment system may perform some self-driving functionality in the event that the main controller(s) (1136) (e.g., the main computer and / or backup computers of the vehicle (1100)) fails. In at least one embodiment, the infotainment SoC (1130) may put the vehicle (1100) into a safe stop mode of operation as described herein.

[0238] In at least one embodiment, the vehicle (1100) may further include a device cluster (1132) (e.g., a digital dash, an electronic device cluster, a digital device panel, etc.). In at least one embodiment, the device cluster (1132) may include, without limitation, a controller and / or a supercomputer (e.g., an individual controller or a supercomputer). In at least one embodiment, the device cluster (1132) may include, without limitation, a set of any number and combination of devices such as a speedometer, fuel level, oil pressure, tachometer, odometer, direction indicators, gear shift position indicator, seat belt warning light(s), parking brake warning light(s), engine malfunction light(s), supplemental restraint system (e.g., airbag) information, lighting controls, safety system controls, navigation information, etc. In some examples, information may be displayed and / or shared between the infotainment SoC (1130) and the device cluster (1132). In at least one embodiment, the device cluster (1132) may be included as part of the infotainment SoC (1130), or vice versa.

[0239] At least one embodiment of FIGS. 1 through 11c performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0240] FIG. 11d is a diagram of a system for communication between the cloud-based server(s) of FIG. 11a and an autonomous vehicle (1100) according to at least one embodiment. In at least one embodiment, the system may include any number and type of vehicles, including server(s) (1178), network(s) (1190), and vehicle (1100), without limitation. In at least one embodiment, the server(s) (1178) may include a plurality of GPUs (1184(A) to 1184(H)) (collectively referred to herein as GPUs (1184)), PCIe switches (1182(A) to 1182(D)) (collectively referred to herein as PCIe switches (1182)), and / or CPUs (1180(A) to 1180(B)) (collectively referred to herein as CPUs (1180)), without limitation. In at least one embodiment, GPUs (1184), CPUs (1180), and PCIe switches (1182) may be interconnected, for example and without limitation, to high-speed interconnects such as NVLink interfaces (1188) developed by NVIDIA and / or PCIe connections (1186). In at least one embodiment, GPUs (1184) are connected via an NVLink and / or NVSwitch SoC, and GPUs (1184) and PCIe switches (1182) are connected via PCIe interconnects. Eight GPUs (1184), two CPUs (1180), and four PCIe switches (1182) are illustrated, but this is not intended to be limiting. In at least one embodiment, each server(s) (1178) may include any number of GPUs (1184), CPUs (1180), and / or PCIe switches (1182) in any combination without limitation. For example, in at least one embodiment, each server(s) (1178) may include 8, 16, 32, and / or more GPUs (1184).

[0241] In at least one embodiment, the server(s) (1178) may receive image data representing images showing unexpected or changed road conditions, such as recently undertaken road work, through the network(s) (1190) and from vehicles. In at least one embodiment, the server(s) (1178) may transmit map information (1194) to the network(s) (1190) and to vehicles, including, without limitation, updated or other types of neural networks (1192) and / or information regarding traffic and road conditions. In at least one embodiment, updates to the map information (1194) may, without limitation, include updates to the HD map (1122), such as information regarding construction sites, potholes, detours, flooding, and / or other obstacles. In at least one embodiment, the neural networks (1192) and / or map information (1194) may be derived from new training and / or experiences represented in data received from any number of vehicles in the environment, and / or at least partially based on training performed in a data center (e.g., using server(s) (1178) and / or other servers).

[0242] In at least one embodiment, server(s) (1178) may be used to train a machine learning model (e.g., a neural network) based at least partially on training data. In at least one embodiment, training data may be generated by vehicles and / or generated in a simulation (e.g., using a game engine). In at least one embodiment, any amount of training data is tagged and / or undergoes other preprocessing (e.g., if the associated neural network has benefited from supervised learning). In at least one embodiment, any amount of training data is not tagged and / or preprocessed (e.g., if the associated neural network has not required supervised learning). In at least one embodiment, once the machine learning models are trained, the machine learning models may be used by vehicles (e.g., transmitted to vehicles via network(s) (1190)) and / or the machine learning models may be used by server(s) (1178) to remotely monitor the vehicles.

[0243] In at least one embodiment, the server(s) (1178) may receive data from vehicles and apply the data to modern real-time neural networks for real-time intelligent inference. In at least one embodiment, the server(s) (1178) may include deep-learning supercomputers and / or dedicated AI computers powered by GPU(s) (1184), such as DGX and DGX Station machines developed by NVIDIA. However, in at least one embodiment, the server(s) (1178) may include a deep-learning infrastructure using CPU-powered data centers.

[0244] In at least one embodiment, the deep-learning infrastructure of the server(s) (1178) may be capable of high-speed, real-time inference and may use its capabilities to evaluate and verify the health of processors, software, and / or associated hardware in the vehicle (1100). For example, in at least one embodiment, the deep-learning infrastructure may receive periodic updates from the vehicle (1100), such as a series of images and / or objects that the vehicle (1100) has placed in the corresponding series of images (e.g., through computer vision and / or other machine learning object classification techniques). In at least one embodiment, the deep-learning infrastructure may run its own neural network to identify objects and compare them with objects identified by the vehicle (1100), and if the results do not match and the deep-learning infrastructure concludes that the AI ​​in the vehicle (1100) is malfunctioning, then the server(s) (1178) may send a signal to the vehicle (1100) commanding the vehicle's safety-assist computer to take control, notify the passengers, and complete a safe parking operation.

[0245] In at least one embodiment, the server(s) (1178) may include GPU(s) (1184) and one or more programmable inference accelerators (e.g., NVIDIA's TensorRT 3 devices). In at least one embodiment, the combination of GPU-powered servers and inference acceleration may enable real-time responsiveness. In at least one embodiment, such as when performance is less critical, servers powered by CPUs, FPGAs, and other processors may be used for inference. In at least one embodiment, hardware structure(s) (815) are used to perform one or more embodiments. Details regarding the hardware structure(s) (815) are provided herein together with FIG. 8a and / or FIG. 8b.

[0246] computer systems

[0247] FIG. 12 is a block diagram illustrating an exemplary computer system that may be a system having interconnected devices and components, a system-on-a-chip (SOC), or a combination thereof, formed together with a processor that may include execution units for executing instructions according to at least one embodiment. In at least one embodiment, the computer system (1200) may, without limitation, include a component such as a processor (1202) that utilizes execution units that include logic for executing algorithms for processing data according to the present disclosure, as in the embodiments described herein. In at least one embodiment, the computer system (1200) may include a PENTIUM® processor family, Xeon available from Intel Corporation, Santa Clara, California. TM , Itanium®, XScale TM and / or StrongARM TM , Intel® Core TM , or Intel® Nervana TM Processors such as microprocessors may be included, but other systems (including PCs, engineering workstations, set-top boxes, etc. with other microprocessors) may also be used. In at least one embodiment, the computer system (1200) may run a version of the WINDOWS operating system available from Microsoft Corporation of Redmond, Wash., but other operating systems (e.g., UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0248] The embodiments may be used in other devices such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol devices, digital cameras, PDAs (personal digital assistants), and handheld PCs. In at least one embodiment, embedded applications may include a microcontroller, a DSP (digital signal processor), a system-on-chip, network computers ("NetPCs"), set-top boxes, network hubs, WAN (wide area network) switches, or any other system capable of executing one or more commands according to at least one embodiment.

[0249] In at least one embodiment, the computer system (1200) may, without limitation, include a processor (1202) which may, without limitation, include one or more execution units (1208) that perform machine learning model training and / or inference according to the techniques described herein. In at least one embodiment, the computer system (1200) is a single-processor desktop or server system, but in another embodiment, the computer system (1200) may be a multiprocessor system. In at least one embodiment, the processor (1202) may, without limitation, include any other processor device, such as, for example, a "complex instruction set computer" (CISC) microprocessor, a "reduced instruction set computing" (RISC) microprocessor, a "very long instruction word" (VLIW) microprocessor, a processor implementing a combination of instruction sets, or a digital signal processor. In at least one embodiment, the processor (1202) may be connected to a processor bus (1210) capable of transmitting data signals between the processor (1202) and other components in the computer system (1200).

[0250] In at least one embodiment, the processor (1202) may include, without limitation, an L1 (Level 1) internal cache memory ("cache") (1204). In at least one embodiment, the processor (1202) may have a single internal cache or multiple levels of internal caches. In at least one embodiment, the cache memory may reside outside the processor (1202). Other embodiments may also include a combination of both internal and external caches depending on specific implementation and needs. In at least one embodiment, the register file (1206) may store different types of data in various registers, without limitation, including integer registers, floating-point registers, status registers, and instruction pointer registers.

[0251] In at least one embodiment, an execution unit (1208) including, without limitation, logic for performing integer and floating-point operations also resides in the processor (1202). In at least one embodiment, the processor (1202) may also include a “ucode” (microcode) “ROM” (read-only memory) for storing microcode for specific macro instructions. In at least one embodiment, the execution unit (1208) may include logic for handling a packed instruction set (1209). In at least one embodiment, by including a packed instruction set (1209) in the instruction set of a general-purpose processor along with associated circuitry for executing instructions, operations used by many multimedia applications can be performed in the processor (1202) using the packed data. In at least one embodiment, many multimedia applications can be accelerated and executed more efficiently by using the full width of the processor's data bus to perform operations on packed data, which can eliminate the need to transmit smaller units of data across the processor's data bus to perform one or more operations on one data element at a time.

[0252] In at least one embodiment, the execution unit (1208) may also be used in microcontrollers, embedded processors, graphics devices, DSPs, and other types of logic circuits. In at least one embodiment, the computer system (1200) may include memory (1220), without limitation. In at least one embodiment, the memory (1220) may be a "DRAM" (Dynamic Random Access Memory) device, a "SRAM" (Static Random Access Memory) device, a flash memory device, or other memory device. In at least one embodiment, the memory (1220) may store instruction(s) (1219) and / or data (1221) represented by data signals that can be executed by the processor (1202).

[0253] In at least one embodiment, the system logic chip may be connected to the processor bus (1210) and the memory (1220). In at least one embodiment, the system logic chip may include, without limitation, a "MCH" (memory controller hub) (1216), and the processor (1202) may communicate with the MCH (1216) via the processor bus (1210). In at least one embodiment, the MCH (1216) may provide a high-bandwidth memory path (1218) to the memory (1220) for instruction and data storage and for the storage of graphic commands, data, and textures. In at least one embodiment, the MCH (1216) may direct data signals between the processor (1202), the memory (1220), and other components within the computer system (1200), and bridge data signals between the processor bus (1210), the memory (1220), and the system I / O interface (1222). In at least one embodiment, the system logic chip may provide a graphics port for connecting to a graphics controller. In at least one embodiment, the MCH (1216) may be connected to memory (1220) via a high-bandwidth memory path (1218), and a graphics / video card (1212) may be connected to the MCH (1216) via an "AGP" (Accelerated Graphics Port) interconnect (1214).

[0254] In at least one embodiment, the computer system (1200) may use a system I / O interface (1222) as a proprietary hub interface bus to connect the MCH (1216) to an ICH (I / O controller hub) (1230). In at least one embodiment, the ICH (1230) may provide direct connections to some I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus may, without limitation, include a high-speed I / O bus for connecting peripherals to memory (1220), a chipset, and a processor (1202). Examples may include, without limitation, an audio controller (1229), a firmware hub ("Flash BIOS") (1228), a wireless transceiver (1226), a legacy I / O controller (1223) including data storage (1224), user input and keyboard interfaces (1225), a serial expansion port (1227), such as a "USB" (Universal Serial Bus) port, and a network controller (1234). In at least one embodiment, the data storage (1224) may include a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage device.

[0255] In at least one embodiment, FIG. 12 illustrates a system comprising interconnected hardware devices or "chips," whereas in other embodiments, FIG. 12 may illustrate an exemplary SoC. In at least one embodiment, the devices illustrated in FIG. 12 may be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of the computer system (1200) are interconnected using CXL (compute express link) interconnects.

[0256] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a computer system (1200) to infer or predict operations based, at least partially, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0257] At least one embodiment of FIGS. 1 through 12 performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0258] FIG. 13 is a block diagram illustrating an electronic device (1300) for using a processor (1310) according to at least one embodiment. In at least one embodiment, the electronic device (1300) may be, for example, and without limitation, a laptop, a tower server, a rack server, a blade server, a laptop, a desktop, a tablet, a mobile device, a telephone, an embedded computer, or any other suitable electronic device.

[0259] In at least one embodiment, the electronic device (1300) may, without limitation, include a processor (1310) that is communically connected to any suitable number or type of components, peripherals, modules, or devices. In at least one embodiment, the processor (1310) is I 2 Connected using a bus or interface such as a C bus, "SMBus" (System Management Bus), LPC (Low Pin Count) bus, "SPI" (Serial Peripheral Interface), "HDA" (High Definition Audio) bus, "SATA" (Serial Advance Technology Attachment) bus, "USB" (Universal Serial Bus) (versions 1, 2, 3, etc.), or "UART" (Universal Asynchronous Receiver / Transmitter) bus. In at least one embodiment, FIG. 13 illustrates a system comprising interconnected hardware devices or "chips," whereas in other embodiments, FIG. 13 may illustrate an exemplary SoC. In at least one embodiment, the devices illustrated in FIG. 13 may be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of FIG. 13 are interconnected using CXL (compute express link) interconnects.

[0260] In at least one embodiment, FIG. 13 includes a display (1324), a touch screen (1325), a touch pad (1330), an NFC (Near Field Communications) unit (1345), a sensor hub (1340), a thermal sensor (1346), an EC (Express Chipset) (1335), a TPM (Trusted Platform Module) (1338), a BIOS, FW Flash (BIOS / firmware / flash) memory (1322), a DSP (1360), a drive (1320) such as an SSD (Solid State Disk) or HDD (Hard Disk Drive), a WLAN (wireless local area network) unit (1350), a Bluetooth unit (1352), a WWAN (Wireless Wide Area Network) unit (1356), a GPS (Global Positioning System) unit (1355), a camera ("USB 3.0 camera") (1354) such as a USB 3.0 camera, and / or, for example, It may include an "LPDDR3" (LPDDR (Low Power Double Data Rate)) memory unit (1315) implemented according to the LPDDR3 standard. Each of these components may be implemented in any suitable manner.

[0261] In at least one embodiment, other components may be communicably connected to the processor (1310) through the components described herein. In at least one embodiment, an accelerometer (1341), an "ALS" (ambient light sensor) (1342), a compass (1343), and a gyroscope (1344) may be communicably connected to a sensor hub (1340). In at least one embodiment, a heat sensor (1339), a fan (1337), a keyboard (1336), and a touchpad (1330) may be communicably connected to an EC (1335). In at least one embodiment, speakers (1363), headphones (1364), and a "mic" (microphone) (1365) may be communicably connected to an audio unit ("audio codec and class D amplifier") (1362), which may eventually be communicably connected to a DSP (1360). In at least one embodiment, the audio unit (1362) may include an audio coder / decoder (“codec”) and a Class D amplifier, for example, and without limitation. In at least one embodiment, a SIM card (“SIM”) (1357) may be communicably connected to the WWAN unit (1356). In at least one embodiment, components such as the WWAN unit (1356), as well as the WLAN unit (1350) and the Bluetooth unit (1352), may be implemented as “NGFF” (Next Generation Form Factor).

[0262] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in an electronic device (1300) to infer or predict operations based, at least partially, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0263] At least one embodiment of FIGS. 1 through 13 performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0264] FIG. 14 illustrates a computer system (1400) according to at least one embodiment. In at least one embodiment, the computer system (1400) is configured to implement various processes and methods described throughout the present disclosure.

[0265] In at least one embodiment, the computer system (1400) includes, without limitation, at least one "CPU" (central processing unit) (1402) connected to a communication bus (1410) implemented using any suitable protocol such as PCI ("Peripheral Component Interconnect"), "PCI-Express" (peripheral component interconnect express), AGP ("Accelerated Graphics Port"), HyperTransport, or any other bus or point-to-point communication protocol(s). In at least one embodiment, the computer system (1400) includes, without limitation, main memory (1404) and control logic (e.g., implemented as hardware, software, or a combination thereof), and data is stored in the main memory (1404), which may take the form of "RAM" (random access memory). In at least one embodiment, a network interface subsystem ("network interface") (1422) provides an interface to other computing devices and networks for receiving data from other systems to the computer system (1400) and transmitting data to other systems.

[0266] In at least one embodiment, the computer system (1400) comprises, without limitation, input devices (1408), a parallel processing system (1412), and display devices (1406) that can be implemented using a conventional "CRT (cathode ray tube), "LCD (liquid crystal display)," "LED (light emitting diode) display, plasma display, or other suitable display technology. In at least one embodiment, user input is received from input devices (1408), such as a keyboard, mouse, touchpad, microphone, etc. In at least one embodiment, each module described herein may be placed on a single semiconductor platform to form a processing system.

[0267] The logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the inference and / or training logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a computer system (1400) to infer or predict operations based, at least partially, on weight parameters calculated using the neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0268] At least one embodiment of FIGS. 1 through 14 performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0269] FIG. 15 illustrates a computer system (1500) according to at least one embodiment. In at least one embodiment, the computer system (1500) includes, without limitation, a computer (1510) and a USB stick (1520). In at least one embodiment, the computer (1510) may include, without limitation, any number and type of processor(s) (not shown) and memory (not shown). In at least one embodiment, the computer (1510) includes, without limitation, a server, a cloud instance, a laptop, and a desktop computer.

[0270] In at least one embodiment, the USB stick (1520) comprises, without limitation, a processing unit (1530), a USB interface (1540), and USB interface logic (1550). In at least one embodiment, the processing unit (1530) may be any instruction execution system, device, or device capable of executing instructions. In at least one embodiment, the processing unit (1530) may comprise, without limitation, any number and type of processing cores (not shown). In at least one embodiment, the processing unit (1530) comprises an "ASIC" (application specific integrated circuit) optimized to perform any amount and type of operations associated with machine learning. For example, in at least one embodiment, the processing unit (1530) is a "TPC" (tensor processing unit) optimized to perform machine learning inference operations. In at least one embodiment, the processing unit (1530) is a "VPU" (vision processing unit) optimized to perform machine vision and machine learning inference operations.

[0271] In at least one embodiment, the USB interface (1540) may be any type of USB connector or USB socket. For example, in at least one embodiment, the USB interface (1540) is a USB 3.0 Type-C socket for data and power. In at least one embodiment, the USB interface (1540) is a USB 3.0 Type-A connector. In at least one embodiment, the USB interface logic (1550) may include any amount and type of logic that enables the processing unit (1530) to interface with devices (e.g., a computer (1510)) through the USB connector (1540).

[0272] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a computer system (1500) to infer or predict operations based, at least partially, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0273] At least one embodiment of FIGS. 1 through 15 performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0274] FIG. 16a illustrates an exemplary architecture in which a plurality of GPUs (1610(1) to 1610(N)) are communicably connected to a plurality of multi-core processors (1605(1) to 1605(M)) via high-speed links (1640(1) to 1640(N)) (e.g., buses, point-to-point interconnects, etc.). In at least one embodiment, the high-speed links (1640(1) to 1640(N)) support communication throughputs of 4 GB / s, 30 GB / s, 80 GB / s or more. In at least one embodiment, various interconnect protocols including, but not limited to, PCIe 4.0 or 5.0 and NVLink 2.0 may be used. In various drawings, "N" and "M" represent positive integers, and their values ​​may vary from drawing to drawing. In at least one embodiment, one or more GPUs in a plurality of GPUs (1610(1) to 1610(N)) comprise one or more graphics cores (also simply referred to as "cores") (1900) as disclosed in FIG. 19a and FIG. 19b. In at least one embodiment, one or more graphics cores (1900) may be referred to as "SMs" (streaming multiprocessors), "SPs" (stream processors), "SPUs" (stream processing units), "CUs" (compute units), "EUs" (execution units), and / or slices, wherein a slice in this context refers to a portion of processing resources in a processing unit (e.g., 16 cores, ray tracing unit, thread director, or scheduler).

[0275] Additionally, and in at least one embodiment, two or more of the GPUs (1610) are interconnected via high-speed links (1629(1) to 1629(2)), which may be implemented using protocols / links similar to or different from those used for the high-speed links (1640(1) to 1640(N)). Similarly, two or more of the multi-core processors (1605) may be connected via high-speed links (1628), which may be symmetric multi-processor (SMP) buses operating at 20 GB / s, 30 GB / s, 120 GB / s, or higher. Alternatively, all communication between the various system components illustrated in FIG. 16a may be achieved using similar protocols / links (e.g., via a common interconnect fabric).

[0276] In at least one embodiment, each multi-core processor (1605) is respectively communicably connected to processor memory (1601(1) to 1601(M)) through memory interconnects (1626(1) to 1626(M)), and each GPU (1610(1) to 1610(N)) is respectively communicably connected to GPU memory (1620(1) to 1620(N)) through GPU memory interconnects (1650(1) to 1650(N)). In at least one embodiment, the memory interconnects (1626 and 1650) may utilize similar or different memory access technologies. Without limitation, for example, processor memories (1601(1) to 1601(M)) and GPU memories (1620) may be volatile memories such as DRAMs (dynamic random access memories) (including stacked DRAMs), GDDR (Graphics DDR SDRAM) (e.g., GDDR5, GDDR6), or HBM (High Bandwidth Memory) and / or non-volatile memories such as 3D XPoint or Nano-Ram. In at least one embodiment, some parts of the processor memories (1601) may be volatile memories, and other parts may be non-volatile memories (e.g., using a 2LM (two-level memory) layer).

[0277] As described in this specification, various multi-core processors (1605) and GPUs (1610) may each be physically connected to specific memories (1601, 1620), and / or, an integrated memory architecture may be implemented in which a virtual system address space (also referred to as an "effective address" space) is distributed among various physical memories. For example, processor memories (1601(1) to 1601(M)) may each contain a system memory address space of 64 GB, and GPU memories (1620(1) to 1620(N)) may each contain a system memory address space of 32 GB, resulting in a total addressable memory of 256 GB when M=2 and N=4. Other values ​​for N and M are possible.

[0278] FIG. 16b illustrates additional details regarding the interconnect between a multi-core processor (1607) and a graphics acceleration module (1646) according to one exemplary embodiment. In at least one embodiment, the graphics acceleration module (1646) may include one or more GPU chips integrated on a line card connected to the processor (1607) via a high-speed link (1640) (e.g., PCIe bus, NVLink, etc.). In at least one embodiment, the graphics acceleration module (1646) may alternatively be integrated on a package or chip with the processor (1607).

[0279] In at least one embodiment, the processor (1607) comprises a plurality of cores (1660A to 1660D) (which may be referred to as "execution units"), each having a "TLB (translation lookaside buffer)" (1661A to 1661D) and one or more caches (1662A to 1662D). In at least one embodiment, the cores (1660A to 1660D) may include various other components for executing instructions not illustrated and processing data. In at least one embodiment, the caches (1662A to 1662D) may include L1 (Level 1) and L2 (Level 2) caches. Additionally, one or more shared caches (1656) may be included in the caches (1662A to 1662D) and shared by sets of cores (1660A to 1660D). For example, one embodiment of the processor (1607) includes 24 cores, each having its own L1 cache, 12 shared L2 caches, and 12 shared L3 caches. In this embodiment, one or more L2 and L3 caches are shared by two adjacent cores. In at least one embodiment, the processor (1607) and the graphics acceleration module (1646) are connected to a system memory (1614) which may include the processor memories (1601(1) to 1601(M)) of FIG. 16a.

[0280] In at least one embodiment, coherence is maintained for data and instructions stored in various caches (1662A to 1662D, 1656) and system memory (1614) through inter-core communication via the coherence bus (1664). In at least one embodiment, for example, each cache may have associated cache coherence logic / circuit to communicate via the coherence bus (1664) in response to detected reads or writes for specific cache lines. In at least one embodiment, a cache snooping protocol is implemented via the coherence bus (1664) to snoop cache accesses.

[0281] In at least one embodiment, the proxy circuit (1625) communicates with the graphics acceleration module (1646) to the coherence bus (1664), allowing the graphics acceleration module (1646) to participate in the cache coherence protocol as a peer of the cores (1660A to 1660D). In particular, in at least one embodiment, the interface (1635) provides connectivity to the proxy circuit (1625) via the high-speed link (1640), and the interface (1637) connects the graphics acceleration module (1646) to the high-speed link (1640).

[0282] In at least one embodiment, the accelerator integration circuit (1636) provides cache management, memory access, context management, and interrupt management services on behalf of the plurality of graphics processing engines (1631(1) to 1631(N)) of the graphics acceleration module (1646). In at least one embodiment, the graphics processing engines (1631(1) to 1631(N)) may each include a separate graphics processing unit (GPU). In at least one embodiment, the plurality of graphics processing engines (1631(1) to 1631(N)) of the graphics acceleration module (1646) include one or more graphics cores (1900) as discussed in relation to FIG. 19a and FIG. 19b. In at least one embodiment, the graphics processing engines (1631(1) to 1631(N)) may alternatively include different types of graphics processing engines within the GPU, such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and bullet engines. In at least one embodiment, the graphics acceleration module (1646) may be a GPU having a plurality of graphics processing engines (1631(1) to 1631(N)), or the graphics processing engines (1631(1) to 1631(N)) may be discrete GPUs integrated on a common package, line card, or chip.

[0283] In at least one embodiment, the accelerator integrated circuit (1636) includes a memory management unit (MMU) (1639) for performing various memory management functions, such as virtual-to-physical memory translations (also referred to as effective-to-actual memory translations) and memory access protocols for accessing system memory (1614). In at least one embodiment, the MMU (1639) may also include a translation lookaside buffer (TLB) (not shown) for caching virtual / effective-to-physical / actual address translations. In at least one embodiment, the cache (1638) may store commands and data for efficient access by graphics processing engines (1631(1) to 1631(N)). In at least one embodiment, data stored in the cache (1638) and graphics memories (1633(1) to 1633(M)) is maintained coherently with the core caches (1662A to 1662D, 1656) and system memory (1614), possibly using a fetch unit (1644). As mentioned, this can be achieved through a proxy circuit (1625) on behalf of the cache (1638) and memories (1633(1) to 1633(M)) (e.g., sending updates related to modifications / accesses of cache lines on the processor caches (1662A to 1662D, 1656) to the cache (1638) and receiving updates from the cache (1638)).

[0284] In at least one embodiment, a set of registers (1645) stores context data for threads executed by graphics processing engines (1631(1) to 1631(N)), and a context management circuit (1648) manages thread contexts. For example, the context management circuit (1648) may perform save and restore operations to save and restore the contexts of various threads during context switches (e.g., a first thread is saved and a second thread is saved so that the second thread can be executed by the graphics processing engine). For example, during a context switch, the context management circuit (1648) may save the current register values ​​to a designated area in memory (e.g., identified by a context pointer). Next, the register values ​​may be restored when returning to the context. In at least one embodiment, an interrupt management circuit (1647) receives and processes interrupts received from a system device.

[0285] In at least one embodiment, virtual / effective addresses from the graphics processing engine (1631) are converted by the MMU (1639) into actual / physical addresses in system memory (1614). In at least one embodiment, the accelerator integration circuit (1636) supports a number of (e.g., 4, 8, 16) graphics accelerator modules (1646) and / or other accelerator devices. In at least one embodiment, a graphics accelerator module (1646) may be dedicated to a single application running on the processor (1607) or may be shared among multiple applications. In at least one embodiment, a virtualized graphics execution environment is presented in which the resources of the graphics processing engines (1631(1) to 1631(N)) are shared with multiple applications or virtual machines (VMs). In at least one embodiment, the resources may be subdivided into "slices" that are allocated to different VMs and / or applications based on processing requirements and priorities associated with the VMs and / or applications.

[0286] In at least one embodiment, the accelerator integration circuit (1636) acts as a bridge to the system for the graphics acceleration module (1646) and provides address translation and system memory cache services. Additionally, in at least one embodiment, the accelerator integration circuit (1636) may provide virtualization facilities for the host processor to manage the virtualization, interrupts, and memory management of the graphics processing engines (1631(1) to 1631(N)).

[0287] In at least one embodiment, since the hardware resources of the graphics processing engines (1631(1) to 1631(N)) are explicitly mapped to the actual address space shown by the host processor (1607), any host processor can directly address these resources using an effective address value. In at least one embodiment, one function of the accelerator integration circuit (1636) is the physical separation of the graphics processing engines (1631(1) to 1631(N)) so that they appear in the system as independent units.

[0288] In at least one embodiment, one or more graphics memories (1633(1) to 1633(M)) are each connected to each of the graphics processing engines (1631(1) to 1631(N)), and N=M. In at least one embodiment, the graphics memories (1633(1) to 1633(M)) store instructions and data processed by each of the graphics processing engines (1631(1) to 1631(N)). In at least one embodiment, the graphics memories (1633(1) to 1633(M)) may be volatile memories such as DRAMs (including stacked DRAMs), GDDR memory (e.g., GDDR5, GDDR6), or HBM, and / or non-volatile memories such as 3D XPoint or Nano-Ram.

[0289] In at least one embodiment, to reduce data traffic through the high-speed link (1640), biasing techniques may be used to ensure that the data stored in the graphics memories (1633(1) to 1633(M)) is data that is most frequently used by the graphics processing engines (1631(1) to 1631(N)) and preferably not used (at least not frequently) by the cores (1660A to 1660D). Similarly, in at least one embodiment, a biasing mechanism attempts to retain data required by the cores (and preferably not by the graphics processing engines (1631(1) to 1631(N))) in the caches (1662A to 1662D, 1656) and system memory (1614).

[0290] FIG. 16c illustrates another exemplary embodiment in which an accelerator integration circuit (1636) is integrated within a processor (1607). In this embodiment, graphics processing engines (1631(1) to 1631(N)) communicate directly with the accelerator integration circuit (1636) via an interface (1637) and an interface (1635) (again, which may be any form of bus or interface protocol) through a high-speed link (1640). In at least one embodiment, the accelerator integration circuit (1636) can perform operations similar to those described in relation to FIG. 16b, but can potentially perform them at a higher throughput if given close proximity to the coherence bus (1664) and caches (1662A to 1662D, 1656). In at least one embodiment, the accelerator integration circuit supports different programming models, including a dedicated-process programming model (without graphics acceleration module virtualization) and shared programming models (with virtualization), which may include programming models controlled by the accelerator integration circuit (1636) and programming models controlled by the graphics acceleration module (1646).

[0291] In at least one embodiment, the graphics processing engines (1631(1) to 1631(N)) are dedicated to a single application or process under a single operating system. In at least one embodiment, a single application can funnel other application requests to the graphics processing engines (1631(1) to 1631(N)) to provide virtualization within a VM / partition.

[0292] In at least one embodiment, the graphics processing engines (1631(1) to 1631(N)) may be shared by a plurality of VM / application partitions. In at least one embodiment, the sharing models may use a system hypervisor to virtualize the graphics processing engines (1631(1) to 1631(N)) to allow access by each operating system. In at least one embodiment, for a single-partition system without a hypervisor, the graphics processing engines (1631(1) to 1631(N)) are owned by the operating system. In at least one embodiment, the operating system may virtualize the graphics processing engines (1631(1) to 1631(N)) to provide access to each process or application.

[0293] In at least one embodiment, a graphics acceleration module (1646) or an individual graphics processing engine (1631(1) to 1631(N)) selects a process element using a process handle. In at least one embodiment, process elements are stored in system memory (1614) and are addressable using the effective address-to-actual address translation technique described herein. In at least one embodiment, the process handle may be an implementation-specific value provided to the host process when registering its context with the graphics processing engine (1631(1) to 1631(N)) (i.e., calling system software to add a process element to a process element linked list). In at least one embodiment, the lower 16 bits of the process handle may be an offset of a process element within the process element linked list.

[0294] FIG. 16d illustrates an exemplary accelerator integration slice (1690). In at least one embodiment, the “slice” comprises a specified portion of the processing resources of the accelerator integration circuit (1636). In at least one embodiment, the application stores process elements (1683) in an effective address space (1682) within system memory (1614). In at least one embodiment, the process elements (1683) are stored in response to GPU calls (1681) from applications (1680) running on the processor (1607). In at least one embodiment, the process element (1683) includes a process state for the corresponding application (1680). In at least one embodiment, the work descriptor (WD) (1684) included in the process element (1683) may be a single task requested by the application or may include a pointer to a queue of tasks. In at least one embodiment, WD (1684) is a pointer to a job request queue in the application's effective address space (1682).

[0295] In at least one embodiment, the graphics acceleration module (1646) and / or individual graphics processing engines (1631(1) to 1631(N)) may be shared by all processes in the system or a subset thereof. In at least one embodiment, infrastructure may be included for setting up process states to start work in a virtualized environment and transmitting WD (1684) to the graphics acceleration module (1646).

[0296] In at least one embodiment, the dedicated-process programming model is implementation-specific. In at least one embodiment, in this model, a single process owns a graphics acceleration module (1646) or a separate graphics processing engine (1631). In at least one embodiment, when the graphics acceleration module (1646) is owned by a single process, the hypervisor initializes the accelerator integration circuit (1636) for the owning partition, and the operating system initializes the accelerator integration circuit (1636) for the owning process when the graphics acceleration module (1646) is assigned.

[0297] In at least one embodiment, during operation, the WD fetch unit (1691) in the accelerator integration slice (1690) fetches the next WD (1684), which includes an indication of the operation to be performed by one or more graphics processing engines of the graphics acceleration module (1646). In at least one embodiment, data from the WD (1684) may be stored in registers (1645) and may be used by the MMU (1639), interrupt management circuit (1647), and / or context management circuit (1648) as illustrated. For example, one embodiment of the MMU (1639) includes a segment / page work circuit for accessing segment / page tables (1686) within the OS virtual address space (1685). In at least one embodiment, the interrupt management circuit (1647) may process interrupt events (1692) received from the graphics acceleration module (1646). In at least one embodiment, when performing graphic operations, the effective address (1693) generated by the graphic processing engine (1631(1) to 1631(N)) is converted into an actual address by the MMU (1639).

[0298] In at least one embodiment, the registers (1645) are duplicated for each graphics processing engine (1631(1) to 1631(N)) and / or graphics acceleration module (1646) and may be initialized by a hypervisor or operating system. In at least one embodiment, each of these duplicated registers may be included in an accelerator integration slice (1690). Exemplary registers that may be initialized by a hypervisor are shown in Table 1.

[0299]

[0300] Exemplary registers that can be initialized by the operating system are shown in Table 2.

[0301]

[0302] In at least one embodiment, each WD (1684) is specific to a particular graphics acceleration module (1646) and / or graphics processing engines (1631(1) to 1631(N)). In at least one embodiment, this may contain all information required by the graphics processing engine (1631(1) to 1631(N)) to perform a task, or this may be a pointer to a memory location that has set up a command queue for the task to be completed by the application. FIG. 16e illustrates additional details for one exemplary embodiment of a shared model. This embodiment includes a hypervisor physical address space (1698) in which a process element list (1699) is stored. In at least one embodiment, the hypervisor physical address space (1698) is accessible through a hypervisor (1696) that virtualizes graphics acceleration module engines for an operating system (1695).

[0303] In at least one embodiment, the shared programming model allows all partitions in the system or all processes from a subset thereof or a subset thereof to use the graphics acceleration module (1646). In at least one embodiment, there are two programming models in which the graphics acceleration module (1646) is shared by a number of processes and partitions, namely time-sliced ​​shared and graphics-oriented shared.

[0304] In at least one embodiment, in this model, the system hypervisor (1696) owns the graphics acceleration module (1646) and makes its functions available to all operating systems (1695). In at least one embodiment, for the graphics acceleration module (1646) to support virtualization by the system hypervisor (1696), the graphics acceleration module (1646) may adhere to certain requirements, such as (1) the application's task request must be autonomous (i.e., no state needs to be maintained between tasks), or the graphics acceleration module (1646) must provide a context save and restore mechanism; (2) the application's task request must be guaranteed by the graphics acceleration module (1646) to be completed within a specified amount of time including any transformation failures, or the graphics acceleration module (1646) must provide the ability to preempt the processing of tasks; and (3) the graphics acceleration module (1646) must be guaranteed fairness between processes when operating in a directional shared programming model.

[0305] In at least one embodiment, the application (1680) is required to make an operating system (1695) system call with a graphics acceleration module type, a work descriptor (WD), an authority mask register (AMR) value, and a context save / restore area pointer (CSRP). In at least one embodiment, the graphics acceleration module type describes a targeted acceleration function for the system call. In at least one embodiment, the graphics acceleration module type may be a system-specific value. In at least one embodiment, the WD may be in the form of a graphics acceleration module (1646) command, a valid address pointer for a user-defined structure, a valid address pointer for a queue of commands, or any other data structure describing the work to be performed by the graphics acceleration module (1646).

[0306] In at least one embodiment, the AMR value is the AMR state to be used for the current process. In at least one embodiment, the value passed to the operating system is similar to the application that sets the AMR. In at least one embodiment, if the accelerator integration circuit (1636) (not shown) and the graphics acceleration module (1646) implementations do not support the User Authority Mask Override Register (UAMOR), the operating system may apply the current UAMOR value to the AMR value before passing the AMR in the hypervisor call. In at least one embodiment, the hypervisor (1696) may optionally apply the current Authority Mask Override Register (AMOR) value before placing the AMR in the process element (1683). In at least one embodiment, the CSRP is one of the registers (1645) containing the valid address of a region in the application's valid address space (1682) for the graphics acceleration module (1646) to save and restore the context state. In at least one embodiment, this pointer is optional when no state needs to be saved between tasks or when a task is preempted. In at least one embodiment, the context save / restore area may be a fixed system memory.

[0307] When receiving a system call, the operating system (1695) can verify that the application (1680) is registered and has been granted permission to use the graphics acceleration module (1646). In at least one embodiment, the operating system (1695) then calls the hypervisor (1696) with the information shown in Table 3.

[0308]

[0309] In at least one embodiment, upon receiving a hypervisor call, the hypervisor (1696) verifies that the operating system (1695) is registered and authorized to use the graphics acceleration module (1646). In at least one embodiment, the hypervisor (1696) then adds the process element (1683) to the process element linked list for the corresponding graphics acceleration module (1646) type. In at least one embodiment, the process element may include information shown in Table 4.

[0310]

[0311] In at least one embodiment, the hypervisor initializes a plurality of accelerator integration slice (1690) registers (1645). As illustrated in FIG. 16f, in at least one embodiment, an integration memory addressable through a common virtual memory address space used to access physical processor memories (1601(1) to 1601(N)) and GPU memories (1620(1) to 1620(N)) is used. In this implementation, operations executed on GPUs (1610(1) to 1610(N)) use the same virtual / effective memory address space to access processor memories (1601(1) to 1601(M)), and vice versa, thereby simplifying programmability. In at least one embodiment, a first portion of the virtual / effective address space is allocated to processor memory (1601(1)), a second portion is allocated to a second processor memory (1601(N)), a third portion is allocated to GPU memory (1620(1)), and so on. In at least one embodiment, the entire virtual / effective memory space (sometimes referred to as the effective address space) is distributed across each of the processor memories (1601) and GPU memories (1620) so as to allow any processor or GPU to access any physical memory having a virtual address mapped to that memory.

[0312] In at least one embodiment, bias / coherence management circuits (1694A to 1694E) within one or more of the MMUs (1639A to 1639E) ensure cache coherence between the caches of one or more host processors (e.g., 1605) and GPUs (1610) and implement biasing techniques that indicate physical memories where specific types of data are to be stored. In at least one embodiment, multiple instances of the bias / coherence management circuits (1694A to 1694E) are illustrated in FIG. 16f, but the bias / coherence circuit may be implemented within the MMU of one or more host processors (1605) and / or within the accelerator integration circuit (1636).

[0313] One embodiment allows GPU memories (1620) to be mapped as part of system memory and accessed using shared virtual memory (SVM) technology without suffering from performance disadvantages associated with full system cache coherence. In at least one embodiment, the ability for GPU memories (1620) to be accessed as system memory without burdensome cache coherence overhead provides a beneficial operating environment for GPU offload. In at least one embodiment, this configuration allows software on the host processor (1605) to set up operands and access computation results without the overhead of traditional I / O DMA data copies. In at least one embodiment, these traditional copies involve driver calls, interrupts, and memory-mapped I / O (MMIO) accesses, all of which are inefficient compared to simple memory accesses. In at least one embodiment, the ability to access GPU memories (1620) without cache coherence overhead may be important for the execution time of offloaded computation. In at least one embodiment, for example, in cases where there is substantial streaming write memory traffic, cache coherence overhead can significantly reduce the effective write bandwidth shown by the GPU (1610). In at least one embodiment, the efficiency of operand setup, the efficiency of result access, and the efficiency of GPU computation may play a role in determining the efficiency of GPU offload.

[0314] In at least one embodiment, the selection of GPU bias and host processor bias is driven by a bias tracker data structure. In at least one embodiment, a bias table may be used, for example, a page-grained structure (e.g., controlled by the grain of memory pages) containing 1 or 2 bits per GPU-attached memory page. In at least one embodiment, the bias table may be implemented in a stolen memory range of one or more GPU memories (1620), with or without a bias cache in the GPU (1610), for example (to cache frequently / recently used entries of the bias table). Alternatively, in at least one embodiment, the entire bias table may be maintained within the GPU.

[0315] In at least one embodiment, a bias table entry associated with each access to the GPU-attached memory (1620) is accessed prior to the actual access to the GPU memory, thereby causing the following operations. In at least one embodiment, local requests from the GPU (1610) to find their pages in GPU bias are delivered directly to the corresponding GPU memory (1620). In at least one embodiment, local requests from the GPU to find their pages in host bias are delivered to the processor (1605) (e.g., via a high-speed link as described herein). In at least one embodiment, requests from the processor (1605) to find the requested pages in host processor bias complete a request such as a normal memory read. Alternatively, requests directed to GPU-biased pages may be delivered to the GPU (1610). In at least one embodiment, the GPU may then transfer the pages to host processor bias if the pages are not currently in use. In at least one embodiment, the bias state of the page may be changed by a software-based mechanism, a hardware-assisted software-based mechanism, or, for a limited set of cases, by a purely hardware-based mechanism.

[0316] In at least one embodiment, a mechanism for changing the bias state uses an API call (e.g., OpenCL), which ultimately calls the GPU device driver, which ultimately sends a message (or encues a command descriptor) instructing the GPU to change the bias state and, for some transitions, to perform a cache flushing operation on the host. In at least one embodiment, the cache flushing operation is used for a transition from the host processor (1605) bias to the GPU bias, but not for the reverse transition.

[0317] In at least one embodiment, cache coherence is maintained by temporarily rendering GPU-biased pages that cannot be cached by the host processor (1605). In at least one embodiment, to access these pages, the processor (1605) may request access from the GPU (1610), which may or may not immediately grant access. Thus, in at least one embodiment, to reduce communication between the processor (1605) and the GPU (1610), it is beneficial to ensure that GPU-biased pages are requested by the GPU rather than the host processor (1605), and vice versa.

[0318] Hardware structure(s) (815) are used to carry out one or more embodiments. Details regarding the hardware structure(s) (815) may be provided in conjunction with FIG. 8a and / or FIG. 8b.

[0319] FIG. 17 illustrates exemplary integrated circuits and associated graphics processors that may be manufactured using one or more IP cores according to various embodiments described herein. In addition to those illustrated, other logic and circuits including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores may be included in at least one embodiment.

[0320] FIG. 17 is a block diagram illustrating an exemplary system-on-chip integrated circuit (1700) that can be manufactured using one or more IP cores according to at least one embodiment. In at least one embodiment, the integrated circuit (1700) includes one or more application processor(s) (1705) (e.g., CPUs), at least one graphics processor (1710), and may additionally include an image processor (1715) and / or a video processor (1720), any of which may be a modular IP core. In at least one embodiment, the integrated circuit (1700) includes a USB controller (1725), a UART controller (1730), an SPI / SDIO controller (1735), and I 2 2S / I 2 It includes peripheral or bus logic including a 2C controller (1740). In at least one embodiment, the integrated circuit (1700) may include a display device (1745) connected to one or more of an HDMI (high-definition multimedia interface) controller (1750) and a MIPI (mobile industry processor interface) display interface (1755). In at least one embodiment, storage may be provided by a flash memory subsystem (1760) including flash memory and a flash memory controller. In at least one embodiment, a memory interface may be provided through a memory controller (1765) for accessing SDRAM or SRAM memory devices. In at least one embodiment, some integrated circuits additionally include an embedded security engine (1770).

[0321] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in an integrated circuit (1700) to infer or predict operations based, at least partially, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0322] At least one embodiment of FIGS. 1 through 17 performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0323] FIGS. 18a and 18b illustrate exemplary integrated circuits and associated graphics processors that may be manufactured using one or more IP cores according to various embodiments described herein. In addition to those illustrated, other logic and circuits, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores, may be included in at least one embodiment.

[0324] FIGS. 18a and FIGS. 18b are block diagrams illustrating exemplary graphics processors for use within an SoC according to embodiments described herein. FIG. 18a illustrates an exemplary graphics processor (1810) of a system-on-chip integrated circuit that can be manufactured using one or more IP cores according to at least one embodiment. FIG. 18b illustrates an additional exemplary graphics processor (1840) of a system-on-chip integrated circuit that can be manufactured using one or more IP cores according to at least one embodiment. In at least one embodiment, the graphics processor (1810) of FIG. 18a is a low-power graphics processor core. In at least one embodiment, the graphics processor (1840) of FIG. 18b is a higher-performance graphics processor core. In at least one embodiment, each of the graphics processors (1810, 1840) may be a variation of the graphics processor (1710) of FIG. 17.

[0325] In at least one embodiment, the graphics processor (1810) comprises a vertex processor (1805) and one or more fragment processor(s) (1815A to 1815N) (e.g., 1815A, 1815B, 1815C, 1815D, to 1815N-1, and 1815N). In at least one embodiment, the graphics processor (1810) may execute different shader programs through separate logic, such that the vertex processor (1805) is optimized to execute operations for vertex shader programs, while one or more fragment processor(s) (1815A to 1815N) execute fragment (e.g., pixel) shading operations for fragment or pixel shader programs. In at least one embodiment, a vertex processor (1805) performs a vertex processing stage of a 3D graphics pipeline and generates primitives and vertex data. In at least one embodiment, fragment processor(s) (1815A to 1815N) use the primitives and vertex data generated by the vertex processor (1805) to produce a frame buffer that is displayed on a display device. In at least one embodiment, the fragment processor(s) (1815A to 1815N) are optimized to execute a fragment shader program provided by the OpenGL API, which can be used to perform operations similar to those of a pixel shader program provided by the Direct 3D API.

[0326] In at least one embodiment, the graphics processor (1810) additionally comprises one or more memory management units (MMUs) (1820A to 1820B), cache(s) (1825A to 1825B), and circuit interconnect(s) (1830A to 1830B). In at least one embodiment, one or more MMU(s) (1820A to 1820B) provide a virtual to physical address mapping for the graphics processor (1810) comprising a vertex processor (1805) and / or fragment processor(s) (1815A to 1815N) capable of referencing vertex or image / texture data stored in memory, in addition to vertex or image / texture data stored in one or more cache(s) (1825A to 1825B). In at least one embodiment, one or more MMU(s) (1820A to 1820B) are synchronized with other MMUs in the system, including one or more MMUs associated with one or more application processor(s) (1705) of FIG. 17, an image processor (1715), and / or a video processor (1720), so that each processor (1705 to 1720) can participate in a shared or integrated virtual memory system. In at least one embodiment, one or more circuit interconnect(s) (1830A to 1830B) enable the graphics processor (1810) to interface with other IP cores in the SoC via the internal bus of the SoC or via a direct connection.

[0327] In at least one embodiment, the graphics processor (1840) comprises one or more shader core(s) (1855A to 1855N) as illustrated in FIG. 18b (e.g., 1855A, 1855B, 1855C, 1855D, 1855E, 1855F, to 1855N-1, and 1855N), and provides an integrated shader core architecture in which a single core or type or a core can execute all types of programmable shader code, including shader program code for implementing vertex shaders, fragment shaders, and / or compute shaders. In at least one embodiment, the number of shader cores may vary. In at least one embodiment, the graphics processor (1840) includes an inter-core task manager (1845) acting as a thread dispatcher that dispatches execution threads to one or more shader cores (1855A to 1855N), and a tiling unit (1858) that accelerates tiling operations for tile-based rendering, for example, by utilizing local spatial coherence in the scene or by subdividing rendering operations for the scene in image space to optimize the use of internal caches.

[0328] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a graphics processor (1810 and / or 1840) to infer or predict operations based, at least partially, on weight parameters calculated using the neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0329] At least one embodiment of FIGS. 1 through 18b performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0330] FIGS. 19a and 19b illustrate additional exemplary graphics processor logic according to embodiments described herein. In at least one embodiment, the components illustrated and described in connection with FIGS. 19a and 19b are integrated within a single system, such as a graphics processing unit (GPU), an SoC, or another type of processor. FIG. 19a illustrates a graphics core (1900) that, in at least one embodiment, may be included within the graphics processor (1710) of FIG. 17 and, in at least one embodiment, may be an integrated shader core (1855A to 1855N) as in FIG. 18b illustrates a highly parallel "GPGPU" (general-purpose graphics processing unit, also referred to as a "graphics processing unit") (1930) suitable for placement on a multi-chip module in at least one embodiment. In at least one embodiment, the graphics processing unit (1930) is a GPGPU comprising a graphics processor. In at least one embodiment, the integrated circuit (1700) includes, for example, a graphics core (1900) to form the integrated circuit and / or to form an SoC, wherein the integrated circuit and / or the SoC performs the operations described herein.

[0331] In at least one embodiment, the graphics core (1900) includes a shared instruction cache (1902), a texture unit (1918), and a cache / shared memory (1920) (e.g., including L1, L2, L3, end-level cache, or other caches) that are common to the execution resources within the graphics core (1900). In at least one embodiment, the graphics core (1900) may include a plurality of slices (1901A to 1901N) or partitions for each core, and the graphics processor may include a plurality of instances of the graphics core (1900). In at least one embodiment, each slice (1901A to 1901N) refers to the graphics core (1900). In at least one embodiment, the slices (1901A to 1901N) have sub-slices that are part of the slices (1901A to 1901N). In at least one embodiment, the slices (1901A to 1901N) are independent of or dependent on other slices. In at least one embodiment, the slices (1901A to 1901N) may include support logic comprising a local instruction cache (1904A to 1904N), a thread scheduler (sequencer) (1906A to 1906N), a thread dispatcher (1908A to 1908N), and a set of registers (1910A to 1910N).In at least one embodiment, the slices (1901A to 1901N) may include a set of AFUs (additional function units) (1912A to 1912N)), FPUs (floating-point units) (1914A to 1914N)), integer ALUs (arithmetic logic units) (1916A to 1916N)), ACUs (address computational units) (1913A to 1913N)), DPFPUs (double-precision floating-point units) (1915A to 1915N)), and MPUs (matrix processing units) (1917A to 1917N)). In at least one embodiment, the MPUs (1917A to 1917N) are referred to as matrix engines.

[0332] In at least one embodiment, each slice (1901A to 1901N) includes one or more engines for floating-point and integer vector operations and one or more engines for accelerating convolution and matrix operations in AI, machine learning, or large dataset workloads. In at least one embodiment, one or more slices (1901A to 1901N) include one or more vector engines for computing vectors (e.g., computing mathematical operations on vectors). In at least one embodiment, the vector engine may compute vector operations in 16-bit floating-point (also referred to as "FP16"), 32-bit floating-point (also referred to as "FP32"), or 64-bit floating-point (also referred to as "FP64"). In at least one embodiment, one or more slices (1901A to 1901N) comprise 16 vector engines paired with 16 matrix mathematical units to compute matrix / tensor operations, wherein the vector engines and mathematical units are exposed through matrix extensions. In at least one embodiment, the slice is a processing unit, e.g., a specified portion of the processing resources of 16 cores and a ray tracing unit or 8 cores, a thread scheduler, a thread dispatcher, and additional functional units for the processor. In at least one embodiment, the graphics core (1900) comprises one or more matrix engines for computing matrix operations, e.g., when computing tensor operations.

[0333] In at least one embodiment, one or more slices (1901A to 1901N) include one or more ray tracing units for computing ray tracing operations (e.g., 16 ray tracing units per slice (1901A to 1901N)). In at least one embodiment, the ray tracing unit computes ray crossing, triangle intersection, bounding box intersection, or other ray tracing operations.

[0334] In at least one embodiment, one or more slices (1901A to 1901N) include a media slice, which encodes, decodes, and / or transcodes data; scales and / or formats data; and / or performs video quality operations on video data.

[0335] In at least one embodiment, one or more slices (1901A to 1901N) are linked to an L2 cache and memory fabric, link connectors, high-bandwidth memory (HBM) (e.g., HBM2e, HDM3) stacks, and a media engine. In at least one embodiment, one or more slices (1901A to 1901N) include a plurality of cores (e.g., 16 cores) and a plurality of ray tracing units (e.g., 16) paired with each core. In at least one embodiment, one or more slices (1901A to 1901N) have one or more L1 caches. In at least one embodiment, one or more slices (1901A to 1901N) include one or more vector engines; one or more instruction caches for storing instructions; one or more L1 caches for caching data; one or more shared local memories (SLMs) for storing data corresponding to instructions, e.g.; and one or more samplers for sampling data. One or more ray tracing units for performing ray tracing operations; one or more geometries for performing operations in geometry pipelines and / or applying geometry transformations to vertices or polygons; one or more rasterizers for describing an image in a vector graphics format (e.g., geometry) and converting it into a raster image (e.g., a series of pixels, dots, or lines that, when displayed together, create an image represented by the geometry); one or more Hierarchical Depth Buffers (Hiz) for buffering data; and / or one or more pixel backends. In at least one embodiment, the slices (1901A to 1901N) include a memory fabric, e.g., an L2 cache.

[0336] In at least one embodiment, FPUs (1914A to 1914N) can perform single-precision (32-bit) and half-precision (16-bit) floating-point operations, whereas DPFPUs (1915A to 1915N) can perform double-precision (64-bit) floating-point operations. In at least one embodiment, ALUs (1916A to 1916N) can perform variable-precision integer operations with 8-bit, 16-bit, and 32-bit precision and can be configured for mixed-precision operations. In at least one embodiment, MPUs (1917A to 1917N) can also be configured for mixed-precision matrix operations, including half-precision floating-point and 8-bit integer operations. In at least one embodiment, the MPUs (1917 to 1917N) can perform various matrix operations to accelerate machine learning application frameworks, including enabling support for accelerated GEMM (general matrix to matrix multiplication). In at least one embodiment, the AFUs (1912A to 1912N) can perform additional logical operations not supported by floating-point or integer units, including trigonometric operations (e.g., sine, cosine, etc.).

[0337] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a graphics core (1900) to infer or predict operations based, at least partially, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0338] In at least one embodiment, the graphics core (1900) includes an interconnect and link fabric sublayer attached to a switch and GPU-GPU bridge that enables a plurality of graphics processors (1900) (e.g., 8) to interlink across the plurality of graphics processors (1900) with LSUs (load / store units), data transfer units, and synchronous semantics without adhesion to one another. In at least one embodiment, the interconnects include standardized interconnects (e.g., PCIe) or a combination thereof.

[0339] In at least one embodiment, the graphics core (1900) includes a plurality of tiles. In at least one embodiment, the tile is an individual die or one or more dies, wherein the individual dies may be connected to an interconnect (e.g., an embedded multi-die interconnect bridge (EMIB)). In at least one embodiment, the graphics core (1900) includes a computing tile, a memory tile (e.g., wherein the memory tile may be exclusively accessed by different tiles or different chipsets, such as a Rambo tile), a substrate tile, a base tile, an HMB tile, a link tile, and an EMIB tile, wherein all tiles are packaged together in the graphics core (1900) as part of the GPU. In at least one embodiment, the graphics core (1900) may include a plurality of tiles in a single package (also referred to as a "multi tile package"). In at least one embodiment, the computing tile may have eight graphics cores (1900), an L1 cache; The base tile may have a host interface with PCIe 5.0, HBM2e, MDFI, and EMIB, a link tile with eight links, and eight ports with an embedded switch. In at least one embodiment, the tiles are connected by face-to-face chip-on-chip bonding through fine-pitch, 36-micron microbumps (e.g., copper pillars). In at least one embodiment, the graphics core (1900) is a tile that includes a memory fabric containing memory and is accessible by a plurality of tiles. In at least one embodiment, the graphics core (1900) stores, accesses, or loads its own hardware contexts in memory, wherein the hardware context is a set of data loaded from registers before the process is resumed, and the hardware context may indicate the state of the hardware (e.g., the state of the GPU).

[0340] In at least one embodiment, the graphics core (1900) includes a SERDES (serializer / deserializer) circuit that converts a serial data stream into a parallel data stream or converts a parallel data stream into a serial data stream.

[0341] In at least one embodiment, the graphics core (1900) includes a high-speed coherent integrated fabric (GPU to GPU), loading / storing units, bulk data transfer and synchronous semantics, and connected GPUs through an embedded switch, wherein the GPU-GPU bridge is controlled by a controller.

[0342] In at least one embodiment, the graphics core (1900) performs an API, said API abstracts the hardware of the graphics core (1900) and accesses libraries with instructions for performing mathematical operations (e.g., a mathematical kernel library), deep neural network operations (e.g., a deep neural network library), vector operations, set communications, thread construction blocks, video processing, data analysis library, and / or ray tracing operations.

[0343] At least one embodiment of FIGS. 1 through 19a performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0344] FIG. 19b illustrates a GPGPU (1930) that, in at least one embodiment, can be configured to enable highly parallel computing operations to be performed by an array of graphics processing units. In at least one embodiment, the GPGPU (1930) can be directly linked to other instances of the GPGPU (1930) to create a multi-GPU cluster to improve the training speed for deep neural networks. In at least one embodiment, the GPGPU (1930) includes a host interface (1932) that enables connection with a host processor. In at least one embodiment, the host interface (1932) is a PCI Express interface. In at least one embodiment, the host interface (1932) may be a vendor-specific communication interface or a communication fabric. In at least one embodiment, the GPGPU (1930) receives commands from a host processor and uses a global scheduler (1934) (which may be referred to as a thread sequencer and / or an asynchronous computing engine) to distribute execution threads associated with these commands to a set of computing clusters (1936A to 1936H). In at least one embodiment, the computing clusters (1936A to 1936H) share a cache memory (1938). In at least one embodiment, the cache memory (1938) may serve as an upper-level cache for the cache memories within the computing clusters (1936A to 1936H). In at least one embodiment, the computing clusters (1936A to 1936H) include slices or are referred to as "slices". In at least one embodiment, the GPGPU (1930) is part of an SoC, such as part of an integrated circuit (1700) (Fig. 17).

[0345] In at least one embodiment, the GPGPU (1930) includes memory (1944A to 1944B) connected to computing clusters (1936A to 1936H) through a set of memory controllers (1942A to 1942B) (e.g., one or more controllers for HBM2e). In at least one embodiment, the memory (1944A to 1944B) may include various types of memory devices, such as dynamic random access memory (DRAM) or graphics random access memory, including graphics double data rate (GDDR) memory, synchronous graphics random access memory (SGRAM).

[0346] In at least one embodiment, computing clusters (1936A to 1936H) each include a set of graphics cores, such as the graphics core (1900) of FIG. 19a, which may include a plurality of integer and floating-point logic units capable of performing computational operations in a range of precisions including those suitable for machine learning computations. For example, in at least one embodiment, at least a subset of floating-point units in each of computing clusters (1936A to 1936H) may be configured to perform 16-bit or 32-bit floating-point operations, whereas a different subset of floating-point units may be configured to perform 64-bit floating-point operations.

[0347] In at least one embodiment, multiple instances of the GPGPU (1930) may be configured to operate as a computing cluster. In at least one embodiment, the communication used by the computing clusters (1936A through 1936H) for synchronization and data exchange varies across embodiments. In at least one embodiment, multiple instances of the GPGPU (1930) communicate through a host interface (1932). In at least one embodiment, the GPGPU (1930) includes an I / O hub (1939) connecting the GPGPU (1930) and a GPU link (1940) that enables direct access to other instances of the GPGPU (1930). In at least one embodiment, the GPU link (1940) is connected to a dedicated GPU-to-GPU bridge that enables communication and synchronization between multiple instances of the GPGPU (1930). In at least one embodiment, the GPU link (1940) is connected to a high-speed interconnect to transmit and receive data to and from other GPGPUs or parallel processors. In at least one embodiment, multiple instances of the GPGPU (1930) are located in a separate data processing system and communicate through a network device accessible via a host interface (1932). In at least one embodiment, the GPU link (1940) may be configured to enable access to a host processor in addition to or as an alternative to the host interface (1932).

[0348] In at least one embodiment, the GPGPU (1930) may be configured to train neural networks. In at least one embodiment, the GPGPU (1930) may be used within an inference platform. In at least one embodiment where the GPGPU (1930) is used for inference, the GPGPU (1930) may include fewer computing clusters (1936A to 1936H) than when the GPGPU (1930) is used to train neural networks. In at least one embodiment, memory technologies associated with memory (1944A to 1944B) may differ between the inference configuration and the training configuration, and higher bandwidth memory technologies are dedicated to the training configurations. In at least one embodiment, the inference configuration of the GPGPU (1930) may support inferring specific instructions. For example, in at least one embodiment, the inference configuration may provide support for one or more 8-bit integer dot product instructions that can be used during inference operations for deployed neural networks.

[0349] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a GPGPU (1930) to infer or predict operations based, at least partially, on weight parameters calculated using the neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0350] At least one embodiment of FIGS. 1 through 19b performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0351] FIG. 20 is a block diagram illustrating a computing system (2000) according to at least one embodiment. In at least one embodiment, the computing system (2000) includes one or more processor(s) (2002) and a processing subsystem (2001) having system memory (2004) that communicates through an interconnect path that may include a memory hub (2005). In at least one embodiment, the memory hub (2005) may be a separate component within a chipset component or may be integrated within one or more processor(s) (2002). In at least one embodiment, the memory hub (2005) is connected to an I / O subsystem (2011) through a communication link (2006). In at least one embodiment, the I / O subsystem (2011) includes an I / O hub (2007) that enables the computing system (2000) to receive input from one or more input device(s) (2008). In at least one embodiment, the I / O hub (2007) may enable a display controller, which may be included in one or more processor(s) (2002), to provide outputs to one or more display device(s) (2010A). In at least one embodiment, one or more display device(s) (2010A) connected to the I / O hub (2007) may include local, internal, or built-in display devices.

[0352] In at least one embodiment, the processing subsystem (2001) includes one or more parallel processor(s) (2012) connected to the memory hub (2005) via a bus or other communication link (2013). In at least one embodiment, the communication link (2013) may use any number of standard-based communication link technologies or protocols, such as PCI Express, but is not limited thereto, or may be a vendor-specific communication interface or communication fabric. In at least one embodiment, the one or more parallel processor(s) (2012) form a computationally intensive parallel or vector processing system that may include a large number of processing cores and / or processing clusters, such as a many-integrated core (MIC) processor. In at least one embodiment, some or all of the parallel processor(s) (2012) form a graphics processing subsystem capable of outputting pixels to one or more display device(s) (2010A) connected via the I / O hub (2007). In at least one embodiment, the parallel processor(s) (2012) may also include a display controller and a display interface (not shown) that enable direct connection to one or more display devices(s) (2010B). In at least one embodiment, the parallel processor(s) (2012) includes one or more cores, such as the graphics cores (1900) discussed herein.

[0353] In at least one embodiment, a system storage unit (2014) may be connected to an I / O hub (2007) to provide a storage mechanism for a computing system (2000). In at least one embodiment, an I / O switch (2016) may be used to provide an interface mechanism that enables connections between the I / O hub (2007) and other components, such as a network adapter (2018) and / or a wireless network adapter (2019) that may be integrated into the platform, and various other devices that may be added via one or more add-in device(s) (2020). In at least one embodiment, the network adapter (2018) may be an Ethernet adapter or another wired network adapter. In at least one embodiment, the wireless network adapter (2019) may include one or more of other network devices, such as Wi-Fi, Bluetooth, NFC (near field communication), or one or more wireless radios.

[0354] In at least one embodiment, the computing system (2000) may include other components not explicitly shown, such as USB or other port connections, optical storage drives, video capture devices, etc., and may also be connected to an I / O hub (2007). In at least one embodiment, communication paths interconnecting the various components in FIG. 20 may be implemented using any suitable protocols, such as PCI (Peripheral Component Interconnect) based protocols (e.g., PCI-Express), or other bus or point-to-point communication interfaces and / or protocol(s), such as NV-Link high-speed interconnect or interconnect protocols.

[0355] In at least one embodiment, the parallel processor(s) (2012) comprises circuits optimized for graphics and video processing, for example, including video output circuits, and constitute a graphics processing unit (GPU), for example, the parallel processor(s) (2012) comprises a graphics core (1900). In at least one embodiment, the parallel processor(s) (2012) comprises circuits optimized for general-purpose processing. In at least one embodiment, the components of the computing system (2000) may be integrated with one or more other system elements on a single integrated circuit. For example, in at least one embodiment, the parallel processor(s) (2012), memory hub (2005), processor(s) (2002), and I / O hub (2007) may be integrated into a system on chip (SoC) integrated circuit. In at least one embodiment, the components of the computing system (2000) may be integrated within a single package to form a system in package (SIP) configuration. In at least one embodiment, at least some of the components of the computing system (2000) may be integrated into a multi-chip module (MCM), which may be interconnected with other multi-chip modules to form a modular computing system.

[0356] Logic (815) is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the logic (815) are provided herein together with FIG. 8a and / or FIG. 8b. In at least one embodiment, the logic (815) may be used in a computing system (2000) to infer or predict operations based, at least partially, on weight parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0357] At least one embodiment of FIGS. 1 through 20 performs an optimized deployment of one or more second neural networks on new platforms, computer systems, and any other hardware resources provided, for example, by using one or more first neural networks. In at least one embodiment, a processor includes one or more circuits that use the one or more first neural networks to generate one or more second versions of the one or more second neural networks based, at least partially, on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. In at least one embodiment, the one or more first neural networks are trained based on hardware features, software features, and software programs associated with a plurality of platforms, computer systems, and any other hardware resources.

[0358] processors

[0359] FIG. 21a illustrates a parallel processor (2100) according to at least one embodiment. In at least one embodiment, various components of the parallel processor (2100) may be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). In at least one embodiment, the illustrated parallel processor (2100) is a variation of one or more parallel processor(s) (2012) illustrated in FIG. 20 according to an exemplary embodiment. In at least one embodiment, the parallel processor (2100) includes one or more graphics cores (1900).

[0360] In at least one embodiment, the parallel processor (2100) includes a parallel processing unit (2102). In at least one embodiment, the parallel processing unit (2102) includes an I / O unit (2104) that enables communication with other devices, including other instances of the parallel processing unit (2102). In at least one embodiment, the I / O unit (2104) may be directly connected to other devices. In at least one embodiment, the I / O unit (2104) connects to other devices through the use of a hub or switch interface, such as a memory hub (2105). In at least one embodiment, the connections between the memory hub (2105) and the I / O unit (2104) form a communication link (2113). In at least one embodiment, the I / O unit (2104) is connected to a host interface (2106) and a memory crossbar (2116), wherein the host interface (2106) receives commands directed to perform processing operations and the memory crossbar (2116) receives commands directed to perform memory operations.

[0361] In at least one embodiment, when the host interface (2106) receives a command buffer through the I / O unit (2104), the host interface (2106) may direct work operations to the front end (2108) to perform these commands. In at least one embodiment, the front end (2108) is connected to a scheduler (2110) (which may be referred to as a sequencer) configured to distribute commands or other work items to a processing cluster array (2112). In at least one embodiment, the scheduler (2110) ensures that the processing cluster array (2112) is properly configured and in an active state before tasks are distributed to clusters of the processing cluster array (2112). In at least one embodiment, the scheduler (2110) is implemented through firmware logic running on a microcontroller. In at least one embodiment, the microcontroller-implemented scheduler (2110) can be configured to perform complex scheduling and task distribution operations in a rough and fine granular manner, thereby enabling rapid preemption and context switching of threads running in the processing array (2112). In at least one embodiment, the host software can identify workloads to be scheduled on the processing cluster array (2112) through one of a plurality of graphic processing paths. In at least one embodiment, the workloads can then be automatically distributed across the processing array cluster (2112) by the scheduler (2110) logic within the microcontroller including the scheduler (2110).

[0362] In at least one embodiment, the processing cluster array (2112) may include up to "N" processing clusters (e.g., cluster (2114A), cluster (2114B), to cluster (2114N)), where "N" represents a positive integer (which may be an integer "N" different from that used in other drawings). In at least one embodiment, each cluster (2114A to 2114N) of the processing cluster array (2112) may execute a large number of concurrent threads. In at least one embodiment, the scheduler (2110) may assign tasks to the clusters (2114A to 2114N) of the processing cluster array (2112) using various scheduling and / or task distribution algorithms that may vary depending on the workload occurring for each type of program or computation. In at least one embodiment, scheduling may be dynamically handled by a scheduler (2110) or partially assisted by compiler logic during the compilation of program logic configured for execution by a processing cluster array (2112). In at least one embodiment, different clusters (2114A to 2114N) of the processing cluster array (2112) may be allocated for processing programs of different types or performing calculations of different types.

[0363] In at least one embodiment, the processing cluster array (2112) may be configured to perform various types of parallel processing operations. In at least one embodiment, the processing cluster array (2112) is configured to perform general-purpose parallel computing operations. For example, in at least one embodiment, the processing cluster array (2112) may include logic for executing processing tasks including filtering video and / or audio data, performing modeling operations including physical operations, and performing data transformations.

[0364] In at least one embodiment, the processing cluster array (2112) is configured to perform parallel graphics processing operations. In at least one embodiment, the processing cluster array (2112) may include additional logic that supports the execution of these graphics processing operations, including texture sampling logic that performs texture operations, as well as tessellation logic and other vertex processing logic, but is not limited thereto. In at least one embodiment, the processing cluster array (2112) may be configured to execute graphics processing-related shader programs, such as vertex shaders, tessellation shaders, geometry shaders, and pixel shaders, but is not limited thereto. In at least one embodiment, the parallel processing unit (2102) may transfer data from system memory via the I / O unit (2104) for processing. In at least one embodiment, during processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory (2122)) during processing and ...

Claims

Claim 1 A processor comprising one or more circuits that use one or more first neural networks to generate one or more second versions of one or more second neural networks—based at least partially on one or more hardware resources to be used to perform one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. Claim 2 In claim 1, one or more first versions of the one or more second neural networks are compressed based at least partially on one or more features of one or more other hardware resources distinct from the one or more hardware resources. Claim 3 In claim 1, the above one or more first neural networks are processors including converter neural networks. Claim 4 In claim 1, the generation of one or more second versions of the one or more second neural networks is further based on a processor that relies at least partially on one or more other hardware resources used to perform one or more first versions of the one or more second neural networks. Claim 5 In claim 1, the one or more circuits further update the one or more first neural networks based at least partially on one or more hardware features of a plurality of hardware resources, and the one or more hardware features include a processor comprising a numeric data type. Claim 6 In claim 1, the one or more circuits further update the one or more first neural networks based, at least partially, on one or more software features of a plurality of hardware resources, and the one or more software features include development tools, runtime environments, or libraries, in a processor. Claim 7 In claim 1, the one or more circuits are a processor that updates the one or more first neural networks at least partially based on one or more first software programs and one or more second software programs used to place the one or more neural networks on additionally different hardware resources. Claim 8 In claim 1, the one or more circuits additionally allow one or more software programs to be executed by the one or more hardware resources, and one or more second versions of the one or more second neural networks are processors implemented by the one or more software programs. Claim 9 A method comprising the step of generating one or more second versions of one or more second neural networks using one or more first neural networks—at least partially based on one or more hardware resources to be used to perform one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. Claim 10 In claim 9, the method wherein one or more first versions of the one or more second neural networks are performed by one or more different hardware resources different from the one or more hardware resources. Claim 11 In claim 9, the method wherein one or more first neural networks include a converter neural network. Claim 12 In claim 9, a method comprising additionally receiving information including one or more features of a plurality of hardware resources—the one or more features being available to update the one or more first neural networks. Claim 13 In claim 9, a method comprising the step of modifying one or more first neural networks at least partially based on one or more first software programs and one or more second software programs executed by different hardware resources to implement one or more compressed neural networks. Claim 14 In claim 9, a method comprising additionally modifying the one or more first neural networks based at least partially on one or more hardware features of a plurality of hardware resources—the one or more hardware features including the number of processing units, clock speed, or memory capacity. Claim 15 A system comprising one or more processors that use one or more first neural networks to generate one or more second versions of one or more second neural networks—based at least partially on one or more hardware resources to be used to execute one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks. Claim 16 In paragraph 15, a system in which one or more first versions of the one or more second neural networks and one or more second versions of the one or more second neural networks are executed by separate hardware resources. Claim 17 In paragraph 15, the system comprises one or more first neural networks that are pre-trained neural networks. Claim 18 In paragraph 15, a system in which one or more processors update the one or more first neural networks at least partially based on a compressed neural network comprising multiple versions corresponding to multiple hardware resources. Claim 19 In paragraph 15, the system wherein the one or more processors further update the one or more first neural networks at least partially based on one or more software programs associated with a plurality of hardware resources. Claim 20 In paragraph 15, the above one or more processors further update the above one or more first neural networks based, at least partially, on one or more software features of a plurality of hardware resources, and the above one or more software features include libraries, runtime environments, or APIs (application programming interfaces).

Images