NEURAL NETWORK FOR PREDICTING THE QUALITY OF WIRELESS SIGNALS

Abstract

Devices, systems, processors, circuits, and techniques for using one or more neural networks to predict the quality of one or more wireless signals, at least partially, based on one or more reference signals. In at least one embodiment, the devices, systems, processors, circuits, and techniques are intended to use one or more neural networks to determine whether a radio link is unstable or stable for a period of time or a number of frames (e.g., 10 consecutive frames).

Description

TECHNICAL FIELD

[0001] At least one embodiment relates to processors, computing systems, devices, non-transitory computing media, and / or methods for using neural networks to monitor and / or adjust (e.g., repair) wireless link communications. For example, a processor includes circuitry for using one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. BACKGROUND

[0002] When a user equipment (UE) connects to a radio access network (RAN), it may monitor a connection between the UE and the RAN (e.g., a connection to a radio unit and / or a base station). This process is commonly known as radio link monitoring (RLM). RLM can have some challenges. For example, if an RLM occurs frequently (e.g., every millisecond), it can use a significant amount of computing resources, which can degrade the performance of devices using the RAN. As another example, RLM may require the use of power to measure radio link quality, which may require a RAN or user equipment to consume a significant amount of power, causing the RAN or user equipment to degrade power efficiency. Accordingly, there is a need to improve radio link monitoring. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 illustrates a computing environment for a user device and a base station according to at least one embodiment; Fig. 2 illustrates a schematic overview of how radio link quality predictions are generated, according to at least one embodiment; Fig. 3 is a process flow diagram for generating a prediction during radio link quality monitoring according to at least one embodiment; Fig. 4 is a process flow diagram for generating a prediction for a radio link and using that prediction to perform a response action, according to at least one embodiment; Fig. 5 is a process flow diagram for generating predictions for a radio link and using those predictions to perform a response action, according to at least one embodiment; Fig. 6 is a schematic block diagram illustrating the use of a neural network to generate predictions about the quality of a radio link, according to at least one embodiment; Fig. 7 is a block diagram illustrating a driver and / or runtime including one or more libraries to provide one or more application programming interfaces (APIs), according to at least one embodiment; Fig. 8A illustrates logic according to at least one embodiment; Fig. 8B illustrates logic according to at least one embodiment; Fig. 9 illustrates the training and deployment of a neural network according to at least one embodiment; Fig. 10 illustrates an example data center system according to at least one embodiment; Fig. 11A illustrates an example of an autonomous vehicle according to at least one embodiment; Fig. Figure 11B illustrates an example of camera positions and fields of view for the autonomous vehicle from Fig. 11A according to at least one embodiment; Fig. Figure 11C is a block diagram illustrating an example system architecture for the autonomous vehicle of Fig. 11A according to at least one embodiment; Fig. 11D is a diagram illustrating a system for communication between one or more cloud-based servers and the autonomous vehicle. Fig. 11A according to at least one embodiment; Fig. 12 is a block diagram illustrating a computer system according to at least one embodiment; Fig. 13 is a block diagram illustrating a computer system according to at least one embodiment; Fig. 14 illustrates a computer system according to at least one embodiment; Fig. 15 illustrates a computer system according to at least one embodiment; Fig. 16A illustrates a computer system according to at least one embodiment; Fig. 16B illustrates a computer system according to at least one embodiment; Fig. 16C illustrates a computer system according to at least one embodiment; Fig. 16D illustrates a computer system according to at least one embodiment; Fig. 16E and Fig. 16F illustrate a shared programming model according to at least one embodiment; Fig. 17 illustrates example integrated circuits and associated graphics processors according to at least one embodiment; Fig. 18A and Fig. 18B illustrate example integrated circuits and associated graphics processors according to at least one embodiment; Fig. 19A and Fig. 19B illustrate additional example graphics processor logic according to at least one embodiment; Fig. 20 illustrates a computer system according to at least one embodiment; Fig. 21A illustrates a parallel processor according to at least one embodiment; Fig. 21B illustrates a partition unit according to at least one embodiment; Fig. 21C illustrates a processing cluster according to at least one embodiment; Fig. 21D illustrates a graphics multiprocessor according to at least one embodiment; Fig. 22 illustrates a multi-graphics processing unit (GPU) system according to at least one embodiment; Fig. 23 illustrates a graphics processor according to at least one embodiment; Fig. 24 is a block diagram illustrating a processor microarchitecture for a processor according to at least one embodiment; Fig. 25 illustrates a deep learning application processor according to at least one embodiment; Fig. 26 is a block diagram illustrating an example neuromorphic processor according to at least one embodiment; Fig. 27 illustrates at least portions of a graphics processor according to one or more embodiments; Fig. 28 illustrates at least portions of a graphics processor according to one or more embodiments; Fig. 29 illustrates at least portions of a graphics processor according to one or more embodiments; Fig. 30 is a block diagram of a graphics processing engine of a graphics processor according to at least one embodiment; Fig. 31 is a block diagram of at least portions of a graphics processor core according to at least one embodiment; Fig. 32A and Fig. 32B illustrates thread execution logic including an array of processing elements of a graphics processor core, according to at least one embodiment; Fig. 33 illustrates a parallel processing unit ("PPU") according to at least one embodiment; Fig. 34 illustrates a general processing cluster (“GPC”) according to at least one embodiment; Fig. 35 illustrates a memory partition unit of a parallel processing unit ("PPU") according to at least one embodiment; Fig. 36 illustrates a streaming multiprocessor according to at least one embodiment; Fig. 37 is an example data flow diagram for an advanced compute pipeline according to at least one embodiment; Fig. 38 is a system diagram for an example system for training, adapting, instantiating, and deploying machine learning models in an advanced compute pipeline, according to at least one embodiment; Fig. 39 includes an exemplary illustration of an advanced compute pipeline 3810A for processing imaging data in accordance with at least one embodiment; Fig. 40A includes an example data flow diagram of a virtual instrument supporting an ultrasound device, according to at least one embodiment; Fig. 40B includes an example data flow diagram of a virtual instrument supporting a CT scanner, according to at least one embodiment; Fig. 41A illustrates a data flow diagram for a process for training a machine learning model according to at least one embodiment; and Fig. 41B is an example illustration of a client-server architecture for enhancing annotation tools with pre-trained annotation models, according to at least one embodiment. DETAILED DESCRIPTION

[0003] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of at least one embodiment. However, it will be apparent to one skilled in the art that the inventive concepts may be practiced without one or more of these specific details.

[0004] In at least one embodiment, techniques for determining whether a radio link should be repaired require a UE to continuously monitor the radio link quality. For example, the user equipment monitors the radio link quality for a given cell, estimates the radio link quality, and compares the estimate to manufacturer-specified thresholds. In at least one embodiment, if the estimated radio link quality does not meet a threshold, the UE determines that a radio link has failed.

[0005] In at least one embodiment, RLM is performed by the UE to detect any failures in the radio link. In at least one embodiment, RLM procedures may include continuous monitoring of physical layers, media access control (MAC) layers, and radio resource control (RRC) layers. In at least one embodiment, a radio link or connection between the UE and a base station may fail for many reasons. For example, lower layers of a radio access network software stack may experience failures due to frequency and / or timing skew. As another example, interference between a UE and a base station may cause wireless signal degradation, poor bearer connections, and loss of synchronization between the UE and base station.As another example, failures may also occur during handover procedures, cell reselection procedures, cyclic redundancy check (CRC) failures, and other initialization procedures.

[0006] In at least one embodiment, during radio link failure (RLF) procedures, the receipt of consecutive out-of-sync indications (e.g., signals or packets cannot be decoded properly, or the message timing is incorrect) from lower layers informs a UE that there is a problem with a downlink radio connection. In at least one embodiment, a duration of a timer allows some action to be taken by the UE or a base station to adjust the radio connection or wait for the radio link quality to exceed an acceptable threshold. In at least one embodiment, a timer may stop upon receipt of consecutive in-sync indications (e.g., signals or packets were decoded correctly or at the correct time) from lower layers or upon receipt of a handover command (e.g., RRCReconfiguration with reconfigurationWithSync).In at least one embodiment, the procedure may have challenges. For example, there may be no consecutive in-sync indications from lower layers when a timer is running. Another example problem is due to unreliable downlink radio link quality when the timer is running; the risk of a handover failure is high (e.g., the UE may not successfully receive the handover command). In at least one embodiment, when a timer expires, a UE may declare a radio link failure, forcing a UE to enter RRC_IDLE or initiate the link reestablishment procedure. In at least one embodiment, while in the RRC_IDLE state, a UE is not registered with a particular cell and does not have a connection with the base station to facilitate data transmission. In at least one embodiment, a UE periodically checks for paging messages from a network.In at least one embodiment, a UE only reestablishes a connection to the network upon successful receipt of a paging message from a network. In at least one embodiment, a UE that is in RRC_IDLE or has initiated a connection reestablishment procedure experiences significant communication disruption.

[0007] In at least one embodiment, a technical problem is that techniques for monitoring radio link quality in fifth-generation (5G) New Radio (NR) networks are reactive, resulting in 5G networks responding too late (e.g., seconds after a failure) or initiating time-consuming repair procedures after determining that the link quality is below a threshold and performance has been degraded for an extended period of time. In at least one embodiment, radio link quality refers to the number of packets successfully decoded by a device (e.g., 99% of packets successfully decoded would be a high-quality radio link). In at least one embodiment, because poor radio link quality is only cured after it is detected, users may experience significant amounts of time with poor connectivity.

[0008] In at least one embodiment, a UE or base station uses a neural network to address at least the above-mentioned technical problem (e.g., reactive or slow responses to radio link problems) to derive a predicted future state of a radio link connection between the UE and a base station (e.g., gNB in ​​a 5G NR network). In at least one embodiment, a neural network may then provide an inference to a UE or base station that informs the UE or base station about a future state of a radio link. With a provided inference, a UE may perform operations to adapt (e.g., repair, modify) a radio link, maintain a radio link, and / or avoid radio link repairs or other (e.g., unnecessary) changes.

[0009] In at least one embodiment, a neural network generates a prediction of the radio link quality based on reference signals. In at least one embodiment, a reference signal is a signal that does not contain user data (e.g., data transmitted by a software program, such as video content), but is used for the purpose of configuring and maintaining a wireless connection between a base station and a UE. In at least one embodiment, there are different types of reference signals used for different purposes, such as synchronization and for determining information about the channel in use (e.g., effects of interference). In at least one embodiment, reference signals correspond to a signal-to-interference ratio, a signal-to-noise ratio, a reference signal received power (RSRP), a reference signal received quality (RSRQ), or a combination thereof.

[0010] In at least one embodiment, the neural network at a UE receives a reference signal, determines properties of the reference signal, such as a signal-to-noise ratio, a signal-to-interference ratio, and frequencies, and then uses these properties to predict the radio link quality. In at least one embodiment, software at a UE can then use a predicted link quality to fix link problems before they occur.

[0011] For example, a neural network may receive a reference signal and then predict that, for the next 100 seconds, a radio link quality will be at or above a threshold (e.g., packets will be decoded with a 99% success rate). In at least one embodiment, an output of the neural network (e.g., predicted link quality and / or duration of predicted link quality) may be used by the UE to proactively adjust the radio link (e.g., change frequencies, adjust power) to improve radio link quality. The UE does not have to wait until low radio link quality is detected; instead, the UE can immediately begin adjusting the radio link when the quality is predicted to fall below a threshold, which may take a long time before performance is affected.In at least one embodiment, a neural network proactively generates a prediction of radio link quality based on receiving reference signals. In at least one embodiment, a UE with an output of the neural network also does not have to wait until low radio link quality is detected to begin adjusting the radio link quality. Instead, a UE can immediately begin adjusting a radio link when quality is predicted to fall below a threshold (e.g., a percentage of a signal cannot be successfully decoded, a signal-to-noise ratio is too low, or a parameter of a 5G reference signal does not meet the threshold). In at least one embodiment, the neural network can be used in 3G, 4G, 5G, 6G, and other wireless communication protocols.In at least one embodiment, a processor including one or more circuits is to use the predicted quality of the one or more wireless signals to immediately start one or more processes to adapt one or more radio links between a user device and a radio unit. In at least one embodiment, immediately includes instantaneous, simultaneous, or without intervening time or space. For example, a UE does not have to wait until another measurement window requires a procedure to initiate a radio link. As another example, a base station may receive information about reference signals from the UE and immediately begin adapting radio links.

[0012] In at least one embodiment, a UE is any device used by an end user to communicate. Examples of user devices include mobile phones, laptops, servers, computers, and any other hardware devices capable of communication. In at least one embodiment, a UE includes one or more processors, one or more circuits, memory, power sources, and any hardware required for the UE to operate within a wireless communications network. For example, the user device includes any device capable of communicating over 3GPP LTE, 4G, 5G New Radio, sixth generation (6G) wireless networks, or any other wireless communications standard from the 3GPP Partnership Project.

[0013] Fig. 1 illustrates a computing environment 100 including user equipment (UE) 101 and base station 102. In at least one embodiment, UE 101 and base station 102 each include neural network 102, radio link monitor 103, reference signal manager 104, and processor 105. In at least one embodiment, UE 101 and base station 102 communicate using a radio link 106, e.g., they may transmit, receive, or otherwise share information such as data packets, reference signals, or other communications (e.g., as part of a wireless communication standard identified by IEEE). In at least one embodiment, a radio link includes radio frequency signals, channels, or other information used to connect to or communicate with a 5G network.In at least one embodiment, devices, systems, methods, and techniques are disclosed herein, wherein reference signals are transmitted by base station 102 to UE 101 in a downlink manner, and training, learning, or inference occurs at UE 101. In at least one embodiment, devices, systems, methods, and techniques are disclosed herein, wherein a UE 101 receives reference signals and transmits them to base station 102 for training, learning, inference, or otherwise using the reference signals.

[0014] In at least one embodiment, processor 105 executes, uses, or implements neural network 102, radio link monitor 103, reference signal manager 104. In at least one embodiment, processor 105 may be any processor, e.g., as described in Fig. 21A to SM 3600. In at least one embodiment, the processor 105 uses a neural network 102 to predict radio link quality, including, for example, non-synchronous and synchronous events. In at least one embodiment, the neural network 102 may be remotely located and configured to be communicatively coupled to the UE 101. In at least one embodiment, the UE 101 collects, receives, or otherwise uses reference signals and provides them to the neural network 102 (e.g., in a downlink operation).

[0015] In at least one embodiment, a processor using the neural network 102 may infer, with reference signal inputs for a future time, whether the UE's radio connection with the base station 102 remains in sync, falls out of sync, or falls out of sync but is capable of being restored. The neural network 102 may then provide its inference to the UE, which may take corrective action if necessary. In at least one embodiment, by inferring a predicted state of a radio connection for a future time, the UE 101 may avoid costs associated with current RLF procedures, provide a stable radio connection, reduce bandwidth usage, reduce computing resource usage, and / or save energy.

[0016] In at least one embodiment, base station 102 is a transceiver that is a communication point for one or more UEs, such as UE 101. In at least one embodiment, base station 102 can transmit and receive wireless signals. In at least one embodiment, base station 102 includes one or more radio frequency (RF) antennas to transmit and receive RF signals to other devices, such as UE 101. In at least one embodiment, base station 102 can transmit and receive 5G NR signals, including reference signals.

[0017] In at least one embodiment, neural network 102 includes a convolutional neural network (CNN), a transform, an encoder, a decoder, a recurrent neural network (RNN), a long short-term memory (LSTM), a radial basis set network (RBN), an autoencoder (AE), a Boltzmann machine (BM), a constrained Boltzmann machine (RBM), a deep belief network (DBN), a deep convolutional network (DCN), an extreme learning machine (ELM), a deep residual network (DRN), and / or any combination thereof. In at least one embodiment, one or more reference signals are one or more inputs to neural network 102, and a predicted quality of one or more wireless signals is one or more outputs of neural network 102.In at least one embodiment, a processor 105 may execute, use, train, or otherwise the neural network 102 to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0018] In at least one embodiment, a processor 105 uses, implements, or performs radio link monitoring 103, which is software, a module, circuitry, logic, or a combination thereof. In at least one embodiment, a processor 105 uses radio link monitoring 103 for radio link monitoring (RLM) on the active DL-BWP of the primary serving cell (PCell) of the master cell group (MCG). When the UE is configured with a secondary cell group (SCG), the UE also monitors the downlink radio link quality on the active DL-BWP of the primary SCG cell (PSCell). In at least one embodiment, the radio link monitoring 103 includes software for performing a physical layer monitoring the DL radio link quality and / or sending measurement results to upper layers.In at least one embodiment, the radio link monitor includes software for performing or using a MAC layer that includes beam failure detection and restoration. In at least one embodiment, the radio link monitor includes an RRC layer and is configured to perform PHY and MAC layer, radio link failure detection, and RRC setup.

[0019] In at least one embodiment, the reference signal manager 104 includes software, logic, circuitry, or a combination thereof for generating, receiving, and analyzing reference signals. In at least one embodiment, a reference signal is a signal that does not contain user data (e.g., data transmitted by a software program, such as video content) but is used for the purpose of configuring and maintaining a wireless connection between a base station and a UE. In at least one embodiment, there are different types of reference signals used for different purposes, such as synchronization and determining information about the channel in use (e.g., effects of interference).In at least one embodiment, PDSCH in 5G-NR is the physical downlink channel carrying user data, and DM-RS and PT-RS are the reference signals associated with PDSCH, with the reference signal manager 104 receiving, analyzing, or otherwise using these signals within the PDSCH allocation, as defined in TS 38.211 Sections 7.4. In at least one embodiment, a processor 105 uses the reference signal manager 104 to receive, analyze, or otherwise use reference signals that include predefined signals occupying specific resource elements within the downlink time-frequency grid (e.g., within a 5G-NR network).

[0020] In at least one embodiment, processor 105 includes one or more circuits for using one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. In at least one embodiment, processor 105 is or includes the processors Fig. 21A to SM 3600.

[0021] Fig. 2 illustrates a schematic overview of how radio link quality predictions are generated, according to at least one embodiment. Fig. 2 illustrates an example 200 of radio link monitoring and predicted radio link state. In at least one embodiment, radio link monitoring 103 of UE 101 monitors (e.g., receives, analyzes) one or more reference signals 201a-201n (e.g., a radio link monitoring reference signal (RLM-RS) resource). In at least one embodiment, the RLM-RS resources may be synchronization signal blocks (SSBs), channel state information reference signals (CSI-RSs), or a mixture of SSBs and CSI-RSs. In at least one embodiment, reference signals 201a-201n are collected at respective past times t[-P]-t[0]. In at least one embodiment, reference signals are collected during measurement window 205. For example, radio link monitoring 103 collects a respective reference signal 201 at past time t[-P].Although only three points in time are shown, the radio link monitor 103 may collect any number of reference signals. In at least one embodiment, the radio link monitor 103 transmits each reference signal to the neural network 102. In at least one embodiment, the neural network 102 infers, based on the reference signals (e.g., 201a-201n), whether the radio link of the UE 101 will experience consecutive out-of-sync events 202-204 (e.g., N310 consecutive out-of-sync events) within a prediction window 206.

[0022] In at least one embodiment, if the neural network 102 predicts that there will be consecutive out-of-sync events 202-204 at future times within a prediction window 206, the UE 101 assumes that the hypothetical timer 207 (e.g., timer T310) would start after the occurrence of consecutive out-of-sync events 202-204. In at least one embodiment, the timer 207 is predicted to start at time 208 and stop at time 209. In at least one embodiment, the timer 207 is predicted to expire at time 210.

[0023] In at least one embodiment, the neural network 102 predicts whether or not there will be consecutive in-sync events 211-213 (e.g., N311 In-Sync) at future times within the prediction window 214 that fall within the duration of the hypothetical runtime of the timer 207.

[0024] In at least one embodiment, if the neural network 102 predicts that there will be consecutive out-of-sync events 202-204 at future times N310, and further predicts that there will be consecutive in-sync events 206 thereafter within the duration of the timer T310 N311, the UE 101 considers the radio connection to become unreliable but will recover and thus takes no further action.

[0025] In at least one embodiment, if the neural network 102 predicts that there will be consecutive out-of-sync events 204-206 at future times, and further predicts that there will be no consecutive in-sync events 211-213 thereafter within the duration of the timer 207, the UE 101 may take one or more of the following actions: The UE 101 notifies its serving base station 102 of the UE's prediction that the UE's radio link would fail; the UE 101 notifies its serving base station 102 of the UE's prediction that the UE's radio link would fail, with an associated probability indicating how likely the radio link would fail;The UE 101 notifies its serving base station 102 of the UE's confidence level for the UE's prediction that the UE's radio link would fail and / or the associated probability indicating how likely the radio link would fail; The UE 101 sends a measurement report to its serving base station 102 including measurement results of reference signals from serving and neighboring cells; The UE 101 requests its serving base station 102 to handover the UE 101's connection from the current serving cell to another serving cell that will provide better radio link quality; The UE 101 autonomously initiates a handover procedure by performing an access procedure to another cell that will provide better radio link quality; The UE 101 initiates the RRC connection re-establishment procedure;and the UE 101 performs the actions when transitioning to RRC_IDLE.;

[0026] In at least one embodiment, the neural network 102 predicts that there will be a number of consecutive out-of-sync events for a downlink radio connection at a number of consecutive future times if the neural network predicts that the radio link quality is worse than the threshold (e.g., Qout) for all resources in the set of resources for the radio link prediction at each of the corresponding consecutive future times. In at least one embodiment, a threshold is defined as a level at which a downlink radio connection cannot be reliably received and should correspond to an out-of-sync block error rate (BLERout).In at least one embodiment, a threshold is derived based on hypothetical transmission parameters of the physical downlink control channel (PDCCH) for the out-of-sync evaluation associated with the corresponding type of RLM-RS, e.g., the value of BLERout is 10%.

[0027] Fig. 3 is a process flow diagram 300 for generating a prediction during radio link quality monitoring according to at least one embodiment. In at least one embodiment, Fig. 1-2 to perform part or all of the process 300, e.g., the processor 105 executes Fig. 1 performs the process 300 using the neural network 102. In at least one embodiment, a processor including one or more circuits, by performing the processor 300, uses one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. In at least one embodiment, in the Fig. 1-2 may perform part or all of the process 300 or be integrated into the process 300. In at least one embodiment, the process 300 may be performed concurrently or sequentially with the system disclosed in Fig. 4. In at least one embodiment, the Fig. 8-41B disclosed systems and processors perform part or all of the process 300.

[0028] In at least one embodiment, part or all of the process 300 (or any other processes described herein or variations and / or combinations thereof) is performed under the control of one or more computer systems configured with computer-executable instructions, and is implemented as code (e.g., computer-executable instructions, one or more computer programs, or one or more applications) that executes collectively on one or more processors by hardware, software, or combinations thereof. In at least one embodiment, the process 300 is implemented by Fig. 1-2, such as processor 105. In at least one embodiment, code is stored on a computer-readable storage medium in the form of a computer program comprising a plurality of computer-readable instructions executable by one or more processors. In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable medium. In at least one embodiment, at least some computer-readable instructions usable for performing process 300 are not stored exclusively using transient signals (e.g., a propagating transient electrical or electromagnetic transmission). In at least one embodiment, a non-transitory computer-readable medium does not necessarily include non-transitory data storage circuitry (e.g., buffers, caches, and queues) within transient signal transceivers.In at least one embodiment, process 300 is performed at least partially on a computer system such as those described elsewhere in this disclosure. In at least one embodiment, logic (e.g., hardware, software, or a combination of hardware and software) performs process 300. In at least one embodiment, process 500 may begin at receive operation 305 and proceed to generate prediction operation 310.

[0029] In receive operation 305, a processor in a UE receives a reference signal. For example, a mobile device using a 5G network may receive a clock synchronization signal. In at least one embodiment, a reference signal is a signal that does not contain user data (e.g., data transmitted by a software program, such as video content) but is used for the purpose of configuring and maintaining a wireless connection between a base station and a UE. In at least one embodiment, there are different types of reference signals used for different purposes, such as synchronization and determining information about the channel in use (e.g., effects of interference). In at least one embodiment, a UE may receive multiple reference signals.In at least one embodiment, a UE may receive a reference signal in the downlink path and then transmit results of receiving the reference signal to a base station. In at least one embodiment, a processor receiving a reference signal occurs during a measurement window, as shown in FIG. Fig. 2 shown.

[0030] In generation operation 310, a processor provides the received reference signal or signals as input to a neural network that generates a prediction based on the reference signal. In at least one embodiment, a processor executing a neural network generates a prediction that a radio link quality is good, high, or above a threshold (e.g., the BLER is less than 2% or 98% of packets can be decoded). In at least one embodiment, a processor executing a neural network generates a prediction that a radio link will have multiple out-of-sync messages (e.g., 10 to 100) or will be out-of-sync for a number of frames (e.g., 10 consecutive frames). In at least one embodiment, a processor executing a neural network generates a prediction that out-of-sync messages will only occur for a few frames or for a short period of time (e.g.,25 milliseconds).

[0031] In the determining response operation 315, a processor, e.g., of a UE 101, generates a response based on the prediction operation 315 (e.g., output of a neural network). In at least one embodiment, the prediction may be provided to a controller or a UE, or may be provided to a base station. For example, a UE notifies its serving base station 102 of the UE's prediction that the UE's radio link would fail. As another example, the UE 101 notifies its serving base station 102 of the UE's prediction that the UE's radio link would fail, with an associated probability indicating how likely the radio link would fail, and takes corrective action, such as restoring a radio link or adjusting parameters.As another example, the UE 101 notifies its serving base station 102 of the UE's confidence level about the UE's prediction that the UE's radio link would fail and / or the associated probability indicating how likely the radio link would fail. As another example, the UE 101 sends to its serving base station 102 a measurement report including measurement results of reference signals from serving and neighboring cells. As another example, the UE 101 requests its serving base station 102 to hand over the UE's 101 connection from a current serving cell to another serving cell that will provide better radio link quality. As another example, the UE 101 autonomously initiates a handover procedure by executing an access procedure to another cell that will provide better radio link quality.As another example, the UE 101 initiates the RRC connection recovery procedure; and the UE 101 executes the actions when it goes to RRC_IDLE.

[0032] At decision operation 320, a processor determines whether it should stop process 320 based on whether it is necessary to continue monitoring the radio link. Based on information from operations 310 and 315, a processor may receive a notification about how long a radio link is predicted to be of good quality or of poor quality (e.g., at or above a 5G stability threshold). In at least one embodiment, a processor may stop measuring until a stability period expires, or it may reuse processor 300 to obtain a different prediction.

[0033] In at least one embodiment, after decision operation 320, a processor including one or more circuits stops or terminates process 300. In at least one embodiment, a processor including one or more circuits may continue to perform process 300. For example, a processor including one or more circuits or one or more processors may continue to monitor a radio link to determine, predict, and make adjustments to the radio link quality, if necessary.

[0034] Fig. 4 is a process flow diagram 400 for generating a prediction for a radio link and using that prediction to perform a response action, according to at least one embodiment. In at least one embodiment, Fig. 1-3 to perform part or all of the process 400, e.g., the processor 105 performs the process 400. In at least one embodiment, a processor including one or more circuits, by performing the processor 400, uses one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. In at least one embodiment, in the Fig. 1-2 may perform part or all of the process 300 or be integrated into the process 300. In at least one embodiment, the process 400 may be performed concurrently or sequentially with the process 400 disclosed in Fig. 3. In at least one embodiment, the Fig. 8-41B disclosed systems and processors perform part or all of the process 400.

[0035] In at least one embodiment, part or all of the process 400 (or any other processes described herein or variations and / or combinations thereof) is performed under the control of one or more computer systems configured with computer-executable instructions, and is implemented as code (e.g., computer-executable instructions, one or more computer programs, or one or more applications) that executes collectively on one or more processors by hardware, software, or combinations thereof. In at least one embodiment, the process 400 is implemented by Fig. 1-2, such as processor 105. In at least one embodiment, code is stored on a computer-readable storage medium in the form of a computer program comprising a plurality of computer-readable instructions executable by one or more processors. In at least one embodiment, a computer-readable storage medium is a non-transitory computer-readable medium. In at least one embodiment, at least some computer-readable instructions usable for performing process 400 are not stored exclusively using transient signals (e.g., a propagating transient electrical or electromagnetic transmission). In at least one embodiment, a non-transitory computer-readable medium does not necessarily include non-transitory data storage circuitry (e.g., buffers, caches, and queues) within transient signal transceivers.In at least one embodiment, process 400 is performed at least partially on a computer system such as those described elsewhere in this disclosure. In at least one embodiment, logic (e.g., hardware, software, or a combination of hardware and software) performs process 400. In at least one embodiment, process 400 may begin at receive operation 405 and proceed to generate prediction operation 410.

[0036] In receive operation 405, a processor in a UE receives a reference signal. For example, a mobile device using a 5G network may receive a clock synchronization signal. In at least one embodiment, a reference signal is a signal that does not contain user data (e.g., data transmitted by a software program, such as video content) but is used for the purpose of configuring and maintaining a wireless connection between a base station and a UE. In at least one embodiment, there are different types of reference signals used for different purposes, such as synchronization and determining information about the channel in use (e.g., effects of interference). In at least one embodiment, a UE may receive multiple reference signals.In at least one embodiment, a UE may receive a reference signal in the downlink path and then transmit results of receiving the reference signal to a base station. In at least one embodiment, a processor receiving a reference signal occurs during a measurement window, as shown in FIG. Fig. 2 shown.

[0037] At generation operation 410, a processor provides the received reference signal or signals as input to a neural network that generates a prediction based on the reference signal. In at least one embodiment, a processor executing a neural network generates a prediction that a radio link quality is good, high, or above a threshold (e.g., the BLER is less than 2% or 98% of packets can be decoded). In at least one embodiment, a processor executing a neural network generates a prediction that a radio link will have multiple out-of-sync messages (e.g., 10 to 100) or will be out-of-sync for a number of frames (e.g., 10 consecutive frames). In at least one embodiment, a processor executing a neural network generates a prediction that out-of-sync messages will only occur for a few frames or for a short period of time (e.g.,25 milliseconds).

[0038] At the determining operation 415, a processor determines that the radio link is below a threshold (e.g., more than 10 out-of-sync messages are expected to be received within the next 20 frames or milliseconds, or the BLER is predicted to be above 2% for an extended period of time).

[0039] At decision operation 420, a processor determines that the prediction below threshold values ​​will only last for a short period of time and then stability will return to the radio link, or that the sub-threshold values ​​will last for a longer period of time and corrective action is required. If a processor determines that no corrective action is required because the instability is only temporary or a timer does not expire before stability is returned, process 400 may end. If instability is predicted to last for a longer period of time, a processor may take immediate action to perform response action 425. For example, UE 101 sends to its serving base station 102 a measurement report that includes measurement results of reference signals from serving and neighboring cells.For example, the UE 101 requests its serving base station 102 to hand over the UE 101's connection from a current serving cell to another serving cell that will provide better radio link quality. As another example, the UE 101 autonomously initiates a handover procedure by executing an access procedure to another cell that will provide better radio link quality. For example, the UE 101 initiates the RRC connection reestablishment procedure.

[0040] In at least one embodiment, after decision operation 420, a processor including one or more circuits stops or terminates process 400. In at least one embodiment, a processor including one or more circuits may continue to perform process 400. For example, a processor including one or more circuits or one or more processors may continue to monitor a radio link to determine, predict, and make adjustments to the radio link quality, if necessary.

[0041] Fig. 5 is a process flow diagram for generating predictions for a radio link and using those predictions to perform a response action, according to at least one embodiment. In at least one embodiment, Fig. 1-2 are used to perform part or all of the process 300, e.g., the processor 105 performs the process 500. In at least one embodiment, a processor including one or more circuits, by performing the processor 300, uses one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. In at least one embodiment, in the Fig. 1-2 may perform part or all of the process 500 or be integrated into the process 500. In at least one embodiment, the process 300 may be performed concurrently or sequentially with the system disclosed in Fig. 4. In at least one embodiment, the Fig. 8-41B perform part or all of the process 500. In at least one embodiment, the process 500 is a specific example of processes 300 and 400. In at least one embodiment, the process 500 includes a measurement operation 505 (e.g., for measuring reference signals for radio link monitoring), a prediction operation 510 (e.g., for predicting the radio link quality at future times in a first prediction window), a decision operation 515 (e.g., for determining if an event is predicted to include consecutive out-of-sync packets or signals), if not, then the radio link is considered stable 520. If there are predicted out-of-sync messages for consecutive frames, then a neural network can predict what will happen in the future for the radio link, and a wait time or timer 525 (e.g.,future times in a second prediction window that is after the first prediction window). If a neural network predicts that an event with N311 consecutive in-sync will occur, then a processor may receive this prediction and consider that a radio link quality will become unreliable but will recover 535, e.g., a controller of the UE or base station does not need to take any action. If a processor determines that a radio link quality will become unstable and will not recover 540, a processor may send signals to a controller or base station to perform preventive actions to avoid a predicted radio link failure. For example, a processor may immediately request that a UE start a recovery process or handover process.As another example, a UE may change base stations or adjust its radio link parameters to improve radio link quality.

[0042] Fig. 6 is a schematic block diagram illustrating the use of a neural network to generate predictions about the quality of a radio link, according to at least one embodiment. Fig. 6 illustrates a reference signal-specific model 600 for predicting consecutive out-of-sync events in multiple units (e.g., milliseconds, frames) in the future. In at least one embodiment, one or more UEs 101 and base stations may be used to train reference signal resource-specific neural networks 602a-602n for out-of-sync prediction outputs 603a-603n for future times and use the out-of-sync prediction outputs 603a-603n for inference, e.g., to predict whether the radio link quality is above a threshold. In at least one embodiment, there is a specific neural network 602a-602n for each reference signal in the measurement windows 601a-601n: The input to the neural networks is the measurement at times {t[-P], ..., t[0]} based on the reference signals 201a-201n. In at least one embodiment, the neural networks 602a-602n generate predictions, e.g.Outputs 603a-603n predict that there will be a number Q of consecutive "out-of-sync" events by applying reference signal-specific neural networks 602a-602n for 1-step ahead, 2-step ahead, and so on to obtain predetermined predictions. In at least one embodiment, if the prediction outputs 603a-603n of the neural network generation, which include the link quality at all times t[1] to t[Q], are worse than Qout, a controller 604, which is software executed by a processor such as processor 105, yields an output bit equal to 1. In at least one embodiment, otherwise, an output bit equal to 0 results. In at least one embodiment, the controller 604 receives output predictions from the neural networks 602a-602n and uses these predictions to perform radio link outage operations (e.g.,Switching to another base station, reconnecting, or reconnecting to the base station). In at least one embodiment, a processor such as processor 102 provides outputs 603a-603n to a controller 604 (e.g., controller in a UE or controller in a base station), where a controller 604 is software that controls parameters, circuits, and other components of a radio link. In at least one embodiment, a controller 604 may use the outputs 603a-603n to generate control signals or adjust parameters of a UE or base station to adjust radio link quality. In at least one embodiment, a processor comprising one or more circuits is to use the predicted quality of the one or more wireless signals as inputs to a control circuit to adjust one or more radio links between a user device and a base node.

[0043] In at least one embodiment, UE 101 and / or base stations 102 are used to train a multiple reference signal input for neural networks 602a-n to predict consecutive out-of-sync events in multiple steps ahead and use them for inference. For example, for the measurement windows 601a-601n, each reference signal 201a-201n (as shown in Fig. 2) is input to a neural network 602a-n, and the neural network 602a-n predicts that there will be Q consecutive out-of-sync events by applying multiple reference signal input neural network 602a-n for 1-step ahead, 2-step ahead, and Q-step ahead predictions. In at least one embodiment, the same multiple reference signal input neural network is applied to 1-step ahead, 2-step ahead, and Q-step ahead predictions. In at least one embodiment, different multiple reference signal input neural networks are applied to different steps ahead predictions. In at least one embodiment, the neural networks 602a-n output 1 at each step t[x], x = 1, 2, 3, and up to Q bits if the prediction is an out-of-sync event, and / or the neural network 602a-n outputs 0 at each step t[x], x = 1, 2, 3, and up to Q bits if the prediction is not an out-of-sync event.In at least one embodiment, the UE 101 using neural networks infers that there will be a number of Q consecutive out-of-sync events when a neural network output is 1, and that there will be no out-of-sync events when a neural network output is 0. In at least one embodiment, the UE 101 performing the neural networks 602a-n on each RS resource 201a-201n for downlink radio link quality prediction infers the downlink radio link quality at a future time based on the neural network's prediction of whether the downlink radio link can be reliably received at the future time (e.g., whether the quality is above the threshold Qin).

[0044] In at least one embodiment, a threshold Qin corresponds to the in-sync block error rate threshold (BLERin). For example, the value of BLERin is 2%. The predicted in-sync block error rate is derived based on the hypothetical PDCCH transmission parameters for the in-sync prediction. In at least one embodiment, the set of hypothetical PDCCH transmission parameters includes one or more of the following: downlink control information (DCI) format, number of orthogonal frequency division multiplexing (OFDM) control symbols, aggregation level with respect to the number of control channel elements (CCE), ratio of the hypothetical PDCCH resource element (RE) energy to the average secondary synchronization signal (SSS) RE energy, ratio of the hypothetical PDCCH RE energy to the average CSI RS RE energy, ratio of the hypothetical PDCCH demodulation RS (DMRS) energy to the average SSS RE energy,Ratio of hypothetical PDCCH DMRS energy to average CSI-RS RE energy, bandwidth relative to the number of physical resource blocks (PRBs), subcarrier spacing, DMRS precoder granularity, resource element group (REG) bundle size, cyclic prefix (CP) length, and REG-to-CCE mapping. In at least one embodiment, for SSB-based radio link prediction, a first set of values ​​for one or more of the above hypothetical PDCCH transmission parameters is used for in-sync block error rate prediction. In at least one embodiment, for CSI-RS-based radio link prediction, a second set of values ​​for one or more of the above hypothetical PDCCH transmission parameters is used for in-sync block error rate prediction.

[0045] In at least one embodiment, the neural networks 602a-n predict that there will be an in-sync event for the downlink radio connection at a future time if the neural network 102 predicts that the radio link quality is better than the threshold Qin for each resource in the set of resources for the radio link prediction.

[0046] In at least one embodiment, neural network 102 assumes that timer T310 would hypothetically start after the occurrence of N310 consecutive out-of-sync events. Neural network 102 then further predicts whether or not there will be N311 consecutive in-sync events at future times that fall within the duration of the hypothetical runtime of T310.

[0047] In at least one embodiment, if the neural network predicts that there will be consecutive out-of-sync (e.g., N310) at future times, and further predicts that there will be consecutive in-sync (e.g., N311) thereafter within the duration of timer T310, the UE 101 considers the radio link to become unreliable but will recover and thus takes no further action at that moment.

[0048] In at least one embodiment, the training data set for the neural networks 602a-n may include one or more prediction windows corresponding to one or more reference signals (e.g., 201a-201n) and the state of the radio link with labels (e.g., high, low, medium, poor, or good radio link quality). In at least one embodiment, a neural network (e.g., the neural network 102) is trained by a processor using labeled training, which includes one or more prediction windows corresponding to one or more reference signals (e.g., 201a-201n) and the state of the radio link with labels (e.g., high, low, medium, poor, or good radio link quality).

[0049] In at least one embodiment, the training dataset for each RLM-RS resource includes thresholds such as Qout and Qin. Qout is a threshold at which the radio link quality cannot be reliably received and corresponds to the out-of-sync error rate (BLERout). In at least one embodiment, Qout is based on the transmission parameters of the physical downlink control channel (PDCCH) for out-of-sync evaluation associated with the corresponding RLM-RS type. For example, a BLERout threshold is 10%.

[0050] In at least one embodiment, Qin is a threshold level at which the radio link quality can be received with higher reliability than Qout. Qin corresponds to the in-sync block error rate (BLERin). The Qin threshold is based on the PDCCH transmission parameters for in-sync evaluation associated with the corresponding type of RLM-RS. For example, a BLERin threshold is 2%. Exceeding Qin or exceeding the Qin threshold indicates that the data transmitted on the radio link can be reliably received. Exceeding Qin or exceeding the Qin threshold also indicates that the radio link between the UE and the base station is in-sync.

[0051] In at least one embodiment, the training data set includes constant N310, where constant N310 can be used to evaluate a maximum number of consecutive out-of-sync or early out-of-sync indications for a primary serving cell (PCell) received from lower layers. In at least one embodiment, upon receiving a threshold number of consecutive N310 out-of-sync indications from lower layers, a UE would start timer T310 for the corresponding cell. Upon expiration of timer T310, a UE would consider a radio link failure detected for the cell, and a UE can then either go to RRC_IDLE or initiate the link restoration procedure.

[0052] In at least one embodiment, the training data set includes constant N311, where constant N311 evaluates a maximum number of consecutive in-sync indications for the PCell received from lower layers. In at least one embodiment, upon receiving N311 consecutive "in-sync" indications for the cell from lower layers while T310 is running, a UE may pause timer T310 for the corresponding cell. In at least one embodiment, a UE does not declare a radio link failure because the N311 consecutive "in-sync" indications imply that the downlink radio connection has recovered to a satisfactory level.

[0053] In at least one embodiment, a processor trains the neural network on training data that includes one or more RLM-RS resources.

[0054] In at least one embodiment, for the SSB-based radio link prediction, a first set of values ​​for one or more of the above hypothetical PDCCH transmission parameters is used for the out-of-sync block error rate prediction.

[0055] In at least one embodiment, for the CSI-RS-based radio link prediction, a second set of values ​​for one or more of the above hypothetical PDCCH transmission parameters is used for the out-of-sync block error rate prediction.

[0056] Fig. 7 is a block diagram illustrating a driver and / or runtime including one or more libraries to provide one or more application programming interfaces (APIs), according to at least one embodiment. In at least one embodiment, software program 702 may be executed by components, systems, or other processes included in Fig. 1- Fig. 7, for example, the processor 105 may execute an API to access, use, or otherwise implement the neural network 102. In at least one embodiment, a software program 702 is a software module. In at least one embodiment, a software program 702 includes one or more software modules. In at least one embodiment, one or more software modules are further not exclusively in Fig. 7. In at least one embodiment, one or more APIs 710 are sets of software instructions that, when executed, cause one or more processors to perform one or more computational operations. In at least one embodiment, one or more APIs 710 are distributed or otherwise provided as part of one or more libraries 706, runtimes 704, drivers 704, and / or any other grouping of software and / or executable code further described herein. In at least one embodiment, one or more APIs 710 perform one or more computational operations in response to being invoked by the software programs 702.In at least one embodiment, a software program 702 is a collection of software code, commands, instructions, or other text sequences to instruct a computing device to perform one or more computational operations and / or to invoke one or more other sets of instructions, such as APIs 710 or API functions 712, to be executed. In at least one embodiment, the functionality provided by one or more APIs 710 includes software functions 712, such as those that can be used to accelerate one or more portions of software programs 702 using one or more parallel processing units (PPUs), such as graphics processing units (GPUs).

[0057] In at least one embodiment, APIs 710 are hardware interfaces to one or more circuits to perform one or more computational operations. In at least one embodiment, one or more software APIs 710 described herein are implemented as one or more circuits to perform one or more techniques described below in connection with Fig. 2-6. In at least one embodiment, one or more software programs 702 include instructions that, when executed, cause one or more hardware devices and / or circuits to perform one or more techniques described below in connection with Fig. 2-6 are further described.

[0058] In at least one embodiment, software programs 702, such as user-implemented software programs, utilize one or more application programming interfaces (APIs) 710 to perform various computational operations, such as memory allocation, matrix multiplication, arithmetic operations, or any computational operation performed by parallel processing units (PPUs), such as graphics processing units (GPUs), as further described herein. In at least one embodiment, one or more APIs 710 provide a set of callable functions 712, referred to herein as APIs, API functions, and / or functions, that individually perform one or more computational operations, such as computational operations related to parallel computing.For example, in one embodiment, one or more APIs 710 provide functions 712 to cause a scheduler 716 to order one or more jobs in a scheduling queue based on credits associated with those jobs.

[0059] In at least one embodiment, one or more software programs 702 interact with or otherwise communicate with one or more APIs 710 to perform one or more computational operations using one or more PPUs, such as GPUs. In at least one embodiment, one or more computational operations using one or more PPUs include at least one or more groups of computational operations to be accelerated by execution at least in part by the one or more PPUs. In at least one embodiment, one or more software programs 702 interact with one or more APIs 710 to enable parallel computing using a remote or local interface.

[0060] In at least one embodiment, an interface is software instructions that, when executed, provide access to one or more functions 712 provided by one or more APIs 710. In at least one embodiment, a software program 702 uses a local interface when a software developer compiles one or more software programs 702 in conjunction with one or more libraries 706 that include or otherwise provide access to one or more APIs 710. In at least one embodiment, one or more software programs 702 are statically compiled in conjunction with precompiled libraries 706 or uncompiled source code that includes instructions for performing one or more APIs 710.In at least one embodiment, one or more software programs 702 are dynamically compiled, and the one or more software programs use a linker to link with one or more precompiled libraries 706 that include one or more APIs 710.

[0061] In at least one embodiment, a software program 702 uses a remote interface when a software developer executes a software program that uses or otherwise communicates with a library 706 comprising one or more APIs 710 over a network or other remote communication medium. In at least one embodiment, one or more libraries 706 comprising one or more APIs 710 are to be executed by a remote computing service, such as a computing resource service provider. In another embodiment, one or more libraries 706 comprising one or more APIs 710 are to be executed by any other computing host that provides the one or more APIs 710 to one or more software programs 702.

[0062] In at least one embodiment, a processor executing or using one or more software programs 702 calls, uses, executes, or otherwise implements one or more APIs 710 to allocate and otherwise manage memory to be used by the software programs 702. In at least one embodiment, one or more software programs 702 use one or more APIs 710 to allocate and otherwise manage memory to be used by one or more portions of the software programs 702 to be accelerated using one or more PPUs, such as GPUs, or any other accelerator or processor further described herein.These software programs 702 request a neural network to perform image generation, feature information generation, and / or depth information generation using functions 712 provided, in one embodiment, by one or more APIs 710.

[0063] In at least one embodiment, an API 710 is an API to enable parallel computing. In at least one embodiment, an API 710 is any other API further described herein. In at least one embodiment, an API 710 is provided by a driver and / or a runtime 704. In at least one embodiment, an API 710 is provided by a CUDA user-mode driver. In at least one embodiment, an API 710 is provided by a CUDA runtime. In at least one embodiment, a driver 704 is data values ​​and software instructions that, when executed, perform or otherwise enable the operation of one or more functions 712 of an API 710 during the loading and execution of one or more portions of a software program 702.In at least one embodiment, a runtime 704 is data values ​​and software instructions that, when executed, perform or otherwise enable the operation of one or more functions 712 of an API 710 during execution of a software program 702. In at least one embodiment, one or more software programs 702 utilize one or more APIs 710 implemented or otherwise provided by a driver and / or a runtime 704 to perform combined arithmetic operations by the one or more software programs 702 during execution by one or more PPUs, such as GPUs.

[0064] In at least one embodiment, one or more software programs 702 utilize one or more APIs 710 provided by a driver and / or a runtime 704 to perform combined arithmetic operations of one or more PPUs, such as GPUs. In at least one embodiment, one or more APIs 710 provide combined arithmetic operations through a driver and / or a runtime 704, as described above. In at least one embodiment, one or more software programs 702 utilize one or more APIs 710 provided by a driver and / or a runtime 704 to allocate or otherwise reserve one or more memory blocks 714 to one or more PPUs, such as GPUs.In at least one embodiment, one or more software programs 702 utilize one or more APIs 710 provided by a driver and / or runtime 704 to allocate or otherwise reserve memory blocks. In at least one embodiment, one or more APIs 710 are intended to perform combined arithmetic operations, as described below in connection with any of the . Fig. 1-6 described.

[0065] To improve the usability and / or optimization of software programs 702 of one or more portions of the software programs 702 to be accelerated by one or more PPUs, such as GPUs, in one embodiment, one or more APIs 710 provide one or more API functions 712 to perform a neural network executed by one or more computing devices, as described above and below in connection with Fig. 1-6 is further described, used, or usable. In at least one embodiment, an example block diagram 700 illustrates a processor including one or more circuits to execute one or more software programs, to combine two or more application programming interfaces (APIs) into a single API. In at least one embodiment, an example block diagram 700 illustrates a system including one or more processors to execute one or more software programs to combine two or more application programming interfaces (APIs) into a single API. In at least one embodiment, a processor uses an API to cause a neural network to select a thread selection mechanism, and / or otherwise perform operations described herein.In at least one embodiment, an example block diagram 700 illustrates an API for invoking a neural network to cause image generation.

[0066] In at least one embodiment, a processor uses an example API to invoke one or more neural networks, the processor including circuitry to use one or more first neural networks to generate one or more first images from a viewpoint external to an object based at least in part on depth information generated using one or more second neural networks and feature information generated by one or more third neural networks and / or to otherwise perform operations described herein. LOGIC FOR NEURAL NETWORKS

[0067] Fig. 8A illustrates logic 815, which, as described elsewhere herein, may be used in one or more devices to perform operations such as those discussed herein, according to at least one embodiment. In at least one embodiment, logic 815 is used to perform reasoning and / or training operations associated with one or more embodiments. In at least one embodiment, logic 815 is reasoning and / or training logic. Details regarding logic 815 are described below in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic refers to any combination of software logic, hardware logic, and / or firmware logic to provide the functionality or operations described herein, where logic, collectively or individually, may be embodied as circuitry that forms part of a larger system, for example, an integrated circuit (IC), system-on-chip (SoC), or one or more processors (e.g., CPU, GPU).

[0068] In at least one embodiment, logic 815 may include, without limitation, code and / or data storage 801 for storing feedforward and / or output weights and / or input / output data and / or other parameters for configuring neurons or layers of a neural network that are trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, logic 815 may include or be coupled to code and / or data storage 801 for storing graph code or other software for controlling timing and / or sequencing, wherein weight and / or other parameter information is to be loaded for configuring logic, including integer and / or floating-point units (collectively, arithmetic logic units (ALUs)).In at least one embodiment, code, such as graph code, loads weight or other parameter information into processor ALUs based on a neural network architecture to which that code corresponds. In at least one embodiment, code and / or data storage 801 stores weight parameters and / or input / output data of each layer of a neural network trained or used in connection with one or more embodiments during forward propagation of input / output data and / or weight parameters during training and / or inference using aspects of one or more embodiments. In at least one embodiment, any portion of code and / or data storage 801 may be included in other on-chip or off-chip data storage, including a processor's L1, L2, or L3 cache or system memory.

[0069] In at least one embodiment, any portion of code and / or data storage 801 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and / or data storage 801 may be cache memory, dynamic random access memory ("DRAM"), static random access memory ("SRAM"), non-volatile memory (e.g., flash memory), or other memory.In at least one embodiment, a choice of whether code and / or data storage 801 is, for example, internal or external to a processor or comprises DRAM, SRAM, flash, or another memory type may depend on available on-chip versus off-chip memory, latency requirements of training and / or inference functions being performed, batch size of data used in inference and / or training of a neural network, or a combination of these factors.

[0070] In at least one embodiment, logic 815 may include, without limitation, code and / or data storage 805 for storing backward and / or output weights and / or input / output data corresponding to neurons or layers of a neural network trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, code and / or data storage 805 stores weight parameters and / or input / output data of each layer of a neural network trained or used in connection with one or more embodiments during backpropagation of input / output data and / or weight parameters during training and / or inference using aspects of one or more embodiments.In at least one embodiment, logic 815 may include or be coupled to code and / or data storage 805 for storing graph code or other software for controlling timing and / or sequencing, wherein weight and / or other parameter information is to be loaded for configuring logic, including integer and / or floating point units (collectively, arithmetic logic units (ALUs)).

[0071] In at least one embodiment, code, such as graph code, causes weight or other parameter information to be loaded into processor ALUs based on a neural network architecture to which that code conforms. In at least one embodiment, any portion of code and / or data storage 805 may be included in other on-chip or off-chip data storage, including a processor's L1, L2, or L3 cache or system memory. In at least one embodiment, any portion of code and / or data storage 805 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, code and / or data storage 805 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other memory.In at least one embodiment, a choice of whether code and / or data storage 805 is, for example, internal or external to a processor or comprises DRAM, SRAM, flash, or another memory type may depend on available on-chip versus off-chip memory, latency requirements of training and / or inference functions being performed, batch size of data used in inference and / or training of a neural network, or a combination of these factors.

[0072] In at least one embodiment, code and / or data storage 801 and code and / or data storage 805 may be separate memory structures. In at least one embodiment, code and / or data storage 801 and code and / or data storage 805 may be a combined memory structure. In at least one embodiment, code and / or data storage 801 and code and / or data storage 805 may be partially combined and partially separate. In at least one embodiment, any portion of code and / or data storage 801 and code and / or data storage 805 may be included in other on-chip or off-chip data storage, including a processor's L1, L2, or L3 cache or system memory.

[0073] In at least one embodiment, logic 815 may include, without limitation, one or more arithmetic logic unit(s) ("ALU(s)") 810, including integer and / or floating-point units, to perform logical and / or mathematical operations based at least in part on or indicated by training and / or reasoning code (e.g., graph code), a result of which may generate activations (e.g., output values ​​of layers or neurons within a neural network) stored in an activation store 820 that are functions of input / output and / or weight parameter data stored in code and / or data store 801 and / or code and / or data store 805.In at least one embodiment, activations stored in activation memory 820 are generated according to linear algebraic and / or matrix-based mathematics performed by ALU(s) 810 in response to the execution of instructions or other code, wherein weight values ​​stored in code and / or data memory 805 and / or data memory 801 are used as operands along with other values, such as deviation values, gradient information, momentum values, or other parameters or hyperparameters, any or all of which may be stored in code and / or data memory 805 or code and / or data memory 801 or other on-chip or off-chip memory.

[0074] In at least one embodiment, ALU(s) 810 are included in one or more processors or other hardware logic devices or circuits, whereas in another embodiment, ALU(s) 810 may be external to a processor or other hardware logic device or circuit that uses them (e.g., a coprocessor). In at least one embodiment, ALU(s) 810 may be included in the execution units of a processor or otherwise in a bank of ALUs that are accessible by the execution units of a processor, either within the same processor or distributed among different processors of different types (e.g., central processing units, graphics processing units, fixed functional units, etc.).In at least one embodiment, code and / or data storage 801, code and / or data storage 805, and activation memory 820 may share a processor or other hardware logic device or circuit, whereas in another embodiment, they may be in different processors or other hardware logic devices or circuits, or a combination of the same and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of activation memory 820 may be included in other on-chip or off-chip data storage, including a processor's L1, L2, or L3 cache or system memory.Furthermore, inference and / or training code may be stored with other code that is accessible by a processor or other hardware logic or circuitry and that is retrieved and / or processed using the fetch, decode, scheduling, execution, retirement, and / or other logic circuitry of a processor.

[0075] In at least one embodiment, activation memory 820 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other memory. In at least one embodiment, activation memory 820 may be entirely or partially internal or external to one or more processors or other logic circuits. In at least one embodiment, a choice of whether activation memory 820 is, for example, internal or external to a processor, or comprises DRAM, SRAM, flash, or another memory type, may depend on available on-chip versus off-chip memory, latency requirements of training and / or inference functions being performed, batch size of data used in inference and / or training of a neural network, or a combination of these factors.

[0076] In at least one embodiment, the Fig. 8A may be used in conjunction with an application-specific integrated circuit (“ASIC”), such as a TensorFlowe processing unit from Google, an inference processing unit (IPU) from Graphcore™, or a Nervana® processor (e.g., “Lake Crest”) from Intel Corp. In at least one embodiment, the logic 815 illustrated in Fig. 8A may be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware, or other hardware such as field programmable gate arrays (“FPGAs”).

[0077] Fig. 8B illustrates logic 815 according to at least one embodiment. In at least one embodiment, logic 815 is reasoning and / or training logic. In at least one embodiment, logic 815 may include, without limitation, hardware logic in which computational resources are dedicated or otherwise used exclusively in connection with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. In at least one embodiment, the logic illustrated in Fig. 8B may be used in conjunction with an application-specific integrated circuit (ASIC), such as a TensorFlow® processing unit from Google, an inference processing unit (IPU) from Graphcore™, or a Nervana® processor (e.g., “Lake Crest”) from Intel Corp. In at least one embodiment, the logic 815 illustrated in Fig. 8B may be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware, such as field-programmable gate arrays (FPGAs). In at least one embodiment, logic 815 includes, without limitation, code and / or data storage 801 and code and / or data storage 805, which may be used to store code (e.g., graph code), weight values, and / or other information, including deviation values, gradient information, pulse values, and / or other parameter or hyperparameter information. In at least one embodiment, Fig. 8B, each of code and / or data memory 801 and code and / or data memory 805 is associated with a dedicated computing resource, such as computing hardware 802 and computing hardware 806, respectively. In at least one embodiment, each of computing hardware 802 and computing hardware 806 includes one or more ALUs that perform mathematical functions, such as linear algebraic functions, only on information stored in code and / or data memory 801 and code and / or data memory 805, respectively, the result of which is stored in activation memory 820.

[0078] In at least one embodiment, each of code and / or data storage 801 and 805, or corresponding compute hardware 802 and 806, corresponds to different layers of a neural network, such that the resulting activation from one memory / compute pair 801 / 802 of code and / or data storage 801 and compute hardware 802 is provided as an input to a next memory / compute pair 805 / 806 of code and / or data storage 805 and compute hardware 806 to mirror a conceptual organization of a neural network. In at least one embodiment, each of memory / compute pairs 801 / 802 and 805 / 806 may correspond to more than one layer of a neural network. In at least one embodiment, additional memory / compute pairs (not shown) may be included after or in parallel with memory / compute pairs 801 / 802 and 805 / 806 in logic 815. TRAINING AND DEPLOYMENT OF A NEURAL NETWORK

[0079] Fig. 9 illustrates the training and deployment of a deep neural network according to at least one embodiment. In at least one embodiment, the untrained neural network 906 is trained using a training data set 902. In at least one embodiment, the training data set 902 includes a database of reference signals. The database of reference signals includes one or more radio link monitoring reference signal (RLM-RS) resources. The RLM-RS resources may include synchronization signal blocks (SSBs) or all channel state information reference signals (CSI-RSs), or a mixture of SSBs and CSI-RSs. The training data set 902 may also include Qout and Qin threshold levels corresponding to the state of the radio link. The state of the radio link may be either out-of-syn or in-syn.The training data set 902 may also include one or more prediction windows corresponding to one or more reference signals and the state of the radio link.

[0080] In at least one embodiment, the training framework 904 is a PyTorch framework, whereas in other embodiments, the training framework 904 is a TensorFlow, Boost, Caffe, Microsoft Cognitive Toolkit / CNTK, MXNet, Chainer, Keras, Deeplearning4j, or another training framework. In at least one embodiment, the training framework 904 trains an untrained neural network 906 and allows it to be trained using processing resources described herein to produce a trained neural network 908. In at least one embodiment, weights may be selected randomly or by pre-training using a deep belief network. In at least one embodiment, the training may be performed in either a supervised, semi-supervised, or unsupervised manner.

[0081] In at least one embodiment, the untrained neural network 906 is trained using supervised learning, where the training data set 902 includes an input paired with a desired output for an input, or where the training data set 902 includes an input with a known output and an output of the neural network 906 is manually ranked. In at least one embodiment, the untrained neural network 906 is trained in a supervised manner and processes inputs from the training data set 902 and compares resulting outputs to a set of expected or desired outputs. In at least one embodiment, errors are then backpropagated through the untrained neural network 906. In at least one embodiment, the training framework 904 adjusts weights that control the untrained neural network 906.In at least one embodiment, the training framework 904 includes tools to monitor how well the untrained neural network 906 converges on a model, such as the trained neural network 908, that is capable of generating correct answers, such as in the result 914, based on input data, such as a new dataset 912. In at least one embodiment, the training framework 904 repeatedly trains the untrained neural network 906 while adjusting weights to refine an output of the untrained neural network 906 using a loss function and an adaptation algorithm, such as stochastic gradient descent. In at least one embodiment, the training framework 904 trains the untrained neural network 906 until the untrained neural network 906 achieves a desired accuracy.In at least one embodiment, the trained neural network 908 may then be used to implement any number of machine learning operations.

[0082] In at least one embodiment, the untrained neural network 906 is trained using unsupervised learning, where the untrained neural network 906 attempts to train itself using unlabeled data. In at least one embodiment, the training dataset 902 for unsupervised learning includes input data without associated output data or ground truth data. In at least one embodiment, the untrained neural network 906 can learn groupings within the training dataset 902 and determine how individual inputs relate to the untrained dataset 902. In at least one embodiment, the unsupervised training can be used to generate a self-organizing map in the trained neural network 908 capable of performing operations useful in reducing the dimensionality of the new dataset 912.In at least one embodiment, the unsupervised training may also be used to perform anomaly detection, which enables identification of data points in the new data set 912 that deviate from normal patterns of the new data set 912.

[0083] In at least one embodiment, semi-supervised learning may be used, which is a technique in which the training dataset 902 includes a mixture of labeled and unlabeled data. In at least one embodiment, the training framework 904 may be used to perform incremental learning, such as through transfer learning techniques. In at least one embodiment, incremental learning allows the trained neural network 908 to adapt to the new dataset 912 without forgetting knowledge embedded in the trained neural network 908 during initial training.

[0084] In at least one embodiment, the training framework 904 is a framework processed in conjunction with a software development toolkit, such as an OpenVINO (Open Visual Inference and Neural Network Optimization) toolkit. In at least one embodiment, an OpenVINO toolkit is a toolkit such as those developed by Intel Corporation of Santa Clara, California. In at least one embodiment, OpenVINO includes logic 815 or uses logic 815 to perform operations described herein. In at least one embodiment, an SoC, integrated circuit, or processor uses OpenVINO to perform operations described herein.

[0085] In at least one embodiment, OpenVINO is a toolkit for facilitating the development of applications, particularly neural network applications, for various tasks and operations, such as human-vision emulation, speech recognition, natural language processing, recommender systems, and / or variations thereof. In at least one embodiment, OpenVINO supports neural networks, such as convolutional neural networks, recurrent and / or attention-based neural networks, and / or various other neural network models. In at least one embodiment, OpenVINO supports various software libraries, such as OpenCV, OpenCL, and / or variations thereof.

[0086] In at least one embodiment, OpenVINO supports neural network models for various tasks and operations, such as classification, segmentation, object detection, face recognition, speech recognition, pose estimation (e.g., humans and / or objects), monocular depth estimation, image inking, style transfer, action recognition, colorization, and / or variations thereof.

[0087] In at least one embodiment, OpenVINO includes one or more software tools and / or modules for model optimization, also referred to as a model optimizer. In at least one embodiment, a model optimizer is a command-line tool that facilitates transitions between training and deployment of neural network models. In at least one embodiment, a model optimizer optimizes neural network models for execution on various devices and / or processing units, such as a GPU, CPU, PPU, GPGPU, and / or variations thereof. In at least one embodiment, a model optimizer generates an internal representation of a model and optimizes the model to generate an intermediate representation. In at least one embodiment, a model optimizer reduces a number of layers of a model. In at least one embodiment, a model optimizer removes layers of a model used for training.In at least one embodiment, a model optimizer performs various neural network operations, such as modifying inputs to a model (e.g., changing the size of inputs to a model), modifying a size of inputs of a model (e.g., modifying a batch size of a model), modifying a model structure (e.g., modifying layers of a model), normalization, standardization, quantization (e.g., converting weights of a model from a first representation, such as floating point, to a second representation, such as an integer), and / or variations thereof.

[0088] In at least one embodiment, OpenVINO includes one or more software libraries for inference, also referred to as an inference engine. In at least one embodiment, an inference engine is a C++ library or any suitable programming language library. In at least one embodiment, an inference engine is used to derive input data. In at least one embodiment, an inference engine implements various classes to derive input data and produce one or more results. In at least one embodiment, an inference engine implements one or more API functions to process an intermediate representation, set input and / or output formats, and / or execute a model on one or more devices.

[0089] In at least one embodiment, OpenVINO provides various capabilities for heterogeneously executing one or more neural network models. In at least one embodiment, heterogeneous execution or heterogeneous computing refers to one or more computing processes and / or systems using one or more types of processors and / or cores. In at least one embodiment, OpenVINO provides various software functions to execute a program on one or more devices. In at least one embodiment, OpenVINO provides various software functions to execute a program and / or portions of a program on different devices. In at least one embodiment, OpenVINO provides various software functions, for example, to execute a first portion of code on a CPU and a second portion of code on a GPU and / or FPGA.In at least one embodiment, OpenVINO provides various software functions to execute one or more layers of a neural network on one or more devices (e.g., a first set of layers on a first device, such as a GPU, and a second set of layers on a second device, such as a CPU).

[0090] In at least one embodiment, OpenVINO includes various functionalities similar to functionalities associated with a CUDA programming model, such as various neural network model operations associated with frameworks such as TensorFlow, PyTorch, and / or variations thereof. In at least one embodiment, one or more CUDA programming model operations are performed using OpenVINO. In at least one embodiment, various systems, methods, and / or techniques described herein are implemented using OpenVINO.

[0091] In at least one embodiment, the untrained neural network 906 is trained on the training dataset 920. The trained neural network 920 can then predict whether a radio link will fail based on inputs from RLM-RS inputs from the UE. For example, in at least one embodiment, on each RS resource for downlink radio link quality prediction, the neural network predicts the downlink radio link quality at a future time by predicting whether the downlink radio link cannot be reliably received at the future time. For example, in at least one embodiment, if the neural network predicts that the quality is below the threshold Qout, the neural network predicts that the radio link cannot be reliably received at the future time.

[0092] In at least one embodiment, the threshold Qout corresponds to the out-of-sync block error rate threshold (BLERout). An exemplary value of BLERout is 10%. The predicted out-of-sync block error rate is derived based on the hypothetical PDCCH transmission parameters for the out-of-sync prediction.

[0093] In at least one embodiment, the set of hypothetical PDCCH transmission parameters includes one or more of the following: downlink control information (DCI) format; number of orthogonal frequency division multiplexing (OFDM) control symbols; aggregation level in terms of the number of control channel elements (CCE); ratio of the hypothetical PDCCH resource element (RE) energy to the average secondary synchronization signal (SSS) RE energy; ratio of the hypothetical PDCCH RE energy to the average CSI RS RE energy; ratio of the hypothetical PDCCH demodulation RS (DMRS) energy to the average SSS RE energy; ratio of the hypothetical PDCCH DMRS energy to the average CSI RS RE energy; bandwidth in terms of the number of physical resource blocks (PRBs); subcarrier spacing; DMRS precoder granularity;Resource element group (REG) bundle size, cyclic prefix (CP) length, and REG to CCE mapping.;

[0094] In at least one embodiment, for the SSB-based radio link prediction, a first set of values ​​for one or more of the above hypothetical PDCCH transmission parameters is used for the out-of-sync block error rate prediction.

[0095] In at least one embodiment, for the CSI-RS-based radio link prediction, a second set of values ​​for one or more of the above hypothetical PDCCH transmission parameters is used for the out-of-sync block error rate prediction.

[0096] In at least one embodiment, the neural network 102 predicts that there will be an out-of-sync for the downlink radio connection at a future time if the UE predicts that the radio link quality is worse than the threshold Qout for all resources in the set of resources for the radio link prediction.

[0097] In at least one embodiment, the UE 101 trains reference signal resource-specific neural networks for out-of-sync prediction at a future time and uses them for inference. This embodiment is described in Fig. 1. For each reference signal n, there is a corresponding neural network n: The input to the neural network n is the measurement results at times {t[-P], ..., t[0]} based on the reference signal n, and the output of the neural network is an x-step-ahead link quality prediction regarding whether the quality based on the reference signal n is below the threshold Qout. If the neural network n predicts that the link quality is below the threshold Qout, the model n yields an output bit equal to 1; otherwise, the model n yields an output bit equal to 0. The N output bits from the N neural networks are then fused using an AND operation. If the result of the AND operation is 1, the UE predicts that there will be an out-of-sync at time t[x]; If the result of the AND operation is 0, the UE predicts that there will be no out-of-sync at time t[x]. DATA CENTER

[0098] Fig. 10 illustrates an example data center 1000 in which at least one embodiment may be used. In at least one embodiment, data center 1000 includes a data center infrastructure layer 1010, a framework layer 1020, a software layer 1030, and an application layer 1040.

[0099] In at least one embodiment, as in Fig. 10, the data center infrastructure layer 1010 may include a resource orchestrator 1012, clustered compute resources 1014, and node compute resources (“node CRs”) 1016(1)-1016(N), where “N” represents a positive integer (which may be a different integer “N” than used in other figures). In at least one embodiment, the node CRs 1016(1)-1016(N) may include, but are not limited to, any number of central processing units ("CPUs") or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, etc.), storage devices 1018(1)-1018(N) (e.g., dynamic read-only memory, solid-state storage, or disk drives), network input / output devices ("NW I / O"), network switches, virtual machines ("VMs"), power modules and cooling modules, etc. In at least one embodiment, one or more node CRss from the node CRs 1016(1)-1016(N) may be a server having one or more of the above-mentioned computing resources.

[0100] In at least one embodiment, the grouped computing resources 1014 may include separate groupings of node CRs housed in one or more racks (not shown) or multiple racks housed in data centers in different geographic locations (also not shown). In at least one embodiment, separate groupings of node CRs within the grouped computing resources 1014 may include grouped computing, networking, storage, or memory resources that may be configured or assigned to support one or more workloads. In at least one embodiment, multiple node CRs including CPUs or processors may be grouped in one or more racks to provide computing resources to support one or more workloads.In at least one embodiment, one or more racks may also include any number of power modules, cooling modules, and network switches in any combination.

[0101] In at least one embodiment, resource orchestrator 1012 may configure or otherwise control one or more node CRs 1016(1)-1016(N) and / or clustered computing resources 1014. In at least one embodiment, resource orchestrator 1012 may include a software design infrastructure ("SDI") management entity for data center 1000. In at least one embodiment, resource orchestrator 1012 may include hardware, software, or a combination thereof.

[0102] In at least one embodiment, as in Fig. 10, the framework layer 1020 includes a job scheduler 1022, a configuration manager 1024, a resource manager 1026, and a distributed file system 1028. In at least one embodiment, the framework layer 1020 may include a framework for supporting software 1032 of the software layer 1030 and / or one or more applications 1042 of the application layer 1040. In at least one embodiment, the software 1032 or application(s) 1042 may each include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 1020 may be some type of free and open source software web application framework, such as, but not limited to, Apache Spark™ (hereinafter "Spark"), which may use the distributed file system 1028 for large-scale data processing (e.g., "Big Data").In at least one embodiment, job scheduler 1022 may include a Spark driver to enable scheduling of workloads supported by different layers of data center 1000. In at least one embodiment, configuration manager 1024 may be capable of configuring different layers, such as software layer 1030 and framework layer 1020, including Spark and distributed file system 1028, to support large-scale computing. In at least one embodiment, resource manager 1026 may be capable of managing clustered or grouped compute resources mapped to or allocated to support distributed file system 1028 and job scheduler 1022. In at least one embodiment, clustered or grouped compute resources may include grouped compute resources 1014 on data center infrastructure layer 1010.In at least one embodiment, the resource manager 1026 may coordinate with the resource orchestrator 1012 to manage these mapped or allocated compute resources.

[0103] In at least one embodiment, the software 1032 included in the software layer 1030 may include software used by at least portions of the node CRs 1016(1)-1016(N), the clustered computing resources 1014, and / or the distributed file system 1028 of the framework layer 1020. In at least one embodiment, one or more types of software may include, but are not limited to, Internet website search software, email virus scanning software, database software, and streaming video content software.

[0104] In at least one embodiment, the application(s) 1042 included in the application layer 1040 may include one or more types of applications used by at least portions of the node CRs 1016(1)-1016(N), the clustered computing resources 1014, and / or the distributed file system 1028 of the framework layer 1020. In at least one embodiment, one or more types of applications may include, but are not limited to, any number of a genomic application, a cognitive computing application, a machine learning application, and a machine learning application, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in connection with one or more embodiments.

[0105] In at least one embodiment, any of the configuration manager 1024, the resource manager 1026, and the resource orchestrator 1012 may implement any number and type of self-modifying actions based on any amount and type of data collected in any technically feasible manner. In at least one embodiment, self-modifying actions may relieve a data center operator of the data center 1000 from making potentially poor configuration decisions and potentially avoiding underutilized and / or poorly performing portions of a data center.

[0106] In at least one embodiment, data center 1000 may include tools, services, software, or other resources to train one or more machine learning models or to predict or infer information using one or more machine learning models according to one or more embodiments described herein. For example, in at least one embodiment, a machine learning model may be trained by calculating weighting parameters according to a neural network architecture using software and computing resources described above with respect to data center 1000.In at least one embodiment, trained machine learning models corresponding to one or more neural networks may be used to infer or predict information using resources described above with respect to data center 1000 by using weighting parameters calculated by one or more training techniques described herein.

[0107] In at least one embodiment, the data center may use CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs, or other hardware to perform training and / or inference using resources described above. Furthermore, one or more of the software and / or hardware resources described above may be configured as a service to enable users to train or perform information inference, such as image recognition, speech recognition, or other artificial intelligence services.

[0108] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in data center 1000 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0109] In at least one embodiment, logic 815 is integrated into components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or utilize logic 815 to use one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. AUTONOMOUS VEHICLE

[0110] Fig. 11A illustrates an example of an autonomous vehicle 1100 according to at least one embodiment. In at least one embodiment, the autonomous vehicle 1100 (alternatively referred to herein as "vehicle 1100") may be, without limitation, a passenger vehicle, such as a car, a truck, a bus, and / or another type of vehicle that accommodates one or more passengers. In at least one embodiment, the vehicle 1100 may be a tractor-trailer truck used to transport cargo. In at least one embodiment, the vehicle 1100 may be an aircraft, a robotic vehicle, or another type of vehicle.

[0111] Autonomous vehicles may be described in terms of automation levels defined by the National Highway Traffic Safety Administration ("NHTSA"), a division of the U.S. Department of Transportation, and the Society of Automotive Engineers ("SAE") "Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles" (e.g., Standard No. J3016-201806, published June 15, 2018, Standard No. J3016-201609, published September 30, 2016, and prior and future versions of that standard). In at least one embodiment, the vehicle 1100 may be capable of functionality according to one or more of Levels 1 through 5 of the autonomous driving levels. For example, in at least one embodiment, the vehicle 1100 may be capable of conditional automation (Level 3), high automation (Level 4), and / or full automation (Level 5), depending on the embodiment.

[0112] In at least one embodiment, vehicle 1100 may include, without limitation, components such as a chassis, a vehicle body, wheels (e.g., 2, 4, 6, 8, 18, etc.), tires, axles, and other components of a vehicle. In at least one embodiment, vehicle 1100 may include, without limitation, a propulsion system 1150 such as an internal combustion engine, a hybrid electric power plant, a pure electric motor, and / or another type of propulsion system. In at least one embodiment, propulsion system 1150 may be connected to a drivetrain of vehicle 1100, which may include, without limitation, a transmission, to enable propulsion of vehicle 1100. In at least one embodiment, propulsion system 1150 may be controlled in response to receiving signals from throttle / accelerator pedal(s) 1152.

[0113] In at least one embodiment, a steering system 1154, which may include, without limitation, a steering wheel, is used to steer the vehicle 1100 (e.g., along a desired path or route) when the propulsion system 1150 is operating (e.g., when the vehicle 1100 is in motion). In at least one embodiment, the steering system 1154 may receive signals from a steering actuator(s) 1156. In at least one embodiment, a steering wheel may be optional for full automation (Level 5) functionality. In at least one embodiment, a brake sensor system 1146 may be used to apply vehicle brakes in response to receiving signals from a brake actuator(s) 1148 and / or brake sensors.

[0114] In at least one embodiment, the controller(s) 1136, which may include, without limitation, one or more system-on-chips (“SoCs”) (in Fig. 11A not shown) and / or graphics processing unit(s) ("GPU(s)"), provide signals (e.g., representative of commands) to one or more components and / or systems of the vehicle 1100. For example, in at least one embodiment, the controller(s) 1136 may send signals to apply vehicle brakes via a brake actuator(s) 1148, to actuate the steering system 1154 via a steering actuator(s) 1156, to actuate the propulsion system 1150 via a throttle / accelerator pedal(s) 1152. In at least one embodiment, the controller(s) 1136 may include one or more on-board (e.g., integrated) computing devices that process sensor signals and issue operational commands (e.g., signals representing commands) to enable autonomous driving and / or assist a human driver in driving the vehicle 1100.In at least one embodiment, the controller(s) 1136 may include a first controller for autonomous driving functions, a second controller for functional safety functions, a third controller for artificial intelligence functionality (e.g., computer vision), a fourth controller for infotainment functionality, a fifth controller for redundancy in emergency conditions, and / or other controllers. In at least one embodiment, a single controller may handle two or more of the above functionalities, two or more controllers may handle a single functionality, and / or any combination thereof.

[0115] In at least one embodiment, the controller(s) 1136 provide signals to control one or more components and / or systems of the vehicle 1100 in response to sensor data received from one or more sensors (e.g., sensor inputs). In at least one embodiment, sensor data may be received, for example, and without limitation, from one or more global navigation satellite system ("GNSS") sensors 1158 (e.g., global positioning system sensors), one or more radar sensors 1160, one or more ultrasonic sensors 1162, one or more lidar sensors 1164, one or more inertial measurement unit ("IMU") sensors 1166 (e.g., accelerometer, gyroscope, a magnetic compass or magnetic compasses, magnetometers, etc.), one or more microphones 1196, one or more stereo cameras 1168, one or more wide-view cameras 1170 (e.g.,fisheye cameras), one or more infrared cameras 1172, one or more surround cameras 1174 (e.g. 360-degree cameras), long-range cameras (not in . Fig. 11A), one or more medium-range cameras (not shown in Fig. 11A), one or more speed sensors 1144 (e.g., for measuring the speed of the vehicle 1100), one or more vibration sensors 1142, one or more steering sensors 1140, one or more brake sensors (e.g., as part of the brake sensor system 1146), and / or other types of sensors.

[0116] In at least one embodiment, one or more of the controller(s) 1136 may receive inputs (e.g., represented by input data) from an instrument cluster 1132 of the vehicle 1100 and provide outputs (e.g., represented by output data, display data, etc.) via a human-machine interface ("HMI") display 1134, an audible indicator, a speaker, and / or via other components of the vehicle 1100. In at least one embodiment, outputs may include information such as vehicle speed, velocity, time, map data (e.g., a high-resolution map (not shown) Fig. 11A), location data (e.g., location of vehicle 1100, such as on a map), direction, location of other vehicles (e.g., an occupancy grid), information about objects and status of objects as perceived by controller(s) 1136, etc. In at least one embodiment, HMI display 1134 may, for example, display information about the presence of one or more objects (e.g., a road sign, warning sign, traffic light change, etc.) and / or information about maneuvers the vehicle has undertaken, is undertaking, or is undertaking (e.g., changing lanes, entering exit 34B in two miles, etc.).

[0117] In at least one embodiment, the vehicle 1100 further includes a network interface 1124 that may utilize one or more wireless antennas 1126 and / or one or more modems to communicate over one or more networks. In at least one embodiment, the network interface 1124 may be capable of communicating over, for example, Long-Term Evolution ("LTE"), Wideband Code Division Multiple Access ("WCDMA"), Universal Mobile Telecommunications System ("UMTS"), Global System for Mobile Communication ("GSM"), IMT-CDMA Multi-Carrier ("CDMA2000") networks, etc. In at least one embodiment, one or more wireless antennas 1126 may also facilitate communication between objects in the environment (e.g., vehicles, mobile devices, etc.) using one or more local area networks, such as Bluetooth, Bluetooth Low Energy ("LE"), Z-Wave, ZigBee, etc.and / or one or more low-power wide area networks (“LPWANs”), such as LoRaWAN, SigFox, etc. protocols.

[0118] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in vehicle 1100 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0119] In at least one embodiment, logic 815 is integrated into components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or utilize logic 815 to use one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0120] Fig. 11B illustrates an example of camera positions and fields of view for the autonomous vehicle 1100 of Fig. 11A according to at least one embodiment. In at least one embodiment, cameras and respective fields of view are exemplary embodiments and are not intended to be limiting. For example, in at least one embodiment, additional and / or alternative cameras may be included and / or cameras may be located at various positions on vehicle 1100.

[0121] In at least one embodiment, camera types may include, but are not limited to, digital cameras that may be adapted for use with components and / or systems of the vehicle 1100. In at least one embodiment, the camera(s) may operate at Automotive Safety Integrity Level (“ASIL”) B and / or another ASIL. In at least one embodiment, camera types may be capable of any image capture rate, such as 60 frames per second (fps), 1220 fps, 240 fps, etc., depending on the embodiment. In at least one embodiment, cameras may be capable of using rolling shutters, global shutters, another type of shutter, or a combination thereof.In at least one embodiment, the color filter array may include a red-clear-clear-clear color filter array ("RCCC"), a red-clear-clear-blue color filter array ("RCCB"), a red-blue-green-clear color filter array ("RBGC"), a Foveon X3 color filter array, a Bayer sensor color filter array ("RGGB"), a monochrome sensor color filter array, and / or another type of color filter array. In at least one embodiment, clear-pixel cameras, such as cameras with an RCCC, an RCCB, and / or an RBGC color filter array, may be used in an effort to increase light sensitivity.

[0122] In at least one embodiment, one or more of the camera(s) may be used to perform advanced driver assistance system ("ADAS") functions (e.g., as part of a redundant or fail-safe design). For example, in at least one embodiment, a multifunction mono camera may be installed to provide functions including lane departure warning, traffic sign assistance, and intelligent headlight control. In at least one embodiment, one or more of the camera(s) (e.g., all cameras) may record and provide image data (e.g., video) simultaneously.

[0123] In at least one embodiment, one or more cameras may be mounted in a mounting assembly, such as a custom-designed (three-dimensional ("3D") printed) assembly, to cut out stray light and reflections from within the vehicle 1100 (e.g., dashboard reflections reflected in windshield mirrors) that may interfere with camera image data collection capabilities. With reference to wing mirror mounting assemblies, in at least one embodiment, wing mirror assemblies may be custom 3D printed such that a camera mounting plate conforms to a shape of a wing mirror. In at least one embodiment, camera(s) may be integrated into wing mirrors. In at least one embodiment, camera(s) for side-view cameras may also be integrated into four pillars at each corner of a cab.

[0124] In at least one embodiment, cameras with a field of view that includes portions of an environment in front of the vehicle 1100 (e.g., forward-facing cameras) may be used for surround vision to help identify forward paths and obstacles, as well as to help provide, using one or more of the controller(s) 1136 and / or control SoCs, information critical for generating an occupancy grid and / or determining preferred vehicle paths. In at least one embodiment, forward-facing cameras may be used to perform many similar ADAS functions as LIDAR, including, but not limited to, emergency braking, pedestrian detection, and collision avoidance.In at least one embodiment, forward-facing cameras may also be used for ADAS features and systems, including, but not limited to, Lane Departure Warning (“LDW”), Autonomous Cruise Control (“ACC”), and / or other features such as Traffic Sign Recognition.

[0125] In at least one embodiment, a plurality of cameras may be used in a forward-facing configuration, including, for example, a monocular camera platform incorporating a CMOS (complementary metal oxide semiconductor) color imager. In at least one embodiment, a far-view camera 1170 may be used to perceive objects coming into view from a periphery (e.g., pedestrians, crossing traffic, or bicycles). Although in Fig. 11B illustrates only one wide-view camera 1170, in other embodiments, there may be any number (including zero) of wide-view cameras on vehicle 1100. In at least one embodiment, any number of long-range camera(s) 1198 (e.g., a long-range stereo camera pair) may be used for depth-based object detection, particularly for objects for which a neural network has not yet been trained. In at least one embodiment, long-range camera(s) 1198 may also be used for object detection and classification, as well as basic object tracking.

[0126] In at least one embodiment, any number of stereo cameras 1168 may also be included in a forward-facing configuration. In at least one embodiment, one or more of the stereo cameras 1168 may include an integrated control unit comprising a scalable processing unit that may provide a field-programmable gate array (“FPGA”) and a multi-core microprocessor with an integrated controller area network (“CAN”) or Ethernet interface on a single chip. In at least one embodiment, such a unit may be used to generate a 3D map of an environment of the vehicle 1100 that includes a range estimate for all points in an image.In at least one embodiment, one or more of the stereo camera(s) 1168 may include, among other things, one or more compact stereo vision sensors, which may include, among other things, two camera lenses (one each on the left and right) and an image processing chip that can measure the distance from the vehicle 1100 to the target object and use generated information (e.g., metadata) to activate autonomous emergency braking and lane departure warning functions. In at least one embodiment, other types of stereo camera(s) 1168 may be used in addition to or alternatively to those described herein.

[0127] In at least one embodiment, cameras with a field of view that includes portions of the environment to the sides of the vehicle 1100 (e.g., side view cameras) may be used for surround vision, providing information used to create and update an occupancy grid, as well as to generate side impact collision warnings. For example, in at least one embodiment, one or more surround cameras 1174 (e.g., four surround cameras, as in Fig. 11B) may be positioned on the vehicle 1100. In at least one embodiment, the surround camera(s) 1174 may include, but are not limited to, any number and combination of wide-view cameras, fisheye cameras, 360-degree cameras, and / or similar cameras. For example, in at least one embodiment, four fisheye cameras may be positioned on a front, a rear, and sides of the vehicle 1100. In at least one embodiment, the vehicle 1100 may utilize three surround camera(s) 1174 (e.g., left, right, and rear) and utilize one or more other cameras (e.g., a forward-facing camera) as a fourth surround-view camera.

[0128] In at least one embodiment, cameras with a field of view that includes portions of an environment behind the vehicle 1100 (e.g., rearview cameras) may be used for parking assistance, surround view, rear collision warnings, and creating and updating an occupancy grid. In at least one embodiment, a wide variety of cameras may be used, including, but not limited to, cameras that are also suitable as forward-facing cameras (e.g., long-range cameras 1198 and / or mid-range camera(s) 1176, stereo camera(s) 1168, infrared camera(s) 1172, etc.), as described herein.

[0129] In at least one embodiment, systems or components of Fig. 11 in components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or utilize vehicle 1100 to use one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0130] Fig. 11C is a block diagram illustrating an example system architecture for the autonomous vehicle 1100 of Fig. 11A according to at least one embodiment. In at least one embodiment, each of the components, features, and systems of the vehicle 1100 is Fig. 11C as connected via a bus 1102. In at least one embodiment, bus 1102 may include, without limitation, a CAN data interface (alternatively referred to herein as a "CAN bus"). In at least one embodiment, a CAN may be a network within vehicle 1100 used to assist in controlling various features and functionality of vehicle 1100, such as brake application, acceleration, braking, steering, windshield wipers, etc. In at least one embodiment, bus 1102 may be configured to have dozens or even hundreds of nodes, each with its own unique identifier (e.g., a CAN ID). In at least one embodiment, bus 1102 may be read to find steering wheel angle, ground speed, engine revolutions per minute ("RPMs"), button positions, and / or other vehicle status indicators.In at least one embodiment, bus 1102 may be a CAN bus that is ASIL B compliant.

[0131] In at least one embodiment, FlexRay and / or Ethernet protocols may be used in addition to or alternatively to CAN. In at least one embodiment, there may be any number of buses that comprise bus 1102, which may include, without limitation, zero or more CAN buses, zero or more FlexRay buses, zero or more Ethernet buses, and / or zero or more other types of buses that use different protocols. In at least one embodiment, two or more buses may be used to perform different functions and / or may be used for redundancy. For example, a first bus may be used for collision avoidance functionality, and a second bus may be used for actuation control.In at least one embodiment, each bus of bus 1102 may communicate with any of the components of vehicle 1100, and two or more buses of bus 1102 may communicate with corresponding components. In at least one embodiment, each of any number of system(s) on chip(s) ("SoC(s)") 1104 (such as SoC 1104(A) and SoC 1104(B)), each of controller(s) 1136, and / or each computer within the vehicle may have access to the same input data (e.g., inputs from sensors of vehicle 1100) and may be connected to a common bus, such as a CAN bus.

[0132] In at least one embodiment, the vehicle 1100 may include one or more controllers 1136, such as those described herein with respect to Fig. 11A. In at least one embodiment, the controller(s) 1136 may be used for a variety of functions. In at least one embodiment, the controller(s) 1136 may be coupled to any of various other components and systems of the vehicle 1100 and may be used to control the vehicle 1100, artificial intelligence of the vehicle 1100, infotainment for the vehicle 1100, and / or other functions.

[0133] In at least one embodiment, vehicle 1100 may include any number of SoCs 1104. In at least one embodiment, each of SoCs 1104 may include, without limitation, central processing units ("CPU(s)") 1106, graphics processing units ("GPU(s)") 1108, processor(s) 1110, cache(s) 1112, accelerators 1114, data storage 1116, and / or other unillustrated components and features. In at least one embodiment, SoC(s) 1104 may be used to control vehicle 1100 in a variety of platforms and systems. For example, in at least one embodiment, SoC(s) 1104 in a system (e.g., system of vehicle 1100) may be combined with a high-definition (“HD”) map 1122 that may receive map updates and / or updates via network interface 1124 from one or more servers (not included in Fig. 11C).

[0134] In at least one embodiment, the CPU(s) 1106 may include a CPU cluster or CPU complex (alternatively referred to herein as a "CCPLEX"). In at least one embodiment, the CPU(s) 1106 may include multiple cores and / or level-two ("L2") caches. For example, in at least one embodiment, the CPU(s) 1106 may include eight cores in a coherent multiprocessor configuration. In at least one embodiment, the CPU(s) 1106 may include four dual-core clusters, each cluster having a dedicated L2 cache (e.g., a 2-megabyte (MB) L2 cache). In at least one embodiment, the CPU(s) 1106 (e.g., CCPLEX) may be configured to support concurrent cluster operations, allowing any combination of clusters of CPU(s) 1106 to be active at any given time.

[0135] In at least one embodiment, one or more of the CPU(s) 1106 may implement power management capabilities, including, without limitation, one or more of the following features: individual hardware blocks may be automatically clocked when idle to conserve dynamic power; each core clock may be clocked when such core is not actively executing due to the execution of wait-for-interrupt ("WFI") / wait-for-event ("WFE") instructions; each core may be independently clocked; each core cluster may be independently clocked when all cores are clocked or unclocked; and / or each core cluster may be independently clocked when all cores are clocked.In at least one embodiment, CPU(s) 1106 may further implement an improved performance state management algorithm, where allowed performance states and expected wake-up times are specified, and hardware / microcode determines which best performance state to enter for core, cluster, and CCPLEX. In at least one embodiment, processing cores may support simplified performance state entry sequences in software with work offloaded to microcode.

[0136] In at least one embodiment, the GPU(s) 1108 may include an integrated GPU (alternatively referred to herein as an "iGPU"). In at least one embodiment, the GPU(s) 1108 may be programmable and efficient for parallel workloads. In at least one embodiment, the GPU(s) 1108 may use an extended tensor instruction set. In at least one embodiment, the GPU(s) 1108 may include one or more streaming microprocessors, where each streaming microprocessor may include a level-one ("L1") cache (e.g., an L1 cache with a memory capacity of at least 96 KB) and two or more streaming microprocessors may share an L2 cache (e.g., an L2 cache with a memory capacity of 512 KB). In at least one embodiment, the GPU(s) 1108 may include at least eight streaming microprocessors.In at least one embodiment, the GPU(s) 1108 may use one or more computing application programming interfaces (APIs). In at least one embodiment, the GPU(s) 1108 may use one or more parallel computing platforms and / or programming models (e.g., NVIDIA's CUDA model).

[0137] In at least one embodiment, one or more of the GPU(s) 1108 may be power-optimized for best performance in automotive and embedded use cases. For example, in at least one embodiment, the GPU(s) 1108 may be fabricated on a fin field-effect transistor ("FinFET") circuit. In at least one embodiment, each streaming microprocessor may include a number of mixed-precision processing cores partitioned into multiple blocks. For example, and without limitation, 64 PF32 cores and 32 FP64 cores may be partitioned into four processing blocks. In at least one embodiment, each processing block could include 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, two NVIDIA mixed-precision Tensor cores for deep learning matrix arithmetic, a level zero (“L0”) instruction cache, a scheduler (e.g.,Warp scheduler) or sequencer, a dispatch unit, and / or a 64 KB register file. In at least one embodiment, streaming microprocessors may include independent parallel integer and floating-point data paths to provide efficient execution of workloads with a mix of computation and addressing calculations. In at least one embodiment, streaming microprocessors may include independent thread scheduling capability to enable finer synchronization and cooperation between parallel threads. In at least one embodiment, streaming microprocessors may include a combined L1 data cache and shared memory unit to improve performance while simplifying programming.

[0138] In at least one embodiment, one or more of the GPU(s) 1108 may include high-bandwidth memory ("HBM") and / or a 16 GB HBM2 memory subsystem to provide, in some examples, a peak memory bandwidth of approximately 900 GB / second. In at least one embodiment, synchronous graphics random access memory ("SGRAM"), such as double data rate type five synchronous graphics random access memory ("GDDR5"), may be used in addition to or alternatively to the HBM memory.

[0139] In at least one embodiment, the GPU(s) 1108 may include unified memory technology. In at least one embodiment, address translation services ("ATS") support may be used to enable the GPU(s) 1108 to directly access page tables of the CPU(s) 1106. In at least one embodiment, when a GPU of the memory management unit ("MMU") of the GPU(s) 1108 experiences a fault, an address translation request may be transmitted to the CPU(s) 1106. In response, two CPUs of the CPU(s) 1106, in at least one embodiment, may look in their page tables for a virtual-to-physical mapping for an address and transmit the translation back to the GPU(s) 1108.In at least one embodiment, the unified memory technology may enable a single unified virtual address space for the memory of both the CPU(s) 1106 and the GPU(s) 1108, thereby simplifying the programming and porting of applications to the GPU(s) 1108 by the GPU(s) 1108.

[0140] In at least one embodiment, the GPU(s) 1108 may include any number of access counters that may track the frequency of access by the GPU(s) 1108 to the memory of other processors. In at least one embodiment, the access counter(s) may help ensure that memory pages are moved to the physical memory of a processor that accesses pages most frequently, thereby improving efficiency for memory shared between processors.

[0141] In at least one embodiment, one or more of the SoC(s) 1104 may include any number of cache(s) 1112, including those described herein. For example, in at least one embodiment, the cache(s) 1112 may include a level-three ("L3") cache available to both the CPU(s) 1106 and the GPU(s) 1108 (e.g., connected to the CPU(s) 1106 and the GPU(s) 1108). In at least one embodiment, the cache(s) 1112 may include a write-back cache that can track states of lines, such as using a cache coherence protocol (e.g., MEI, MESI, MSI, etc.). In at least one embodiment, an L3 cache may include 4 MB of memory or more, depending on the embodiment, although smaller cache sizes may be used.

[0142] In at least one embodiment, one or more of the SoC(s) 1104 may include one or more accelerators 1114 (e.g., hardware accelerators, software accelerators, or a combination thereof). In at least one embodiment, the SoC(s) 1104 may include a hardware acceleration cluster, which may include optimized hardware accelerators and / or large on-chip memory. In at least one embodiment, large on-chip memory (e.g., 4 MB SRAM) may enable a hardware acceleration cluster to accelerate neural networks and other computations. In at least one embodiment, a hardware acceleration cluster may be used to supplement the GPU(s) 1108 and offload some tasks from the GPU(s) 1108 (e.g., to free up more cycles of the GPU(s) 1108 to perform other tasks). In at least one embodiment, the accelerator(s) 1114 may be configured for targeted workloads (e.g.,Perception, convolutional neural networks (Cneural networks), recurrent neural networks (Rneural networks), etc.) that are robust enough to be capable of acceleration may be used. In at least one embodiment, a Cneural network may include a region-based or regional convolutional neural network (RCneural networks) and fast RCneural networks (such as those used for object detection) or another type of CNN.

[0143] In at least one embodiment, the accelerator(s) 1114 (e.g., hardware acceleration clusters) may include one or more deep learning accelerators ("DLAs"). In at least one embodiment, the DLA(s) may include, without limitation, one or more tensor processing units ("TPUs") that may be configured to provide an additional tens of trillion operations per second for deep learning applications and inference. In at least one embodiment, the TPUs may be accelerators configured and optimized to perform image processing functions (e.g., for C neural networks, RC neural networks, etc.). In at least one embodiment, the DLA(s) may be further optimized for a specific set of neural network types and floating-point operations, as well as inference.In at least one embodiment, the design of DLA(s) can provide more performance per millimeter than a typical general-purpose GPU and typically significantly exceeds the performance of a CPU. In at least one embodiment, the TPU(s) can perform multiple functions, including a single-instance convolution function, supporting, for example, INT8, INT16, and FP16 data types for both features and weights, as well as post-processing functions.In at least one embodiment, the DLA(s) may quickly and efficiently execute neural networks, particularly Cneural networks, on processed or unprocessed data for any of a variety of functions, including, for example and without limitation: a Cneural network for object identification and detection using data from camera sensors; a Cneural network for range estimation using data from camera sensors; a Cneural network for emergency vehicle detection and identification using data from microphones; a Cneural network for facial recognition and vehicle owner identification using data from camera sensors; and / or a Cneural network for safety and / or security-related events.

[0144] In at least one embodiment, the DLA(s) may perform any function of the GPU(s) 1108, and by using an inference accelerator, for example, a designer may target either the DLA(s) or the GPU(s) 1108 for any function. For example, in at least one embodiment, a designer may focus neural network processing and floating-point operations on the DLA(s) and leave other functions to the GPU(s) 1108 and / or the accelerator(s) 1114.

[0145] In at least one embodiment, the accelerator(s) 1114 may include a programmable vision accelerator ("PVA"), which may alternatively be referred to herein as a computer vision accelerator. In at least one embodiment, the PVA may be designed and configured to accelerate computer vision algorithms for the advanced driver assistance system ("ADAS") 1138, autonomous driving, augmented reality ("AR") applications, and / or virtual reality ("VR") applications. In at least one embodiment, the PVA may provide a balance between performance and flexibility. For example, in at least one embodiment, each PVA may include, for example and without limitation, any number of reduced instruction set computing ("RISC") cores, direct memory access ("DMA") cores, and / or any number of vector processors.

[0146] In at least one embodiment, RISC cores may interact with image sensors (e.g., image sensors of cameras described herein), image signal processor(s), etc. In at least one embodiment, each RISC core may include any amount of memory. In at least one embodiment, RISC cores may use any of a number of protocols, depending on the embodiment. In at least one embodiment, RISC cores may execute a real-time operating system ("RTOS"). In at least one embodiment, RISC cores may be implemented using one or more integrated circuit devices, application-specific integrated circuits ("ASICs"), and / or memory devices. For example, in at least one embodiment, RISC cores may include an instruction cache and / or tightly coupled RAM.

[0147] In at least one embodiment, DMA may enable components of PVA to access system memory independent of CPU(s) 1106. In at least one embodiment, DMA may support any number of features used to provide optimization for a PVA, including, but not limited to, supporting multi-dimensional addressing and / or circular addressing. In at least one embodiment, DMA may support up to six or more dimensions of addressing, which may include, without limitation, block width, block height, block depth, horizontal block striding, vertical block striding, and / or depth striding.

[0148] In at least one embodiment, vector processors may be programmable processors that may be designed to efficiently and flexibly execute programming for computer vision algorithms and provide signal processing capabilities. In at least one embodiment, a PVA may include a PVA core and two vector processing subsystem partitions. In at least one embodiment, a PVA core may include a processor subsystem, DMA engine(s) (e.g., two DMA engines), and / or other peripherals. In at least one embodiment, a vector processing subsystem may operate as a primary processing engine of a PVA and may include a vector processing unit ("VPU"), an instruction cache, and / or vector memory (e.g., "VMEM").In at least one embodiment, the VPU core may include a digital signal processor, such as a single instruction, multiple data ("SIMD"), very long instruction word ("VLIW") digital signal processor. In at least one embodiment, a combination of SIMD and VLIW may improve throughput and speed.

[0149] In at least one embodiment, each of the vector processors may include an instruction cache and may be coupled to dedicated memory. Consequently, in at least one embodiment, each of the vector processors may be configured to execute independently of other vector processors. In at least one embodiment, vector processors included in a particular PVA may be configured to employ data parallelism. For example, in at least one embodiment, a plurality of vector processors included in a single PVA may execute a common computer vision algorithm, but on different regions of an image.In at least one embodiment, vector processors included in a particular PVA may concurrently execute different computer vision algorithms on an image, or even execute different algorithms on consecutive images or portions of an image. In at least one embodiment, among other things, any number of PVAs may be included in a hardware acceleration cluster, and any number of vector processors may be included in each PVA. In at least one embodiment, PVAs may include additional error correction code ("ECC") memory to improve overall system security.

[0150] In at least one embodiment, the accelerator(s) 1114 may include an on-chip computer vision network and static random access memory ("SRAM") to provide high-bandwidth, low-latency SRAM for the accelerator(s) 1114. In at least one embodiment, on-chip memory may include at least 4 MB of SRAM, including, for example, and without limitation, eight field-configurable memory blocks accessible by both a PVA and a DLA. In at least one embodiment, each pair of memory blocks may include an advanced peripheral bus ("APB") interface, configuration circuitry, a controller, and a multiplexer. In at least one embodiment, any type of memory may be used.In at least one embodiment, a PVA and a DLA may access memory via a backbone that provides a PVA and a DLA with high-speed access to memory. In at least one embodiment, a backbone may include an on-chip computer vision network that connects a PVA and a DLA to memory (e.g., using APB).

[0151] In at least one embodiment, an on-chip computer vision network may include an interface that determines that both a PVA and a DLA are providing ready and valid signals before transmitting any control signal / address / data. In at least one embodiment, an interface may provide separate phases and separate channels for transmitting control signals / addresses / data, as well as burst-type communications for continuous data transmission. In at least one embodiment, an interface may conform to International Organization for Standardization ("ISO") 26262 or International Electrotechnical Commission ("IEC") 61508 standards, although other standards and protocols may be used.

[0152] In at least one embodiment, one or more of the SoC(s) 1104 may include a real-time ray tracing hardware accelerator. In at least one embodiment, the real-time ray tracing hardware accelerator may be used to quickly and efficiently determine positions and extents of objects (e.g., within a world model) to generate real-time visualization simulations for radar signal interpretation, sound propagation synthesis and / or analysis, simulation of sonar systems, general wave propagation simulation, comparison with lidar data for localization and / or other functions, and / or for other uses.

[0153] In at least one embodiment, the accelerator(s) 1114 may have a wide range of uses for autonomous driving. In at least one embodiment, a PVA may be used for key processing stages in ADAS and autonomous vehicles. In at least one embodiment, the capabilities of a PVA are a good match for algorithmic domains that require predictable processing, low power, and low latency. In other words, a PVA performs well on semi-dense or dense regular computation, even on small datasets, which might require predictable runtimes with low latency and low power. In at least one embodiment, such as in vehicle 1100, PVAs could be designed to execute classical computer vision algorithms because they can be efficient at object detection and operate on integer mathematics.

[0154] For example, according to at least one embodiment of the technology, a PVA is used to perform computer stereo vision. In at least one embodiment, a semi-global matching-based algorithm may be used in some examples, although this is not intended to be limiting. In at least one embodiment, Level 3-5 autonomous driving applications use on-the-fly motion estimation / stereo matching (e.g., structure from motion, pedestrian detection, lane detection, etc.). In at least one embodiment, a PVA may perform computer stereo vision functions on inputs from two monocular cameras.

[0155] In at least one embodiment, a PVA may be used to perform dense optical flow. For example, in at least one embodiment, a PVA may process raw radar data (e.g., using a 4D Fast Fourier Transform) to provide processed radar data. In at least one embodiment, a PVA is used for runtime depth processing, for example, by processing raw runtime data to provide processed runtime data.

[0156] In at least one embodiment, a DLA may be used to execute any type of network to improve control and driving safety, including, for example and without limitation, a neural network that outputs a confidence measure for each object detection. In at least one embodiment, the confidence may be represented or interpreted as a probability or as providing a relative "weight" of each detection compared to other detections. In at least one embodiment, a confidence measure allows a system to make further decisions about which detections should be considered true positives rather than false positives. In at least one embodiment, a system may set a confidence threshold and consider only detections that exceed the threshold as true positives.In an embodiment using an automatic emergency braking ("AEB") system, false positive detections would cause the vehicle to automatically perform emergency braking, which is clearly undesirable. In at least one embodiment, highly confident detections may be considered triggers for AEB. In at least one embodiment, a DLA may execute a neural network to reduce the confidence score. In at least one embodiment, the neural network may take as its input at least a subset of parameters, such as bounding box dimensions, ground plane estimation obtained (e.g., from another subsystem), output from IMU sensor(s) 1166, which correlates with orientation, range, 3D position estimates of an object of the vehicle 1100 obtained from, among other things, the neural network and / or other sensors (e.g., LIDAR sensor(s) 1164 or RADAR sensor(s) 1160).

[0157] In at least one embodiment, one or more of the SoC(s) 1104 may include data storage 1116 (e.g., memory). In at least one embodiment, the data storage(s) 1116 may be on-chip memory of the SoC(s) 1104, which may store neural networks to be executed on the GPU(s) 1108 and / or a DLA. In at least one embodiment, the data storage(s) 1116 may be large enough in capacity to store multiple instances of neural networks for redundancy and security. In at least one embodiment, the data storage(s) 1116 may include L2 or L3 cache(s).

[0158] In at least one embodiment, one or more of the SoC(s) 1104 may include any number of processor(s) 1110 (e.g., embedded processors). In at least one embodiment, the processor(s) 1110 may include a boot and power management processor, which may be a dedicated processor and subsystem to handle boot power and management functions and associated security enforcement. In at least one embodiment, a boot and power management processor may be part of a boot sequence of SoC(s) 1104 and may provide runtime power management services. In at least one embodiment, a boot power and management processor may provide clock and voltage programming, low power state system transition support, management of thermal and temperature sensors of SoC(s) 1104, and / or management of power states of SoC(s) 1104.In at least one embodiment, each temperature sensor may be implemented as a ring oscillator whose output frequency is proportional to temperature, and SoC(s) 1104 may use ring oscillators to detect temperatures of CPU(s) 1106, GPU(s) 1108, and / or accelerator(s) 1114. In at least one embodiment, if temperatures are determined to exceed a threshold, a boot and power management processor may enter a temperature fault routine and place SoC(s) 1104 into a lower power state and / or place vehicle 1100 into a chauffeur-into-safe-stop mode (e.g., bring vehicle 1100 to a safe stop).

[0159] In at least one embodiment, processor(s) 1110 may further include a set of embedded processors that may serve as an audio processing engine, which may be an audio subsystem enabling full hardware support for multi-channel audio across multiple interfaces and a wide and flexible range of audio I / O interfaces. In at least one embodiment, an audio processing engine is a dedicated processor core with a digital signal processor with dedicated RAM.

[0160] In at least one embodiment, the processor(s) 1110 may further include an always-on processor engine that may provide necessary hardware features to support low-power sensor management and wake-use cases. In at least one embodiment, an always-on processor engine may include, without limitation, a processor core, tightly coupled RAM, supporting peripherals (e.g., timers and interrupt controllers), various I / O controller peripherals, and routing logic.

[0161] In at least one embodiment, the processor(s) 1110 may further include a security cluster engine, including, without limitation, a dedicated processor subsystem to handle security management for automotive applications. In at least one embodiment, a security cluster engine may include, without limitation, two or more processor cores, tightly coupled RAM, supporting peripherals (e.g., timers, an interrupt controller, etc.), and / or routing logic. In a security mode, two or more cores, in at least one embodiment, may operate in a lockstep mode, functioning as a single core with comparison logic to detect any differences between their operations.In at least one embodiment, the processor(s) 1110 may further include a real-time camera engine, which may include, without limitation, a dedicated processor subsystem for handling real-time camera management. In at least one embodiment, the processor(s) 1110 may further include a high dynamic range signal processor, which may include, without limitation, an image signal processor that is a hardware engine that is part of a camera processing pipeline.

[0162] In at least one embodiment, processor(s) 1110 may include a video image compositor, which may be a processing block (e.g., implemented on a microprocessor) that implements video post-processing functions required by a video playback application to produce a final image for a playback window. In at least one embodiment, a video image compositor may perform lens distortion correction on one or more wide-view cameras 1170, one or more surround cameras 1174, and / or one or more in-cabin monitoring camera sensors. In at least one embodiment, the in-cabin monitoring camera sensor(s) is / are preferably monitored by a neural network running on another instance of SoC 1104 configured to identify and respond to in-cabin events.In at least one embodiment, an in-cabin system may perform, without limitation, lip reading to activate cellular service and place a phone call, dictate emails, change a vehicle's destination, activate or change a vehicle's infotainment system and settings, or provide voice-activated web browsing. In at least one embodiment, certain features are available to a driver when a vehicle is operating in an autonomous mode and are otherwise disabled.

[0163] In at least one embodiment, a video image compositor may include enhanced temporal noise reduction for both spatial and temporal noise reduction. For example, in at least one embodiment, when motion occurs in a video, the noise reduction appropriately weights spatial information, thereby reducing weights of information provided by neighboring frames. In at least one embodiment, when an image or a portion of an image does not include motion, the temporal noise reduction performed by the video image compositor may use information from a previous image to reduce noise in a current image.

[0164] In at least one embodiment, a video image compositor may also be configured to perform stereo rectification on input stereo lens frames. In at least one embodiment, a video image compositor may further be used for user interface composition when using an operating system desktop, and one or more GPUs 1108 are not required to continuously render new surfaces. In at least one embodiment, when one or more GPUs 1108 are powered on and actively performing 3D rendering, a video image compositor may be used to offload one or more GPUs 1108 to improve performance and responsiveness.

[0165] In at least one embodiment, one or more of SoC(s) 1104 may further include a Mobile Industrial Processor Interface ("MIPI") serial camera interface for receiving video and inputs from cameras, a high-speed interface, and / or a video input block that may be used for a camera and related pixel input functions. In at least one embodiment, one or more of SoC(s) 1104 may further include one or more input / output controllers that may be controlled by software and may be used to receive I / O signals that are not committed to a specific role.

[0166] In at least one embodiment, one or more of SoC(s) 1104 may further include a wide range of peripheral interfaces to enable communication with peripherals, audio encoders / decoders ("codecs"), power management, and / or other devices. In at least one embodiment, SoC(s) 1104 may be used to process data from cameras (e.g., connected via Gigabit Multimedia Serial Link and Ethernet channels), sensors (e.g., LIDAR sensor(s) 1164, RADAR sensor(s) 1160, etc., which may be connected via Ethernet channels), data from bus 1102 (e.g., speed of vehicle 1100, steering wheel position, etc.), data from GNSS sensor(s) 1158 (e.g., connected via an Ethernet bus or a CAN bus), etc.In at least one embodiment, one or more of SoC(s) 1104 may further include dedicated high-performance mass storage controllers, which may include their own DMA engines and which may be used to free CPU(s) 1106 from routine data management tasks.

[0167] In at least one embodiment, SoC(s) 1104 may be an end-to-end platform with a flexible architecture spanning Levels 3-5 of automation, providing a comprehensive functional safety architecture that leverages and efficiently utilizes computer vision and ADAS techniques for diversity and redundancy, and providing a platform for a flexible, reliable driving software stack along with deep learning tools. In at least one embodiment, SoC(s) 1104 may be faster, more reliable, and even more power and space efficient than traditional systems. For example, in at least one embodiment, accelerators 1114, when combined with CPU(s) 1106, GPU(s) 1108, and data storage(s) 1116, may provide a fast, efficient platform for Level 3-5 autonomous vehicles.

[0168] In at least one embodiment, computer vision algorithms may be executed on CPUs, which may be configured using a high-level programming language, such as C, to perform a wide variety of processing algorithms over a wide variety of visual data. However, in at least one embodiment, CPUs are often unable to meet performance requirements of many computer vision applications, such as those related to, for example, execution time and power consumption. In at least one embodiment, many CPUs are unable to execute complex object detection algorithms in real time, which is used in in-vehicle ADAS applications and in practical Level 3-5 autonomous vehicles.

[0169] Embodiments described herein enable multiple neural networks to be executed concurrently and / or sequentially and for results to be combined to enable Level 3-5 autonomous driving functionality. For example, in at least one embodiment, a neural network executing on a DLA or a discrete GPU (e.g., GPU(s) 1120) may include text and word recognition, enabling the reading and understanding of traffic signs, including signs for which a neural network has not been specifically trained. In at least one embodiment, a DLA may further include a neural network capable of identifying, interpreting, and providing a semantic understanding of a sign and passing that semantic understanding to path planning modules executing on a CPU complex.

[0170] In at least one embodiment, multiple neural networks may be executed simultaneously, such as during Level 3, 4, or 5 driving. For example, in at least one embodiment, a warning sign indicating "Warning: Flashing lights indicate icy conditions" along with an electrical light may be interpreted independently or jointly by multiple neural networks. In at least one embodiment, such a warning sign may itself be identified as a traffic sign by a first deployed neural network (e.g., a neural network that has been trained). Text "Flashing lights indicate icy conditions" may be interpreted by a second deployed neural network, which informs a vehicle's path planning software (preferably executing on a CPU complex) that, if flashing lights are detected, icy conditions exist.In at least one embodiment, a flashing light may be identified by operating a third deployed neural network over multiple frames, which informs a vehicle's path planning software of the presence (or absence) of flashing lights. In at least one embodiment, all three neural networks may be executed concurrently, such as within a DLA and / or on GPU(s) 1108.

[0171] In at least one embodiment, a neural network for facial recognition and vehicle owner identification may use data from camera sensors to identify the presence of an authorized driver and / or owner of vehicle 1100. In at least one embodiment, an always-on sensor processing engine may be used to unlock a vehicle when an owner approaches a driver's door and turns on the lights, and to disable such a vehicle in a security mode when an owner exits such a vehicle. In this way, SoC(s) 1104 provide security against theft and / or carjacking.

[0172] In at least one embodiment, a Cneural network for emergency vehicle detection and identification may use data from microphones 1196 to detect and identify emergency vehicle sirens. In at least one embodiment, SoC(s) 1104 use a Cneural network to classify ambient and urban noise, as well as to classify visual data. In at least one embodiment, a Cneural network running on a DLA is trained to identify a relative closing speed of an emergency vehicle (e.g., by using a Doppler effect). In at least one embodiment, a Cneural network may also be trained to identify emergency vehicles specific to a local area in which a vehicle is operating, as identified by GNSS sensor(s) 1158.In at least one embodiment, a Cneural network operating in Europe will attempt to detect European sirens, and a Cneural network operating in North America will attempt to identify only North American sirens. In at least one embodiment, once an emergency vehicle is detected, a control program may be used to execute an emergency vehicle safety routine, slow a vehicle, pull a vehicle to the side of the road, park a vehicle, and / or idle a vehicle using ultrasonic sensor(s) 1162 until emergency vehicles pass by.

[0173] In at least one embodiment, vehicle 1100 may include CPU(s) 1118 (e.g., discrete CPU(s) or dCPU(s)) that may be coupled to SoC(s) 1104 via a high-speed interconnect (e.g., PCIe). In at least one embodiment, CPU(s) 1118 may include, for example, an x86 processor. The CPU(s) 1118 may be used to perform any of a variety of functions, including, for example, arbitrating potentially inconsistent results between ADAS sensors and SoC(s) 1104 and / or monitoring the status and integrity of the controller(s) 1136 and / or an infotainment system on a chip (“Infotainment SoC”) 1130. In at least one embodiment, the SoC(s) 1104 include one or more interconnects, and an interconnect may include a Peripheral Component Interconnect Express (PCIe).

[0174] In at least one embodiment, vehicle 1100 may include GPU(s) 1120 (e.g., discrete GPU(s) or dGPU(s)) that may be coupled to SoC(s) 1104 via a high-speed interconnect (e.g., NVIDIA's NVLINK channel). In at least one embodiment, GPU(s) 1120 may provide additional artificial intelligence functionality, such as by executing redundant and / or distinct neural networks, and may be used to train and / or update neural networks based at least in part on inputs (e.g., sensor data) from sensors of vehicle 1100.

[0175] In at least one embodiment, the vehicle 1100 may further include a network interface 1124, which may include, without limitation, one or more wireless antennas 1126 (e.g., one or more wireless antennas for different communication protocols, such as a cellular antenna, a Bluetooth antenna, etc.). In at least one embodiment, the network interface 1124 may be used to enable wireless connectivity to internet cloud services (e.g., to server(s) and / or other network devices), to other vehicles, and / or to computing devices (e.g., passenger client devices). In at least one embodiment, to communicate with other vehicles, a direct connection may be established between the vehicle 1100 and another vehicle and / or an indirect connection may be established (e.g., via networks and over the internet).In at least one embodiment, direct connections may be provided using a vehicle-to-vehicle communication link. In at least one embodiment, a vehicle-to-vehicle communication link may provide information to vehicle 1100 about vehicles in the vicinity of vehicle 1100 (e.g., vehicles in front of, to one side of, and / or behind vehicle 1100). In at least one embodiment, such aforementioned functionality may be part of a cooperative adaptive cruise control functionality of vehicle 1100.

[0176] In at least one embodiment, network interface 1124 may include an SoC that provides modulation and demodulation functionality and enables controller(s) 1136 to communicate over wireless networks. In at least one embodiment, network interface 1124 may include a radio frequency front-end for upconversion from baseband to radio frequency and downconversion from radio frequency to baseband. In at least one embodiment, frequency conversions may be performed in any technically feasible manner. For example, frequency conversions could be performed by well-known processes and / or using superheterodyne processes. In at least one embodiment, radio frequency front-end functionality may be provided by a separate chip.In at least one embodiment, network interfaces may include wireless functionality for communicating over LTE, WCDMA, UMTS, GSM, CDMA2000, Bluetooth, Bluetooth LE, Wi-Fi, Z-Wave, ZigBee, LoRaWAN, and / or other wireless protocols.

[0177] In at least one embodiment, the vehicle 1100 may further include data storage 1128, which may include, but is not limited to, off-chip memory (e.g., off-SoC(s) 1104). In at least one embodiment, the data storage(s) 1128 may include, but are not limited to, one or more memory elements, including RAM, SRAM, dynamic random access memory ("DRAM"), video random access memory ("VRAM"), flash memory, hard drives, and / or other components and / or devices capable of storing at least one bit of data.

[0178] In at least one embodiment, vehicle 1100 may further include GNSS sensor(s) 1158 (e.g., GPS and / or assisted GPS sensors) to assist with mapping, sensing, occupancy grid generation, and / or path planning functions. In at least one embodiment, any number of GNSS sensor(s) 1158 may be used, including, for example and without limitation, a GPS using a USB connector with an Ethernet-to-serial bridge (e.g., RS-232).

[0179] In at least one embodiment, vehicle 1100 may further include radar sensor(s) 1160. In at least one embodiment, radar sensor(s) 1160 may be used by vehicle 1100 for long-range vehicle detection, even in darkness and / or severe weather conditions. In at least one embodiment, radar functional safety levels may be ASIL B. In at least one embodiment, the RADAR sensor(s) 1160 may use a CAN bus and / or bus 1102 (e.g., to transmit data generated by the RADAR sensor(s) 1160 for control and to access object tracking data, with some examples accessing Ethernet channels to access raw data). In at least one embodiment, a wide variety of RADAR sensor types may be used. For example, and among other things, the RADAR sensor(s) 1160 may be suitable for front, rear, and side RADAR use.In at least one embodiment, one or more sensors of RADAR sensors 1160 is / are a pulse Doppler RADAR sensor.

[0180] In at least one embodiment, the RADAR sensor(s) 1160 may include various configurations, such as long-range narrow field of view, short-range wide field of view, short-range side coverage, etc. In at least one embodiment, long-range RADAR may be used for adaptive cruise control functionality. In at least one embodiment, long-range RADAR systems may provide a wide field of view realized by two or more independent scans, such as within a range of 250 m (meters). In at least one embodiment, the RADAR sensor(s) 1160 may help distinguish between static and moving objects and may be used by the ADAS system 1138 for emergency braking assistance and forward collision warning.In at least one embodiment, the sensor(s) 1160 included in a long-range RADAR system may include, without limitation, monostatic multimodal RADAR with multiple (e.g., six or more) fixed RADAR antennas and a high-speed CAN and FlexRay interface. In at least one embodiment, a central four-antenna array with six antennas may produce a focused beam pattern configured to record the surroundings of the vehicle 1100 at higher speeds with minimal interference from traffic in adjacent lanes. In at least one embodiment, two additional antennas may expand the field of view, making it possible to quickly detect vehicles entering or exiting a lane of the vehicle 1100.

[0181] For example, in at least one embodiment, medium-range radar systems may include a range of up to 160 m (front) or 80 m (rear) and a field of view of up to 42 degrees (front) or 150 degrees (rear). In at least one embodiment, short-range radar systems may include, without limitation, any number of radar sensors 1160 configured to be installed at either end of a rear bumper. When installed at either end of a rear bumper, in at least one embodiment, a radar sensor system may generate two beams that continuously monitor blind spots in a rearward direction and to the side of a vehicle. In at least one embodiment, short-range radar systems may be used in ADAS system 1138 for blind spot detection and / or lane change assistance.

[0182] In at least one embodiment, the vehicle 1100 may further include ultrasonic sensor(s) 1162. In at least one embodiment, the ultrasonic sensor(s) 1162, which may be positioned at a front, rear, and / or side location of the vehicle 1100, may be used for parking and / or for creating and updating an occupancy grid. In at least one embodiment, a wide variety of ultrasonic sensor(s) 1162 may be used, and different ultrasonic sensor(s) 1162 may be used for different detection ranges (e.g., 2.5 m, 4 m). In at least one embodiment, the ultrasonic sensor(s) 1162 may operate at functional safety levels of ASIL B.

[0183] In at least one embodiment, the vehicle 1100 may include LIDAR sensor(s) 1164. In at least one embodiment, the LIDAR sensor(s) 1164 may be used for object and pedestrian detection, emergency braking, collision avoidance, and / or other functions. In at least one embodiment, the LIDAR sensor(s) 1164 may operate at ASIL B functional safety level. In at least one embodiment, the vehicle 1100 may include multiple LIDAR sensors 1164 (e.g., two, four, six, etc.) that may use an Ethernet channel (e.g., to provide data to a Gigabit Ethernet switch).

[0184] In at least one embodiment, the LIDAR sensor(s) 1164 may be capable of providing a list of objects and their distances for a 360-degree field of view. In at least one embodiment, the commercially available LIDAR sensor(s) 1164 may have an advertised range of approximately 100 m with an accuracy of 2 cm to 3 cm and with support for a 100 Mbps Ethernet connection. In at least one embodiment, the LIDAR sensor(s) 1164 may be capable of achieving a range of approximately 100 m. In at least one embodiment, one or more non-proximate LIDAR sensors may be used. In such an embodiment, the LIDAR sensor(s) 1164 may include a small device that may be embedded in a front, rear, side, and / or corner location of the vehicle 1100.In at least one embodiment, the LIDAR sensor(s) 1164 in such an embodiment can provide up to a horizontal and vertical field of view of 120 degrees and 35 degrees, respectively, with a range of 200 m, even for objects with low reflectivity. In at least one embodiment, the front-mounted LIDAR sensor(s) 1164 can be configured for a horizontal field of view between 45 degrees and 135 degrees.

[0185] In at least one embodiment, LIDAR technologies, such as 3D flash LIDAR, may also be used. In at least one embodiment, 3D flash LIDAR uses a laser flash as a transmission source to illuminate the surroundings of the vehicle 1100 up to approximately 200 m. In at least one embodiment, a flash LIDAR unit includes, without limitation, a receiver that records the laser pulse travel time and reflected light at each pixel, which in turn corresponds to a range from the vehicle 1100 to objects. In at least one embodiment, flash LIDAR may enable highly accurate and distortion-free images of the surroundings to be generated with each laser flash. In at least one embodiment, four flash LIDAR sensors may be deployed, one on each side of the vehicle 1100.In at least one embodiment, 3D flash lidar systems include, without limitation, a solid-state 3D staring array lidar camera with no moving parts other than a fan (e.g., a non-scanning lidar device). In at least one embodiment, the flash lidar device may use a 5-nanosecond Class I (eye-safe) laser pulse per frame and may collect reflected laser light as a 3D range point cloud and simultaneously registered intensity data. In at least one embodiment, the flash lidar device may include a 3D flash lidar device using a 5-nanosecond Class I (eye-safe) laser pulse per frame.

[0186] In at least one embodiment, the vehicle 1100 may further include IMU sensor(s) 1166. In at least one embodiment, the IMU sensor(s) 1166 may be located at a center of a rear axle of the vehicle 1100. In at least one embodiment, the IMU sensor(s) 1166 may include, for example, and without limitation, accelerometers, magnetometers, gyroscopes, a magnetic compass, magnetic compasses, and / or other types of sensors. In at least one embodiment, such as in six-axis applications, the IMU sensor(s) 1166 may include, without limitation, accelerometers and gyroscopes. In at least one embodiment, such as in nine-axis applications, the IMU sensor(s) 1166 may include, without limitation, accelerometers, gyroscopes, and magnetometers.

[0187] In at least one embodiment, the IMU sensor(s) 1166 may be implemented as a miniaturized, high-performance GPS-based inertial navigation system ("GPS / INS") that combines microelectromechanical system ("MEMS") inertial sensors, a high-sensitivity GPS receiver, and advanced Kalman filtering algorithms to provide estimates of position, velocity, and attitude. In at least one embodiment, the IMU sensor(s) 1166 may enable the vehicle 1100 to estimate its heading without requiring input from a magnetic sensor by directly observing and correlating changes in velocity from a GPS to the IMU sensor(s) 1166. In at least one embodiment, the IMU sensor(s) 1166 and the GNSS sensor(s) 1158 may be combined into a single integrated unit.

[0188] In at least one embodiment, the vehicle 1100 may include microphone(s) 1196 placed in and / or around the vehicle 1100. In at least one embodiment, the microphone(s) 1196 may be used for, among other things, emergency vehicle detection and identification.

[0189] In at least one embodiment, the vehicle 1100 may further include any number of camera types, including stereo camera(s) 1168, wide-view camera(s) 1170, infrared camera(s) 1172, surround camera(s) 1174, long-range camera(s) 1198, medium-range camera(s) 1176, and / or other camera types. In at least one embodiment, cameras may be used to capture image data around an entire periphery of the vehicle 1100. In at least one embodiment, the type of cameras used depends on the vehicle 1100. In at least one embodiment, any combination of camera types may be used to provide necessary coverage around the vehicle 1100. In at least one embodiment, the number of cameras employed may vary depending on the embodiment.For example, in at least one embodiment, vehicle 1100 could include six cameras, seven cameras, ten cameras, twelve cameras, or another number of cameras. In at least one embodiment, cameras can support, by way of example and without limitation, Gigabit Multimedia Serial Link ("GMSL") and / or Gigabit Ethernet communications. In at least one embodiment, each camera could be configured as previously described with respect to [ ]. Fig. 11A and Fig. 11B will be described in more detail.

[0190] In at least one embodiment, the vehicle 1100 may further include vibration sensor(s) 1142. In at least one embodiment, the vibration sensor(s) 1142 may measure vibrations of components of the vehicle 1100, such as axle(s). In at least one embodiment, the vibration sensor(s) 1142 may measure vibrations of components of the vehicle 1100, such as For example, in at least one embodiment, changes in vibrations may indicate a change in road surfaces. In at least one embodiment, when two or more vibration sensors 1142 are used, differences between vibrations may be used to determine friction or slippage of the road surface (e.g., when there is a difference in vibration between a power-driven axle and a free-spinning axle).

[0191] In at least one embodiment, vehicle 1100 may include ADAS system 1138. In at least one embodiment, ADAS system 1138 may include, among other things, an SoC in some examples. In at least one embodiment, ADAS system 1138 may include, among other things, any number and combination of an autonomous / adaptive / automatic cruise control (“ACC”) system, a cooperative adaptive cruise control (“CACC”) system, a forward collision warning (“FCW”) system, an automatic emergency braking (“AEB”) system, a lane departure warning (“LDW”) system, a lane keep assist (“LKA”) system, a blind spot warning (“BSW”) system, a rear cross traffic alert (“RCTW”) system, a collision warning (“CW”) system, a lane centering (“LC”) system, and / or other systems, features, and / or functionality.

[0192] In at least one embodiment, the ACC system may use RADAR sensor(s) 1160, LIDAR sensor(s) 1164, and / or any number of cameras. In at least one embodiment, the ACC system may include a longitudinal ACC system and / or a lateral ACC system. In at least one embodiment, a longitudinal ACC system monitors and controls the distance to another vehicle immediately ahead of the vehicle 1100 and automatically adjusts the speed of the vehicle 1100 to maintain a safe distance from preceding vehicles. In at least one embodiment, a lateral ACC system performs distance monitoring and notifies the vehicle 1100 to change lanes if necessary. In at least one embodiment, lateral ACC relates to other ADAS applications, such as LC and CW.

[0193] In at least one embodiment, a CACC system utilizes information from other vehicles that may be received via network interface 1124 and / or one or more wireless antennas 1126 from other vehicles over a wireless connection or indirectly via a network connection (e.g., over the Internet). In at least one embodiment, direct connections may be provided through a vehicle-to-vehicle ("V2V") communication link, while indirect connections may be provided through an infrastructure-to-vehicle ("I2V") communication link. Generally, V2V communication provides information about immediately ahead vehicles (e.g., vehicles immediately in front of and in the same lane as vehicle 1100), while I2V communication provides information about traffic farther ahead.In at least one embodiment, a CACC system may include one or both of the I2V and V2V information sources. In at least one embodiment, a CACC system may be more reliable given information from vehicles ahead of vehicle 1100 and has the potential to improve traffic flow smoothness and reduce congestion on the road.

[0194] In at least one embodiment, an FCW system is designed to warn a driver of a hazard so that such a driver can take corrective action. In at least one embodiment, an FCW system uses a forward-facing camera and / or RADAR sensor(s) 1160 coupled to a dedicated processor, DSP, FPGA, and / or ASIC electrically coupled to provide driver feedback, such as a display, speaker, and / or vibration component. In at least one embodiment, an FCW system can provide a warning, such as in the form of a sound, a visual warning, a vibration, and / or a rapid braking pulse.

[0195] In at least one embodiment, an AEB system detects an impending forward collision with another vehicle or other object and may automatically apply braking if a driver does not take corrective action within a specified time or distance parameter. In at least one embodiment, the AEB system may utilize one or more forward-facing cameras and / or RADAR sensors 1160 coupled to a dedicated processor, DSP, FPGA, and / or ASIC. In at least one embodiment, when an AEB system detects a hazard, it will typically first alert a driver to take corrective action to avoid a collision, and if that driver does not take corrective action, that AEB system may automatically apply braking to prevent or at least mitigate the impact of a predicted collision.In at least one embodiment, an AEB system may include techniques such as dynamic brake support and / or imminent collision braking.

[0196] In at least one embodiment, an LDW system provides visual, audible, and / or tactile warnings, such as steering wheel or seat vibrations, to alert the driver when the vehicle 1100 crosses lane markings. In at least one embodiment, an LDW system is not activated when a driver indicates an intended lane departure, such as by activating a turn signal. In at least one embodiment, an LDW system may utilize forward-facing cameras coupled to a dedicated processor, DSP, FPGA, and / or ASIC electrically coupled to provide driver feedback, such as a display, speaker, and / or vibration component. In at least one embodiment, an LKA system is a variation of an LDW system. In at least one embodiment, an LKA system provides steering input or braking to correct the vehicle 1100 when the vehicle 1100 begins to depart from its lane.

[0197] In at least one embodiment, a BSW system detects and warns a driver of vehicles in an automobile's blind spot. In at least one embodiment, a BSW system may provide a visual, audible, and / or tactile warning to indicate that merging or changing lanes is unsafe. In at least one embodiment, a BSW system may provide an additional warning when a driver uses a turn signal. In at least one embodiment, a BSW system may utilize one or more rear-facing cameras and / or RADAR sensor(s) 1160 coupled to a dedicated processor, DSP, FPGA, and / or ASIC electrically coupled to driver feedback, such as a display, speaker, and / or vibration component.

[0198] In at least one embodiment, an RCTW system may provide visual, audible, and / or tactile notification when an object is detected outside of a rear camera range when the vehicle 1100 is reversing. In at least one embodiment, an RCTW system includes an AEB system to ensure that vehicle braking is applied to avoid a collision. In at least one embodiment, an RCTW system may utilize one or more rear-facing RADAR sensors 1160 coupled to a dedicated processor, DSP, FPGA, and / or ASIC electrically coupled to provide driver feedback, such as a display, speaker, and / or vibration component.

[0199] In at least one embodiment, conventional ADAS systems may be prone to false positives, which may be annoying and distracting for a driver, but are typically not catastrophic because conventional ADAS systems warn a driver and allow that driver to decide whether a safety condition actually exists and act accordingly. In at least one embodiment, the vehicle 1100 itself decides, in the case of conflicting results, whether the result from a primary computer or a secondary computer (e.g., a first controller or a second controller of the controllers 1136) should be considered. For example, in at least one embodiment, the ADAS system 1138 may be a backup and / or secondary computer for providing perception information to a backup computer rationality module.In at least one embodiment, a backup computer rationality monitor may execute redundant, diverse software on hardware components to detect errors in perception and dynamic driving tasks. In at least one embodiment, outputs from the ADAS system 1138 may be provided to a monitoring MCU. If outputs from a primary computer and outputs from a secondary computer conflict, a monitoring MCU, in at least one embodiment, determines how to reconcile the conflict to ensure safe operation.

[0200] In at least one embodiment, a primary computer may be configured to provide a monitoring MCU with a confidence score indicating the confidence of that primary computer in a selected outcome. If that confidence score exceeds a threshold, in at least one embodiment, that monitoring MCU may follow the instruction of that primary computer regardless of whether that secondary computer provides a conflicting or inconsistent outcome. If a confidence score does not meet a threshold and if primary and secondary computers indicate different outcomes (e.g., a conflict), in at least one embodiment, a monitoring MCU may arbitrate between computers to determine an appropriate outcome.

[0201] In at least one embodiment, a monitoring MCU may be configured to execute neural network(s) trained and configured to determine, based at least in part on outputs from a primary computer and outputs from a secondary computer, conditions under which that secondary computer provides false alarms. In at least one embodiment, neural network(s) in a monitoring MCU may learn when the output of a secondary computer can be trusted and when it cannot. For example, if that secondary computer is a radar-based FCW system, neural network(s) in that monitoring MCU may learn when an FCW system identifies metallic objects that are not actually hazards, such as a drainage grate or manhole cover, that trigger an alarm.When a secondary computer is a camera-based LDW system, in at least one embodiment, a neural network in a monitoring MCU may learn to override LDW when cyclists or pedestrians are present and lane departure is actually the safest maneuver. In at least one embodiment, a monitoring MCU may include at least one of a DLA or a GPU capable of executing a neural network(s) with associated memory. In at least one embodiment, a monitoring MCU may comprise and / or be included as a component of SoC(s) 1104.

[0202] In at least one embodiment, ADAS system 1138 may include a secondary computer that performs ADAS functionality using conventional computer vision rules. In at least one embodiment, this secondary computer may use classical computer vision (if-then) rules, and the presence of a neural network(s) in a monitoring MCU may improve reliability, safety, and performance. For example, in at least one embodiment, diverse implementation and intended non-identity makes an overall system more fault-tolerant, particularly against errors caused by software (or software-to-hardware interface) functionality.For example, in at least one embodiment, if there is a software bug or error in software executing on a primary computer and non-identical software code executing on a secondary computer provides a consistent overall result, then a monitoring MCU may have greater confidence that an overall result is correct and a bug in software or hardware on that primary computer does not cause a significant error.

[0203] In at least one embodiment, an output of the ADAS system 1138 may be fed into the perception block of a primary computer and / or the dynamic driving task block of a primary computer. For example, in at least one embodiment, if the ADAS system 1138 indicates a forward collision warning due to an object immediately ahead, a perception block may use this information when identifying objects. In at least one embodiment, a secondary computer may have its own neural network trained, thus reducing the risk of false positives, as described herein.

[0204] In at least one embodiment, the vehicle 1100 may further include an infotainment SoC 1130 (e.g., an in-vehicle infotainment system (IVI)). Although illustrated and described as an SoC, in at least one embodiment, the infotainment system SoC 1130 may not be an SoC and may include, among other things, two or more discrete components. In at least one embodiment, the infotainment SoC 1130 may include, among other things, a combination of hardware and software that may be used to provide the vehicle 1100 with audio (e.g., music, a personal digital assistant, navigation directions, news, radio, etc.), video (e.g., television, movies, streaming, etc.), phone (e.g., hands-free calling), network connectivity (e.g., LTE, Wi-Fi, etc.), and / or information services (e.g.,Navigation systems, backup assist, a radio data system, vehicle-related information such as fuel level, total distance traveled, brake fuel level, oil level, door opening / closing, air filter information, etc.). For example, the infotainment SoC 1130 could include radios, record players, navigation systems, video players, USB and Bluetooth connectivity, car putters, in-vehicle entertainment, Wi-Fi, steering wheel audio controls, hands-free voice control, a head-up display ("HUD"), an HMI display 1134, a telematics device, a control panel (e.g., for controlling and / or interacting with various components, features, and / or systems), and / or other components. In at least one embodiment, the infotainment SoC 1130 may be further used to provide information (e.g.,visual and / or audible), such as information from the ADAS system 1138, autonomous driving information such as planned vehicle maneuvers, trajectories, environmental information (e.g., intersection information, vehicle information, road information, etc.), and / or other information.

[0205] In at least one embodiment, the infotainment SoC 1130 may include any amount and type of GPU functionality. In at least one embodiment, the infotainment SoC 1130 may communicate with other devices, systems, and / or components of the vehicle 1100 via the bus 1102. In at least one embodiment, the infotainment SoC 1130 may be coupled to a supervisory MCU so that a GPU of an infotainment system can perform some self-driving functions in the event that the primary controller(s) 1136 (e.g., primary and / or backup computers of the vehicle 1100) fail. In at least one embodiment, the infotainment SoC 1130 may place the vehicle 1100 into a chauffeur-in-safe-stop mode, as described herein.

[0206] In at least one embodiment, the vehicle 1100 may further include an instrument cluster 1132 (e.g., a digital instrument panel, an electronic instrument cluster, a digital instrument panel, etc.). In at least one embodiment, the instrument cluster 1132 may include, among other things, a controller and / or a supercomputer (e.g., a discrete controller or a supercomputer). In at least one embodiment, the instrument cluster 1132 may include, among other things, any number and combination of a set of instruments, such as a speedometer, fuel level, oil pressure, tachometer, odometer, turn signals, gear shift position indicators, seat belt warning light(s), parking brake warning light(s), engine failure light(s), supplemental restraint system (e.g., airbag) information, lighting controls, safety system controls, navigation information, etc.In some examples, information may be displayed and / or shared between the infotainment SoC 1130 and the instrument cluster 1132. In at least one embodiment, the instrument cluster 1132 may be included as part of the infotainment SoC 1130, or vice versa.

[0207] In at least one embodiment, systems or components of Fig. 11 in components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or utilize vehicle 1100 to use one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0208] Fig. 11D is a diagram of a system for communication between one or more cloud-based servers and the autonomous vehicle 1100 of Fig. 11A according to at least one embodiment. In at least one embodiment, the system may include, among other things, one or more servers 1178, one or more networks 1190, and any number and type of vehicles, including vehicle 1100. In at least one embodiment, server(s) 1178 may include, among other things, a plurality of GPUs 1184(A)-1184(H) (collectively referred to herein as GPUs 1184), PCIe switches 1182(A)-1182(D) (collectively referred to herein as PCIe switches 1182), and / or CPUs 1180(A)-1180(B) (collectively referred to herein as CPUs 1180). In at least one embodiment, the GPUs 1184, the CPUs 1180, and the PCIe switches 1182 may be connected to high-speed interconnects, such as, but not limited to, NVLink interfaces 1188 developed by NVIDIA and / or PCIe interconnects 1186.In at least one embodiment, the GPUs 1184 are connected via an NVLink and / or an NVSwitch SoC, and the GPUs 1184 and the PCIe switches 1182 are connected via PCIe connections. Although eight GPUs 1184, two CPUs 1180, and four PCIe switches 1182 are illustrated, this is not intended to be limiting. In at least one embodiment, each of the servers 1178 may include, among other things, any number of GPUs 1184, CPUs 1180, and / or PCIe switches 1182 in any combination. For example, in at least one embodiment, the server(s) 1178 could each include eight, sixteen, thirty-two, and / or more GPUs 1184.

[0209] In at least one embodiment, the server(s) 1178 may receive, via one or more networks 1190 and from vehicles, image data representative of images depicting unexpected or changed road conditions, such as recently commenced roadwork. In at least one embodiment, the server(s) 1178 may transmit, via one or more networks 1190 and to vehicles, updated or updated neural networks 1192 and / or map information 1194 including, among other things, information related to traffic and road conditions. In at least one embodiment, updates to the map information 1194 may include, among other things, updates to the HD map 1122, such as information related to construction, potholes, detours, flooding, and / or other obstacles.In at least one embodiment, neural networks 1192 and / or map information 1194 may result from new training and / or experience represented in data received from any number of vehicles in an environment and / or may be based at least in part on training performed in a data center (e.g., using server(s) 1178 and / or other servers).

[0210] In at least one embodiment, the server(s) 1178 may be used to train machine learning models (e.g., neural networks) based at least in part on training data. In at least one embodiment, training data may be generated by vehicles and / or may be generated in a simulation (e.g., using a gaming machine). In at least one embodiment, any amount of training data is labeled (e.g., if the associated neural network benefits from supervised learning) and / or undergoes other preprocessing. In at least one embodiment, any amount of training data is unlabeled and / or preprocessed (e.g., if the associated neural network does not require supervised learning). In at least one embodiment, machine learning models may be used by vehicles (e.g.,transmitted to vehicles via network(s) 1190) once machine learning models are trained, and / or machine learning models may be used by server(s) 1178 to remotely monitor vehicles.

[0211] In at least one embodiment, servers 1178 may receive data from vehicles and apply the data to real-time neural networks for intelligent real-time inference. In at least one embodiment, servers 1178 may include deep learning supercomputers and / or dedicated AI computers powered by one or more GPUs 1184, such as a DGX and DGX Station machine developed by NVIDIA. However, in at least one embodiment, servers 1178 may include a deep learning infrastructure using CPU-powered data centers.

[0212] In at least one embodiment, the deep learning infrastructure of server(s) 1178 may be capable of fast, real-time inference and may use this capability to evaluate and verify the integrity of processors, software, and / or associated hardware in vehicle 1100. For example, in at least one embodiment, the deep learning infrastructure may receive periodic updates from vehicle 1100, such as a sequence of images and / or objects that vehicle 1100 has been located in that sequence of images (e.g., via computer vision and / or other machine learning object classification techniques).In at least one embodiment, the deep learning infrastructure may run its own neural network to identify objects and compare them to objects identified by the vehicle 1100, and if the results do not match and the deep learning infrastructure concludes that the AI ​​in the vehicle 1100 is malfunctioning, then servers 1178 may transmit a signal to the vehicle 1100 instructing a fail-safe computer of the vehicle 1100 to take over control, notify passengers, and complete a safe parking maneuver.

[0213] In at least one embodiment, servers 1178 may include one or more GPUs 1184 and one or more programmable inference accelerators (e.g., TensorRT 3 devices from NVIDIA). In at least one embodiment, a combination of GPU-powered servers and inference acceleration may enable real-time responsiveness. In at least one embodiment, such as when performance is less critical, servers powered by CPUs, FPGAs, and other processors may be used for inference. In at least one embodiment, one or more hardware structures 815 are used to perform one or more embodiments. Details of, e.g., the hardware structures 815 are described herein in connection with Fig. 8A and / or Fig. 8B provided. COMPUTER SYSTEM

[0214] Fig. 12 is a block diagram illustrating an example computer system, which may be a system with interconnected devices and components, a system-on-a-chip (SOC), or a combination thereof, formed with a processor that may include execution units for executing an instruction, according to at least one embodiment. In at least one embodiment, a computer system 1200 may include, without limitation, a component such as a processor 1202 for employing execution units that include logic for performing algorithms on process data, according to the present disclosure, such as in the embodiment described herein.In at least one embodiment, computer system 1200 may include processors such as the PENTIUM® processor family, Xeon™, Itanium®, XScale™ and / or StrongARM™, Intel® Core™, or Intel® Nervana™ microprocessors available from Intel Corporation of Santa Clara, California, although other systems (including PCs with other microprocessors, engineering workstations, set-top boxes, and the like) may be used. In at least one embodiment, computer system 1200 may run a version of the WINDOWS operating system available from Microsoft Corporation of Redmond, California, although other operating systems (e.g., UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0215] Embodiments may be used in other devices such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol devices, digital cameras, personal digital assistants ("PDAs"), and handheld PCs. In at least one embodiment, embedded applications may include a microcontroller, a digital signal processor ("DSP"), a system-on-a-chip, network computers ("NetPCs"), set-top boxes, network hubs, wide area network ("WAN") switches, or any other system capable of performing one or more instructions according to at least one embodiment.

[0216] In at least one embodiment, computer system 1200 may include, but is not limited to, a processor 1202, which may include, but is not limited to, one or more execution units 1208 to perform machine learning model training and / or inference according to techniques described herein. In at least one embodiment, computer system 1200 is a single-processor desktop or server system, but in another embodiment, computer system 1200 may be a multiprocessor system.In at least one embodiment, processor 1202 may include, but is not limited to, a complex instruction set computer ("CISC") microprocessor, a reduced instruction set computer ("RISC") microprocessor, a very long instruction word ("VLIW") microprocessor, a processor implementing a combination of instruction sets, or any other processor device, such as a digital signal processor. In at least one embodiment, processor 1202 may be coupled to a processor bus 1210 that may communicate data signals between processor 1202 and other components in computer system 1200.

[0217] In at least one embodiment, the processor 1202 may include, but is not limited to, an internal Level 1 ("L1") cache memory ("cache") 1204. In at least one embodiment, the processor 1202 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the processor 1202 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache memory may be external to the processor 1202. Other embodiments may also include a combination of both internal and external caches depending on a particular implementation and requirements.In at least one embodiment, a register file 1206 may store various types of data in various registers, including, but not limited to, integer registers, floating point registers, status registers, and an instruction pointer register.

[0218] In at least one embodiment, execution unit 1208, including, but not limited to, logic for performing integer and floating-point operations, is also located in processor 1202. In at least one embodiment, processor 1202 may also include microcode ("ucode") read-only memory ("ROM") that stores microcode for certain macroinstructions. In at least one embodiment, execution unit 1208 may include logic for handling a packed instruction set 1209. In at least one embodiment, by incorporating packed instruction set 1209 into an instruction set of a general-purpose processor, along with associated instruction-executing circuitry, operations used by many multimedia applications may be performed using packed data in processor 1202.In at least one embodiment, many multimedia applications may be accelerated and executed more efficiently by using a full width of a processor's data bus to perform operations on packed data, which may eliminate a need to transfer smaller units of data across that processor's data bus to perform one or more operations one data item at a time.

[0219] In at least one embodiment, execution unit 1208 may also be used in microcontrollers, embedded processors, graphics devices, DSPs, and other types of logic circuits. In at least one embodiment, computer system 1200 may include, among other things, a memory 1220. In at least one embodiment, memory 1220 may be a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, a flash memory device, or another storage device. In at least one embodiment, memory 1220 may store instruction(s) 1219 and / or data 1221 represented by data signals that may be executed by processor 1202.

[0220] In at least one embodiment, a system logic chip may be coupled to the processor bus 1210 and the memory 1220. In at least one embodiment, a system logic chip may include, among other things, a memory controller hub ("MCH") 1216, and the processor 1202 may communicate with the MCH 1216 via the processor bus 1210. In at least one embodiment, the MCH 1216 may provide a high-bandwidth memory path 1218 to the memory 1220 for instruction and data storage and for storing graphics commands, data, and textures. In at least one embodiment, the MCH 1216 may route data signals between the processor 1202, the memory 1220, and other components in the computer system 1200, and may bridge data signals between the processor bus 1210, the memory 1220, and a system I / O interface 1222. In at least one embodiment, a system logic chip may provide a graphics port for coupling to a graphics controller.In at least one embodiment, the MCH 1216 may be coupled to the memory 1220 through the high-bandwidth memory path 1218, and a graphics / video card 1212 may be coupled to the MCH 1216 through an Accelerated Graphics Port ("AGP") connection 1214.

[0221] In at least one embodiment, computer system 1200 may use system I / O interface 1222 as a proprietary hub interface bus to couple MCH 1216 to an I / O controller hub ("ICH") 1230. In at least one embodiment, ICH 1230 may provide direct connections to some I / O devices via a local I / O bus. In at least one embodiment, a local I / O bus may include, without limitation, a high-speed I / O bus for connecting peripherals to memory 1220, a chipset, and processor 1202.Examples may include, without limitation, an audio controller 1229, a firmware hub ("Flash BIOS") 1228, a wireless transceiver 1226, a data storage 1224, a legacy I / O controller 1223 containing user input and keyboard interfaces 1225, a serial expansion port 1227, such as a Universal Serial Bus ("USB") port, and a network controller 1234. In at least one embodiment, the data storage 1224 may comprise a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage device.

[0222] In at least one embodiment, Fig. 12 a system that includes interconnected hardware devices or ‘chips’, whereas Fig. 12 may illustrate an exemplary SoC in other embodiments. In at least one embodiment, Fig. 12 may be connected using proprietary connections, standardized connections (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components of computer system 1200 are interconnected using Compute Express Link (CXL) interconnects.

[0223] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in computer system 1200 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0224] In at least one embodiment, systems or components of Fig. 8A or Fig. 8B in components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or utilize logic 815 to use one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0225] Fig. 13 is a block diagram illustrating an electronic device 1300 for using a processor 1310 according to at least one embodiment. In at least one embodiment, the electronic device 1300 may be, for example and without limitation, a notebook, a master server, a rack server, a blade server, a laptop, a desktop, a tablet, a mobile device, a phone, an embedded computer, or any other suitable electronic device.

[0226] In at least one embodiment, electronic device 1300 may include, without limitation, processor 1310 communicatively coupled to any suitable number or type of components, peripherals, modules, or devices. In at least one embodiment, processor 1310 is coupled using a bus or interface, such as an I 2C-Bus, a System Management Bus (“SM-Bus”), a Low Pin Count (LPC) Bus, a Serial Peripheral Interface (“SPI”), a High Definition Audio (“HDA”) Bus, a Serial Advanced Technology Port (“SATA”) Bus, a Universal Serial Bus (“USB”) (versions 1, 2, 3, etc.), or a Universal Asynchronous Receiver / Transmitter (“UART”) Bus. In at least one embodiment, Fig. 13 a system that includes interconnected hardware devices or ‘chips’, whereas Fig. 13 may illustrate an exemplary SoC in other embodiments. In at least one embodiment, Fig. 13 may be connected to proprietary connections, standardized connections (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components of Fig. 13 are connected using Compute Express Link (CXL) connections.

[0227] In at least one embodiment, Fig. 13 a display 1324, a touchscreen 1325, a touchpad 1330, a near field communication unit (“NFC”) 1345, a sensor hub 1340, a thermal sensor 1346, an express chipset (“EC”) 1335, a trusted platform module (“TPM”) 1338, BIOS / firmware / flash memory (“BIOS, FW Flash”) 1322, a DSP 1360, a drive 1320, such as a solid-state disk (“SSD”) or a hard disk drive (“HDD”), a wireless local area network unit (“WLAN”) 1350, a Bluetooth unit 1352, a wireless wide area network unit (“WWAN”) 1356, a global positioning system (GPS) unit 1355, a camera (“USB 3.0 camera”) 1354, such as a USB 3.0 camera, and / or a low-density dual data rate ("LPDDR") ("LPDDR3") memory device 1315, implemented, for example, in an LPDDR3 standard. These components may each be implemented in any suitable manner.

[0228] In at least one embodiment, other components may be communicatively coupled to processor 1310 through components described herein. In at least one embodiment, an accelerometer 1341, an ambient light sensor ("ALS") 1342, a compass 1343, and a gyroscope 1344 may be communicatively coupled to sensor hub 1340. In at least one embodiment, a thermal sensor 1339, a fan 1337, a keyboard 1336, and a touchpad 1330 may be communicatively coupled to EC 1335. In at least one embodiment, speakers 1363, headphones 1364, and a microphone ("mic") 1365 may be communicatively coupled to an audio unit ("audio codec and class-D amplifier") 1362, which in turn may be communicatively coupled to DSP 1360. In at least one embodiment, the audio unit 1362 may include, for example and without limitation, an audio encoder / decoder ("codec") and a Class D amplifier.In at least one embodiment, a SIM card ("SIM") 1357 may be communicatively coupled to the WWAN unit 1356. In at least one embodiment, components such as the WLAN unit 1350 and the Bluetooth unit 1352, as well as the WWAN unit 1356, may be implemented in a next-generation form factor ("NGFF").

[0229] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in electronic device 1300 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0230] In at least one embodiment, systems or components of Fig. 8A or Fig. 8B in components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or use logic 815 to implement one or more neural networks (e.g., network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0231] Fig. Figure 14 illustrates a computer system 1400 according to at least one embodiment. In at least one embodiment, the computer system 1400 is configured to implement various processes and methods described in this disclosure.

[0232] In at least one embodiment, computer system 1400 includes, without limitation, at least one central processing unit ("CPU") 1402 connected to a communications bus 1410 implemented using any suitable protocol, such as PCI ("Peripheral Component Interconnect"), Peripheral Component Interconnect Express ("PCI-Express"), AGP ("Accelerated Graphics Port"), HyperTransport, or any other bus or point-to-point communications protocol(s). In at least one embodiment, computer system 1400 includes, without limitation, main memory 1404 and control logic (e.g., implemented as hardware, software, or a combination thereof), and data is stored in main memory 1404, which may take the form of random access memory ("RAM").In at least one embodiment, a network interface subsystem ("network interface") 1422 provides an interface to other computing devices and networks for receiving data from and transmitting data to other systems with computer system 1400.

[0233] In at least one embodiment, computer system 1400 includes, without limitation, input devices 1408, a parallel processing system 1412, and display devices 1406, which may be implemented using a conventional cathode ray tube ("CRT"), a liquid crystal display ("LCD"), a light-emitting diode ("LED"), a plasma display, or other suitable display technology. In at least one embodiment, user input is received from input devices 1408, such as a keyboard, mouse, touchpad, microphone, etc. In at least one embodiment, each module described herein may be arranged on a single semiconductor platform to form a processing system.

[0234] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding inference and / or training logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in computer system 1400 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0235] In at least one embodiment, systems or components of Fig. 8A or Fig. 8B in components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or use logic 815 to implement one or more neural networks (e.g., network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0236] Fig. 15 illustrates a computer system 1500 according to at least one embodiment. In at least one embodiment, the computer system 1500 includes, among other things, a computer 1510 and a USB flash drive 1520. In at least one embodiment, the computer 1510 may include, among other things, any number and type of processor(s) (not shown) and memory (not shown). In at least one embodiment, the computer 1510 includes, among other things, a server, a cloud instance, a laptop, and a desktop computer.

[0237] In at least one embodiment, the USB flash drive 1520 includes, among other things, a processing unit 1530, a USB interface 1540, and USB interface logic 1550. In at least one embodiment, the processing unit 1530 may be any instruction execution system, instruction execution apparatus, or instruction execution device capable of executing instructions. In at least one embodiment, the processing unit 1530 may include, among other things, any number and type of processing cores (not shown). In at least one embodiment, the processing unit 1530 comprises an application-specific integrated circuit ("ASIC") optimized to perform any set and type of operations associated with machine learning.For example, in at least one embodiment, processing unit 1530 is a tensor processing unit ("TPC") optimized to perform inference operations for machine learning. In at least one embodiment, processing unit 1530 is a vision processing unit ("VPU") optimized to perform inference operations for machine vision and machine learning.

[0238] In at least one embodiment, USB interface 1540 may be any type of USB connector or USB receptacle. For example, in at least one embodiment, USB interface 1540 is a USB 3.0 Type-C receptacle for data and power. In at least one embodiment, USB interface 1540 is a USB 3.0 Type-A connector. In at least one embodiment, USB interface logic 1550 may include any amount and type of logic that enables processing unit 1530 to interface with devices (e.g., computer 1510) via USB connector 1540.

[0239] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in computer system 1500 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0240] In at least one embodiment, systems or components of Fig. 8A or Fig. 8B in components, systems and / or processes that are Fig. 1-7. For example, the processor 105 may include or use logic 815 to implement one or more neural networks (e.g., the network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0241] Fig. 16A illustrates an example architecture in which a plurality of GPUs 1610(1)-1610(N) are communicatively coupled to a plurality of multi-core processors 1605(1)-1605(M) via high-speed interconnects 1640(1)-1640(N) (e.g., buses, point-to-point interconnects, etc.). In at least one embodiment, high-speed interconnects 1640(1)-1640(N) support communication throughput of 4 GB / s, 30 GB / s, 80 GB / s, or higher. In at least one embodiment, various interconnect protocols may be used, including, but not limited to, PCIe 4.0 or 5.0 and NVLink 2.0. In various figures, "N" and "M" represent positive integers, the values ​​of which may vary from figure to figure. In at least one embodiment, one or more GPUs in a plurality of GPUs 1610(1)-1610(N) include one or more graphics cores (also referred to simply as “cores”) 1900, as shown in Fig. 19A and Fig. 19B. In at least one embodiment, one or more graphics cores 1900 may be referred to as streaming multiprocessors ("SMs"), stream processors ("SPs"), stream processing units ("SPUs"), compute units ("CUs"), execution units ("EUs"), and / or slices, where a slice in this context may refer to a portion of processing resources in a processing unit (e.g., 16 cores, a ray tracing unit, a thread director, or scheduler).

[0242] Additionally, and in at least one embodiment, two or more of the GPUs 1610 are interconnected via high-speed interconnects 1629(1)-1629(2), which may be implemented using similar or different protocols / connections than those used for high-speed interconnects 1640(1)-1640(N). Likewise, two or more of the multi-core processors 1605 may be interconnected via a high-speed interconnect 1628, which may be symmetric multiprocessor (SMP) buses operating at 20 GB / s, 30 GB / s, 120 GB / s, or higher. Alternatively, all communication between different Fig. 16A using similar protocols / connections (e.g., via a common connection structure).

[0243] In at least one embodiment, each multi-core processor 1605 is communicatively coupled to a processor memory 1601(1)-1601(M) via memory interconnects 1626(1)-1626(M), respectively, and each GPU 1610(1)-1610(N) is communicatively coupled to GPU memory 1620(1)-1620(N) via GPU memory interconnects 1650(1)-1650(N), respectively. In at least one embodiment, memory interconnects 1626 and 1650 may use similar or different memory access technologies. For example, and not by way of limitation, the processor memories 1601(1)-1601(M) and the GPU memories 1620 may be volatile memories such as dynamic random access memories (DRAMs) (including stacked DRAMs), graphics DDR SDRAM (GDDR) (e.g., GDDR5, GDDR6), or high bandwidth memories (HBM), and / or may be non-volatile memories such as 3D XPoint or Nano-Ram.In at least one embodiment, a portion of processor memory 1601 may be volatile memory and another portion may be non-volatile memory (e.g., using a two-level memory (2LM) hierarchy).

[0244] As described herein, various multi-core processors 1605 and GPUs 1610 may each be physically coupled to a particular memory 1601, 1620, and / or a unified memory architecture may be implemented in which a virtual system address space (also referred to as "effective address space") is distributed across different physical memories. For example, processor memories 1601(1)-1601(M) may each comprise 64 GB of system memory address space, and GPU memories 1620(1)-1620(N) may each comprise 32 GB of system memory address space, resulting in a total of 256 GB of addressable memory when M = 2 and N = 4. Other values ​​for N and M are possible.

[0245] Fig. 16B illustrates additional details for a connection between a multi-core processor 1607 and a graphics acceleration module 1646 according to an example embodiment. In at least one embodiment, the graphics acceleration module 1646 may include one or more GPU chips integrated on a line card coupled to the processor 1607 via a high-speed interconnect 1640 (e.g., a PCIe bus, NVLink, etc.). Alternatively, in at least one embodiment, the graphics acceleration module 1646 may be integrated on a package or die with the processor 1607.

[0246] In at least one embodiment, processor 1607 includes a plurality of cores 1660A-1660D (which may be referred to as "execution units"), each having a translation buffer ("TLB") 1661A-1661D and one or more caches 1662A-1662D. In at least one embodiment, cores 1660A-1660D may include various other components for executing instructions and processing data, not illustrated. In at least one embodiment, caches 1662A-1662D may include Level 1 (L1) and Level 2 (L2) caches. Additionally, one or more shared caches 1656 may be included within caches 1662A-1662D and shared among sets of cores 1660A-1660D. For example, one embodiment of processor 1607 includes 24 cores, each with its own L1 cache, twelve shared L2 caches, and twelve shared L3 caches.In this embodiment, one or more L2 and L3 caches are shared between two adjacent cores. In at least one embodiment, processor 1607 and graphics acceleration module 1646 are coupled to system memory 1614, which includes processor memories 1601(1)-1601(M) of FIG. Fig. 16A may include.

[0247] In at least one embodiment, coherency for data and instructions stored in various caches 1662A-1662D, 1656, and system memory 1614 is maintained via intercore communication over a coherency bus 1664. For example, in at least one embodiment, each cache may have cache coherency logic / circuitry associated with it to communicate over the coherency bus 1664 in response to detected reads or writes to particular cache lines. In at least one embodiment, a cache sniffing protocol is implemented over the coherency bus 1664 to sniff cache accesses.

[0248] In at least one embodiment, a proxy circuit 1625 communicatively couples the graphics acceleration module 1646 to the coherence bus 1664, enabling the graphics acceleration module 1646 to participate in a cache coherence protocol as a peer of the cores 1660A-1660D. Specifically, in at least one embodiment, an interface 1635 provides connectivity to the proxy circuit 1625 via the high-speed interconnect 1640, and an interface 1637 connects the graphics acceleration module 1646 to the high-speed interconnect 1640.

[0249] In at least one embodiment, an accelerator integration circuit 1636 provides cache management, memory access, context management, and interrupt management services for a plurality of graphics processing engines 1631(1)-1631(N) of the graphics acceleration module 1646. In at least one embodiment, the graphics processing engines 1631(1)-1631(N) may each include a separate graphics processing unit (GPU). In at least one embodiment, the plurality of graphics processing engines 1631(1)-1631(N) of the graphics acceleration module 1646 includes one or more graphics cores 1900, as described in connection with Fig. 19A and Fig. 19B. In at least one embodiment, the graphics processing engines 1631(1)-1631(N) may alternatively comprise various types of graphics processing engines within a GPU, such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and blit engines. In at least one embodiment, the graphics acceleration module 1646 may be a GPU with a plurality of graphics processing engines 1631(1)-1631(N), or the graphics processing engines 1631(1)-1631(N) may be individual GPUs integrated on a common package, line card, or die.

[0250] In at least one embodiment, accelerator integration circuit 1636 includes a memory management unit (MMU) 1639 for performing various memory management functions, such as virtual-to-physical memory translations (also referred to as effective-to-real memory translations) and memory access protocols for accessing system memory 1614. In at least one embodiment, MMU 1639 may also include a translation buffer (TLB) (not shown) for caching virtual / effective to physical / real address translations. In at least one embodiment, a cache 1638 may store instructions and data for efficient access by graphics processing engines 1631(1)-1631(N).In at least one embodiment, data stored in cache 1638 and graphics memories 1633(1)-1633(M) is kept coherent with core caches 1662A-1662D, 1656, and system memory 1614, possibly using a fetch unit 1644. As noted, this may be accomplished via proxy circuitry 1625 for cache 1638 and memories 1633(1)-1633(M) (e.g., sending updates to cache 1638 regarding modifications / accesses to cache lines on processor caches 1662A-1662D, 1656 and receiving updates from cache 1638).

[0251] In at least one embodiment, a set of registers 1645 stores context data for threads executed by graphics processing engines 1631(1)-1631(N), and a context management circuit 1648 manages thread contexts. For example, context management circuit 1648 may perform save and restore operations to save and restore contexts of different threads during context switches (e.g., saving a first thread and saving a second thread so that a second thread can be executed by a graphics processing engine). For example, on a context switch, context management circuit 1648 may save current register values ​​to a designated region in memory (e.g., identified by a context pointer). It may then restore register values ​​when returning to a context.In at least one embodiment, an interrupt management circuit 1647 receives and processes interrupts received from system devices.

[0252] In at least one embodiment, virtual / effective addresses from a graphics processing engine 1631 are translated by the MMU 1639 into real / physical addresses in system memory 1614. In at least one embodiment, the accelerator integration circuit 1636 supports multiple (e.g., 4, 8, 16) graphics accelerator modules 1646 and / or other accelerator devices. In at least one embodiment, the graphics accelerator module 1646 may be dedicated to a single application executing on the processor 1607 or may be shared among multiple applications. In at least one embodiment, a virtualized graphics execution environment is depicted in which resources of the graphics processing engines 1631(1)-1631(N) are shared among multiple applications or virtual machines (VMs).In at least one embodiment, resources may be divided into "slices" that are assigned to different VMs and / or applications based on processing requirements and priorities associated with VMs and / or applications.

[0253] In at least one embodiment, accelerator integration circuitry 1636 operates as a bridge to a system for graphics acceleration module 1646, providing address translation and system memory caching services. Additionally, in at least one embodiment, accelerator integration circuitry 1636 may provide virtualization facilities to a host processor to manage the virtualization of graphics processing engines 1631(1)-1631(N), interrupts, and memory management.

[0254] Because hardware resources of graphics processing engines 1631(1)-1631(N) are explicitly mapped to a real address space seen by host processor 1607, each host processor can directly address these resources using an effective address value. In at least one embodiment, a function of accelerator integration circuit 1636 is to physically separate graphics processing engines 1631(1)-1631(N) so that they appear to a system as independent entities.

[0255] In at least one embodiment, one or more graphics memories 1633(1)-1633(M) are each coupled to each of the graphics processing engines 1631(1)-1631(N), and N = M. In at least one embodiment, the graphics memories 1633(1)-1633(M) store instructions and data processed by each of the graphics processing engines 1631(1)-1631(N). In at least one embodiment, the graphics memories 1633(1)-1633(M) may be volatile memories such as DRAMs (including stacked DRAMs), GDDR memories (e.g., GDDR5, GDDR6), or HBM, and / or may be non-volatile memories such as 3D XPoint or Nano-Ram.

[0256] To reduce data traffic over high-speed interconnect 1640, in at least one embodiment, biasing techniques may be used to ensure that data stored in graphics memories 1633(1)-1633(M) is data most frequently used by graphics processing engines 1631(1)-1631(N) and preferably not used (at least not frequently) by cores 1660A-1660D. Similarly, in at least one embodiment, a biasing mechanism attempts to keep data needed by the cores (and preferably not by graphics processing engines 1631(1)-1631(N)) within caches 1662A-1662D, 1656, and system memory 1614.

[0257] Fig. 16C illustrates another exemplary embodiment in which accelerator integration circuitry 1636 is integrated within processor 1607. In this embodiment, graphics processing engines 1631(1)-1631(N) communicate directly over high-speed interconnect 1640 with accelerator integration circuitry 1636 via interface 1637 and interface 1635 (which may again be any form of bus or interface protocol). In at least one embodiment, accelerator integration circuitry 1636 may perform similar operations to those described with respect to Fig. 16B, but possibly with higher throughput given their close proximity to the coherence bus 1664 and the caches 1662A-1662D, 1656. In at least one embodiment, an accelerator integration circuit supports various programming models, including a dedicated process programming model (no graphics acceleration module virtualization) and shared programming models (with virtualization), which may include programming models controlled by the accelerator integration circuit 1636 and programming models controlled by the graphics acceleration module 1646.

[0258] In at least one embodiment, graphics processing engines 1631(1)-1631(N) are dedicated to a single application or process under a single operating system. In at least one embodiment, a single application can forward other application requests to graphics processing engines 1631(1)-1631(N), thereby providing virtualization within a VM / partition.

[0259] In at least one embodiment, the graphics processing engines 1631(1)-1631(N) may be shared between multiple VM / application partitions. In at least one embodiment, shared models may use a system hypervisor to virtualize the graphics processing engines 1631(1)-1631(N) to provide access by any operating system. In at least one embodiment, for single-partition systems without a hypervisor, the graphics processing engines 1631(1)-1631(N) are owned by an operating system. In at least one embodiment, an operating system may virtualize the graphics processing engines 1631(1)-1631(N) to provide access to any process or application.

[0260] In at least one embodiment, the graphics acceleration module 1646 or an individual graphics processing engine 1631(1)-1631(N) selects a process element using a process handle. In at least one embodiment, process elements are stored in system memory 1614 and are addressable using an effective address-to-real address translation technique described herein. In at least one embodiment, a process handle may be an implementation-specific value provided to a host process when its context is registered with the graphics processing engine 1631(1)-1631(N) (i.e., system software is invoked to add a process element to a linked list of process elements). In at least one embodiment, a lower 16-bit of a process handle may be an offset of a process element within a linked list of process elements.

[0261] Fig. 16D illustrates an exemplary accelerator integration slice 1690. In at least one embodiment, a "slice" comprises a specified portion of processing resources of accelerator integration circuitry 1636. In at least one embodiment, an application stores process elements 1683 in an effective address space 1682 within system memory 1614. In at least one embodiment, process elements 1683 are stored in response to GPU calls 1681 from applications 1680 executing on processor 1607. In at least one embodiment, a process element 1683 contains process state for the corresponding application 1680. In at least one embodiment, a work descriptor (WD) 1684 contained within process element 1683 may be a single job requested by an application or may contain a pointer to a queue of jobs.In at least one embodiment, the WD 1684 is a pointer to a job request queue in the effective address space 1682 of an application.

[0262] In at least one embodiment, the graphics acceleration module 1646 and / or individual graphics processing engines 1631(1)-1631(N) may be shared by all or a subset of processes in a system. In at least one embodiment, an infrastructure for establishing process states and sending a WD 1684 to a graphics acceleration module 1646 to start a job in a virtualized environment may be included.

[0263] In at least one embodiment, a dedicated process programming model is implementation-specific. In at least one embodiment, in this model, a single process owns the graphics acceleration module 1646 or a single graphics processing engine 1631. In at least one embodiment, when the graphics acceleration module 1646 is owned by a single process, a hypervisor initializes the accelerator integration circuit 1636 for an ownership partition, and an operating system initializes the accelerator integration circuit 1636 for an ownership process when the graphics acceleration module 1646 is assigned.

[0264] In operation, in at least one embodiment, a WD fetch unit 1691 in the accelerator integration slice 1690 fetches the next WD 1684, which includes an indication of work to be performed by one or more graphics processing engines of the graphics acceleration module 1646. In at least one embodiment, data from the WD 1684 may be stored in registers 1645 and used by the MMU 1639, the interrupt management circuitry 1647, and / or the context management circuitry 1648, as illustrated. For example, one embodiment of the MMU 1639 includes segment / page browsing circuitry for accessing segment / page tables 1686 within an OS virtual address space 1685. In at least one embodiment, the interrupt management circuitry 1647 may process interrupt events 1692 received from the graphics acceleration module 1646.In at least one embodiment, when performing graphics operations, an effective address 1693 generated by a graphics processing engine 1631(1)-1631(N) is translated into a real address by the MMU 1639.

[0265] In at least one embodiment, registers 1645 are duplicated for each graphics processing engine 1631(1)-1631(N) and / or each graphics acceleration module 1646 and may be initialized by a hypervisor or an operating system. In at least one embodiment, each of these duplicated registers may be included in an accelerator integration slice 1690. Example registers that may be initialized by a hypervisor are shown in Table 1. Table 1 - Hypervisor-initialized registers Register # Beschreibung 1 Scheibensteuerregister 2 Real Address (RA) Scheduled Processes Area Pointer 3 Authority Mask Override Register 4 Interrupt Vector Table Entry Offset 5 Interrupt Vector Table Entry Limit 6 State Register 7 Logical Partition ID 8 Real Address (RA) Hypervisor Accelerator Utilization Record Pointer 9 Storage Description Register

[0266] Example registers that can be initialized by an operating system are shown in Table 2. Table 2 - Operating system initialized registers Register # Description 1 Process and thread identification 2 Effective Address (EA) Context Save / Restore Pointer 3 Virtual Address (VA) Accelerator Utilization Record Pointer 4 Virtual Address (VA) Storage Segment Table Pointer 5 Authority Mask 6 Work descriptor

[0267] In at least one embodiment, each WD 1684 is specific to a particular graphics acceleration module 1646 and / or graphics processing engines 1631(1)-1631(N). In at least one embodiment, it contains all the information required by a graphics processing engine 1631(1)-1631(N) to perform work, or it may be a pointer to a memory location where an application has established a command queue for work to be performed.

[0268] Fig.16E illustrates additional details for an exemplary embodiment of a shared model. This embodiment includes a hypervisor real address space 1698 in which a process element list 1699 is stored. In at least one embodiment, the hypervisor real address space 1698 is accessible via a hypervisor 1696 that virtualizes graphics acceleration engine engines for the operating system 1695.

[0269] In at least one embodiment, shared programming models enable all or a subset of processes from all or a subset of partitions in a system to use a graphics acceleration module 1646. In at least one embodiment, there are two programming models where the graphics acceleration module 1646 is shared among multiple processes and partitions: shared time slices and shared graphics controllers.

[0270] In at least one embodiment, in this model, the system hypervisor 1696 owns the graphics acceleration module 1646 and makes its functionality available to all operating systems 1695. In at least one embodiment, for a graphics acceleration module 1646 that supports virtualization by the system hypervisor 1696, the graphics acceleration module 1646 may meet certain requirements, such as (1) the job request of an application must be autonomous (i.e.the state does not need to be maintained between jobs), or the graphics acceleration module 1646 must provide a context save and restore mechanism, (2) an application's job request is guaranteed by the graphics acceleration module 1646 to complete in a certain amount of time, including any translation errors, or the graphics acceleration module 1646 provides an ability to prevent a job from being processed, and (3) the graphics acceleration module 1646 must be guaranteed fairness between processes when operating in a controlled shared programming model.

[0271] In at least one embodiment, the application 1680 is required to make an operating system 1695 system call with a graphics acceleration module type, a work descriptor (WD), an authority mask register (AMR) value, and a context save / restore area pointer (CSRP). In at least one embodiment, the graphics acceleration module type describes a targeted acceleration function for a system call. In at least one embodiment, the graphics acceleration module type may be a system-specific value.In at least one embodiment, the WD is specifically formatted for the graphics acceleration module 1646 and may be in the form of an instruction of the graphics acceleration module 1646, an effective address pointer to a user-defined structure, an effective address pointer to a queue of instructions, or any other data structure to describe work to be performed by the graphics acceleration module 1646.

[0272] In at least one embodiment, an AMR value is an AMR state to be used for a current process. In at least one embodiment, a value passed to an operating system is similar to an application setting an AMR. In at least one embodiment, when accelerator integration circuitry 1636 (not shown) and implementations of graphics acceleration module 1646 do not support a User Authority Mask Override Register (UAMOR), an operating system may apply a current UAMOR value to an AMR value before passing an AMR in a hypervisor call. In at least one embodiment, hypervisor 1696 may optionally apply a current Authority Mask Override Register (AMOR) value before placing an AMR into process element 1683.In at least one embodiment, CSRP is one of the registers 1645 that contains an effective address of a region in the effective address space 1682 of an application for the graphics acceleration module 1646 to save and restore context state. In at least one embodiment, this pointer is optional when no state needs to be saved between jobs or when a job is preempted. In at least one embodiment, the context save / restore region may be pinned system memory.

[0273] Upon receiving a system call, the operating system 1695 may verify that the application 1680 has been registered and granted permission to use the graphics acceleration module 1646. In at least one embodiment, the operating system 1695 then calls the hypervisor 1696 with the information shown in Table 3. Table 3 - OS-to-Hypervisor call parameters Parameters # Description 1 A work descriptor (WD) 2 An Authority Mask Register (AMR) value (possibly masked) 3 An Effective Address (EA) Context Save / Restore Area Pointer (CSRP) 4 A process ID (PID) and optional thread ID (TID) 5 A Virtual Address (VA) Accelerator Utilization Record Pointer (AURP) 6 Virtual Address of Storage Segment Table Pointer (SSTP) 7 A Logical Interrupt Service Number (LISN)

[0274] In at least one embodiment, upon receiving a hypervisor call, the hypervisor 1696 verifies that the operating system 1695 has been registered and authorized to use the graphics acceleration module 1646. In at least one embodiment, the hypervisor 1696 then places the process element 1683 in a linked list of process elements for a corresponding graphics acceleration module type 1646. In at least one embodiment, a process element may include information shown in Table 4. Table 4 - Process element information Item # Description 1 A work descriptor (WD) 2 An Authority Mask Register (AMR) value (possibly masked) 3 An Effective Address (EA) Context Save / Restore Area Pointer (CSRP) 4 A process ID (PID) and optional thread ID (TID) 5 Ein Virtual Address (VA) Accelerator Utilization Record Pointer (AURP) 6 Virtual Address of Storage Segment Table Pointer (SSTP) 7 Eine Logical Interrupt Service Number (LISN) 8 Interrupt Vector Table, abgeleitet aus Hypervisor-Aufrufparametern 9 Ein State Register (SR) Wert 10 Eine Logical Partition ID (LPID) 11 Ein Real Address (RA) Hypervisor Accelerator Utilization Record Pointer 12 Storage Descriptor Register (SDR)

[0275] In at least one embodiment, the hypervisor initializes a plurality of accelerator integration slices 1690 registers 1645.

[0276] As in Fig. 16F, in at least one embodiment, a unified memory addressable via a common virtual memory address space is used to access physical processor memories 1601(1)-1601(N) and GPU memories 1620(1)-1620(N). In this implementation, operations performed on GPUs 1610(1)-1610(N) use a same virtual / effective memory address space to access processor memories 1601(1)-1601(M) and vice versa, thereby simplifying programmability. In at least one embodiment, a first portion of a virtual / effective address space is assigned to processor memory 1601(1), a second portion is assigned to second processor memory 1601(N), a third portion is assigned to GPU memory 1620(1), and so on.In at least one embodiment, this distributes an entire virtual / effective memory space (sometimes referred to as effective address space) across each of the processor memories 1601 and GPU memories 1620, allowing any processor or GPU to access any physical memory with a virtual address associated with that memory.

[0277] In at least one embodiment, bias / coherence management circuitry 1694A-1694E within one or more of MMUs 1639A-1639E ensures cache coherence between caches of one or more host processors (e.g., 1605) and GPUs 1610 and implements bias techniques that indicate physical memories in which certain data types should be stored. In at least one embodiment, while multiple instances of bias / coherence management circuitry 1694A-1694E in Fig. 16F, the bias / coherence circuit may be implemented within an MMU of one or more host processors 1605 and / or within the accelerator integration circuit 1636.

[0278] One embodiment enables GPU memories 1620 to be allocated as part of system memory and accessed using shared virtual memory (SVM) technology, but without incurring performance penalties associated with full system cache coherence. In at least one embodiment, an ability for GPU memories 1620 to be accessed as system memory without costly cache coherence overhead provides a favorable operating environment for GPU offloading. In at least one embodiment, this arrangement enables host processor 1605 software to set up operands and access computation results without the overhead of traditional I / O DMA data copies. In at least one embodiment, such traditional copies involve driver calls, interrupts, and memory-mapped I / O (MMIO) accesses, all of which are inefficient compared to simple memory accesses.In at least one embodiment, the ability to access GPU memory 1620 without cache coherence overheads may be critical to the execution time of an offloaded computation. For example, in at least one embodiment, in cases with significant streaming write memory traffic, the cache coherence overhead may significantly reduce the effective write bandwidth seen by a GPU 1610. In at least one embodiment, operand facility efficiency, result access efficiency, and GPU computation efficiency may play a role in determining the effectiveness of GPU offloading.

[0279] In at least one embodiment, the selection of the GPU bias and the host processor bias is controlled by a bias tracker data structure. For example, in at least one embodiment, a bias table may be used, which may be a page-granular structure (e.g., controlled at a memory page granularity) including 1 or 2 bits per GPU-bound memory page. In at least one embodiment, a bias table may be implemented in a stolen memory region of one or more GPU memories 1620 with or without a bias cache in a GPU 1610 (e.g., to cache frequently / recently used bias table entries). Alternatively, in at least one embodiment, an entire bias table may be maintained within a GPU.

[0280] In at least one embodiment, a bias table entry associated with each access to GPU-bound memory 1620 is accessed prior to the actual access to GPU memory, causing the following operations. In at least one embodiment, local requests from a GPU 1610 that find their page in the GPU bias are forwarded directly to a corresponding GPU memory 1620. In at least one embodiment, local requests from a GPU that find their page in the host bias are forwarded to processor 1605 (e.g., over a high-speed interconnect as described herein). In at least one embodiment, requests from processor 1605 that find a requested page in the host processor bias complete a request like a normal memory read. Alternatively, requests directed to a GPU-biased page may be forwarded to GPU 1610.In at least one embodiment, a GPU may then transition a page to a host processor bias if it is not currently using a page. In at least one embodiment, a page's bias state may be changed either through a software-based mechanism, a hardware-assisted software-based mechanism, or, for a limited set of cases, a purely hardware-based mechanism.

[0281] In at least one embodiment, a mechanism for changing the bias state uses an API call (e.g., OpenCL), which in turn invokes a GPU's device driver, which in turn sends a message to a GPU (or queues a command descriptor) instructing it to change a bias state, and for some transitions, performs a cache flush operation in a host. In at least one embodiment, a cache flush operation is used for a transition from the host processor 1605 bias to the GPU bias, but is not used for an opposite transition.

[0282] In at least one embodiment, cache coherence is maintained by temporarily uncaching GPU-biased pages by host processor 1605. In at least one embodiment, to access these pages, processor 1605 may request access from GPU 1610, which may or may not grant access immediately. Therefore, in at least one embodiment, to reduce communication between processor 1605 and GPU 1610, it is advantageous to ensure that GPU-biased pages are those required by a GPU but not by host processor 1605, and vice versa.

[0283] One or more hardware structures 815 are used to perform one or more embodiments. Details regarding one or more hardware structures 815 may be described herein in connection with Fig. 8A and / or Fig. 8B are provided.

[0284] Fig. Figure 17 illustrates exemplary integrated circuits and associated graphics processors that may be fabricated using one or more IP cores, according to various embodiments described herein. In addition to what is illustrated, other logic and circuitry may be included in at least one embodiment, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0285] Fig. 17 is a block diagram illustrating an exemplary system-on-chip integrated circuit 1700 that may be manufactured using one or more IP cores, according to at least one embodiment. In at least one embodiment, the integrated circuit 1700 includes one or more application processors 1705 (e.g., CPUs), at least one graphics processor 1710, and may additionally include an image processor 1715 and / or a video processor 1720, each of which may be a modular IP core. In at least one embodiment, the integrated circuit 1700 includes peripheral or bus logic, including a USB controller 1725, a UART controller 1730, an SPI / SDIO controller 1735, and an I 2 2S / I 22C controller 1740. In at least one embodiment, integrated circuit 1700 may include a display device 1745 coupled to one or more of a high-definition multimedia interface (HDMI) controller 1750 and a mobile industry processor interface (MIPI) display interface 1755. In at least one embodiment, memory may be provided by a flash memory subsystem 1760 that includes flash memory and a flash memory controller. In at least one embodiment, a memory interface may be provided via a memory controller 1765 for accessing SDRAM or SRAM memory devices. In at least one embodiment, some integrated circuits additionally include an embedded security engine 1770.

[0286] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in integrated circuit 1700 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0287] In at least one embodiment, systems or components of Fig. 8A or Fig. 8B in components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or use logic 815 to implement one or more neural networks (e.g., network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0288] Fig. 18A and Fig. 18B illustrate example integrated circuits and associated graphics processors that may be fabricated using one or more IP cores, according to various embodiments described herein. In addition to what is illustrated, other logic and circuitry may be included in at least one embodiment, including additional graphics processors / cores, peripheral interface controllers, or general-purpose processor cores.

[0289] Fig. 18A and Fig. 18B are block diagrams illustrating example graphics processors for use within an SoC according to embodiments described herein. Fig. 18A illustrates an exemplary graphics processor 1810 of a system-on-chip integrated circuit that may be manufactured using one or more IP cores, according to at least one embodiment. Fig. 18B illustrates an additional exemplary graphics processor 1840 of a system-on-chip integrated circuit that may be fabricated using one or more IP cores, according to at least one embodiment. In at least one embodiment, the graphics processor 1810 is Fig. 18A is a low-power graphics processor core. In at least one embodiment, the graphics processor 1840 is Fig. 18B, a higher performance graphics processor core. In at least one embodiment, each of the graphics processors 1810, 1840 may be variants of the graphics processor 1710 of Fig. be 17.

[0290] In at least one embodiment, graphics processor 1810 includes a vertex processor 1805 and one or more fragment processors 1815A-1815N (e.g., 1815A, 1815B, 1815C, 1815D through 1815N-1, and 1815N). In at least one embodiment, graphics processor 1810 may execute different shader programs via separate logic, such that vertex processor 1805 is optimized to perform operations for vertex shader programs, while one or more fragment processors 1815A-1815N perform fragment (e.g., pixel) shading operations for fragment or pixel shader programs. In at least one embodiment, vertex processor 1805 performs a vertex processing stage of a 3D graphics pipeline and generates primitives and vertex data.In at least one embodiment, the fragment processor(s) 1815A-1815N use primitive and vertex data generated by the vertex processor 1805 to generate a frame buffer displayed on a display device. In at least one embodiment, the fragment processor(s) 1815A-1815N are optimized to execute fragment shader programs such as those provided in an OpenGL API, which can be used to perform similar operations as a pixel shader program such as those provided in a Direct 3D API.

[0291] In at least one embodiment, graphics processor 1810 additionally includes one or more memory management units (MMUs) 1820A-1820B, cache(s) 1825A-1825B, and circuit interconnect(s) 1830A-1830B. In at least one embodiment, one or more MMU(s) 1820A-1820B provide virtual-to-physical address mapping for graphics processor 1810, including vertex processor 1805 and / or fragment processor(s) 1815A-1815N, which may reference vertex or image / texture data stored in memory in addition to vertex or image / texture data stored in one or more cache(s) 1825A-1825B. In at least one embodiment, one or more MMU(s) 1820A-1820B may be synchronized with other MMUs within a system, including one or more MMUs associated with one or more application processors 1705, image processors 1715, and / or video processors 1720 of Fig. 17, so that each processor 1705-1720 can participate in a shared or unified virtual memory system. In at least one embodiment, one or more circuit interconnects 1830A-1830B enable the graphics processor 1810 to interface with other IP cores within the SoC, either via an internal bus of the SoC or via a direct connection.

[0292] In at least one embodiment, the graphics processor 1840 includes one or more shader cores 1855A-1855N (e.g., 1855A, 1855B, 1855C, 1855D, 1855E, 1855F through 1855N-1 and 1855N), as shown in Fig. 18B, providing a unified shader core architecture in which a single core or type of core can execute all types of programmable shader code, including shader program code to implement vertex shaders, fragment shaders, and / or computational shaders. In at least one embodiment, a number of shader cores may vary. In at least one embodiment, the graphics processor 1840 includes an intercore task manager 1845 acting as a thread dispatcher to dispatch execution threads to one or more shader cores 1855A-1855N, and a tiling unit 1858 to accelerate tiling operations for tile-based rendering in which rendering operations for a scene are partitioned into image space, for example, to exploit local spatial coherence within a scene or to optimize the use of internal caches.

[0293] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in graphics processor 1810 and / or 1840 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0294] In at least one embodiment, systems or components of Fig. 8A or Fig. 8B in components, systems and / or processes that are Fig. 1-7. For example, processor 105 may include or use logic 815 to implement one or more neural networks (e.g., network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0295] Fig. 19A and Fig. 19B illustrate additional exemplary graphics processor logic according to embodiments described herein. In at least one embodiment, components associated with Fig. 19A and Fig. 19B are integrated into a single system, such as a graphics processing unit (GPU), an SoC, or other type of processor. Fig. 19A illustrates a graphics core 1900 included in at least one embodiment in the graphics processor 1710 of Fig. 17 and in at least one embodiment, a unified shader core 1855A -1855N as in Fig. 18B can be. Fig. 19B illustrates a highly parallel general-purpose graphics processing unit ("GPGPU," which may also be referred to as a "graphics processing unit") 1930, which in at least one embodiment is suitable for deployment on a multi-chip module. In at least one embodiment, the graphics processing unit 1930 is a GPGPU that includes a graphics processor. In at least one embodiment, the integrated circuit 1700 includes a graphics core 1900, e.g., to form an integrated circuit and / or to form an SoC, such an integrated circuit and / or SoC performing operations described herein.

[0296] In at least one embodiment, the graphics core 1900 includes a shared instruction cache 1902, a texture unit 1918, and a cache / shared memory 1920 (e.g., including L1, L2, L3, last-level cache, or other caches) that are common execution resources within the graphics core 1900. In at least one embodiment, the graphics core 1900 may include multiple slices 1901A-1901N or a partition for each core, and a graphics processor may include multiple instances of the graphics core 1900. In at least one embodiment, each slice 1901A-1901N refers to the graphics core 1900. In at least one embodiment, the slices 1901A-1901N include subslices that are part of a slice 1901A-1901N. In at least one embodiment, the disks 1901A-1901N are independent of other disks or dependent on other disks.In at least one embodiment, slices 1901A-1901N may include support logic including a local instruction cache 1904A-1904N, a thread scheduler (sequencer) 1906A-1906N, a thread dispatcher 1908A-1908N, and a set of registers 1910A-1910N. In at least one embodiment, the slices 1901A-1901N may include a set of additional functional units (AFUs 1912A-1912N), floating-point units (FPUs 1914A-1914N), integer arithmetic logic units (ALUs 1916A-1916N), address calculation units (ACUs 1913A-1913N), double-precision floating-point units (DPFPUs 1915A-1915N), and matrix processing units (MPUs 1917A-1917N). In at least one embodiment, the MPUs 1917A-1917N are referred to as matrix engines.

[0297] In at least one embodiment, each slice 1901A-1901N includes one or more engines for floating-point and integer vector operations and one or more engines for accelerating convolution and matrix operations in AI, machine learning, or large dataset workloads. In at least one embodiment, one or more slices 1901A-1901N include one or more vector engines for computing a vector (e.g., computing mathematical operations on vectors). In at least one embodiment, a vector engine can compute a vector operation in 16-bit floating point (also referred to as "FP16"), 32-bit floating point (also referred to as "FP32"), or 64-bit floating point (also referred to as "FP64").In at least one embodiment, one or more slices 1901A-1901N include 16 vector engines paired with 16 matrix math units for computing matrix / tensor operations, where vector engines and math units are exposed via matrix extensions. In at least one embodiment, a slice comprises a specified portion of processing resources of a processing unit, e.g., 16 cores and a ray tracing unit, or 8 cores, a thread scheduler, a thread dispatcher, and additional functional units for a processor. In at least one embodiment, the graphics core 1900 includes one or more matrix engines for computing matrix operations, e.g., when computing tensor operations.

[0298] In at least one embodiment, one or more slices 1901A-1901N include one or more ray tracing units for computing ray tracing operations (e.g., 16 ray tracing units per slice 1901A-1901N). In at least one embodiment, a ray tracing unit computes ray tracing, triangle crossing, bounding box crossing, or other ray tracing operations.

[0299] In at least one embodiment, one or more slices 1901A-1901N include a media slice that encodes, decodes, and / or transcodes data; scales and / or format converts data; and / or performs video quality operations on video data.

[0300] In at least one embodiment, one or more slices 1901A-1901N are associated with L2 cache and memory fabric, interconnects, high-bandwidth memory (HBM) stacks (e.g., HBM2e, HDM3), and a media engine. In at least one embodiment, one or more slices 1901A-1901N include multiple cores (e.g., 16 cores) and multiple ray tracing units (e.g., 16) paired with each core. In at least one embodiment, one or more slices 1901A-1901N include one or more L1 caches. In at least one embodiment, one or more slices 1901A-1901N include one or more vector machines; one or more instruction caches for storing instructions; one or more L1 caches for caching data; one or more shared local memories (SLMs) for storing data, e.g., B.instructions; one or more samplers for sampling data; one or more ray tracing units for performing ray tracing operations; one or more geometries for performing operations in geometry pipelines and / or applying geometric transformations to vertices or polygons; one or more rasterizers for describing an image in vector graphic format (e.g., shape) and converting it to a raster image (e.g., a series of pixels, points, or lines that, when displayed together, create an image represented by shapes); one or more hierarchical depth buffers (Hiz) for buffering data; and / or one or more pixel backends. In at least one embodiment, a slice 1901A-1901N includes a memory structure, e.g., an L2 cache.

[0301] In at least one embodiment, FPUs 1914A-1914N can perform single-precision (32-bit) and half-precision (16-bit) floating-point operations, while DPFPUs 1915A-1915N can perform double-precision (64-bit) floating-point operations. In at least one embodiment, ALUs 1916A-1916N can perform variable-precision integer operations with 8-bit, 16-bit, and 32-bit precision and can be configured for mixed-precision operations. In at least one embodiment, MPUs 1917A-1917N can also be configured for mixed-precision matrix operations, including half-precision floating-point and 8-bit integer operations.In at least one embodiment, the MPUs 1917-1917N may perform a variety of matrix operations to accelerate machine learning application frameworks, including enabling support for accelerated general matrix-matrix multiplication (GEMM). In at least one embodiment, the AFUs 1912A-1912N may perform additional logic operations not supported by floating-point or integer units, including trigonometric operations (e.g., sine, cosine, etc.).

[0302] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in graphics core 1900 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0303] In at least one embodiment, the graphics core 1900 includes an interconnect and an interconnect fabric sublayer tied to a switch and a GPU-GPU bridge that enables multiple graphics processors 1900 (e.g., 8) to be connected without gluing together, with load / store units (LSUs), data transfer units, and synchronization semantics across multiple graphics processors 1900. In at least one embodiment, interconnects include standardized interconnects (e.g., PCIe) or a combination thereof.

[0304] In at least one embodiment, the graphics core 1900 includes multiple tiles. In at least one embodiment, a tile is a single die or one or more dies, where individual dies may be connected with an interconnect (e.g., embedded multi-die interconnect bridge (EMIB)). In at least one embodiment, the graphics core 1900 includes a compute tile, a memory tile (e.g., where a memory tile may be exclusively accessed by different tiles or different chipsets, such as a Rambo tile), a substrate tile, a base tile, an HMB tile, an interconnect tile, and an EMIB tile, all of which tiles are packaged together as part of a GPU in the graphics core 1900. In at least one embodiment, the graphics core 1900 may include multiple tiles in a single package (also referred to as a "multi-tile package").In at least one embodiment, a compute tile may include 8 graphics cores 1900, an L1 cache; and a base tile may include a host interface with PCIe 5.0, HBM2e, MDFI, and EMIB, an interconnect tile with 8 interconnects, 8 ports, and an embedded switch. In at least one embodiment, the graphics core 1900 may include an L1 cache; and a base tile may include a host interface with PCIe 5.0, HBM2e, MDFI, and EMIB, an interconnect tile with 8 interconnects, 8 ports, and an embedded switch. In at least one embodiment, tiles are connected by face-to-face (F2F) chip-on-chip bonding through finely spaced 36-micron microbumps (e.g., copper pillars). In at least one embodiment, the graphics core 1900 includes a memory structure that includes memory and is a tile that can be accessed by multiple tiles.In at least one embodiment, the graphics core 1900 stores, accesses, or loads its own hardware contexts in memory, where a hardware context is a set of data loaded from registers before a process continues, and where a hardware context may indicate a state of the hardware (e.g., the state of a GPU).

[0305] In at least one embodiment, the graphics core 1900 includes a serializer / deserializer (SERDES) circuit that converts a serial data stream to a parallel data stream or converts a parallel data stream to a serial data stream.

[0306] In at least one embodiment, the graphics core 1900 includes a coherent unified high-speed fabric (GPU to GPU), load / store units, bulk data transfer and synchronization semantics, and connected GPUs through an embedded switch, with a GPU-GPU bridge controlled by a controller.

[0307] In at least one embodiment, the graphics core 1900 implements an API, where the API abstracts the hardware of the graphics core 1900 and accesses libraries of instructions for performing mathematical operations (e.g., a math core library), deep neural network operations (e.g., a deep neural network library), vector operations, collective communications, thread building blocks, video processing, a data analysis library, and / or ray tracing operations.

[0308] In at least one embodiment, the graphics core 1900 is divided into components, systems and / or processes that are Fig. 1-7. For example, the processor 105 may include or use the graphics core 1900 to implement one or more neural networks (e.g., the network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0309] Fig. 19B illustrates the GPGPU 1930, which may be configured to enable highly parallel computational operations to be performed by an array of graphics processing units, in at least one embodiment. In at least one embodiment, the GPGPU 1930 may be directly linked to other instances of the GPGPU 1930 to create a multi-GPU cluster to improve training speed for deep neural networks. In at least one embodiment, the GPGPU 1930 includes a host interface 1932 to enable connection to a host processor. In at least one embodiment, the host interface 1932 is a PCI Express interface. In at least one embodiment, the host interface 1932 may be a vendor-specific communication interface or communication fabric.In at least one embodiment, the GPGPU 1930 receives instructions from a host processor and uses a global scheduler 1934 (which may be referred to as a thread sequencer and / or an asynchronous computing engine) to dispatch execution threads associated with those instructions to a set of compute clusters 1936A-1936H. In at least one embodiment, the compute clusters 1936A-1936H utilize a cache 1938. In at least one embodiment, the cache 1938 may serve as a higher-level cache for caches within the compute clusters 1936A-1936H. In at least one embodiment, the GPGPU 1930 is part of an SoC, such as part of the integrated circuit 1700 (. Fig. 17).

[0310] In at least one embodiment, GPGPU 1930 includes memory 1944A-1944B coupled to compute clusters 1936A-1936H via a set of memory controllers 1942A-1942B (e.g., one or more controllers for HBM2e). In at least one embodiment, memory 1944A-1944B may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including double data rate graphics memory (GDDR).

[0311] In at least one embodiment, the compute clusters 1936A-1936H each include a set of graphics cores, such as the graphics core 1900 of Fig. 19A, which may include multiple types of integer and floating-point logic units capable of performing computational operations with a range of precisions, including those suitable for machine learning computations. For example, in at least one embodiment, at least a subset of floating-point units in each of compute clusters 1936A-1936H may be configured to perform 16-bit or 32-bit floating-point operations, while another subset of floating-point units may be configured to perform 64-bit floating-point operations.

[0312] In at least one embodiment, multiple instances of the GPGPU 1930 may be configured to operate as a compute cluster. In at least one embodiment, the communication used by the compute clusters 1936A-1936H for synchronization and data exchange varies across embodiments. In at least one embodiment, multiple instances of the GPGPU 1930 communicate via the host interface 1932. In at least one embodiment, the GPGPU 1930 includes an I / O hub 1939 that couples the GPGPU 1930 to a GPU link 1940 that enables direct connection to other instances of the GPGPU 1930. In at least one embodiment, the GPU link 1940 is coupled to a dedicated GPU-to-GPU bridge that enables communication and synchronization between multiple instances of the GPGPU 1930.In at least one embodiment, GPU link 1940 couples to a high-speed connection to transmit and receive data to other GPGPUs or parallel processors. In at least one embodiment, multiple instances of GPGPU 1930 reside in separate computing systems and communicate via a network device accessible via host interface 1932. In at least one embodiment, GPU link 1940 may be configured to enable connection to a host processor in addition to, or alternatively to, host interface 1932.

[0313] In at least one embodiment, the GPGPU 1930 may be configured to train neural networks. In at least one embodiment, the GPGPU 1930 may be used within an inference platform. In at least one embodiment where the GPGPU 1930 is used for inference, the GPGPU 1930 may include fewer compute clusters 1936A-1936H than when the GPGPU 1930 is used to train a neural network. In at least one embodiment, the memory technology associated with the memory 1944A-1944B may differ between inference and training configurations, with higher-bandwidth memory technologies dedicated to training configurations. In at least one embodiment, an inference configuration of the GPGPU 1930 may support inference-specific instructions.For example, in at least one embodiment, an inference configuration may provide support for one or more 8-bit integer dot product instructions that may be used during inference operations for deployed neural networks.

[0314] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in GPGPU 1930 may be used to infer or predict operations based at least in part on weighting parameters computed using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0315] In at least one embodiment, the GPGPU 1930 is integrated into components, systems and / or processes that are Fig. 1-7. For example, the processor 105 may include or use the GPGPU 1930 to implement one or more neural networks (e.g., network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0316] Fig. 20 is a block diagram illustrating a computer system 2000 according to at least one embodiment. In at least one embodiment, computer system 2000 includes a processing subsystem 2001 having one or more processors 2002 and a system memory 2004 communicating via an interconnect path that may include a memory hub 2005. In at least one embodiment, memory hub 2005 may be a separate component within a chipset component or may be integrated within one or more processors 2002. In at least one embodiment, memory hub 2005 couples to an I / O subsystem 2011 via a communications link 2006. In at least one embodiment, I / O subsystem 2011 includes an I / O hub 2007 that may enable computer system 2000 to receive input from one or more input devices 2008.In at least one embodiment, the I / O hub 2007 may enable a display controller, which may be included in one or more processors 2002, to provide outputs to one or more display devices 2010A. In at least one embodiment, one or more display devices 2010A coupled to the I / O hub 2007 may include a local, internal, or embedded display device.

[0317] In at least one embodiment, the processing subsystem 2001 includes one or more parallel processors 2012 coupled to the storage hub 2005 via a bus or other communication link 2013. In at least one embodiment, the communication link 2013 may utilize any number of standards-based communication link technologies or protocols, such as, but not limited to, PCI Express, or may be a vendor-specific communication interface or communication structure. In at least one embodiment, one or more parallel processors 2012 form a compute-focused parallel or vector processing system, which may include a large number of processing cores and / or processing clusters, such as an integrated multiple core (MIC) processor.In at least one embodiment, some or all of the parallel processor(s) 2012 form a graphics processing subsystem that can output pixels to one or more display devices 2010A coupled via the I / O hub 2007. In at least one embodiment, the parallel processor(s) 2012 may also include a display controller and a display interface (not shown) to enable direct connection to one or more display devices 2010B. In at least one embodiment, the parallel processor(s) 2012 include one or more cores, such as the graphics cores 1900 discussed herein.

[0318] In at least one embodiment, a system storage unit 2014 may connect to the I / O hub 2007 to provide a storage mechanism for the computing system 2000. In at least one embodiment, an I / O switch 2016 may be used to provide an interface mechanism to enable connections between the I / O hub 2007 and other components, such as a network adapter 2018 and / or a wireless network adapter 2019, that may be integrated into the platform, and various other devices that may be added via one or more add-on devices 2020. In at least one embodiment, the network adapter 2018 may be an Ethernet adapter or other wired network adapter.In at least one embodiment, the wireless network adapter 2019 may include one or more of a Wi-Fi, Bluetooth, near field communication (NFC), or other network device that includes one or more wireless radios.

[0319] In at least one embodiment, the computing system 2000 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, and the like, which may also be connected to the I / O hub 2007. In at least one embodiment, communication paths connecting various components in Fig. 20 may be implemented using any suitable protocols, such as PCI (Peripheral Component Interconnect)-based protocols (e.g., PCI Express) or other bus or point-to-point communication interfaces and / or protocol(s), such as NV-Link high-speed interconnect or link protocols.

[0320] In at least one embodiment, the parallel processor(s) 2012 include circuitry optimized for graphics and video processing, including, for example, video output circuitry, and form a graphics processing unit (GPU), e.g., the parallel processor(s) 2012 include the graphics core 1900. In at least one embodiment, the parallel processor(s) 2012 include circuitry optimized for general-purpose processing. In at least one embodiment, components of the computing system 2000 may be integrated with one or more other system elements on a single integrated circuit. For example, in at least one embodiment, the parallel processor(s) 2012, the memory hub 2005, the processor(s) 2002, and the I / O hub 2007 may be integrated into a system-on-chip (SoC) integrated circuit.In at least one embodiment, components of computing system 2000 may be integrated into a single package to form a system-in-package (SIP) configuration. In at least one embodiment, at least a portion of components of computing system 2000 may be integrated into a multi-chip module (MCM), which may be interconnected with other multi-chip modules to form a modular computing system.

[0321] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in computing system 2000 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0322] In at least one embodiment, the computing system 2000 is divided into components, systems and / or processes that are Fig. 1-7. For example, the computing system 2000 may include processor 105 to implement one or more neural networks (e.g., network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals. PROCESSORS

[0323] Fig. 21A illustrates a parallel processor 2100 according to at least one embodiment. In at least one embodiment, various components of the parallel processor 2100 may be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). In at least one embodiment, the illustrated parallel processor 2100 is a variant of one or more in Fig. 20, according to an exemplary embodiment. In at least one embodiment, a parallel processor 2100 includes one or more graphics cores 1900.

[0324] In at least one embodiment, parallel processor 2100 includes a parallel processing unit 2102. In at least one embodiment, parallel processing unit 2102 includes an I / O unit 2104 that enables communication with other devices, including other instances of parallel processing unit 2102. In at least one embodiment, I / O unit 2104 may be directly connected to other devices. In at least one embodiment, I / O unit 2104 connects to other devices using a hub or switch interface, such as storage hub 2105. In at least one embodiment, connections between storage hub 2105 and I / O unit 2104 form a communication link 2113.In at least one embodiment, the I / O unit 2104 connects to a host interface 2106 and a memory crossbar 2116, where the host interface 2106 receives commands directed to performing processing operations and the memory crossbar 2116 receives commands directed to performing memory operations.

[0325] In at least one embodiment, when host interface 2106 receives a command buffer via I / O unit 2104, host interface 2106 may direct work operations to a front end 2108 to perform those commands. In at least one embodiment, front end 2108 couples to a scheduler 2110 (which may be referred to as a sequencer) configured to dispatch commands or other work items to a processing cluster array 2112. In at least one embodiment, scheduler 2110 ensures that processing cluster array 2112 is properly configured and in a valid state before dispatching tasks to a cluster of processing cluster array 2112. In at least one embodiment, scheduler 2110 is implemented via firmware logic executing on a microcontroller.In at least one embodiment, the microcontroller-implemented scheduler 2110 is configurable to perform complex scheduling and work distribution operations at coarse and fine granularity, enabling rapid preemption and context switching of threads executing on the processing array 2112. In at least one embodiment, host software may allocate workloads for scheduling on the processing cluster array 2112 via one of several graphics processing paths. In at least one embodiment, workloads may then be automatically distributed across the processing array cluster 2112 by the logic of the scheduler 2110 within a microcontroller that includes the scheduler 2110.

[0326] In at least one embodiment, the processing cluster array 2112 may include up to "N" processing clusters (e.g., cluster 2114A, cluster 2114B through cluster 2114N), where "N" represents a positive integer (which may be a different integer "N" than used in other figures). In at least one embodiment, each cluster 2114A-2114N of the processing cluster array 2112 may execute a large number of concurrent threads. In at least one embodiment, the scheduler 2110 may assign work to the clusters 2114A-2114N of the processing cluster array 2112 using various scheduling and / or work distribution algorithms, which may vary depending on the workload incurred for each type of program or computation.In at least one embodiment, scheduling may be handled dynamically by scheduler 2110 or may be partially assisted by compiler logic during compilation of program logic configured for execution by processing cluster array 2112. In at least one embodiment, different clusters 2114A-2114N of processing cluster array 2112 may be assigned to process different types of programs or to perform different types of computations.

[0327] In at least one embodiment, processing cluster array 2112 may be configured to perform various types of parallel processing operations. In at least one embodiment, processing cluster array 2112 is configured to perform parallel, multi-purpose computing operations. For example, in at least one embodiment, processing cluster array 2112 may include logic for performing processing tasks, including filtering video and / or audio data, performing modeling operations, including physical operations, and performing data transformations.

[0328] In at least one embodiment, processing cluster array 2112 is configured to perform parallel graphics processing operations. In at least one embodiment, processing cluster array 2112 may include additional logic to support the execution of such graphics processing operations, including, but not limited to, texture sampling logic for performing texture operations, as well as tiling logic and other vertex processing logic. In at least one embodiment, processing cluster array 2112 may be configured to execute graphics processing-related shader programs, such as, but not limited to, vertex shaders, tile shaders, geometry shaders, and pixel shaders. In at least one embodiment, parallel processing unit 2102 may transfer data from system memory via I / O unit 2104 for processing.In at least one embodiment, data transferred during processing may be stored in on-chip memory (e.g., parallel processor memory 2122) during processing and then written back to system memory.

[0329] In at least one embodiment, when the parallel processing unit 2102 is used to perform graphics processing, the scheduler 2110 may be configured to divide a processing workload into approximately equally sized tasks to better facilitate distribution of graphics processing operations across multiple clusters 2114A-2114N of the processing cluster array 2112. In at least one embodiment, portions of the processing cluster array 2112 may be configured to perform different types of processing. For example, in at least one embodiment, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tile and geometry shading, and a third portion may be configured to perform pixel shading or other screen-space operations to generate a rendered image for display.In at least one embodiment, intermediate data generated by one or more of the clusters 2114A-2114N may be stored in buffers to enable intermediate data to be transferred between the clusters 2114A-2114N for further processing.

[0330] In at least one embodiment, processing cluster array 2112 may receive processing tasks to be executed via scheduler 2110, which receives commands defining processing tasks from front-end 2108. In at least one embodiment, processing tasks may include indices of data to be processed, e.g., surface (patch) data, primitive data, vertex data, and / or pixel data, as well as state parameters and commands that define how data is to be processed (e.g., which program is to be executed). In at least one embodiment, scheduler 2110 may be configured to retrieve indices corresponding to tasks or may receive indices from front-end 2108. In at least one embodiment, front-end 2108 may be configured to ensure that processing cluster array 2112 is configured to a valid state before initiating a workload defined by incoming command buffers (e.g.,Stack buffer, push buffer, etc.) is specified.

[0331] In at least one embodiment, each of one or more instances of parallel processing unit 2102 may couple to a parallel processor memory 2122. In at least one embodiment, parallel processor memory 2122 may be accessed via memory crossbar 2116, which may receive memory requests from processing cluster array 2112 as well as from I / O unit 2104. In at least one embodiment, memory crossbar 2116 may access parallel processor memory 2122 via a memory interface 2118. In at least one embodiment, memory interface 2118 may include multiple partition units (e.g., partition unit 2120A, partition unit 2120B, through partition unit 2120N), each of which may couple to a portion (e.g., memory unit) of parallel processor memory 2122.In at least one embodiment, a number of partition units 2120A-2120N is configured to equal a number of storage units, such that a first partition unit 2120A has a corresponding first storage unit 2124A, a second partition unit 2120B has a corresponding storage unit 2124B, and an Nth partition unit 2120N has a corresponding Nth storage unit 2124N. In at least one embodiment, a number of partition units 2120A-2120N may not equal a number of storage units.

[0332] In at least one embodiment, the memory units 2124A-2124N may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including double data rate graphics memory (GDDR). In at least one embodiment, the memory units 2124A-2124N may also include stacked 3D memory, including, but not limited to, high bandwidth memory (HBM), HBM2e, or HDM3. In at least one embodiment, render targets, such as frame buffers or texture maps, may be stored across the memory units 2124A-2124N, allowing the partition units 2120A-2120N to write portions of each render target in parallel to efficiently utilize the available bandwidth of the parallel processor memory 2122.In at least one embodiment, a local instance of parallel processor memory 2122 may be eliminated in favor of a unified memory design that uses system memory in conjunction with local cache memory.

[0333] In at least one embodiment, any of the clusters 2114A-2114N of the processing cluster array 2112 may process data written to any of the memory units 2124A-2124N within the parallel processor memory 2122. In at least one embodiment, the memory crossbar 2116 may be configured to transfer an output of each cluster 2114A-2114N to any partition unit 2120A-2120N or to another cluster 2114A-2114N that may perform additional processing operations on an output. In at least one embodiment, each cluster 2114A-2114N may communicate with the memory interface 2118 via the memory crossbar 2116 to read from or write to various external memory devices.In at least one embodiment, memory crossbar 2116 includes a connection to memory interface 2118 to communicate with I / O unit 2104, as well as a connection to a local instance of parallel processor memory 2122, enabling processing units within different processing clusters 2114A-2114N to communicate with system memory or other memory that is not local to parallel processing unit 2102. In at least one embodiment, memory crossbar 2116 may use virtual channels to separate traffic flows between clusters 2114A-2114N and partition units 2120A-2120N.

[0334] In at least one embodiment, multiple instances of the parallel processing unit 2102 may be provided on a single add-in card, or multiple add-in cards may be interconnected. In at least one embodiment, different instances of the parallel processing unit 2102 may be configured to interoperate, even if different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences. In at least one embodiment, the parallel processing unit 2102 may be configured to interoperate with one or more instances of the parallel processing unit 2102. For example, in at least one embodiment, some instances of the parallel processing unit 2102 may include higher-precision floating-point units relative to other instances.In at least one embodiment, systems including one or more instances of the parallel processing unit 2102 or the parallel processor 2100 may be implemented in a variety of configurations and form factors, including, but not limited to, desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.

[0335] Fig. 21B is a block diagram of a partition unit 2120 according to at least one embodiment. In at least one embodiment, the partition unit 2120 is an instance of one of the partition units 2120A-2120N of Fig. 21A. In at least one embodiment, partition unit 2120 includes an L2 cache 2121, a frame buffer interface 2125, and a ROP 2126 (raster operation unit). In at least one embodiment, L2 cache 2121 is a read / write cache configured to perform load and store operations received from memory crossbar 2116 and ROP 2126. In at least one embodiment, read misses and urgent write-back requests are issued by L2 cache 2121 to frame buffer interface 2125 for processing. In at least one embodiment, updates may also be sent to a frame buffer via frame buffer interface 2125 for processing. In at least one embodiment, frame buffer interface 2125 interfaces with one of the memory units in parallel processor memory, such as memory units 2124A-2124N of Fig. 21A (e.g. within the parallel processor memory 2122).

[0336] In at least one embodiment, ROP 2126 is a processing unit that performs raster operations such as stencil, z-test, blending, etc. In at least one embodiment, ROP 2126 then outputs processed graphics data stored in graphics memory. In at least one embodiment, ROP 2126 includes compression logic to compress depth or color data written to memory and decompress depth or color data read from memory. In at least one embodiment, the compression logic may be lossless compression logic using one or more of several compression algorithms. In at least one embodiment, a type of compression performed by ROP 2126 may vary based on statistical properties of data to be compressed.For example, in at least one embodiment, delta color compression is performed on depth and color data on a per-tile basis.

[0337] In at least one embodiment, the ROP 2126 is within each processing cluster (e.g., clusters 2114A-2114N of Fig. 21A) rather than within the partition unit 2120. In at least one embodiment, read and write requests for pixel data rather than pixel fragment data are transmitted across the memory crossbar 2116. In at least one embodiment, processed graphics data may be displayed on a display device, such as one of one or more display devices 2010 of Fig. 20, for further processing by the processor(s) 2002 or for further processing by one of the processing entities within the parallel processor 2100 of Fig. 21A.

[0338] Fig. 21C is a block diagram of a processing cluster 2114 within a parallel processing unit according to at least one embodiment. In at least one embodiment, a processing cluster is an instance of one of the processing clusters 2114A-2114N of Fig. 21A. In at least one embodiment, the processing cluster 2114 may be configured to execute many threads in parallel, where "thread" refers to an instance of a particular program executing on a particular set of input data. In at least one embodiment, single-instruction-multiple-data (SIMD) instruction issue techniques are used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In at least one embodiment, single-instruction-multiple-thread (SIMT) techniques are used to support the parallel execution of a large number of generally synchronized threads using a common instruction unit configured to issue instructions to a set of processing engines within each of the processing clusters.

[0339] In at least one embodiment, the operation of the processing cluster 2114 may be controlled by a pipeline manager 2132 that distributes processing tasks to SIMT parallel processors. In at least one embodiment, the pipeline manager 2132 receives instructions from the scheduler 2110 of Fig. 21A and manages the execution of these instructions via a graphics multiprocessor 2134 and / or a texture unit 2136. In at least one embodiment, the graphics multiprocessor 2134 is an exemplary instance of a SIMT parallel processor. However, in at least one embodiment, various types of SIMT parallel processors of different architectures may be included in the processing cluster 2114. In at least one embodiment, one or more instances of the graphics multiprocessor 2134 may be included in a processing cluster 2114. In at least one embodiment, the graphics multiprocessor 2134 may process data, and a data crossbar 2140 may be used to distribute processed data to one of several possible destinations, including other shader units.In at least one embodiment, the pipeline manager 2132 may facilitate the distribution of processed data by specifying destinations for processed data to be distributed across the data crossbar 2140.

[0340] In at least one embodiment, each graphics multiprocessor 2134 within the processing cluster 2114 may include an identical set of function execution logic (e.g., arithmetic logic units, load-store units, etc.). In at least one embodiment, the function execution logic may be configured in a pipelined manner, in which new instructions may be issued before previous instructions complete. In at least one embodiment, the function execution logic supports a variety of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, bit shifting, and the calculation of various algebraic functions. In at least one embodiment, the same functional unit hardware may be employed to perform different operations, and any combination of functional units may be present.

[0341] In at least one embodiment, instructions transferred to processing cluster 2114 constitute a thread. In at least one embodiment, a set of threads executing across a set of parallel processing engines is a thread group. In at least one embodiment, a thread group executes a common program on different input data. In at least one embodiment, each thread within a thread group may be assigned to a different processing engine within a graphics multiprocessor 2134. In at least one embodiment, a thread group may contain fewer threads than a number of processing engines within the graphics multiprocessor 2134.In at least one embodiment, when a thread group contains fewer threads than a number of processing engines, one or more processing engines may be idle during cycles in which that thread group is processing. In at least one embodiment, a thread group may also contain more threads than a number of processing engines within the graphics multiprocessor 2134. In at least one embodiment, when a thread group contains more threads than a number of processing engines within the graphics multiprocessor 2134, processing may be performed over consecutive clock cycles. In at least one embodiment, multiple thread groups may execute concurrently on a graphics multiprocessor 2134.

[0342] In at least one embodiment, the graphics multiprocessor 2134 includes an internal cache to perform load and store operations. In at least one embodiment, the graphics multiprocessor 2134 may forgo an internal cache and use a cache (e.g., L1 cache 2148) within the processing cluster 2114. In at least one embodiment, each graphics multiprocessor 2134 also has access to L2 caches within partition units (e.g., partition units 2120A-2120N of Fig. 21A) that are shared by all processing clusters 2114 and can be used to transfer data between threads. In at least one embodiment, graphics multiprocessor 2134 can also access off-chip global memory, which can include one or more of local parallel processor memory and / or system memory. In at least one embodiment, any memory external to parallel processing unit 2102 can be used as global memory. In at least one embodiment, processing cluster 2114 includes multiple instances of graphics multiprocessor 2134 and can share common instructions and data that can be stored in L1 cache 2148.

[0343] In at least one embodiment, each processing cluster 2114 may include a memory management unit (MMLT) 2145 configured to map virtual addresses to physical addresses. In at least one embodiment, one or more instances of the MMU 2145 may reside within the memory interface 2118 of Fig. 21A. In at least one embodiment, the MMU 2145 includes a set of page table entries (PTEs) used to map a virtual address to a physical address of a tile and optionally a cache line index. In at least one embodiment, the MMU 2145 may include address translation buffers (TLBs) or caches that may be located within the graphics multiprocessor 2134 or the L1 cache 2148 or the processing cluster 2114. In at least one embodiment, a physical address is processed to distribute surface data access locally to enable efficient request interleaving between partition units. In at least one embodiment, a cache line index may be used to determine whether a request for a cache line is a hit or miss.

[0344] In at least one embodiment, a processing cluster 2114 may be configured such that each graphics multiprocessor 2134 is coupled to a texture unit 2136 for performing texture mapping operations, such as determining texture sample positions, reading texture data, and filtering texture data. In at least one embodiment, texture data is read from an internal texture L1 cache (not shown) or from an L1 cache within the graphics multiprocessor 2134 and retrieved from an L2 cache, local parallel processor memory, or system memory as needed. In at least one embodiment, each graphics multiprocessor 2134 issues processed tasks to the data crossbar 2140 to provide a processed task to another processing cluster 2114 for further processing or to store the processed task in an L2 cache, local parallel processor memory, or system memory via the memory crossbar 2116.In at least one embodiment, a pre-ROP 2142 (pre-raster operation unit) is configured to receive data from the graphics multiprocessor 2134 and pass data to ROP units that may be located with partition units as described herein (e.g., partition units 2120A-2120N of FIG. Fig. 21A). In at least one embodiment, the pre-ROP 2142 unit may perform optimizations for color mixing, organizing pixel color data, and performing address translations.

[0345] Logic 815 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding logic 815 are described herein in connection with Fig. 8A and / or Fig. 8B. In at least one embodiment, logic 815 in graphics processing cluster 2114 may be used to infer or predict operations based at least in part on weighting parameters calculated using neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0346] In at least one embodiment, processing cluster 2114 is divided into components, systems and / or processes that are Fig. 1-7. For example, processing cluster 2114 may include processor 105 to implement one or more neural networks (e.g., network 102 of Fig. 1) to predict a quality of one or more wireless signals based at least in part on one or more reference signals.

[0347] Fig. 21D illustrates a graphics multiprocessor 2134 according to at least one embodiment. In at least one embodiment, the graphics multiprocessor 2134 is coupled to the pipeline manager 2132 of the processing cluster 2114. In at least one embodiment, the graphics multiprocessor 2134 has an execution pipeline including an instruction cache 2152, an instruction unit 2154, an address mapping unit 2156, a register file 2158, one or more general purpose graphics processing unit (GPGPU) cores 2162, and one or more load / store units 2166, where one or more load / store units 2166 may perform load / store operations to load / store instructions according to performing an operation.In at least one embodiment, the GPGPU cores 2162 and the load / store units 2166 are coupled to the cache memory 2172 and the shared memory 2170 via a memory and cache interconnect 2168. In at least one embodiment, the GPGPU cores 2162 are part of an SoC, such as part of the integrated circuit 1700 in FIG. Fig. 17.

[0348] In at least one embodiment, instruction cache 2152 receives a stream of instructions to be executed from pipeline manager 2132. In at least one embodiment, instructions are cached in instruction cache 2152 and dispatched for execution by an instruction unit 2154. In at least one embodiment, instruction unit 2154 may dispatch instructions as thread groups (e.g., warps, wavefronts, waves), where each thread of the thread group is assigned to a different execution unit within GPGPU cores 2162. In at least one embodiment, an instruction may access any of a local, shared, or global address space by specifying an address within a unified address space.In at least one embodiment, address mapping unit 2156 may be used to translate addresses in a unified address space into a unique memory address that may be accessed by load / store units 2166.

[0349] In at least one embodiment, register file 2158 provides a set of registers for functional units of graphics multiprocessor 2134. In at least one embodiment, register file 2158 provides temporary storage for operands associated with data paths of functional units (e.g., GPGPU cores 2162, load / store units 2166) of graphics multiprocessor 2134. In at least one embodiment, register file 2158 is partitioned between each of the functional units such that each functional unit is assigned a dedicated portion of register file 2158. In at least one embodiment, register file 2158 is partitioned between different warps (which may be referred to as wavefronts and / or waves) executed by graphics multiprocessor 2134.

[0350] In at least one embodiment, GPGPU cores 2162 may each include floating-point units (FPUs) and / or integer arithmetic logic units (ALUs) used to execute instructions of graphics multiprocessor 2134. In at least one embodiment, GPGPU cores 2162 may be similar in architecture or different in architecture. In at least one embodiment, a first portion of GPGPU cores 2162 includes a single-precision FPU and an integer ALU, while a second portion of GPGPU cores includes a double-precision FPU. In at least one embodiment, FPUs may implement the floating-point arithmetic of the IEEE 754-2008 standard or enable variable-precision floating-point arithmetic.In at least one embodiment, graphics multiprocessor 2134 may additionally include one or more fixed-function or special-function units to perform specific functions, such as copy rectangle or pixel blending operations. In at least one embodiment, one or more of GPGPU cores 2162 may also include fixed-function or special-function logic.

[0351] In at least one embodiment, GPGPU cores 2162 include SIMD logic capable of performing a single instruction on multiple data sets. In at least one embodiment, GPGPU cores 2162 may physically execute instructions for SIMD4, SIMD8, and SIMD16, and logically execute instructions for SIMD1, SIMD2, and SIMD32. In at least one embodiment, SIMD instructions for GPGPU cores may be generated at compile time by a shader compiler or automatically generated when executing programs written and compiled for single-program multiple data (SPMD) or SIMT architectures. In at least one embodiment, multiple threads of a program configured for a SIMT execution model may execute from a single SIMD instruction.For example, in at least one embodiment, eight SIMT threads performing the same or similar operations may be executed in parallel via a single SIMD8 logic unit.

[0352] In at least one embodiment, the memory and cache interconnect 2168 is an interconnect network that connects each functional unit of the graphics multiprocessor 2134 to the register file 2158 and to the shared memory 2170. In at least one embodiment, the memory and cache interconnect 2168 is a crossbar interconnect that enables the load / store unit 2166 to implement load and store operations between the shared memory 2170 and the register file 2158. In at least one embodiment, the register file 2158 may operate at the same frequency as the GPGPU cores 2162, so that data transfer between the GPGPU cores 2162 and the register file 2158 may have very low latency.In at least one embodiment, shared memory 2170 may be used to enable communication between threads executing on functional units within graphics multiprocessor 2134. In at least one embodiment, cache 2172 may be used, for example, as a data cache to cache texture data communicated between the functional units and texture unit 2136. In at least one embodiment, shared memory 2170 may also be used as a program-managed cache. In at least one embodiment, threads executing on GPGPU cores 2162 may programmatically store data within shared memory in addition to automatically cached data stored within cache 2172.

[0353] In at least one embodiment, a parallel processor or GPGPU, as described herein, is communicatively coupled to host processor cores to accelerate graphics operations, machine learning operations, pattern analysis...

Claims

[1] Processor comprising: one or more circuits for using one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. [2] The processor of claim 1, wherein the reference signals are transmitted by a radio unit and include predefined values ​​used by one or more user devices to generate an estimate of the radio link quality between a user device and the radio unit. [3] The processor of claim 1, wherein the one or more reference signals are one or more inputs to the neural networks and the predicted quality of the one or more wireless signals are one or more outputs of the one or more neural networks. [4] The processor of claim 1, wherein the one or more circuits are to use the predicted quality of the one or more wireless signals to immediately start one or more processes to adjust one or more radio links between a user device and a radio unit. [5] The processor of claim 1, wherein the one or more circuits are to use the predicted quality of the one or more wireless signals as inputs to a control circuit to adjust one or more radio links between a user device and a base node. [6] The processor of claim 1, wherein the predicted quality of the one or more wireless signals includes whether received signals at a user device should be out of synchronous or synchronous. [7] The processor of claim 1, wherein the predicted quality of the one or more wireless signals includes whether received signals at a user device are expected to be out of synchronization and for how long they will be out of synchronization or how many consecutive frames of the received signals. [8] System comprising: one or more processors to use one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. [9] The system of claim 8, wherein the one or more reference signals are associated with synchronization signal block signals. [10] The system of claim 8, wherein the one or more reference signals are associated with channel state information reference signals. [11] The system of claim 8, wherein the one or more reference signals are associated with both synchronization signal block signals and channel state information reference signals. [12] The system of claim 8, wherein the one or more reference signals are used by the one or more networks to generate a predicted block error rate (BLER). [13] The system of claim 8, wherein the one or more reference signals are used by the one or more neural networks to generate one or more predicted transmission parameters of the physical downlink control channel (PDCCH). [14] The system of claim 8, wherein one or more processors are to perform preemptive operations to adjust the quality of a radio link based at least in part on the predicted quality of the one or more reference signals. [15] Method comprising: Using one or more neural networks to predict a quality of one or more wireless signals based at least in part on one or more reference signals. [16] The method of claim 15, further comprising: Identifying that a radio link is likely to fail based at least in part on the predicted quality of the one or more wireless signals; and in response to the probability of the radio link failing, generating control signals or sending information to a radio unit to be used by a node to adjust the radio link. [17] The method of claim 15, further comprising: Identifying that a radio link is likely to be unstable for the next few frames and then become stable again, based at least in part on the predicted quality of the one or more wireless signals. [18] The method of claim 15, wherein the one or more reference signals are transmitted periodically. [19] The method of claim 15, further comprising: Identifying that a radio link is likely to be unstable in a number of consecutive frames based at least in part on the predicted quality of the one or more wireless signals. [20] The method of claim 15, further comprising: Transmitting, by a user device, the predicted quality of the one or more wireless signals to a base station node in a fifth generation (5G) network.

Images