Infrared and other coloring via generative neural networks

Abstract

In various examples, infrared image data (e.g., images from an infrared video transmission) can be colorized by applying the infrared image data and / or a corresponding edge map to a generator of a generative adversarial network (GAN). The GAN can be trained with or without paired RGB and infrared images (and / or edge maps) using basic truth. In an example of the latter scenario, a first generator G(IR)→RGB and a second generator G(RGB)→IR can be trained in a first chain, their positions can be swapped in a second chain, and the second chain can then be trained. In some embodiments, edges can be emphasized by weighting edge pixels (e.g., determined from a corresponding edge map) more heavily than non-edge pixels when backpropagating the loss.After training, G(IR)→RGB can be used to generate RGB image data from infrared image data (and / or a corresponding edge map).

Description

BACKGROUNDColoring infrared images may be a fascinating way to visualize a world that is normally hidden from natural human perception. Infrared (IR) photography captures a spectrum of light invisible to the human eye, and by coloring into the visible spectrum, breath-threatening images can be created from rarely or never previously viewed perspectives or details. Moreover, IR photography can visualize details and patterns (e.g., in vegetation and landscapes) that are not visible in visible light, making them useful for scientific and ecological research. By adding color to these images, researchers and photographs can create more visually pleasing and more informative representations of the world around us. Generally, coloring infrared images can be a conservative and curative way of experimenting with traditional visual representations.Conventional techniques for coloring infrared images have a number of disadvantages. For example, one prior technique uses a convolutional neural network (CNN) to learn an association from an IR image to an RGB image. However, since any color in an image may typically be associated with a number of possible valid colors, conventional techniques may effectively average the various valid color candidates to produce an RGB image that is often blurred or produce RGB images that include segments of different colors that do not conform to ground truth (e.g., a single-color shirt could be colored with a plurality of different colors). Therefore, there is a need for improved infrared colorizing techniques that color IR images more accurately and / or with more stringent details.BRIEF DESCRIPTION OF THE INVENTIONThe invention is defined by the claims. To illustrate the invention, aspects and embodiments are described herein which may or may not fall within the scope of the claims.In various examples, infrared image data (e.g., images of an infrared video transmission) may be colored by applying the infrared image data and / or a corresponding edge map to a generative opposing network (GAN) generator. The GAN may be trained with or without paired ground truth RGB and infrared images (and / or edge maps). In an example of the latter scenario, a first generator G(IR)→RGB and a second generator G(RGB)→IR may be trained in a first string, their positions may be interchanged in a second string, and the second string may be trained. In some embodiments, edges may be emphasized by weighting edge pixels (e.g., determined from a corresponding edge map) higher than non-edge pixels when back propagating the loss. After training, G(IR)→RGB may be used to generate RGB image data from infrared image data (and / or a corresponding edge map).In various examples, infrared image data (e.g., images of an infrared video transmission) may be colored by transmitting color statistics from an RGB image with an overlapping field of view, by modifying one or more dimensions of an encoded representation of a generated RGB image, and / or in other ways. For example, segmentation may be applied to the IR and RGB image data, and one or more colors or statistics may be transferred from a segmented region of the RGB image data to a corresponding segmented region of the IR image data. In some embodiments, synthesized RGB image data may be finely tuned by transferring color or one or more color statistics from corresponding real RGB image data and / or by modifying one or more dimensions of an encoded representation of the synthesized RGB image data.Embodiments of the present disclosure relate to infrared coloring. Systems and methods are disclosed that color IR image data (e.g., images of an IR video transmission). In contrast to conventional systems such as those described above, an IR image (e.g., images of an IR video transmission) may be colored using a conditional generative adversarial network (cGAN) generator by transmitting color statistics from an RGB image with an overlapping field of view, modifying one or more dimensions of a coded representation of a synthesized RGB image, and / or otherwise.In some embodiments, a generative machine learning model (e.g., a generative opposing network (GAN) generator G(IR)→RGB, such as a cGAN), may be trained to generate an RGB image from an input IR image (and / or corresponding edge map) with or without paired ground truth RGB and IR images (and / or edge maps). In an example of the latter scenario, a first generator G(IR)→RGB and a second generator G(RGB)→IR may be trained in a first string, their positions may be interchanged in a second string, and the second string may be trained. In some embodiments, edges may be emphasized by weighting edge pixels (e.g., determined from a corresponding edge map) higher than non-edge pixels in the backpropagation of the loss. After training, G(IR)→RGB may be used to synthesize an RGB image from an infrared image (and / or corresponding edge map).In some embodiments, segmentation may be applied to an IR image generated with an IR camera and to an RGB image generated with an RGB or RGB / IR camera to identify one or more corresponding regions. In some embodiments (e.g., where the cameras are part of an occupant monitoring system (OMS)), any known technique may be used to identify facial features and / or segment (e.g., corresponding) areas such as eyes, skin, face, hair, neck, arms, other parts of the face or body, vehicle parts (e.g., seat), background, and / or other areas (which may depend on the application). Thus, segmented regions from the RGB image may be mapped to the IR image and / or vice versa to identify pairs of corresponding segmented regions present in both images. For each pair of corresponding segmented regions, the color may be transferred from the segmented region in the RGB image to a corresponding segmented region in the infrared image in any suitable manner. In some embodiments, a stroke-based colorization algorithm may be used to propagate color.In some embodiments where an infrared camera and an RGB or RGB / IR camera have at least partially overlapping fields of view, a convolutional neural network (CNN) may be used to blindly generate an RGB image from an infrared image (and / or a corresponding edge map), and this synthesized RGB image may be fine tuned by transferring color from a segmented region in the RGB image to a corresponding segmented region in the synthesized RGB image (for some or all pairs of segmented regions). For example, in some embodiments, a synthesized RGB image and a corresponding RGB image may be segmented with at least partially overlapping fields of view, corresponding regions may be paired, and for each pair, the one or more colors of the corresponding regions of the real and synthesized RGB images may be compared. If the difference exceeds a certain threshold (e.g., if the person's hair is known to be black but not black in the synthesized image, in which case the difference in the average color values of the regions may exceed a threshold, for example), color may be transferred from the segmented region in the RGB image to the corresponding segmented region in the synthesized RGB image. In some embodiments, a synthesized RGB image (e.g., generated using a GAN or CNN) may be fine tuned by selectively modifying one or more dimensions of a latent representation of the synthesized RGB image and decoding the modified latent representation to modify a corresponding visual attribute. For example, a pair of encoder-decoders may be trained to encode and then decode an RGB image (e.g., using loss of reconstruction). Thus, the trained encoder may be used to encode a real RGB image into a latent vector, wherein different dimensions of the latent vector may be selectively modified, and the trained decoder may be used to decode the modified latent vector into a corresponding RGB image to determine the correlation between one or more dimensions of the latent vector and a corresponding visual attribute of a decoded RGB image (e.g., hair color) controlled by the one or more dimensions. Thus, in some embodiments, a synthesized RGB image and a corresponding RGB image may be segmented with at least partially overlapping fields of view, corresponding regions may be paired, and for each pair of one or more pairs, the one or more colors of the corresponding regions of the real and synthesized RGB may be compared. If the difference (e.g., in the average or mean color values of the regions) exceeds a certain threshold, the synthesized RGB image may be encoded, one or more dimensions of the latent vector that control a corresponding visual attribute may be modified (e.g., to a determined value that encodes black hairs), and the modified latent vector may be decoded to produce a modified synthesized RGB image.The techniques described herein may therefore be used for coloring infrared images and / or videos, such as in infrared photography, scientific or ecological research, OMS (e.g., an IR video feed for video calls, emergency scenarios, intrusion scenarios), and / or other types of systems.The disclosure extends to all novel aspects or features described and / or illustrated herein.Further features of the disclosure are characterized by the independent and dependent claims.Each feature of an aspect of the disclosure may be applied to other aspects of the disclosure in any suitable combination. In particular, method aspects may be applied to device or system aspects, and vice versa.Further, features implemented in hardware may be implemented in software, and vice versa. Any reference to software and hardware features herein should be construed accordingly.Any system or device feature described herein may also be provided as a method feature, and vice versa. System and / or device aspects that are functionally described (including means plus features) may alternatively be expressed in terms of their corresponding structure, such as as a suitably programmed processor and associated memory.It will also be appreciated that certain combinations of the various features described and defined in each aspect of the disclosure may be implemented and / or provided and / or used independently of one another.The disclosure also provides computer programs and computer program products comprising software code adapted, when executed on a data processing device, to perform any of the methods described herein and / or to embody any of the device and system features described herein, including any or all of the component steps of a method.The disclosure also includes a computer or computer system (including networked or distributed systems) having an operating system that supports a computer program for performing the methods described herein and / or for embodying the apparatus or system features described herein.The disclosure also provides a computer readable medium having stored thereon one or more of the aforementioned computer programs.The disclosure also provides a signal carrying one or more of the aforementioned computer programs.The disclosure extends to methods and / or apparatuses and / or systems as described herein with reference to the accompanying drawings.Aspects and embodiments of the disclosure will now be described, purely by way of example, with reference to the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGSThe present infrared colorizing systems and methods are described in detail below with reference to the accompanying drawing figures, wherein: FIG. 1 illustrates a data flow through an example infrared colorizing system trained using paired infrared and RGB ground truth images, in accordance with some embodiments of the present disclosure; FIG. 2 illustrates a data flow through an example infrared coloring system trained without paired infrared and RGB ground truth images, in accordance with some embodiments of the present disclosure; FIG. 3 illustrates a data flow through an example infrared coloring system for coloring segmented regions of an infrared image using corresponding regions of a real RGB image, in accordance with some embodiments of the present disclosure; FIG. 4 illustrates a flow of data through an example infrared colorizing system for fine tuning segmented regions of a synthesized RGB image using corresponding regions of a real RGB image, in accordance with some embodiments of the present disclosure; FIG. 5 illustrates a flow of data through an example infrared colorizing system for fine tuning segmented regions of a synthesized RGB image by modifying a latent representation of the synthesized RGB image, in accordance with some embodiments of the present disclosure; FIG. 6 is a flow chart illustrating a method for blind colorizing infrared image data according to some embodiments of the present disclosure; FIG. 7 is a flow chart illustrating a method for colorizing a frame of infrared image data using a corresponding frame of RGB image data, in accordance with some embodiments of the present disclosure; FIG. 8A is an illustration of an example autonomous vehicle, in accordance with some embodiments of the present disclosure; FIG. 8B is an example of camera locations and fields of view for the example autonomous vehicle of FIG. 8A, in accordance with some embodiments of the present disclosure; FIG. 8C is a block diagram of an example system architecture for the example autonomous vehicle of FIG. 8A, in accordance with some embodiments of the present disclosure; FIG. 8D is a system diagram for communication between the one or more cloud-based servers and the example autonomous vehicle of FIG. 8A, in accordance with some embodiments of the present disclosure; FIG. 9 is a block diagram of an example computing device suitable for use in implementing some embodiments of the present disclosure; and FIG. 10 is a block diagram of an example data center suitable for use in implementing some embodiments of the present disclosure.DETAILED DESCRIPTIONSystems and methods are disclosed in the context of infrared coloring. For example, systems and methods are disclosed that color an infrared image (e.g., images of an infrared video transmission) using a conditional generative opposing network generator by transmitting color statistics from an RGB image having an overlapping field of view, modifying one or more dimensions of an encoded representation of a generated RGB image, and / or otherwise. The present techniques may be used for colorizing infrared images or videos in systems for infrared photography, scientific or environmental research, occupant monitoring systems, and / or other types of systems.Although the present disclosure may be described with respect to an example autonomous or semi-autonomous vehicle or machine 800 (alternatively referred to herein as "vehicle 800" or "ego machine 800", an example of which is described with respect to FIGS. 8A-8D ), this is not to be understood as limiting. For example, the systems and methods described herein may be used without limitation: non-autonomous vehicles or machines, semi-autonomous vehicles or machines (e.g., in one or more advanced driver assistance systems (ADAS)), autonomous vehicles or machines, piloted and unpilotated robots or robot platforms, storage vehicles, off-road vehicles, vehicles coupled to one or more trailers, flying vessels, boats, shuttles, emergency vehicles, motor bicycles, electric or motorized bicycles, aircraft, construction vehicles, trains, underwater vehicles, remote controlled vehicles such as drones, and / or other types of vehicles. Moreover, while the present disclosure may be described in relation to infrared coloring for OMS, this is not intended to be limiting, and the systems and methods described herein may be used in photography, scientific or environmental research, augmented reality, virtual reality, mixed reality, robotics, safety and monitoring, autonomous or semi-autonomous machine applications, and / or in other fields of technology where coloring may be used in monochrome colors, gray levels, black-and-white, infrared, or coloring of scientific data (e.g., pseudo-coloring of available or missing hyperspectral or multispectral data, colored images astronomic observations, satellite images, or imaging spectroscopy).An application for infrared images is an occupant monitoring system (OMS) that can be deployed in a vehicle cabin to assess, in real-time, the presence of occupants and / or operators, their gaze, attention, and / or other conditions. OMSs often rely on observations from multiple OMS cameras (e.g., RGB cameras, infrared cameras, combined RGB / infrared (RGB / IR) cameras) positioned at various locations in the vehicle interior. Vehicle manufacturers tend to vary the number of OMS cameras from model to model and depending on the level of equipment. Basic models typically have an infrared camera facing the driver (e.g., in a steering column, a vehicle column, or an infotainment console). Higher levels of equipment may include any number of additional cameras (e.g., one in the steering column facing the driver, one in the rearview mirror facing the driver or cabin, one in a vehicle column facing a particular row of seats, etc.).For example, since OMSs often include an infrared camera directed to the driver, the infrared color may be used to generate and color an infrared video feed of the driver (e.g., for video calls, emergency scenarios, intrusion scenarios, and / or others). Generally, depending on the configuration of the one or more cameras (e.g., OMS), a number of scenarios are possible. For example, an infrared colorization can be performed "blind", i.e. using an infrared image without using a corresponding RGB image. In another example scenario, an infrared camera may be coupled to or associated with an RGB and / or an RGB / IR camera, the field of view of which at least partially overlaps that of the infrared camera. In another example, an infrared camera may be coupled to or associated with an RGB and / or an RGB / IR camera whose field of view does not overlap that of the infrared camera. Depending on the scenario, the infrared coloring can be performed in various ways.For example, in some embodiments, a generative machine learning model (e.g., a generative opposing network (GAN), such as a conditional generative opposing network (cGAN), a transformer, a convolutional neural network (CNN), etc.) may be trained to generate an RGB image from an input infrared image (and / or a corresponding edge map) using paired RGB and infrared ground truth images (and / or edge maps). For example, the paired ground truth RGB and infrared (and / or edge maps) images may be applied as examples of pairs of real images to a discriminator, the infrared ground truth image (and / or edge map) may be paired with an RGB image synthesized by a generator based on the infrared image (and / or edge map) of the ground truth and applied to the discriminator as examples of paired real and counterfeit images, and the generator may be trained to decept the discriminator using a suitable loss function (e.g., reconstruction loss). In some embodiments, edges may be emphasized by weighting edge pixels (e.g., determined from a corresponding edge map) higher than non-edge pixels when back propagating the loss. In some embodiments (e.g., for coloring an edge map extracted from an infrared image, for example, for each image of a video feed), the coloring may be performed (e.g., using a generator of a GAN or cGAN) at a remote location. As a result, extracting edges from an infrared image and transmitting the edge map to the remote location (e.g., via a wireless channel) for colorization should reduce bandwidth consumption as compared to transmitting and colorizing the full infrared image.In some embodiments, a GAN or cGAN may be trained to generate an RGB image from an infrared image (and / or corresponding edge map) without having paired ground truth RGB and infrared (and / or edge map) images. For example, in a first branch or chain, two generators may be concatenated, a G(IR)→RGB that generates an RGB image from an infrared image (and / or corresponding edge map) and a G(RGB)→IR that generates an infrared image (and / or corresponding edge map) from an RGB image. During training, a real infrared image (and / or corresponding edge map) may be applied to G(IR)→RGB, and the synthesized RGB image may be paired with the real infrared image (and / or corresponding edge map) and provided to a first discriminator as examples of pairs of real images. Additionally or alternatively, the synthesized RGB image may be applied to G(RGB)→IR to synthesize an infrared image (and / or corresponding edge map), and one or both generators may be updated using any suitable loss function (reconstruction loss based on the pair of real synthesized infrared images with edge pixels emphasized higher weights than non-edge pixels). In a second branch or chain, the two generators may be exchanged and a real RGB image may be applied to G(RGB)→IR to synthesize an infrared image (and / or corresponding edge map) that may be paired with the real RGB image and fed to a second discriminator as an example of pairs of real images. Additionally or alternatively, the synthesized infrared image (and / or corresponding edge map) may be applied to G(IR)→RGB to synthesize an RGB image, and one or both generators may be updated using any suitable loss function (reconstruction loss based on the pair of real-synthesized RGB images with edge pixels emphasized higher weights than non-edge pixels). After training the first and second branches or strings in any order, G(IR)→RGB may be used to blindly generate an RGB image from an infrared image (and / or corresponding edge map).Turning now to example scenarios where an infrared camera and an RGB or RGB / IR camera have at least partially overlapping fields of view; in some embodiments, segmentation may be applied to an infrared image generated with the infrared camera and an RGB image generated with the RGB or RGB / IR camera to identify one or more corresponding regions. In some embodiments (e.g., where the cameras are part of an OMS), any known technique may be used to identify facial features and / or segment (e.g., corresponding) areas such as eyes, skin, face, hair, neck, arms, other parts of the face or body, vehicle parts (e.g., seat), background, and / or other areas (which may depend on the application). Thus, segmented regions may be mapped from the RGB image to the infrared image and / or vice versa to identify pairs of corresponding segmented regions present in both images. For each pair of corresponding segmented regions, the color may be transferred from the segmented region in the RGB image to a corresponding segmented region in the infrared image in any suitable manner. In some embodiments, a stroke-based colorization algorithm may be used to propagate color. For example, in order to color the forehead of a person, an output color may be selected from the forehead area in the RGB image, and this output color may be transferred to the forehead area in the infrared image and used as an input stroke indicating a target area for a stroke-based color algorithm for colorizing. In some embodiments, one or more color statistics (e.g., a statistical moment or characteristic of one or more color channels) for the segmented region of the RGB image (or a portion thereof, such as a portion in the overlapping field of view and / or a majority cluster of the segmented region or the portion in the overlapping field of view) may be calculated, and the one or more color statistics from the segmented region of the RGB image may be used to adjust or change the colors of the corresponding segmented region in the infrared image such that the one or more color statistics match those from the corresponding region of the RGB image. The process may be repeated for each pair of segmented regions. Thus, each region of the infrared image may be colored using a corresponding region of the RGB image.In some embodiments where an infrared camera and an RGB or RGB / IR camera have at least partially overlapping fields of view, a CNN may be used to blindly generate an RGB image from an infrared image (and / or corresponding edge map), and this synthesized RGB image may be fine tuned by transferring color from a segmented region in the RGB image to a corresponding segmented region in the synthesized RGB image (for some or all pairs of segmented regions). For example, in some embodiments, a synthesized RGB image and a corresponding RGB image may be segmented with at least partially overlapping fields of view, corresponding regions may be paired, and for each pair, the one or more colors of the corresponding regions of the real and synthesized RGB images may be compared. If the difference exceeds a certain threshold (e.g., if the person's hair is known to be black but not black in the synthesized image, in which case the difference in the average color values of the regions may exceed a threshold, for example), color may be transferred from the segmented region in the RGB image to the corresponding segmented region in the synthesized RGB image. Additional fine tuning may additionally or alternatively be performed, such as sharpening edges, smoothing noise (e.g., with a bilateral filter), and / or other techniques.Additionally or alternatively, in some embodiments, a synthesized RGB image (e.g., generated using a GAN or CNN) may be fine tuned by selectively modifying one or more dimensions of a latent representation of the synthesized RGB image and decoding the modified latent representation to modify a corresponding visual attribute. For example, a pair of encoder-decoders may be trained to encode and then decode an RGB image (e.g., using loss of reconstruction). Thus, the trained encoder may be used to encode a real RGB image into a latent vector, wherein different dimensions of the latent vector may be selectively modified, and the trained decoder may be used to decode the modified latent vector into a corresponding RGB image to determine the correlation between one or more dimensions of the latent vector and a corresponding visual attribute of a decoded RGB image (e.g., hair color) controlled by the one or more dimensions. Thus, in some embodiments, a synthesized RGB image and a corresponding RGB image may be segmented with at least partially overlapping fields of view, corresponding regions may be paired, and for each pair of one or more pairs, the color(s) of the corresponding regions of the real and synthesized RGB may be compared. If the difference (e.g., in the average or mean color values of the regions) exceeds a certain threshold, the synthesized RGB image may be encoded, one or more dimensions of the latent vector that control a corresponding visual attribute may be modified (e.g., to a determined value that encodes black hairs), and the modified latent vector may be decoded to produce a modified synthesized RGB image.In some embodiments where the infrared camera is coupled to an RGB / IR camera having at least partially overlapping fields of view and where paired RGB and infrared ground truth images (and / or corresponding edge map) are used in training, the paired ground truth images may be generated using the RGB / IR camera (e.g., positioned at the same installation location as in inference). In general, training with paired RGB and infrared ground truth images (and / or corresponding edge maps) generated with a camera that is at the same installation location as inference should improve network performance as compared to using training images that were not generated with a camera at that installation location.In some embodiments (e.g., in scenarios where an infrared camera and an RGB or RGB / IR camera do not have overlapping fields of view), an infrared image generated with the infrared camera (or a portion thereof, e.g., a segmented region thereof) may be colored using a known or captured cabin color (e.g., known by the design based on the make and model of the vehicle captured from a segmented region of the cabin), a captured exterior color (e.g., a segmented region of an image of the cabin that also shows exterior space such as outside a window, a captured color from an exterior camera, etc.), and / or otherwise. Additionally or alternatively, a color, illumination, or weather condition may be detected outside the cabin (e.g., from an image generated with an exterior or OMS camera) and used to control colorization of an infrared image (e.g., colorization with higher illumination at night, and vice versa). In some embodiments, face, age, and / or ethnicity sensing may be applied (e.g., to an infrared image, a synthesized RGB image, a corresponding edge map), and one or more segmented regions (e.g., skin regions, hairs) may be colored using colors selected from a predetermined range of candidate colors for that region for a person of a sensed age and / or ethnicity. These are just a few examples, and other techniques may be used within the scope of the present disclosure.Thus, the techniques described herein may be used for coloring infrared images and / or videos, such as in infrared photography, scientific or environmental research, OMS (e.g., an infrared video feed for video calls, emergency scenarios, intrusion scenarios), and / or other types of systems. With the techniques described herein, such as those that color infrared image data (e.g., images of an infrared video transmission) using a conditional generative opposing network generator, by transmitting color statistics from an RGB image with overlapping field of view, and / or by fine tuning a generated RGB image, for example, by modifying one or more dimensions of a encoded representation of the generated RGB image, infrared images can be colored more accurately and with sharper details than with previous techniques.Referring to FIG. 1, FIG. 1 is an example of an infrared coloring system 100, in accordance with some embodiments of the present disclosure. It should be appreciated that these and other arrangements described herein are presented by way of example only. Other arrangements and elements (e.g., machines, interfaces, functions, arrangements, groupings of functions, etc.) may be used in addition to or in place of those shown, and some elements may be omitted altogether. In addition, many of the elements described herein are functional entities that can be implemented as single or distributed components or in conjunction with other components and in any suitable combination and location. Various functions described herein that are performed by entities may be performed by hardware, firmware, and / or software. Various functions may be performed, for example, by a processor executing instructions stored in memory. In some embodiments, the systems, methods, and processes described herein may be performed using similar components, features, and / or functionality to that of the example autonomous vehicle 800 of FIGS. 8A-8D, the example computing device 900 of FIG. 9, and / or the example data center 1000 of FIG. 10.FIG. 1 illustrates a data flow through an example infrared coloring system 100 trained using paired infrared and RGB ground truth images, in accordance with some embodiments of the present disclosure. More specifically, FIG. 1 illustrates a GAN (e.g., a cGAN) that includes a generator G(IR)→RGB 120 and a discriminator DRGB 140, wherein the generator G(IR)→RGB 120 synthesizes an RGB image 130 based on and / or conditional on an infrared image 110, the discriminator DRGB 140 learns to discriminate true and false (synthesized) RGB images, and the generator G(IR)→RGB 120 learns to synthesize RGB images that deceive the discriminator DRGB 140.FIG. 1 illustrates an example technique in which a machine learning model (e.g., generator G(IR)→RGB 120) may be trained to blindly generate an RGB image from an input infrared image using paired RGB and infrared ground truth images. For example, if an example pair of RGB and infrared ground truth images is assumed, the pair of ground truth images may be combined (e.g., stacked, concatenated) and applied to the discriminator DRGB 140 as an example of a pair of real images, and the discriminator DRGB 140 may be updated to generate an output indicating that the pair of input images is true. Assume that a real infrared image is now used as the infrared image 110 and applied to the generator G(IR)→RGB 120 to synthesize an RGB image 130. The synthesized RGB image 130 may be combined (e.g., stacked, concatenated) with the infrared image 110 and applied to the discriminator DRGB 140; as an example of paired genuine and counterfeit images, the discriminator DRGB 140 may be updated to generate an output indicating that the pair contains a counterfeit image and the generator G(IR)→RGB 120 may be updated to generate an RGB image that simulates the discriminator DRGB 140. Any suitable loss function may be used, such as reconstruction loss, edge loss, etc. In some embodiments, an edge map may be extracted from the infrared image 110, the edge map may be used to identify edge pixels and / or non-edge pixels, and the loss for edge pixels may be weighted more heavily than the loss for non-edge pixels to highlight edges during training. The generator G(IR)→RGB 120 and / or the discriminator DRGB 140 may be trained with any number of genuine and counterfeit examples in any order. The generator G(IR)→RGB 120 can therefore be used to blindly generate an RGB image from an input infrared image in any desired application.For example, in some embodiments, infrared color may be applied to sensor data generated using one or more sensors (e.g., cameras) of a monitoring system (e.g., an OMS), such as the one or more OMS sensors 801 of the vehicle 800 in FIG. 8C. In an example cabin or cockpit monitoring system, such as a vehicle OMS, one or more optical sensors may be positioned to observe a scene in the cabin, cockpit, or other interior. An OMS may include a driver monitoring system (DMS), a system that monitors non-driver occupants, or a system that monitors driver occupants and / or non-driver occupants. Optical sensors (e.g., RGB sensors, infrared sensors, depth sensors, cameras, etc.) may be positioned and / or distributed in various ways. For example, one or more DMS cameras may be positioned in a steering column, a vehicle pillar, or an infotainment console and oriented with their field of view directed toward the driver's seat. One or more OMS cameras may be positioned upside down and / or in a pillar and oriented with a field of view downward toward the front row of the vehicle and / or a rear row of the vehicle. Generally, occupant and / or driver monitoring systems may include any number of cameras (e.g., 4 DMS cameras and 16 OMS cameras) positioned throughout the vehicle interior. These are just a few examples of possible sensor layouts, and other sensor layouts within a suitable scene may be implemented within the scope of the present disclosure.One or all of the sensors may be used to generate a frame of sensor data (e.g., an image generated with one or more OMS cameras) for each time slice (e.g., at a particular frame rate, such as 30 images per second (fps)), and the frame of sensor data for each time slice may be used to color a video feed (e.g., at the same frame rate as one or more of the sensors, at the lowest common image of the sensor / sensors, etc.). For example, in some embodiments, an infrared image generated for each time slice may be colored (e.g., by applying the infrared image to the generator G(IR)→RGB 120 of FIG. 1 or by using any of the other infrared coloring techniques described herein).In some embodiments, the one or more machine learning models used to color the infrared image may be located at a location remote from the infrared camera used to generate the infrared image. For example, an infrared camera (e.g., an OMS system of the vehicle 800 in FIG. 8C ) may be used to generate images (e.g., a video feed) of infrared images at a location (e.g., in a vehicle cabin), and the infrared images (and / or extracted edge maps) may be forwarded or streamed via a transport system to the one or more machine learning models (e.g., via one or more wireless and / or other types of communication channels) at a remote location (e.g., a server in a data center). The transport system may use any streaming technology to transport the infrared images and / or extracted edge maps to a remote location over a wireless communication channel. As such, the one or more machine learning models may synthesize an RGB image for each infrared image, where each synthesized RGB image may be used as a corresponding frame and assembled into a video stream, and the video stream may be stored and / or distributed to enable various remote experiences, such as streaming to a remote viewer (e.g., a friend or relative, a meeting party), streaming to a remote or fleet operator, streaming to a mobile app configured to park itself or invoke an ego object, and / or others.FIG. 1 illustrates an example technique in which one or more machine learning models (e.g., generator G(IR)→RGB 120) are trained to blindly generate an RGB image from an input infrared image. Depending on the embodiment, the one or more machine learning models may include a generative machine learning model, such as a GAN or cGAN, a transformer neural network, diffusion models, a CNN, and / or others. Although certain embodiments are described with one or all of the machine learning models implemented using one or more neural networks, this is not intended to be limiting. For example, some or all of the machine learning models may include any type of machine learning model, such as one or more machine learning models, linear regression, logistic regression, decision trees, support vector machines (SVM), naive bayes, k-nearest neighbor (Knn), k-mean clustering, random forest, dimensionality reduction algorithms, gradient boosting algorithms, neural networks (e.g., auto-encoders, convolutional, transformer, recurrent, perceptrons, long / short term memory (LSTM), hopfield, boltzmann, deep belief, The OS may be implemented using de-convolutional, generative adventrial, liquid state machine, etc.) and / or other types of machine learning models.In the example illustrated in FIG. 1, input to the one or more machine learning models (e.g., generator G(IR)→RGB 120) is an infrared image 110. Additionally or alternatively, an edge map may be extracted from the infrared image 110 using any known technique, and the edge map (e.g., alone or stacked, concatenated, or otherwise combined with the infrared image 110) may be applied to the one or more machine learning models (e.g., the G(IR)→RGB 120 generator and / or the DRGB 140 discriminator) such that the one or more machine learning models learn to generate the RGB image 130 (e.g., deceiving the DRGB 140 discriminator) based on an input edge map instead of or in addition to an input infrared image.The example illustrated in FIG. 1 illustrates a scenario in which the infrared coloring system 100 may be trained using paired infrared and RGB ground truth images (and / or corresponding edge maps in embodiments in which the generator G(IR)→RGB 120 and / or the discriminator DRGB 140 accept an input representation of an edge map). FIG. 2 now illustrates a data flow through an example infrared coloring system 200 trained without paired ground truth infrared and RGB images (and / or corresponding edge maps) in accordance with some embodiments of the present disclosure.In particular, FIG. 2 illustrates a GAN (or cGAN) including a generator G(IR)→RGB 220 that synthesizes an RGB image (e.g., RGB image 230, RGB image 295) based on and / or conditional on an infrared image (e.g., infrared image 210, infrared image 280), a generator G(RGB)→IR 250 that synthesizes an infrared image (e.g., infrared image 260, infrared image 280) based on and / or conditional on an RGB image (e.g., RGB image 230, RGB image 270), a discriminator DRGB 240 that classifies an RGB image as true or false, and a discriminator DIR 290 which classifies an infrared image as genuine or counterfeit. In the upper branch of the infrared colorizing system 200, a real infrared training image may be used as the infrared image 210 and applied to the generator G(IR)→RGB 220 to synthesize an RGB image (e.g., RGB image 230); the synthesized RGB image 230 may be combined (e.g., stacked, concatenated) with the real infrared training image (e.g., the infrared image 210) and applied to the discriminator DRGB 240 as an example of paired real and counterfeit images, and the discriminator DRGB 240 may be updated to generate an output indicating that the input (e.g., the pair) includes a counterfeit image. Additionally or alternatively, the synthesized RGB image 230 may be applied to the generator G(RGB)→IR 250 to synthesize an infrared image (e.g., the infrared image 260), and one or both of the generators G(IR)→RGB 220 and / or G(RGB)→IR 250 may be updated based on the difference between the synthesized infrared image 260 and the real infrared image 210 using any suitable loss function (reconstruction loss, emphasis on edge pixels having weights higher than for non-edge pixels).In the lower branch of the infrared colorizing system 200, the two generators may be exchanged, a real RGB training image may be used as the RGB image 270 and applied to the generator G(RGB)→IR 250 to synthesize an infrared image (e.g., the infrared image 280), the synthesized infrared image 2 280 may be combined (e.g., stacked, concatenated) with the real RGB training image (e.g., the RGB image 270) and applied to the discriminator DIR 290 as an example of paired real and counterfeit images, and the discriminator DRGB 240 may be updated to generate an output indicating that the input (e.g., the pair) includes a counterfeit image. Additionally or alternatively, the synthesized infrared image 280 may be applied to the generator G(IR)→RGB 220 to synthesize an RGB image (e.g., the RGB image 295), and one or both of the generators G(IR)→RGB 220 and / or G(RGB)→IR 250 may be updated based on the difference between the synthesized RGB image 295 and the true RGB image 270 using any suitable loss function (reconstruction loss, enhancement of edge pixels having weights higher than those for non-edge pixels).The upper and lower branches of the infrared coloring system 200 may be trained in any order, and the G(IR)→RGB 220 generator may be used to blindly generate an RGB image from an infrared image.In the example illustrated in FIG. 2, the input to the generator G(IR)→RGB 220 is an infrared image 210. Additionally or alternatively, an edge map may be extracted from the infrared image 210 using any known technique, and the edge map (e.g., alone or stacked, concatenated, or otherwise combined with the infrared image 210) may be applied to the generator G(IR)→RGB 220 and / or the discriminator DRGB 240 such that the generator G(IR)→RGB 220 learns to generate an RGB image that deceives the discriminator DRGB 240 based on an input edge map instead of or in addition to an input infrared image.FIG. 3 now illustrates the flow of data through an example infrared coloring system 300 for coloring segmented regions of an infrared image 310 using corresponding regions of a real RGB image 320, in accordance with some embodiments of the present disclosure. In some embodiments, the infrared image 310 and the RGB image 320 include at least partially overlapping fields of view. For example, the infrared image 310 may be generated using an infrared DMS camera directed to the driver of a vehicle, and the RGB image 320 may be generated using an OMS camera (e.g., an RGB camera, an RGB / IR camera) directed to the front row of seats in the vehicle. In the example illustrated in FIG. 3, the infrared coloring system 300 includes an area segmentation component 315 that segments the infrared image 310, an area segmentation component 325 that segments the RGB image 320, an area mapping component 330 that associates segmented areas of the infrared image 310 with corresponding segmented areas of the RGB image 320 (and / or vice versa), a color statistics extraction component 340 that extracts one or more colors or color statistics from a segmented area of the RGB image 320, and a coloring component 350 that transfers the one or more colors or color statistics to a corresponding segmented area of the infrared image 310 to color the infrared image 310.In general, region segmentation component 315 and / or region segmentation component 325 may apply any known method (e.g., using one or more machine learning models, such as one or more deep neural networks) to segment one or more regions of a corresponding image. For example, in some embodiments, the infrared image 310 and / or the RGB image 320 may represent a human face and / or other body parts, and the region segmentation component 315 and / or the region segmentation component 325 may detect facial features and identify corresponding regions delimited by particular facial features (eyes, nose, forehead, chin, hair, etc.). Additionally or alternatively, the region segmentation component 315 and / or the region segmentation component 325 may segment (e.g., corresponding) regions such as eyes, skin, face, hair, neck, arms, other portions of the occupant or driver's face or body, vehicle portions (e.g., seat), background, and / or other regions (which may depend on the application). In FIG. 3, the components for region segmentation 315 and 325 are shown as separate components (e.g., separate neural networks) because they can accept and process different types of inputs (e.g., infrared vs. RGB images); however, in some embodiments, region segmentation component 315 and region segmentation component 325 can be implemented using a common network (e.g., having multiple input channels or input heads that accept the different inputs). Depending on the application, the region segmentation component 315 and / or the region segmentation component may include any number of classifiers (e.g., one for each region class to be segmented) that may be implemented using separate networks and / or using a common network (e.g., having multiple output channels or output headers predicting the various outputs). As such, region segmentation component 315 and / or region segmentation component may segment and generate a representation of any number of regions of a corresponding image.In general, the region mapping component 330 may evaluate the representation of the segmented regions of the infrared image 310 generated by the region segmentation component 315 and the representation of the segmented regions of the RGB image 320 generated by the region segmentation component 325 to determine whether the same type of segmented region is present in both the infrared image 310 and the RGB image 320. In an example implementation where the representations of the segmented regions of the infrared image 310 and the RGB image 320 are respective N-channel tensors, each channel segmenting a respective class, the region mapping component 330 may determine which of the N channels identified a respective region in each image (e.g., based on one or more pixels of a respective class having a predicted classification value that exceeds a threshold, a predicted classification value of the average or median that exceeds a threshold, a region that exceeds a threshold, etc.), and for each such channel, the region mapping component 330 may generate a representation that a respective type of region is present in each image, and / or a representation of the correspondence between the corresponding areas in the infrared image 310 and the RGB image 320. In some embodiments, the RGB image 320 and / or the infrared image 310 may be first transformed into a common coordinate system (e.g., cabin coordinates) to facilitate the region mapping component 330 determining whether the same type of segmented region is present in both images. In some embodiments, the infrared image 310 and the RGB image 320 may represent the same person (e.g., the head of the person) from different perspectives, such that the region mapping component 330 uses one or more n-point (PnP) perspective solvers to estimate the different poses in a common coordinate system and apply head normalization using these poses to transform one of the images to correspond to the view depicted in the other. In this way, both images (e.g., an original image and a transformed image) may effectively present the head from a similar perspective, which may facilitate the region mapping component 330 to identify corresponding segments presented in both the infrared image 310 and the RGB image 320.For each region type identified by the region mapping component 330 as being present in each image, the color statistics extraction component 340 may extract one or more colors and / or color statistics from the segmented region of the RGB image 320, and the colorization component 350 may transmit the one or more colors and / or color statistics to the corresponding segmented region of the infrared image 310.In general, color and / or one or more color statistics may be transferred from a segmented region of the RGB image 320 to a corresponding segmented region of the infrared image 310 in any suitable manner. For example, the color statistics extraction component 340 may sample one or more colors from the segmented region (e.g., a detected forehead region) of the RGB image 320, and the colorization component 350 may transfer the one or more sampled colors to a corresponding segmented region (e.g., a detected forehead region) of the infrared image 310 and use them as the output color by being referred to as an input stroke that identifies a target region to be colored with any known stretch-based colorization algorithm, such as described in Luan et al., Natural Image Colorization, Erographics Symposium on Rendering (2007). In some embodiments, the colorizing component 350 may apply a stroke-based colorizing algorithm to transfer color from a segmented region of the RGB image 320 to a corresponding segmented region of the infrared image 310.In another example, the color statistics extraction component 340 may calculate one or more color statistics (e.g., a statistical moment or characteristic of one or more color channels) from the segmented region of the RGB image 320 (or a portion thereof, such as a portion in the overlapping field of view and / or a majority cluster of the segmented region or the overlapping field of view), and the colorization component 350 may use the one or more color statistics from the segmented region of the RGB image 320 to adjust or modify the colors of the corresponding segmented region of the infrared image 310 to have a color statistics that match that of the corresponding region of the RGB image 320. These are just a few examples, and other techniques for transferring color or color statistics may be implemented within the scope of the present disclosure.Therefore, the process may be repeated for each pair of segmented regions to color the infrared image 310 using corresponding regions of the RGB image 320.FIG. 4 illustrates a flow of data through an example infrared coloring system 400 for fine tuning segmented regions of a synthesized RGB image 430 using corresponding regions of a real RGB image 450, in accordance with some embodiments of the present disclosure; in some embodiments, the infrared image 410 and the RGB image 450 include at least partially overlapping fields of view. For example, the infrared image 410 may be generated using an infrared DMS camera directed to the driver of a vehicle, and the RGB image 450 may be generated using an OMS camera (e.g., an RGB camera, an RGB / IR camera) directed to the front row of seats in the vehicle.In some embodiments, the infrared coloring system 400 includes one or more machine learning models (e.g., a CNN 420, a GAN such as a cGAN, etc.) that synthesize an RGB image 430 from an input infrared image 410. In the example illustrated in FIG. 4, the input to the CNN 420 is an infrared image 410. Additionally or alternatively, an edge map may be extracted from the infrared image 410 using any known technique, and the edge map (e.g., alone or stacked, concatenated, or otherwise associated with the infrared image 410) may be applied to one or more machine learning models (e.g., CNN 420) to generate the RGB image 450 based on an input edge map instead of or in addition to an input infrared image. However, in some cases, the synthesized RGB image 430 may be blurred. Thus, in some embodiments, a post-processing component 440 may apply one or more fine tuning operations to the synthesized RGB image 430 to generate an RGB image 490.For example, in the embodiment illustrated in FIG. 4, the post-processing component 440 may transmit color and / or one or more color statistics from a segmented region in a real RGB image 450 to a corresponding segmented region in the synthesized RGB image 430 (for some or all pairs of segmented regions). For example, in some embodiments, a region segmentation component 460 (which may correspond to region segmentation components 315 and / or 325 of FIG. 3, for example) may apply any known technique to segment regions of the real RGB image 450 and regions of the synthesized RGB image 430, wherein the color statistics extraction component (which may correspond to the color statistics extraction component 340 of FIG. 3 ) may extract one or more colors or color statistics from a segmented region of the real RGB image 450, the region mapping component 480 (which may correspond to the region mapping component 330 of FIG. 3, for example) may map segmented regions of the real RGB image 450 to corresponding segmented regions of the synthesized RGB image 430 (and / or vice versa), and post-processing component 440 (which may include functionality corresponding to colorizing component 350 of FIG. 3 ) may transmit one or more colors or color statistics to a corresponding segmented region of synthesized RGB image 430.In some embodiments, the post-processing component 440 may compare corresponding regions of the synthesized RGB image 430 and the real RGB image 450 using any suitable metric (e.g., by comparing average or median color values) and determine whether the difference exceeds a certain threshold. In some embodiments, the post-processing component 440 only transmits color and / or one or more color statistics from the segmented region in the real RGB image 450 to the corresponding segmented region in the synthesized RGB image 430 when the difference between these segmented regions exceeds the threshold. Therefore, the process may be repeated for each pair of segmented regions to fine tune the synthesized RGB image 430 using corresponding regions of the true RGB image 450.In some embodiments, the post-processing component 440 may additionally or alternatively apply other types of fine tuning to the synthesized RGB image 430. For example, the post-processing component 440 may employ any known edge sharpening technique, noise suppression technique (e.g., using a bilateral filter), and / or other techniques. In some embodiments, post-processing component 440 may encode synthesized RGB image 430 into a latent representation, modify one or more dimensions of the latent representation, and decode the modified latent representation to generate an RGB image 490 that modifies one or more visual attributes of synthesized RGB image 430.For example, FIG. 5 illustrates a flow of data through an example infrared colorizing system 500 for fine tuning segmented regions of a synthesized RGB image 510 (e.g., which may correspond to the synthesized image 430 of FIG. 4 ) by modifying a latent representation of the synthesized RGB image 510, in accordance with some embodiments of the present disclosure. In some embodiments, a pair of encoder-decoders (e.g., encoder 520, decoder 540) may be trained using any known technique to encode and then decode an RGB image (e.g., using reconstruction loss). Thus, the trained encoder (e.g., encoder 520) may be used to encode a real (e.g., training) RGB image (e.g., RGB image 510) into a latent vector (e.g., latent vector Z 530), wherein different dimensions of latent vector Z 530 may be selectively modified, and the trained decoder (e.g., decoder 540) may be used to decode the modified latent vector into a corresponding RGB image (e.g., attribute-modified RGB image 550) to determine the correlation between one or more dimensions of latent vector Z 530 and a corresponding visual attribute of a decoded RGB image (e.g., hair color), controlled by the one or more dimensions. For example, it may be determined by observing that one dimension of the latent vector Z 530 controls the hair color, another controls the forehead color, another controls the skin color, another controls the color of an interior or exterior cabin (e.g., the color of the interior roof, seats), etc.Referring back to FIG. 4, in some embodiments, the region segmentation component 460 may segment (e.g., with at least partially overlapping fields of view) a synthesized RGB image 430 and a corresponding RGB image 450, the color statistics extraction component 470 may extract one or more colors or color statistics from one or more segmented regions of the synthesized RGB image 430 and the real RGB image 450, the region mapping component 480 may couple corresponding regions, and for each pair of one or more pairs, the post-processing component 440 may compare the one or more colors and / or one or more color statistics of the corresponding regions of the synthesized RGB image 430 and the real RGB image 450. If the difference (e.g., in the average or mean color values of the regions) exceeds a certain threshold, the post-processing component 440 may use the encoder 520 of FIG. 5 to encode the synthesized RGB image 430 into a latent vector, and the post-processing component 440 may determine which of the previously determined dimensions of the latent vector controls a corresponding visual attribute, determine which previously detected value encodes a desired modification of that visual attribute (e.g., a detected value encoding the one or more colors and / or one or more color statistics for the corresponding region of the real RGB image 450, such as a value encoding black hair), and / or modify the one or more dimensions of the latent vector to have the selected value. Therefore, the post-processing component 440 may use the decoder 540 of FIG. 5 to decode the modified latent vector and generate the RGB image 490 with the modified visual attribute.In some embodiments (e.g., in scenarios where an infrared camera and an RGB or RGB / IR camera do not have overlapping fields of view), an infrared image (or a portion thereof, e.g., a segmented region) generated with the infrared camera may be colored in various additional or alternative ways. For example, known or acquired interior or exterior cabin colors and / or color statistics may be used to color at least a portion of an infrared image (e.g., infrared image 310 in FIG. 3 ) and / or a synthesized RGB image (e.g., RGB image 130 in FIG. 1, synthesized RGB image 430 in FIG. 4, synthesized RGB image 510). For example, one or more interior colors and / or color statistics (e.g., the color of the interior roof, seats, etc.) may be known based on the make and model of the vehicle such that the region segmentation component 315 segments the infrared image 310 into a corresponding region (regions), and the colorization component 350 may transmit one or more colors and / or color statistics to a corresponding segmented region of the infrared image 310. Additionally or alternatively, one or more interior and / or exterior colors and / or color statistics may be captured and transmitted to a corresponding region of the infrared image 310. For example, the segmenting component 325 may segment the RGB image 320, the region segmenting component 315 may segment the infrared image 310, and the region mapping component 330 may identify corresponding segmented regions (e.g., a region within a cabin such as the color of the interior roof or seats, a region outside a cabin such as a portion of the environment visible through a vehicle window). Thus, the color statistics extraction component 340 may extract one or more colors and / or color statistics from the one or more segmented regions of the RGB image 320, and the colorization component 350 may transmit the one or more colors and / or one or more color statistics to one or more corresponding segmented regions of the infrared image 310.In some embodiments, known (e.g., predetermined) or acquired interior or exterior cabin colors and / or color statistics may be used to color or modify a segmented region of an infrared image or a synthesized RGB image (e.g., generated from an infrared image generated using a corresponding RGB image and / or infrared image having a field of view that at least partially overlaps with the synthesized RGB image, etc.). For example, if one or more interior colors and / or color statistics (e.g., the color of the interior roof, the color of the seats, etc.) are known based on the make and model of the vehicle, a synthesized RGB image may be segmented (e.g., by the region segmentation component 460) to identify one or more regions, and one or more known colors and / or color statistics for each region may be applied (e.g., by the post-processing component 440) to color that region. For example, the post-processing component 440 may generate or modify a fill that includes one or more known colors and / or color statistics for a segmented region using any suitable technique. In some embodiments, the post-processing component 440 may apply any known stroke-based fill technique to generate or modify a fill. In some embodiments, the post-processing component 440 may optimize an energy function (e.g., including texture similarity, intensity smoothness) to generate a fill having a color or texture that resembles a target color or texture, promotes spatial color continuity, and / or otherwise. In another example, in some embodiments where a synthesized RGB image is generated using a GAN or cGAN and may be represented as a corresponding latent vector, post-processing component 440 may determine which dimension(s) of the latent vector control a corresponding visual attribute, may determine which previously detected value encodes a desired modification of that visual attribute (e.g., a detected value encoding the one or more known colors and / or one or more color statistics for the corresponding region), and / or may modify the dimension(s) of the latent vector to have the value encoding the one or more known colors and / or one or more color statistics. In some embodiments, any known blending technique may be employed (e.g., to smooth the boundary and / or surrounding area of the generated fill). These are just a few examples, and other ways of applying one or more known colors and / or one or more color statistics are also provided within the scope of the present disclosure.In some embodiments, a color, illumination, or weather condition may be detected outside the cabin (e.g., from an image generated with an exterior or OMS camera) and used to control colorization of an infrared image (e.g., colorization with higher illumination at night, and vice versa). For example, the segmenting component 325 may segment the RGB image 320 and the color statistics extracting component 340 may extract a representation of one or more colors and / or color statistics from the one or more segmented regions. Additionally or alternatively, the segmentation component 325 may include one or more classifiers that extract a representation of a captured lighting or weather condition from the RGB image 320. Thus, in some embodiments, one or more of these extracted representations of color, illumination, or weather may be used as a guide for infrared coloring. For example, the G(IR)→RGB 120 generator of FIG. 1 and / or the G(IR)→RGB 220 generator of FIG. 2 and / or the CNN 420 of FIG. 4 may be conditioned on one or more of these extracted representations (e.g., one or more of the extracted representations may be used as input). In some embodiments, colorizing component 350 may transmit one or more colors and / or one or more color statistics in a manner that depends on one or more of the extracted representations (e.g., by applying higher illumination at night, at poorer lighting conditions, at occluded weather conditions, and vice versa).In some embodiments, the detection of face, age, and / or ethnic membership may be applied to an infrared image (e.g., the infrared image 310 of FIG. 3, the infrared image 410 of FIG. 4 ), a synthesized RGB image (e.g., generated using any of the techniques described herein), and / or a corresponding edge map, and one or more segmented regions (e.g., skin regions, hairs) may be colored (e.g., by the colorizing component 350 of FIG. 3 ) using one or more colors and / or one or more color statistics selected from a predetermined range of candidate colors for that region for a person of a corresponding detected age and / or ethnicity.In some embodiments, the one or more colors and / or one or more color statistics of a particular type of segmented region (e.g., a portion of a vehicle interior) and / or segmented region associated with a person with a captured identity may be stored and used to color these segmented regions for future images. For example, the region mapping component 330 and / or the colorizing component 350 may assign an identification value or values to a particular type of segmented region and / or particular detected identities (e.g., particular detected persons) and store a representation of the one or more colors and / or one or more color statistics used to color a corresponding segmented region. In some embodiments, the region mapping component 330 and / or the colorizing component 350 may use these stored colors and / or color statistics to color these segmented regions for future frames. These are just a few examples, and other techniques may be used within the scope of the present disclosure.Referring to FIGS. 6 and 7, each block of the methods 600 and 700 described herein includes a computing process that may be performed with any combination of hardware, firmware, and / or software. Various functions may be performed, for example, by a processor executing instructions stored in memory. The methods may also be embodied as computer usable instructions stored on computer storage media. The methods may be provided by a stand-alone application, service or hosted service (stand-alone or in combination with another hosted service), or plug-in for another product, to name just a few. Moreover, methods 600 and 700 are described by way of example with respect to infrared colorizing systems 100, 200, 300, 400, and / or 500 of FIGS. 1-5. However, these methods may additionally or alternatively be performed by any system or combination of systems, including, but not limited to, the systems described herein.FIG. 6 is a flow diagram illustrating a method 600 for blind colorizing infrared image data according to some embodiments of the present disclosure; the method 600 includes generating infrared image data at block B 602. For example, referring to FIG. 8C, one or more infrared cameras of a monitoring system (e.g., an OMS) such as the one or more OMS sensors 801 of the vehicle 800 in FIG. 8C may be used to generate corresponding infrared images.The method 600, at block B 604, includes generating RGB image data corresponding to the infrared image data based at least on applying a representation of the infrared image data to a generative opposing network generator. For example, referring to FIGS. 1 and 2, once the generator G(IR)→RGB 120 and / or the generator G(IR)→RGB 220 are trained, an infrared image (and / or corresponding edge map) may be applied to G(IR)→RGB 120 and / or G(IR)→RGB 220 to synthesize a corresponding RGB image that effectively colorizes the infrared image.FIG. 7 is a flow diagram illustrating a method 700 of colorizing a frame of infrared image data using a corresponding frame of RGB image data, in accordance with some embodiments of the present disclosure. The method 700 includes, at block B 702, generating a frame of infrared image data and a frame of RGB image data having at least partially overlapping fields of view. For example, referring to FIG. 8C, one or more infrared cameras and / or one or more RGB or RGB / IR cameras of a monitoring system (e.g., an OMS) such as the one or more OMS sensors 801 of the vehicle 800 in FIG. 8C may be used to generate corresponding infrared images and RGB images. As a non-limiting example, one or more infrared DMS cameras may be positioned in a steering column, a vehicle column, and / or an infotainment console and oriented with their fields of view facing the driver's seat. Moreover, one or more RGB and / or RGB / IR OMS cameras may be positioned upside down and / or in a pillar and oriented with their fields of view toward the front row of the vehicle and / or a rear row of the vehicle.The method 700 includes, at block B 704, generating synthesized RGB image data based at least on coloring the frame of infrared image data using the frame of RGB image data. For example, referring to FIG. 3, the infrared colorizing system 300 may synthesize an RGB image that effectively colorizes the infrared image 310 using the RGB image 320. Referring to FIG. 4, the infrared coloring system 400 may synthesize an RGB image 490 that effectively colorizes the infrared image 410 using the RGB image 450. Referring to FIG. 5, the infrared colorizing system 500 may fine tune one of the synthesized RGB images to generate the attribute modified RGB image 550.The systems and methods described herein may be used without limitation of non-autonomous vehicles, semi-autonomous vehicles (e.g., in one or more Adaptive Driver Assistance Systems (ADAS)), steered and unrouted robots or robot platforms, storage vehicles, off-road vehicles, vehicles coupled to one or more trailers, flying ships, boats, shuttles, emergency vehicles, motor bicycles, electric or motorized bicycles, aircrafts, construction vehicles, trains, underwater vehicles, remote controlled vehicles such as drones and / or other types of vehicles. Further, the systems and methods described herein may be used for a variety of purposes, including, but not limited to, machine control, machine travel, machine drive, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, safety and monitoring, simulation and digital twinning, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation, and / or digital twinning, data center processing, conversational AI, light transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation for 3D assets, Cloud computing and / or other suitable applications.Embodiments presented herein may include a variety of different systems, such as automotive systems (e.g., an autonomous or semi-autonomous machine control system, an autonomous or semi-autonomous machine perception system), systems implemented with a robot, aeronautical systems, medial systems, boot systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twining operations, systems implemented using an edge device, systems including one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least in part in a data center, Systems for performing AI operations in dialog, systems for performing light transport simulations, systems for performing collaborative content creation for 3D assets, systems implemented at least in part using cloud computing resources, and / or other types of systems.EXAMPLE AUTONOMOUS VEHICLEFIG. 8A is an illustration of an example autonomous vehicle 800, in accordance with some embodiments of the present disclosure. The autonomous vehicle 800 (alternatively referred to herein as "vehicle 800") may include, without limitation, a passenger vehicle, such as a car, truck, bus, emergency service vehicle, shuttle, electric or motorized bicycle, motorcycle, fire truck, police vehicle, lift truck, boat, construction vehicle, underwater vehicle, robotic vehicle, drone, plane, vehicle coupled to a trailer (e.g., semi-trailer used for transporting cargo), and / or another type of vehicle (e.g., unmanned and / or receiving one or more passengers). Autonomous vehicles are generally described in terms of automation levels covered by the National Highway Traffic Safety Administration (NHTSA), a division of the U.S. Ministry of Traffic, and the Society of Automotive Engineers (SAE) "Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Engine Vehicles (Taxonomy and Definitions of Terms Related to Automated Driving Systems for Road Vehicles)" (Standard No. J3016-201806 published June 15, 2018, Standard No. J3016-20169 published September 30, 2016, as well as earlier and future versions of this standard). The vehicle 800 may have functionality according to one or more of the level 3 - level 5 of the autonomous driving levels. The vehicle 800 may have functionality according to one or more of the level 1 - level 5 of the autonomous driving levels. For example, depending on the embodiment, the vehicle 800 may be capable of providing driver assistance (level 1), partial automation (level 2), conditional automation (level 3), high automation (level 4) and / or full automation (level 5). The term "autonomous" as used herein may include any and / or all types of endoscopy for the vehicle 800 or other machine, such as fully autonomous, highly autonomous, conditionally autonomous, partially autonomous, supporting endoscopy, semi-autonomous, primarily autonomous, or other designation.The vehicle 800 may include components such as a chassis, a vehicle body, wheels (e.g., 2, 4, 6, 8, 18, etc.), tires, axles, and other components of a vehicle. The vehicle 800 may include a propulsion system 850, such as an internal combustion engine, a hybrid electric power plant, a pure electric motor, and / or another type of propulsion. The propulsion system 850 may be connected to a powertrain of the vehicle 800, which may include a transmission to enable propulsion of the vehicle 800. The propulsion system 850 may be controlled in response to receiving signals from the accelerator 852.A steering system 854, which may include a steering wheel, may be used to steer the vehicle 800 (e.g., along a desired path or route) when the propulsion system 850 is operating (e.g., when the vehicle is in motion). The steering system 854 may receive signals from a steering actuator 856. The steering wheel can optionally be for full automation (level 5).The brake sensor system 846 may be used to actuate the vehicle brakes in response to receiving signals from the brake actuators 848 and / or the brake sensors.The one or more controllers 836, which may include one or more system-on-chips (SoCs) 804 (FIG. 8C ) and / or GPUs, may provide signals (e.g., representative of instructions) to one or more components and / or systems of the vehicle 800. For example, the one or more controllers may send signals to actuate vehicle brakes via one or more brake actuators 848, to actuate the steering system 854 via one or more steering actuators 856, to actuate the propulsion system 850 via one or more throttle / accelerator devices 852. The one or more controllers 836 may include one or more onboard (e.g., integrated) computing devices (e.g., supercomputers) that process sensor signals and issue operating commands (e.g., signals representing commands) to enable autonomous driving and / or to assist a human driver in driving the vehicle 800. The one or more controllers 836 may include a first autonomous driving function controller 836, a second safety function controller 836, a third artificial intelligence (e.g., computer vision) function controller 836, a fourth infotainment function controller 836, a fifth emergency redundancy controller 836, and / or other controllers. In some examples, a single controller 836 may take over two or more of the above-mentioned functionalities, two or more controllers 836 may take over a single functionality, and / or any combination thereof.The one or more controllers 836 may provide the signals to control one or more components and / or systems of the vehicle 800 in response to sensor data received from one or more sensors (e.g., sensor inputs). The sensor data may be received, for example and without limitation, from one or more of global navigation satellite systems ("GNSS") sensor(s) 858 (e.g., global positioning system sensor(s) (global positioning system sensor(s)), RADAR sensor(s) 860, ultrasonic sensor(s) 862, LIDAR sensor(s) 864, inertial measurement unit ("IMU") sensor(s) 866 (e.g., accelerometer, gyroscope(s), magnetic compass(s), magnetometer(s), etc.), microphone(s) 896, stereo camera(s) 868, Wide-angle camera(s) 870 (e.g., fisheye cameras), infrared camera(s) 872, environmental camera(s) 874 (e.g., 360-degree cameras), long-range and / or mid-range camera(s) 898, speed sensor(s) 844 (e.g., for measuring the speed of the vehicle 800), vibration sensor(s) 842, steering sensor(s) 840, brake sensor(s) (e.g., as part of the brake sensor system 846), one or more sensor(s) of the occupant monitoring system (OMS) 801 (e.g., one or more interior cameras), and / or other types of sensors.One or more of the controllers 836 may receive input (e.g., in the form of input data) from an instrument cluster 832 of the vehicle 800 and provide output (e.g., in the form of output data, display data, etc.) via a human-machine interface (HMI) display 834, an acoustic detector, a speaker, and / or via other components of the vehicle 800. The outputs may include information such as vehicle speed, speed, time, map data (e.g., the high definition ("HD") map 822 of FIG. 8C ), location data (e.g., the location of the vehicle 800, e.g., on a map), direction, location of other vehicles (e.g., an occupancy grid), information about objects and status of objects as perceived by the one or more controllers 836, etc. For example, information about the presence of one or more objects (e.g., a road sign, a warning sign, a changing traffic light, etc.), and / or information about driving maneuvers that the vehicle has performed may be displayed on the HMI display 834, The time that the lane is being guided through or guided through is being performed (e.g., now changing lanes, take exit 34B in two miles, etc.).The vehicle 800 further includes a network interface 824 that may use one or more wireless antennas 826 and / or modems to communicate over one or more networks. The network interface 824 may be suitable for long-term evolution ("LTE"), wideband code division multiple access ("WCDMA"), universal mobile telecommunications system ("UMTS"), global system for mobile communication ("GSM"), IMT-CDMA multi-carrier ("CDMA2000"), etc., communication, for example. The one or more wireless antennas 826 may also enable communication between objects in the environment (e.g., vehicles, mobile devices, etc.) using local area networks such as Bluetooth, Bluetooth Low Energy ("LE"), Z-Wave, ZigBee, etc., and / or low power wide area networks ("LPWANs") such as LoRaWAN, SigOx, etc.FIG. 8B is an example of camera locations and fields of view for the example autonomous vehicle 800 of FIG. 8A, in accordance with some embodiments of the present disclosure; the cameras and the respective fields of view are an example of one embodiment and are not to be understood as limiting. For example, additional and / or alternative cameras may be included and / or the cameras may be located at different locations of the vehicle 800.The camera types for the cameras may include, but are not limited to, digital cameras that may be configured for use with the components and / or systems of the vehicle 800. The one or more cameras may operate with the Automotive Safety Integrity Level (ASIL) B and / or with another ASIL. The camera types may have any image capture rate, such as 60 images per second (fps), 120 fps, 240 fps, etc. The cameras may use shutters, global shutters, another type of shutter, or a combination thereof, depending on the embodiment. In some examples, the color filter array may include a red clear clear clear color filter array (RCCC), a red clear blue color filter array (RCCB), a red blue green clear color filter array (RBGC), a foveon X3 color filter array, a bayer sensor color filter array (RGGB), a monochrome sensor color filter array, and / or another type of color filter array. In some embodiments, clear pixel cameras, such as cameras with an RCCC, RCCB, and / or RBGC color filter array, may be used to increase photosensitivity.In some examples, one or more of the cameras may be used to perform advanced driver assistance system (ADAS) functions (e.g., as part of a redundant or fail-safe design). For example, a multi-function monocamera may be installed to provide functions such as lane departure warning, road sign assist, and smart headlight control. One or more of the cameras (e.g., all cameras) may simultaneously record and provide image data (e.g., video).One or more of the cameras may be mounted in a mount, e.g., in a specially designed (three-dimensional ("3D") printed) mount, to eliminate stray light and reflections from the vehicle interior (e.g., reflections from the dashboard that mirror in the mirrors of the windshield) that could interfere with image data capture of the camera. Referring to the mounting of exterior mirrors, the exterior mirrors may be printed individually in 3D so that the camera mounting plate conforms to the shape of the exterior mirror. In some examples, the one or more cameras may be integrated into the exterior mirror. In side cameras, the one or more cameras may also be integrated into the four columns at each corner of the cabin.Cameras having a field of view that includes portions of the environment in front of the vehicle 800 (e.g., forward facing cameras) may be used for the environment view to help identify forward facing paths and obstacles, and to provide information relevant to the creation of an occupancy grid and / or determination of preferred vehicle paths using one or more controllers 836 and / or control SoCs. Forward facing cameras may be used to perform many of the same ADAS functions as LiDAR, including emergency braking, pedestrian detection, and collision avoidance. Forward facing cameras may also be used for ADAS functions and systems that include lane departure warnings ("LDW"), autonomous cruise control ("ACC"), and / or other functions such as road sign detection.A variety of cameras may be used in a forward-facing configuration, including, for example, a monocular camera platform that includes a complementary metal oxide semiconductor ("CMOS") color imager. Another example is the wide-angle cameras 870 that can be used to capture objects that come into view from the periphery (e.g., pedestrians, crossing vehicles, or bicycles). Although only one wide-angle camera is illustrated in FIG. 8B, the vehicle 800 may include any number (including zero) of wide-angle cameras 870. Moreover, any number of long-range cameras 898 (e.g., a long-range stereo camera pair) may be used for depth-based object detection, particularly for objects for which a neural network has not yet been trained. The one or more long-range cameras 898 may also be used for object detection and classification as well as for basic object tracking.Any number of stereo cameras 868 may also be included in a forward-facing configuration. In at least one embodiment, one or more of stereo cameras 868 may include an integrated control unit that includes a scalable processing unit that may provide programmable logic ("FPGA") and a multi-core microprocessor with a controller area network ("CAN") or Ethernet interface on a single chip. Such a unit may be used to generate a 3D map of the environment of the vehicle that includes a range estimate for all points in the image. An alternative stereo camera 868 may include a compact stereo vision sensor that may include two camera lenses (one left and right each) and an image processing chip that may measure the distance between the vehicle and the target object and use the generated information (e.g., metadata) to enable the autonomous emergency braking and lane departure warning functions. Other types of stereo cameras 868 may be used in addition to or alternatively to those described herein.Cameras having a field of view that includes portions of the environment lateral to the vehicle 800 (e.g., side cameras) may be used for the environment view and provide information used to create and update the occupancy grid as well as to generate collision warnings in the event of a side impact. For example, the one or more environmental cameras 874 (e.g., four environmental cameras 874 as illustrated in FIG. 8B ) may be positioned on the vehicle 800. The one or more environmental cameras 874 may include one or more wide-angle cameras 870, one or more fisheye cameras, one or more 360-degree cameras, and / or the like. For example, four fish eye cameras may be mounted on the front, rear, and sides of the vehicle. In an alternative arrangement, the vehicle may use three surround cameras 874 (e.g., left, right, and rear) and use one or more other camera(s) (e.g., a front facing camera) as the fourth surround view camera.Cameras having a field of view that includes portions of the environment behind the vehicle 800 (e.g., backup cameras) may be used for parking assist, environmental view, rear end collision warnings, and occupancy grid creation and update. A variety of different cameras may be used, including, but not limited to, cameras also suitable as one or more forward facing cameras (e.g., one or more long-range and / or mid-range cameras 898, one or more stereo cameras 868, one or more infrared cameras 872, etc.), as described herein.Cameras having a field of view including portions of the interior environment in the cabin of the vehicle 800 (e.g., one or more OMS sensors 801) may be used as part of, but not limited to, an occupant monitoring system (OMS) such as a driver monitoring system (DMS). For example, OMS sensors (e.g., the one or more OMS sensors 801) may be used (e.g., by the one or more controllers 836) to track occupant and / or driver gaze direction, head posture, and / or blink. This gaze information may be used to determine the degree of attention of the occupant or driver (e.g., to detect drowsiness, tiredness, and / or distraction) and / or take appropriate actions to prevent damage to the occupant or driver. In some embodiments, data from OMS sensors may be used to enable gaze-controlled operations triggered by the driver and / or other occupant, such as adjusting cabin temperature and / or airflow, opening and closing windows, controlling cabin lighting, controlling entertainment systems, adjusting mirrors, adjusting seat positions, and / or other operations. In some embodiments, an OMS may be used for applications such as determining whether objects and / or occupants have been left in a vehicle cabin (e.g., by detecting the presence of occupants after the driver has left the vehicle).FIG. 8C is a block diagram of an example system architecture for the example autonomous vehicle 800 of FIG. 8A, in accordance with some embodiments of the present disclosure. It should be appreciated that these and other arrangements described herein are presented by way of example only. Other arrangements and elements (e.g., machines, interfaces, functions, arrangements, groupings of functions, etc.) may be used in addition to or in place of those shown, and some elements may be omitted altogether. In addition, many of the elements described herein are functional entities that can be implemented as single or distributed components or in conjunction with other components and in any suitable combination and location. Various functions described herein that are performed by entities may be performed by hardware, firmware, and / or software. Various functions may be performed, for example, by a processor executing instructions stored in memory.Each of the components, features, and systems of vehicle 800 in FIG. 8C is illustrated as being connected via bus 802. Bus 802 may include a controller area network (CAN) data interface (alternatively referred to herein as a "CAN bus"). A CAN may be a network within the vehicle 800 that serves to assist in controlling various features and functions of the vehicle 800, such as actuation of brakes, acceleration, brakes, steering, windshield wipers, etc. A CAN bus may be configured to have dozens or even hundreds of nodes, each of which has its own unique identifier (e.g., a CAN ID). The CAN bus may be read to determine the steering wheel angle, the vehicle speed, the engine speed (U / min), the button positions, and / or other vehicle status indicators. The CAN bus may be ASIL B compliant.Although bus 802 is described herein as a CAN bus, this is not to be understood as limiting. For example, FlexRay and / or Ethernet may be used in addition or alternatively to the CAN bus. In addition, while a single line is used to represent bus 802, this is not intended to be limiting. For example, there may be any number of buses 802, which may include one or more CAN buses, one or more FlexRay buses, one or more Ethernet buses, and / or one or more other types of buses using a different protocol. In some examples, two or more buses 802 may be used to perform different functions and / or may be used for redundancy. For example, a first bus 802 may be used for the collision avoidance functionality and a second bus 802 may be used for the actuation control. In each example, each bus 802 may communicate with one of the components of the vehicle 800, and two or more buses 802 may communicate with the same components. In some examples, each SoC 804, controller 836, and / or computer within the vehicle may have access to the same input data (e.g., inputs from sensors of the vehicle 800) and may be connected to a common bus, such as the CAN bus.The vehicle 800 may include one or more controllers 836 as described herein with respect to FIG. 8A. The one or more controllers 836 may be used for a variety of functions. The one or more controllers 836 may be coupled to one of the various other components and systems of the vehicle 800 and may be used to control the vehicle 800, artificial intelligence of the vehicle 800, infotainment for the vehicle 800, and / or the like.The vehicle 800 may include one system (or more systems) on a chip (SoC) 804. The SoC 804 may include one or more CPUs 806, one or more GPUs 808, one or more processors 810, one or more caches 812, one or more accelerators 814, one or more data stores 816, and / or other components and features not illustrated. The one or more SoCs 804 may be used to control the vehicle 800 in a variety of platforms and systems. For example, the one or more SoCs 804 may be combined in a system (e.g., the vehicle 800 system) with a HD card 822, which may receive card refreshes and / or updates from one or more servers (e.g., the one or more servers 878 of FIG. 8D ) via a network interface 824.The one or more CPUs 806 may include a CPU cluster or complex (alternatively referred to herein as a "CCPLEX"). The one or more CPUs 806 may include multiple cores and / or L2 caches. In some embodiments, the one or more CPUs 806 may include, for example, eight cores in a coherent multiprocessor configuration. In some embodiments, the one or more CPUs 806 may include four dual core clusters, each cluster having a dedicated L2 cache (e.g., a 2 MB L2 cache). The one or more CPUs 806 (e.g., the CCPLEX) may be configured to support concurrent operation of clusters such that any combination of the clusters of the one or more CPUs 806 may be active at a particular time.The one or more CPUs 806 may implement power management functions that include one or more of the following features: individual hardware blocks may be automatically clock controlled at idle to conserve dynamic power; each core clock may be enabled when the core is not actively executing commands due to execution of WFI / WFE commands; each core may be independently power controlled; each core cluster may be independently clock controlled when all cores are clock controlled or power controlled; and / or each core cluster may be independently power controlled when all cores are power controlled. The one or more CPUs 806 may also implement an enhanced power state management algorithm in which allowed power states and expected wake-up times are set and the hardware / microcode determines the best power state to enter for the core, cluster, and CCPLEX. The processing cores may support simplified sequences for input of the power state in software, offloading work to microcode.The one or more GPUs 808 may include an integrated GPU (alternatively referred to herein as "iG"). The one or more GPUs 808 may be programmable and may be efficient for parallel workloads. The one or more GPUs 808 may use an extended tensor instruction set in some examples. The one or more GPUs 808 may include one or more streaming microprocessors, where each streaming microprocessor may include an L1 cache (e.g., an L1 cache with at least 96 KB storage capacity), and two or more of the streaming microprocessors may share an L2 cache (e.g., an L2 cache with 512 KB storage capacity). In some embodiments, the one or more GPUs 808 may include at least eight streaming microprocessors. The one or more GPUs 808 may use one or more application programming interfaces (API) for computations. Moreover, the one or more GPUs 808 may use one or more parallel computing platforms and / or programming models (e.g., CUDA from NVIDIA).The one or more GPUs 808 may be energy optimized for best performance in automotive and embedded use cases. The one or more GPUs 808 may be fabricated on a fin field effect transistor (FinFET), for example. However, this is not to be understood as limiting, and the one or more GPUs 808 may also be fabricated with other semiconductor fabrication processes. Each streaming microprocessor may include a series of composite precision processing messages divided into multiple blocks. For example and without limitation, 64 PF32 cores and 32 PF64 cores may be divided into four processing blocks. In such an example, each processing block may be allocated 16 FP32 cores, 8 FP64 cores, 16 INT32 cores, two mixed precision NVIDIA TENSOR COREs for deep leaming matrix arithmetic, an L0 instruction cache, a warp scheduler, a dispatch unit, and / or a 64 KB register file. In addition, the streaming microprocessors may include independent parallel integer and floating point data paths to enable efficient execution of workloads with a mixture of computations and addressing computations. The streaming microprocessors may include an independent thread scheduling function to enable fine grain synchronization and cooperation between parallel threads. The streaming microprocessors may include a combined L1 data cache and a common memory unit to improve performance while simplifying programming.The one or more GPUs 808 may include high bandwidth memory (HBM) and / or a 16 GB HBM2 memory subsystem to provide a peak memory bandwidth of about 900 GB / second, in some examples. In some examples, in addition to or as an alternative to HBM memory, a synchronous graphics random-access memory (SGRAM) may be used, such as a double data rate synchronous graphics random-access memory type five (GDDR5).The one or more GPUs 808 may include unified memory technology that includes access counters to enable more accurate migration of memory pages to the processor that most frequently accesses them, thereby improving efficiency for processor shared memory areas. In some examples, Address Translation Services (ATS) support may be used to allow the one or more GPUs 808 to directly access the page tables of the one or more CPUs 806. In such examples, if the memory management unit (MMU) of the one or more GPUs 808 fails, an address translation request may be sent to the one or more CPUs 806. In response, the one or more CPUs 806 may search in their page tables for the virtual physical mapping for the address and send translation back to the one or more GPUs 808. Thus, the unified memory technology may enable a single unified virtual address space for the memory of both the one or more CPUs 806 and the one or more GPUs 808, thus facilitating programming of the one or more GPUs 808 and porting applications to the one or more GPUs 808.Additionally, the one or more GPUs 808 may include an access counter that may track the frequency of access of the one or more GPUs 808 to memory of other processors. The access counter may help to move memory pages into physical memory of the processor that most frequently accesses the pages.The one or more SoCs 804 may include any number of cache(s) 812 including those described herein. For example, the one or more caches 812 may include an L3 cache available to both the one or more CPUs 806 and the one or more GPUs 808 (e.g., connected to both the one or more CPUs 806 and the one or more GPUs 808). The one or more caches 812 may include a write back cache that may track the states of the lines, e.g., by using a cache coherency protocol (e.g., MEI, MESI, MSI, etc.). The L3 cache may contain 4 MB or more, depending on embodiment, although smaller cache sizes may also be used.The one or more SoCs 804 may include the one or more arithmetic logic units (ALUs) that may be used in performing processing related to any of the various tasks or operations of the vehicle 800, such as processing DNNs. Additionally, the one or more SoCs 804 may include one or more floating point units (FPUs) or other mathematical co-processors or numerical co-processors for performing mathematical operations within the system. For example, the one or more SoCs 804 may include one or more FPUs integrated as execution units into one or more CPUs 806 and / or one or more GPUs 808.The one or more SoCs 804 may include one or more accelerators 814 (e.g., hardware accelerators, software accelerators, or a combination thereof). The one or more SoCs 804 may include, for example, a hardware acceleration cluster that may include optimized hardware accelerators and / or large memory on-chip. The large memory on-chip (e.g., 4MB SRAM) may allow the hardware acceleration cluster to accelerate neural networks and other computations. The hardware acceleration cluster may be used as a supplement to the one or more GPUs 808 and offload some of the tasks of the one or more GPUs 808 (e.g., to enable more cycles of the one or more GPUs 808 to perform other tasks). The one or more accelerators 814 may be used, for example, for targeted workloads (e.g., perception, convolutional neural networks (CNNs), etc.) that are stable enough to be suitable for acceleration. As used herein, the term "CNN" may include all types of CNNs, including region-based or regional convolutional neural networks (RCNNs) and fast RCNNs (e.g., for object detection).The one or more accelerators 814 (e.g., the hardware acceleration cluster) may include a deep learning accelerator (DLA). The one or more DLAs may include one or more tensor processing units (TPUs) configured to provide additional ten billion operations per second for deep learning applications and inferencing. The TPUs may be accelerators configured and optimized for performing image processing functions (e.g., for CNNs, RCNNs, etc.). The one or more DLAs may also be optimized for a particular set of neural network types and floating point operations, as well as inferencing. The design of the one or more DLAs can provide more performance per millimeter than a general purpose GPU and far exceeds the performance of a CPU. The one or more TPUs may perform multiple functions including a single instance convolution function that supports, for example, INT8, INT16, and FP16 data types for both features and weights, and postprocessor functions.The one or more DLAs may quickly and efficiently execute neural networks, particularly CNNs, on processed or unprocessed data for a variety of functions including, for example and without limitation: a CNN for identifying and detecting objects using camera sensor data; a CNN for distance estimation using camera sensor data; a CNN for detecting and identifying emergency vehicles using microphone data; a CNN for face detection and identifying vehicle owners using camera sensor data; and / or a CNN for security and / or protection relevant events.The one or more DLAs may perform each function of the one or more GPUs 808, and by using an inference accelerator, for example, a developer may provide either the one or more DLAs or the one or more GPUs 808 to each function. For example, the developer may focus the processing of CNNs and floating point operations on the one or more DLAs and leave other functions of the one or more GPUs 808 and / or other accelerators 814.The one or more accelerators 814 (e.g., the hardware acceleration cluster) may include a programmable vision accelerator (PVA), which may also be referred to herein as a computer vision accelerator. The one or more PVAs may be developed and configured to accelerate computer vision algorithms for advanced driver assistance systems (ADAS), autonomous driving, and / or augmented reality (AR) and / or virtual reality (VR) applications. The one or more PVAs may provide a balance between performance and flexibility. Each PVA may include, for example and without limitation, any number of reduced instruction set computer (RISC) cores, direct memory access (DMA), and / or any number of vector processors.The RISC cores may interact with image sensors (e.g., the image sensors of any of the cameras described herein), image signal processors, and / or the like. Each of the RISC cores may include any amount of memory. The RISC cores may use any number of protocols, depending on the embodiment. In some examples, the RISC cores may execute a real-time operating system (RTOS). The RISC cores may be implemented with one or more integrated circuits, application specific integrated circuits (ASICs), and / or memory devices. The RISC cores may include, for example, an instruction cache and / or a close coupled RAM.The DMA may allow components of the PVA(s) to access the memory of the system independent of the one or more CPUs 806. The DMA may support any number of features that serve to optimize the PVA, including, but not limited to, support multidimensional addressing and / or circular addressing. In some examples, DMA may support up to six or more dimensions of addressing, which may include block width, block height, block depth, horizontal block locking, vertical block locking, and / or depth locking.The vector processors may be programmable processors that may be designed to efficiently and flexibly execute programming for computer vision algorithms and provide signal processing functions. In some examples, the PVA may include a PVA core and two vector processing subsystem partitions. The PVA core may include a processor subsystem, one or more DMA engines (e.g., two DMA engines), and / or other peripheral devices. The vector processing subsystem may operate as a primary processing engine of the PVA and may include a vector processing unit (VPU), an instruction cache, and / or a memory (e.g., VMEM). A VPU core may include a digital signal processor, such as a single instruction, multiple data (SIMD) and very long instruction words (VLIW). The combination of SIMD and VLIW may increase throughput and speed.Each of the vector processors may include an instruction cache and may be coupled to dedicated memory. Thus, in some examples, each of the vector processors may be configured to operate independently of the other vector processors. In other examples, the vector processors included in a particular PVA may be configured to use data parallelism. For example, in some embodiments, the plurality of vector processors included in a single PVA may execute the same computer vision algorithm, but on different regions of an image. In other examples, the vector processors included in a particular PVA may execute different computers concurrently, vision algorithms on the same image, or even execute different algorithms on successive images or portions of an image. Among other things, any number of PVAs may be included in the hardware acceleration cluster and any number of vector processors in each of the PVAs. Moreover, the one or more PVAs may include additional Error Correcting Code (ECC) memories to increase overall system security.The one or more accelerators 814 (e.g., the hardware acceleration cluster) may include a computer vision network on-chip and SRAM to provide high bandwidth, low latency SRAM to the one or more accelerators 814. In some examples, the on-chip memory may include at least 4 MB SRAM, which may consist of, for example and without limitation, eight field configurable memory blocks that may be accessed by both the PVA and the DLA. Each pair of memory blocks may include an advanced peripheral bus (APB) interface, configuration circuits, a controller, and a multiplexer. Any type of memory may be used. The PVA and the DLA may access the memory via a backbone that enables the PVA and the DLA to access the memory at high speed. The backbone may include an on-chip computer vision network that connects the PVA and the DLA to the memory (e.g., using the APB).The computer vision network on-chip may include an interface that determines that both the PVA and the DLA provide ready and valid signals prior to transmission of control signals / addresses / data. Such an interface may provide separate phases and separate channels for the transmission of control signals / addresses / data as well as bursty communication for continuous data transmission. This type of interface may conform to the ISO 26262 or IEC 61508 standards, although other standards and protocols may also be used.In some examples, the one or more SoCs 804 may include a real-time ray tracing hardware accelerator as described in U.S. patent application Ser. No. 16 / 101,232, filed Aug. 10, 2018. The real-time ray tracing hardware accelerator may be used to quickly and efficiently determine the positions and extents of objects (e.g., within a world model), to generate real-time visualization simulations, for RADAR signal interpretation, for sound propagation synthesis and / or analysis, for simulation of SONAR systems, for general wave propagation simulation, for comparison with LIDAR data for purposes of localization and / or other functions and / or other purposes. In some embodiments, one or more Tree Traversal Units (TTUs) may be used for performing one or more operations associated with ray tracing.The one or more accelerators 814 (e.g., the hardware accelerator cluster) have a wide range of uses for autonomous driving. The PVA may be a programmable image processing accelerator that may be used for important processing steps in ADAS and autonomous vehicles. The capabilities of the PVA fit well to algorithmic areas that require predictable processing with low power consumption and low latency. In other words, the PVA is well suited for semi-dense or dense regular computations, even for small data sets requiring predictable low latency, low power consumption runtimes. Thus, in the context of autonomous vehicle platforms, the PVAs are designed to execute classical computer vision algorithms because they are efficient in object detection and operate with integer mathematics.According to one embodiment of the technology, the PVA is used, for example, to perform computer stereo vision. In some examples, a semi-global matching based algorithm may be used, although this is not intended to be limiting. Many autonomous driving level 3-5 applications require spontaneous motion estimation (e.g., structure of motion, pedestrian detection, lane detection, etc.). The PVA may perform a computer stereo vision function on inputs from two monocular cameras.In some examples, the PVA may be used to perform a dense optical flow. According to processing raw RADAR data (e.g., using a 4D fast Fourier transform) to provide processed RADAR. In other examples, the PVA is used for processing time of flight depth, e.g., by processing time of flight raw data to provide processed time of flight data.The DLA may be used to operate any type of network to improve control and driving safety; for example, this includes a neural network that outputs a confidence measure for each object detection. Such a confidence value may be interpreted as a probability or as a relative "weight" of each detection compared to other detections. This confidence value allows the system to make further decisions about which detections should be considered true positive detections and not false positive detections. For example, the system may set a threshold for confidence and consider only the detections exceeding the threshold as true positive detections. In an automatic emergency braking (AEB) system, false positive detections would result in the vehicle automatically performing emergency braking, which is, of course, undesirable. Therefore, only the most safe detections should be considered as triggers for AEB. The DLA may employ a neural network to regression the confidence value. The neural network may use as input at least a subset of parameters, such as bounding box dimensions, ground plane estimate obtained (e.g., from another subsystem), swing measurement unit (IMU) sensor 866 output correlated with the orientation of the vehicle 800, range, 3D position estimates of the object obtained from the neural network and / or other sensors (e.g., LIDAR sensors 864 or RADAR sensors 860), among others.The one or more SoCs 804 may include the one or more data stores 816 (e.g., memory). The one or more data stores 816 may be on-chip memory on the one or more SoCs 804 in which neural networks to execute on the GPU and / or the DLA may be stored. In some examples, the one or more data stores 816 may be large enough to store multiple instances of neural networks for redundancy and security. The one or more data stores 816 may include L2 or L3 cache(s) 812. The reference to the one or more of the data stores 816 may include a reference to memory associated with the PVA, the DLA, and / or one or more other accelerators 814 as described herein.The one or more SoCs 804 may include one or more processors 810 (e.g., embedded processors). The one or more processors 810 may include a boot and power management processor, which may be a dedicated processor and subsystem to handle the boot power and management functions and associated security enforcement. The boot and power management processor may be part of the boot sequence of the one or more SoCs 804 and may provide runtime power management services. The boot and power management processor may provide clock and voltage programming, support for transitions of the system to a low power state, management of the thermals and temperature sensors of the one or more SoCs 804, and / or management of the one or more SoCs 804 power states. Each temperature sensor may be implemented as a ring oscillator whose output frequency is proportional to temperature, and the one or more SoCs 804 may use the ring oscillators to sense the temperatures of the one or more CPUs 806, the one or more GPUs 808, and / or the one or more accelerators 814. When it is determined that the temperatures exceed a threshold, the boot and power management processor may enter a temperature fault routine and place the one or more SoCs 804 in a lower power state and / or place the vehicle 800 in a chauffeur-for-safe-stop mode (e.g., place the vehicle 800 in a safe stop).The one or more processors 810 may also include a number of embedded processors, which may serve as an audio processing engine. The audio processing engine may be an audio subsystem that enables full hardware support for multi-channel audio over multiple interfaces and a wide and flexible range of audio I / O interfaces. In some examples, the audio processing engine is a dedicated processor core with a dedicated RAM digital signal processor.The one or more processors 810 may also include an Ways-On processor engine that may provide the necessary hardware functions to support sensor management with low power consumption and waking up use cases. The "Ways-On" processor engine may include a processor core, close-coupled RAM, supporting peripherals (e.g., timers and interrupt controllers), various I / O controller peripherals, and routing logic.The one or more processors 810 may also include a safety cluster machine that includes a dedicated processor subsystem for the safety management of automotive applications. The safety cluster machine may include two or more processor cores, close coupled RAM, supporting peripherals (e.g., timers, an interrupt controller, etc.), and / or routing logic. In a safety mode, the two or more cores may operate in a lockstep mode and function as a single core with comparison logic that detects all differences between their operations.The one or more processors 810 may also include a real-time camera engine, which may include a dedicated processor subsystem for managing the real-time camera.The one or more processors 810 may further include a high dynamic range signal processor, which may include an image signal processor that is a hardware machine that is part of the camera processing pipeline.The one or more processors 810 may include a video image combiner, which may be a processing block (e.g., implemented on a microprocessor) that implements video post-processing functions needed by a video playback application to generate the final image for the player window. The video image combiner may make a lens distortion correction at the one or more wide angle cameras 870, the one or more surround cameras 874 and / or at the sensors of the surveillance camera in the cabin. The sensor of the surveillance camera in the cabin is preferably monitored by a neural network running on another instance of the enhanced SoC and configured to detect events in the cabin and react accordingly. A system in the cabin may perform lipread to activate the cellular service and place a call, dictate emails, change destination, activate or change infotainment system and vehicle settings, or enable voice-controlled browsing on the Internet. Certain functions are available to the driver only when the vehicle is operating in an autonomous mode and are otherwise disabled.The video image combiner may include improved temporal noise suppression for both spatial and temporal noise suppression. For example, when motion occurs in a video, the noise suppression weights the spatial information accordingly and reduces the weight of the information provided by adjacent images. If an image or portion of an image does not contain motion, the temporal noise suppression performed by the video image combiner may use information from the previous image to reduce the noise in the current image.The video image combiner may also be configured to perform stereo equalization of the input stereo objective images. The video image combiner may also be used for user interface design when the operating system desktop is in use and the one or more GPUs 808 do not need to continually render new surfaces. Even when the one or more GPUs 808 are turned on and actively operate 3D rendering, the video image combiner may be used to offload the one or more GPUs 808 and thus improve performance and responsiveness.The one or more SoCs 804 may also include a Mobile Industry Processor Interface (MIPI) serial camera interface for receiving video and input from cameras, a high-speed interface, and / or a video input block that may be used for camera and related pixel input functions. The one or more SoCs 804 may also include one or more input / output controllers, which may be controlled by software and used to receive I / O signals that are not associated with any particular role.The one or more SoCs 804 may further include a wide range of peripheral interfaces to enable communication with peripheral devices, audio codecs, power management, and / or other devices. The one or more SoCs 804 may be used to process data from cameras (e.g., via gigabit multimedia serial link and Ethernet), sensors (e.g., one or more LIDAR sensors 864, one or more RADAR sensors 860, etc., which may be connected via Ethernet), data from bus 802 (e.g., speed of vehicle 800, steering wheel position, etc.), data from one or more GNSS sensors 858 (e.g., connected via Ethernet or CAN bus). The one or more SoCs 804 may further include dedicated high-performance mass storage controllers, which may include their own DMA engines and which may be used to offload the one or more CPUs 806 from routine data management tasks.The one or more SoCs 804 may be an end-to-end platform with a flexible architecture that extends across automation levels 3- 5 and thereby provides a comprehensive functional safety architecture that uses computer vision and ADAS techniques for diversity and redundancy and provides a platform for a flexible, reliable driving software stack along with deep learning tools. The one or more SoCs 804 may be faster, more reliable, and even more energy and space saving than conventional systems. For example, the one or more accelerators 814 in combination with the one or more CPUs 806, the one or more GPUs 808, and the one or more data stores 816 may form a fast, efficient Level 3-5 autonomous vehicle platform.The technology thus provides capabilities and functions that cannot be achieved by conventional systems. For example, computer vision algorithms can be executed on CPUs that can be configured using a high-level programming language, such as C programming language, to execute a variety of processing algorithms for a variety of visual data. However, CPUs are often unable to meet the performance requirements of many computer vision applications, such as execution time and power consumption requirements. In particular, many CPUs are unable to execute complex object detection algorithms in real-time, which is a prerequisite for in-vehicle ADAS applications and a prerequisite for practical level 3-5 autonomous vehicles.In contrast to conventional systems, the technology described herein, by providing a CPU complex, a GPU complex, and a hardware acceleration cluster, enables multiple neural networks to be executed simultaneously and / or sequentially and the results combined to enable the level 3-5 autonomous driving functionality. A CNN executing on the DLA or dG (e.g., the one or more GPUs 820) may include, for example, text and word detection that enables the supercomputer to read and understand traffic signs, including signs for which the neural network has not been specifically trained. The DLA may further include a neural network capable of identifying, interpreting, and providing a semantic understanding and passing this semantic understanding to the path planning modules running on the CPU complex.Another example is that multiple neural networks may run simultaneously, as required for level 3, 4, or 5 driving. For example, a warning sign labeled "Caution: flashing lights indicate Smoothis" may be interpreted independently or jointly together with an electrical light from multiple neural networks. The sign itself may be identified as a traffic sign by a first deployed neural network (e.g., a trained neural network), the text "flashing lights indicate smoothis" may be interpreted by a second deployed neural network informing the path planning software of the vehicle (preferably executed on the CPU complex) that when flashing lights are detected smoothis is present. The turn signal light may be identified by operating a third neural network across multiple images informing the path planning software of the vehicle of the presence (or absence) of turn signals. All three neural networks may run simultaneously, e.g., within the DLA and / or on the one or more GPUs 808.In some examples, a CNN for face detection and vehicle owner identification may use data from camera sensors to identify the presence of an authorized driver and / or owner of the vehicle 800. The Ways-On-Sensor processing engine may be used to unlock the vehicle when the owner approaches the driver door and turn on the lights, and to disable the vehicle in the security mode when the owner leaves the vehicle. In this way, the one or more SoCs 804 provide security against theft and / or carjacking.In another example, a emergency vehicle detection and identification CNN may use data from microphones 896 to detect and identify emergency vehicle sirens. Unlike conventional systems that use general classifiers to detect sirens and manually extract features, the one or more SoCs 804 use the CNN to classify environmental and urban sounds as well as to classify visual data. In a preferred embodiment, the CNN running on the DLA is trained to detect the relative approach speed of the emergency vehicle (e.g., using the Doppler effect). The CNN may also be trained to identify emergency vehicles specific to the local area in which the vehicle operates as identified by one or more GNSS sensors 858. For example, the CNN will attempt to detect European sirens when operating in Europe, and when operating in the United States, the CNN will attempt to identify only North American sirens. Once an emergency vehicle is detected, a controller may be used to execute an emergency vehicle safety routine, slow the vehicle, drive to the roadside, park the vehicle, and / or idle the vehicle using ultrasonic sensors 862 until the one or more emergency vehicles pass.The vehicle may include one or more CPUs 818 (e.g., discrete one or more CPUs or one or more dCPUs), which may be coupled to the one or more SoCs 804 via a high-speed link (e.g., PCIe). The CPUs 818 may include, for example, an X86 processor. For example, the CPUs 818 may be used to perform a variety of functions including arbitration for potentially inconsistent results between ADAS sensors and the one or more SoCs 804 and / or monitoring the status and state of the one or more controllers 836 and / or the infotainment SoC 830.The vehicle 800 may include one or more GPUs 820 (e.g., one or more discrete GPUs or one or more dGs), which may be coupled to the one or more SoCs 804 via a high-speed link (e.g., NVIDIAs NVLINK). The one or more GPUs 820 may provide additional artificial intelligence functions, e.g., through execution of redundant and / or different neural networks, and may be used to train and / or update neural networks based on inputs (e.g., sensor data) from sensors of the vehicle 800.The vehicle 800 may further include the network interface 824, which may include one or more wireless antennas 826 (e.g., one or more wireless antennas for various communication protocols, such as a cellular antenna, a Bluetooth antenna, etc.). The network interface 824 may be used to enable wireless connection over the Internet to the cloud (e.g., to the one or more servers 878 and / or other network devices), to other vehicles, and / or to computing devices (e.g., passenger client devices). To communicate with other vehicles, a direct connection between the two vehicles and / or an indirect connection may be established (e.g., via networks and the Internet). Direct connections may be established via vehicle-to-vehicle communication. The vehicle-to-vehicle communication may provide information to the vehicle 800 about vehicles in the vicinity of the vehicle 800 (e.g., vehicles in front of, beside, and / or behind the vehicle 800). This functionality may be part of a cooperative adaptive cruise control function of the vehicle 800.The network interface 824 may include an SoC that provides modulation and demodulation functions and allows the controller(s) 836 to communicate over wireless networks. The network interface 824 may include a radio frequency front end for up-converting from baseband to radio frequency and down-converting from radio frequency to baseband. The frequency conversions can be carried out using known methods and / or using super-heterodyne methods. In some examples, the radio frequency front-end functionality may be provided by a separate chip. The network interface may include wireless functionality for communicating over LTE, WCDMA, UMTS, GSM, CDMA2000, Bluetooth, Bluetooth LE, Wi-Fi, Z-Wave, ZigBee, LoRaWA, and / or other wireless protocols.The vehicle 800 may further include one or more data stores 828, which may be off-chip (e.g., off-chip from the SoCs 804). The one or more data stores 828 may include one or more memory elements, including RAM, SRAM, DRAM, VRAM, flash, hard drives, and / or other components and / or devices capable of storing at least one bit of data.The vehicle 800 may also include one or more GNSS sensors 858. The one or more GNSS sensors 858 (e.g., GPS, assisted GPS sensors, differential GPS (DGPS) sensors, etc.) assist in mapping, perception, occupancy grid creation, and / or path planning. Any number of GNSS sensors 858 may be used, including, for example and without limitation, a GPS using a USB port with an Ethernet to Serial (RS-232) bridge.The vehicle 800 may also include one or more RADAR sensors 860. The one or more RADAR sensors 860 may be used by the vehicle 800 to detect long-range vehicles, even in darkness and / or bad weather conditions. The functional safety plane of the RADAR may be ASIL B. The one or more RADAR sensors 860 may use the CAN and / or bus 802 (e.g., to transmit the data generated by the one or more RADAR sensors 860) to control and access object tracking data, in some examples, the raw data being accessed via Ethernet. A variety of RADAR sensor types may be used. The one or more RADAR sensors 860 may be suitable for, for example, front, rear, and side RADARs without limitation. In some examples, one or more pulse Doppler RADAR sensors are used.The one or more RADAR sensors 860 may include various configurations, such as a long range narrow field of view, a short range wide field of view, a side coverage short range, etc. In some examples, a long range RADAR may be used for the adaptive cruise control function. The long range RADAR systems may provide a wide field of view realized by two or more independent scans, e.g., within a range of 250 m. The one or more RADAR sensors 860 may aid in distinguishing between static and moving objects and may be used by ADAS systems for emergency braking assist and forward collision warning. Long range RADAR sensors may include a monostatic multimode RADAR with multiple (e.g., six or more) fixed RADAR antennas and a high speed CAN and FlexRay interface. In a six antenna example, the middle four antennas may generate a focused beam pattern that is to detect the surroundings of the vehicle 800 at higher speeds with minimal interference from traffic in the adjacent lanes. The other two antennas may extend the field of view so that vehicles entering or leaving the lane of vehicle 800 may be quickly detected.For example, mid range RADAR systems may include a range of up to 860 m (front) or 80 m (rear) and a field of view of up to 42 degrees (front) or 850 degrees (rear). Short range RADAR systems may include, among other things, RADAR sensors provided for installation at both ends of the rear bumper. When such a RADAR sensor system is installed at both ends of the rear bumper, it can generate two beams that constantly monitor blind spot in the rear area and adjacent to the vehicle.Short range RADAR systems may be used in an ADAS blind spot detection system and / or as a lane change assist.The vehicle 800 may also include one or more ultrasonic sensors 862. The one or more ultrasonic sensors 862, which may be one or more mounted on the front, rear and / or sides of the vehicle 800, may be used for parking assistance and / or for creating and updating an occupancy grid. A plurality of ultrasonic sensors 862 may be used, and different ultrasonic sensors 862 may be used for different sensing ranges (e.g., 2.5 m, 4 m). The one or more ultrasonic sensors 862 may operate with functional safety levels of ASIL B.The vehicle 800 may include one or more LIDAR sensors 864. The one or more LIDAR sensors 864 may be used for object and pedestrian detection, emergency braking, collision avoidance, and / or other functions. The one or more LIDAR sensors 864 may correspond to the functional safety plane ASIL B. In some examples, the vehicle 800 may include multiple LIDAR sensors 864 (e.g., two, four, six, etc.) that may use Ethernet (e.g., to provide data to a gigabit Ethernet switch).In some examples, the one or more LIDAR sensors 864 may be capable of providing a list of objects and their distances for a 360 degree field of view. Commercially available LIDAR sensors 864 may have, for example, an announced range of about 800 m, with an accuracy of 2 cm-3 cm, and with support for an 800 Mbps Ethernet connection. In some examples, one or more non-protruding LIDAR sensors 864 may be used. In such examples, the one or more LIDAR sensors 864 may be implemented as a small device that may be embedded in the front, rear, sides, and / or corners of the vehicle 800. The one or more LIDAR sensors 864 may provide a horizontal field of view of up to 120 degrees and a vertical field of view of up to 35 degrees, with a range of 200 m, even for low reflectivity objects, in such examples. The one or more front-mounted LIDAR sensors 864 may be configured for a horizontal field of view between 45 degrees and 135 degrees.In some examples, LIDAR technologies such as 3D flash LIDAR may also be used. 3D flash LIDAR uses a laser flash as a transmission source to illuminate the environment of the vehicle up to about 200 m. A flash LIDAR unit includes a receptor that records the time of flight of the laser pulse and the reflected light on each pixel, which in turn corresponds to the distance between the vehicle and the objects. Flash LIDAR may allow highly accurate and distortion-free images of the environment to be generated with each laser flash. In some examples, four flash LIDAR sensors may be employed, one on each side of the vehicle 800. Available 3D flash LIDAR systems include a solid state 3D star-array LIDAR camera that does not include moving parts (e.g., a non-scanning LIDAR device) other than a blower. The flash LIDAR device may use a 5 nanosecond class I (eye proof) laser pulse per image and acquire the reflected laser light in the form of 3D range point clouds and co-registered intensity data. By using flash LIDAR and because flash LIDAR is a solid state device without moving parts, the one or more LIDAR sensors 864 may be less susceptible to motion blur, vibrations, and / or shocks.The vehicle may also include one or more IMU sensors 866. The one or more IMU sensors 866 may be located in the center of the rear axle of the vehicle 800 in some examples. The one or more IMU sensors 866 may include, for example and without limitation, one or more accelerometers, one or more magnetometers, one or more gyroscopes, one or more magnetic compasses, and / or other types of sensors. In some examples, such as in six-axis applications, the one or more IMU sensors 866 may include accelerometers and gyroscopes, while in nine-axis applications the one or more IMU sensors 866 may include accelerometers, gyroscopes, and magnetometers.In some embodiments, the one or more IMU sensors 866 may be implemented as a miniaturized, high performance GPS-based inertial navigation system (GPS) combining micro-electro-mechanical system (MEMS) inertial sensors, a high sensitivity GPS receiver, and advanced Kalman filter algorithms to provide estimates of position, velocity, and attitude. Thus, in some examples, the one or more IMU sensors 866 may enable the vehicle 800 to estimate the heading without requiring input from a magnetic sensor by directly observing the changes in speed of GPS and correlating with the one or more IMU sensors 866. In some examples, the one or more IMU sensors 866 and the one or more GNSS sensors 858 may be combined into a single integrated unit.The vehicle may include one or more microphones 896 mounted in and / or around the vehicle 800. The one or more microphones 896 may be used to capture and identify emergency vehicles, among other things.The vehicle may further include any number of camera types, including one or more stereo cameras 868, one or more wide-angle cameras 870, one or more infrared cameras 872, one or more surround cameras 874, one or more long-range and / or mid-range cameras 898, and / or other camera types. The cameras may be used to capture image data around the entire perimeter of the vehicle 800. Which types of cameras are used will depend on the embodiments and requirements of the vehicle 800, and any combination of camera types may be used to ensure the necessary coverage around the vehicle 800. Moreover, the number of cameras may be different depending on the embodiment. For example, the vehicle may include six cameras, seven cameras, ten cameras, twelve cameras, and / or another number of cameras. The cameras may support gigabit multimedia serial link (GMSL) and / or gigabit Ethernet, by way of example and without limitation. Each of the one or more cameras will be described in more detail herein with reference to FIGS. 8A and 8B.The vehicle 800 may further include one or more vibration sensors 842. The one or more vibration sensors 842 may measure vibrations of components of the vehicle, such as the one or more axles. For example, changes in vibrations may be indicative of a change in road surface. In another example, when two or more vibration sensors 842 are used, the differences between the vibrations may be used to determine friction or slippage of the road surface (e.g., when the difference in vibration exists between a driven axle and a freely rotating axle).The vehicle 800 may include an ADAS system 838. The ADAS system 838 may include a SoC in some examples. The ADAS system 838 may include autonomous / adaptive / automatic cruise control (ACC), cooperative adaptive cruise control (CACC), forward crash warning (FCW), automatic emergency braking (AEB), lane departure warning (LDW), lane keep assist (LKA), blind spot warning (BSW), Rear Cross-Traffic Warning (RCTW), Collision Warning Systems (CWS), Lane Centering (LC), and / or other features and functions.The ACC systems may use one or more RADAR sensors 860, one or more LIDAR sensors 864, and / or one or more cameras. The ACC systems may include longitudinal ACC and / or lateral ACC. The longitudinal ACC monitors and controls the distance to the vehicle immediately ahead of the vehicle 800 and automatically adjusts the vehicle speed to maintain a safe distance to preceding vehicles. Lateral ACC performs the distance keeping and notifies the vehicle 800 to change lanes when necessary. Lateral ACC is related to other ADAS applications such as LCA and CWS.CACC uses information from other vehicles that may be received via network interface 824 and / or the one or more wireless antennas 826 from other vehicles via a wireless connection or indirectly via a network connection (e.g., via the Internet). Direct connections may be provided via a vehicle-to-vehicle (V2V) communication connection, while indirect connections may be an infrastructure-to-vehicle (I2V) communication connection. In general, the V2V communication concept provides information about the immediately preceding vehicles (e.g., vehicles that are immediately in front of and in the same lane as vehicle 800) while the I2V communication concept provides information about the traffic that is further preceding. CACC systems may include both I2V and V2V information sources. Given the information about the vehicles in front of the vehicle 800, CACC may be more reliable and has the potential to improve traffic flow and reduce road jams.FCW systems are designed to warn the driver of a risk so that he can intervene in a corrective manner. FCW systems use a front-facing camera and / or one or more RADAR sensors 860 coupled to a special purpose processor, DSP, FPGA, and / or ASIC electrically coupled to feedback to the driver, e.g., a display, speaker, and / or vibrating component. FCW systems may issue a warning, e.g., in the form of a sound, a visual warning, a vibration, and / or a rapid brake pulse.AEB systems detect an imminent forward collision with another vehicle or object and can automatically apply the brakes if the driver does not correctively intervene within a certain time or distance parameter. AEB systems may use one or more forward-facing cameras and / or one or more RADAR sensors 860 coupled to a special purpose processor, DSP, FPGA, and / or ASIC. When the AEB system senses a risk, it typically first alerts the driver to take corrective action to avoid the collision; when the driver does not take corrective action, the AEB system may automatically apply the brakes to prevent or at least mitigate the effects of the predicted collision. AEB systems may include techniques such as dynamic brake assist and / or imminent accident braking.LDW systems provide visual, audible, and / or tactile warnings, such as steering wheel or seat vibrations, to warn the driver when the vehicle 800 crosses the lane markings. An LDW system is not activated when the driver indicates an intentional lane departure by operating a turn signal. LDW systems may use front-facing cameras coupled to a dedicated processor, DSP, FPGA, and / or ASIC electrically coupled to feedback to the driver, e.g., a display, speaker, and / or vibrating component.LKA systems are a variant of the LDW systems. LKA systems provide steering inputs or brakes to correct the vehicle 800 when the vehicle 800 begins to exit the lane.BSW systems detect and warn the driver of vehicles in the blind spot of the car. BSW systems may output an optical, acoustic, and / or tactile warning signal to indicate that merging or changing the lane is uncertain. The system may issue an additional warning when the driver actuates a turn signal. BSW systems may use one or more rear-facing cameras and / or RADAR sensors 860 coupled to a dedicated processor, DSP, FPGA, and / or ASIC electrically coupled to feedback to the driver, such as a display, speaker, and / or vibrating component.RCTW systems may provide visual, audible, and / or tactile notification when an object is detected outside of the range of the rear camera when the vehicle 800 is backing up. Some RCTW systems include AEB to ensure that the vehicle brakes are applied to avoid an accident. RCTW systems may use one or more rear-facing RADAR sensors 860 coupled to a special purpose processor, DSP, FPGA, and / or ASIC electrically coupled to feedback to the driver, e.g., a display, speaker, and / or vibrating component.Conventional ADAS systems may lead to false positive results, which may be annoying and distracted to the driver, but are generally not catastrophic because the ADAS systems warn the driver and allow him to decide whether a safety condition is actually present and act accordingly. However, in an autonomous vehicle 800, in the case of conflicting results, the vehicle 800 itself must decide whether to observe the result of a primary computer or a secondary computer (e.g., a first controller 836 or a second controller 836). In some embodiments, the ADAS system 838 may be, for example, a backup and / or secondary computer that provides perception information to a rationality module of the backup computer. The backup computer rationality monitor may run redundant various software on hardware components to detect errors in perception and dynamic driving tasks. The outputs of the ADAS system 838 may be provided to a monitoring MCU. When the outputs of the primary computer and the secondary computer conflict, the monitoring MCU must determine how the conflict can be placed to ensure safe operation.In some examples, the primary computer may be configured to provide the monitoring MCU with a confidence value that indicates the confidence of the primary computer in the selected result. If the confidence value exceeds a threshold, the monitoring MCU may follow the primary computer's instruction, regardless of whether the secondary computer provides an conflicting or inconsistent result. If the confidence value does not meet the threshold and the primary and secondary computers indicate different results (e.g., conflict), the monitoring MCU may mediate between the computers to determine the appropriate result.The monitoring MCU may be configured to operate one or more neural networks trained and configured to determine, based on the results of the primary and secondary computers, the conditions under which the secondary computer triggers false alarms. Thus, the one or more neural networks in the monitoring MCU may learn when the output of the secondary computer can be trusted and when not. For example, if the secondary computer is a RADAR-based FCW system, a neural network in the monitoring MCU may learn when the FCW system identifies metallic objects that are not actually hazardous, such as a drain grill or a channel cap that triggers an alarm. Similarly, a neural network in the monitoring MCU may learn, when the secondary computer is a camera-based LDW system, to override the LDW system when cyclists or pedestrians are present and lane departure is indeed the safest maneuver. In embodiments including one or more neural networks running on the monitoring MCU, the monitoring MCU may include at least one DLA or GPU suitable for executing the one or more neural networks with associated memory. In preferred embodiments, the monitoring MCU may include and / or be included as a component of the one or more SoCs 804.In other examples, the ADAS system 838 may include a secondary computer that executes the ADAS functionality according to traditional rules of computer vision. Thus, the secondary computer may use classic computer vision rules (if-then), and the presence of one or more neural networks in the monitoring MCU may improve reliability, security, and performance. For example, the multiple implementation and intentional non-identity makes the overall system more fault tolerant, particularly to faults caused by software (or software-hardware interfaces). For example, if a software bug or error occurs in the software on the primary computer and the non-identical software code on the secondary computer provides the same overall result, the monitoring MCU may have a greater confidence that the overall result is correct and the bug in the software or hardware on the primary computer does not cause a significant error.In some examples, the output of the ADAS system 838 may be fed to the primary computer perception block and / or the primary computer dynamic driving task block. For example, if the ADAS system 838 indicates a forward collision warning due to an object immediately ahead, the perception block may use this information in identifying objects. In other examples, the secondary computer may have its own neural network trained to reduce the risk of false positives, as described herein.The vehicle 800 may further include the infotainment SoC 830 (e.g., an in-vehicle infotainment system (IVI)). Although illustrated and described as SoC, the infotainment system may not be SoC and may include two or more discrete components. Infotainment SoC 830 may include a combination of hardware and software that may be used to provide audio (e.g., music, a personal digital assistant, navigation instructions, messages, radio, etc.), video (e.g., TV, movies, streaming, etc.), phone (e.g., hands-free calling), network connectivity (e.g., LTE, Wi-Fi, etc.), and / or information services (e.g., navigation systems, rear parking assist, a radio data system, vehicle related information such as fuel level, total distance traveled, brake fuel level, oil level, door open / close, air filter information, etc.) to vehicle 800. The infotainment SoC 830 may include, for example, radios, disk players, navigation systems, video players, USB and Bluetooth connectivity, carputers, in-car entertainment, Wi-Fi, steering wheel audio controls, hands-free kit, heads-up display (HUD), HMI display 834, telematics device, control panel (e.g., for control and / or interaction with various components, functions, and / or systems), and / or other components. The infotainment SoC 830 may be further used to provide information (e.g., visual and / or audible) to one or more users of the vehicle, such as information from the ADAS system 838, autonomous driving information, such as scheduled vehicle maneuvers, road patterns, environmental information (e.g., intersection information, vehicle information, road information, etc.), and / or other information.The infotainment SoC 830 may include GPU functionality. The infotainment SoC 830 may communicate with other devices, systems, and / or components of the vehicle 800 via the bus 802 (e.g., CAN bus, Ethernet, etc.). In some examples, the infotainment SoC 830 may be coupled to a monitoring MCU such that the infotainment system GPU may perform some self-driving functions if the one or more primary controllers 836 (e.g., the primary and / or backup computers of the vehicle 800) fail. In such an example, the infotainment SoC 830 may place the vehicle 800 in a chauffeur for safe stop mode as described herein.The vehicle 800 may further include an instrument cluster 832 (e.g., a digital dashboard, an electronic instrument cluster, a digital instrument panel, etc.). The instrument cluster 832 may include a controller and / or supercomputer (e.g., a discrete controller or supercomputer). The instrument cluster 832 may include a number of instruments, such as tachometer, fuel level, oil pressure, tachometer, odometer, flasher, shift position indicator, seatbelt warning light(s), parking brake warning light(s), engine malfunction light(s), airbag system (SRS) information, lighting controls, safety system controls, navigation information, etc. In some examples, information may be displayed and / or shared by the infotainment SoC 830 and the instrument cluster 832. In other words, instrument cluster 832 may be included as part of infotainment SoC 830, or vice versa.FIG. 8D is a system diagram for communication between the one or more cloud-based servers and the example autonomous vehicle 800 of FIG. 8A, in accordance with some embodiments of the present disclosure. The system 876 may include the one or more servers 878, the one or more networks 890, and the vehicles, including the vehicle 800. The server or servers 878 may include a plurality of GPUs 884(A)-884(H) (collectively referred to herein as GPUs 884), PCIe switches 882(A)-882(H) (collectively referred to herein as PCIe switches 882), and / or CPUs 880(A)-880(B) (collectively referred to herein as CPUs 880). The GPUs 884, the CPUs 880, and the PCIe switches may be interconnected with high speed links such as, but not limited to, the NVLink interfaces 888 developed by NVIDIA and / or PCIe links 886. In some examples, the GPUs 884 are connected via NVLink and / or NVSwitch-SoC, and the GPUs 884 and the PCIe switches 882 are connected via PCIe connections. Although eight GPUs 884, two CPUs 880, and two PCIe switches are illustrated, this is not to be understood as limiting. Depending on the embodiment, each of the servers 878 may include any number of GPUs 884, CPUs 880, and / or PCIe switches. For example, the one or more servers 878 may each include eight, sixteen, thirty-two, and / or more GPUs 884.The one or more servers 878 may receive, via the one or more networks 890 and from the vehicles, image data representative of images showing unexpected or altered road conditions, such as recently initiated road operations. The one or more servers 878 may transmit, via the one or more networks 890 and to the vehicles, neural networks 892, updated neural networks 892, and / or map information 894 containing traffic and road condition information. The updates to the map information 894 may include updates to the HD map 822, e.g., information about construction sites, holes, diversions, floods, and / or other obstacles. In some examples, neural networks 892, updated neural networks 892, and / or map information 894 may result from new training and / or experiences represented in data received from any number of vehicles in the environment, and / or based on training performed in a data center (e.g., using the one or more server(s) 878 and / or other servers).The one or more servers 878 may be used to train machine learning models (e.g., neural networks) based on training data. The training data may be generated by the vehicles and / or in a simulation (e.g., using a game machine). In some examples, the training data is tagged (e.g., when the neural network benefits from supervised learning) and / or subjected to other preprocessing, while in other examples, the training data is not tagged and / or preprocessed (e.g., when the neural network does not require supervised learning). Training may be performed according to one or more classes of machine learning techniques, including, without limitation, classes such as: supervised training, semi-supervised training, unsupervised training, self-learning, reinforcement learning, federated learning, transfer learning, feature learning (including principal component and cluster analyses), multilinear subspace learning, multi-learning, representation learning (including replacement dictionary learning), rule-based machine learning, anomaly detection, and any variants or combinations thereof. Once the machine learning models are trained, the machine learning models may be used by the vehicles (e.g., transmitted to the vehicles via the one or more networks 890 and / or the machine learning models may be used by the one or more servers 878 to remotely monitor the vehicles.In some examples, the server or servers 878 may receive data from the vehicles and apply the data to current neural networks in real-time to draw smart inferencing in real-time. The one or more servers 878 may include deep learning supercomputers and / or dedicated AI computers powered by GPUs 884, such as the DGX and DGX station machines developed by NVIDIA. However, in some examples, the one or more servers 878 may include a deep learning infrastructure that uses only CPU-powered data centers.The deep learning infrastructure of the one or more servers 878 may be capable of performing fast real-time inferencing and may utilize this capability to assess and verify the state of the processors, software, and / or associated hardware in the vehicle 800. For example, the deep learning infrastructure may receive periodic updates from the vehicle 800, such as a sequence of images and / or objects that the vehicle 800 has localized in this sequence of images (e.g., via computer vision and / or other machine learning object classification techniques). The deep learning infrastructure may execute its own neural network to identify the objects and compare them to the objects identified by the vehicle 800, and if the results do not match and the infrastructure concludes that the AI in the vehicle 800 is malfunctioning, the one or more servers 878 may send a signal to the vehicle 800 instructing a fail-safe computer of the vehicle 800 to take control, notify the passengers, and perform a safe parking maneuver.For inferencing, the one or more servers 878 may include the GPUs 884 and one or more programmable inference accelerators (e.g., NVIDIA's TensorRT). The combination of GPU-controlled servers and inference accelerators may enable real-time responsiveness. In other examples, e.g., where performance is less critical, servers driven by CPUs, FPGAs, and other processors may be used for inferencing.EXAMPLE COMPUTING DEVICEFIG. 9 is a block diagram of one or more example computing devices 900 suitable for use in implementing some embodiments of the present disclosure. Computing device 900 may include an interconnect system 902 that directly or indirectly couples the following devices: memory 904, one or more central processing units (CPUs) 906, one or more graphics processing units (GPUs) 908, a communication interface 910, input / output (I / O) ports 912, input / output components 914, a power supply 916, one or more presentation components 918 (e.g., displays), and one or more logic units 920. In at least one embodiment, one or more computing devices 900 may include one or more virtual machines (VMs), and / or each of components thereof may include virtual components (e.g., virtual hardware components). As non-limiting examples, one or more of the GPUs 908 may include one or more vPUs, one or more of the CPUs 906 may include one or more vCPUs, and / or one or more of the logic units 920 may include one or more virtual logic units. Thus, a computing device 900 may include discrete components (e.g., a full GPU associated with the computing device 900), virtual components (e.g., a portion of a GPU associated with the computing device 900), or a combination thereof.Although the various blocks of FIG. 9 are shown as being connected to lines via the connection system 902, this is not intended to be limiting and is for clarity only. For example, in some embodiments, a presentation component 918, such as a display device, may be considered an I / O component 914 (e.g., when the display is a touch screen). As another example, CPUs 906 and / or GPUs 908 may include memory (e.g., memory 904 may represent a storage device in addition to storing GPUs 908, CPUs 906, and / or other components). In other words, the computing device of FIG. 9 is illustrative only. Categories such as "workstation," "server," "laptop," "desktop," "tablet," "client device," "mobile device," "handheld device," "game console," "electronic control unit (ECU)," "virtual reality system," and / or other device or system types are not distinguished, as all are within the scope of the computing device of FIG. 9.The interconnect system 902 may represent one or more interconnects or buses, such as an address bus, a data bus, a control bus, or a combination thereof. The interconnect system 902 may include one or more types of bus or interconnects, such as an industry standard architecture (ISA) bus, an extended ISA (EISA) bus, a video electronics standards association (VESA) bus, a peripheral component interconnect (PCI) bus, a PCI express (PCIe) bus, and / or another type of bus or interconnect. In some embodiments, there are direct connections between components. For example, the CPU 906 may be directly connected to the memory 904. Further, the CPU 906 may be directly connected to the GPU 908. In a direct or point-to-point connection between components, the interconnect system 902 may include a PCIe link to establish the connection. In these examples, a PCI bus need not be included in computing device 900.The memory 904 may include a variety of computer readable media. The computer readable media may be any available media that can be accessed by computing device 900. The computer readable media may include both volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, the computer readable media may include computer storage media and communication media.The computer storage media may include both volatile and non-volatile media and / or removable and non-removable media implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules, and / or other types of data. For example, memory 904 may store computer readable instructions (e.g., representing one or more programs and / or one or more program elements, such as an operating system. Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the computing device 900. As used herein, computer storage media does not include signals per se.The computer storage media may embody computer readable instructions, data structures, program modules, and / or other types of data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any media for conveying information. The term "modulated data signal" may refer to a signal in which one or more of its characteristics are adjusted or altered to encode information in the signal. The computer storage media may include, for example, and is not limited to, wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of the above should also be included within the scope of computer readable media.The one or more CPUs 906 may be configured to execute at least some of the computer readable instructions to control one or more components of the computing device 900 to perform one / more of the methods and / or processes described herein. The one or more CPUs 906 may each include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a plurality of software threads simultaneously. The one or more CPUs 906 may include any type of processor and may include different types of processors (e.g., processors with fewer cores for mobile devices and processors with more cores for servers), depending on the type of computing device 900 implemented. Depending on the type of computing device 900, the processor may be, for example, an Advanced RISC Machines (ARM) processor implemented with Reduced Instruction Set Computing (RISC) or an x86 processor implemented with Complex Instruction Set Computing (CISC). Computing device 900 may include one or more CPUs 906, in addition to one or more microprocessors or additional co-processors, such as mathematical co-processors.In addition to, or as an alternative to, the one or more CPUs 906, the one or more GPUs 908 may be configured to execute at least some of the computer readable instructions to control one or more components of the computing device 900 to perform one or more of the methods and / or processes described herein. One or more of the GPUs 908 may be an integrated GPU (e.g., with one or more of the CPUs 906 and / or one or more of the GPUs 908 may be a discrete GPU. In embodiments, one or more of the GPUs 908 may be a coprocessor of one or more of the CPUs 906. The one or more GPUs 908 may be used by the computing device 900 to render graphics (e.g., 3D graphics) or perform general computations. The one or more GPUs 908 may be used for general-purpose computing on GPUs (GPGPUs)), for example. The one or more GPUs 908 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. The one or more GPUs 908 may generate pixel data for output images in response to rendering commands (e.g., rendering commands from the one or more CPUs 906 received via a host interface). The one or more GPUs 908 may include graphics memory, such as display memory, for storing pixel data or other suitable data, such as GPGPU data. The display memory may be included as part of the memory 904. The one or more GPUs 908 may include two or more parallel operating GPUs (e.g., via a link). The link may connect the GPUs directly (e.g., to NVLINK) or connect the GPUs via a switch (e.g., to NVSwit). When combined together, each GPU 908 may generate pixel data or GPGPU data for different portions of an output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU may contain its own memory or share memory with other GPUs.In addition to, or as an alternative to, the one or more CPUs 906 and / or the one or more GPUs 908, the one or more logic units 920 may be configured to execute at least some of the computer readable instructions to control one or more components of the computing device 900 to perform one / more of the methods and / or processes described herein. In embodiments, the one or more CPUs 906, the GPUs 908, and / or the one or more logic units 920 may discretely or jointly execute any combination of the methods, processes, and / or portions thereof. One or more of the logic units 920 may be part of or integrated with one or more of the CPUs 906 and / or one or more of the GPUs 908 and / or one or more of the logic units 920 may be discrete components or otherwise external to the CPUs 906 and / or the GPUs 908. In embodiments, one or more of the logic units 920 may be a coprocessor of one or more of the CPUs 906 and / or one or more of the GPUs 908.Examples of the one or more logic units 920 include one or more processing cores and / or components thereof, such as data processing units (DPUs), tensor cores (TCs), tensor processing units (TPUs), visual cores (PVCs), vision processing units (VPUs), graphics processing clusters (GPCs), texture processing clusters (TPCs), Streaming multiprocessors (SMs), tree traversal units (TTUs), artificial intelligence accelerators (AIAs), deep learning accelerators (DLAs), arithmetic logic units (ALUs), application-specific integrated circuits (ASICs), floating point units (FPUs), input / output (I / O) elements, peripheral component interconnect (ALUs, PCI) or PCI Express (PCIe) elements, and / or the like.Communication interface 910 may include one or more receivers, transmitters, and / or transceivers that enable computing device 900 to communicate with other computing devices over an electronic network, including wired and / or wireless communication. The communication interface 910 may include components and functionality that enables communication over a variety of networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication over Ethernet or InfiniBand), low-power wide area networks (e.g., LoRaWA, SigOx, etc.), and / or the Internet. In one or more embodiments, the one or more logic units 920 and / or the communication interface 910 may include one or more computing devices (DPUs) to directly transmit data received over a network and / or via the connection system 902 to one or more GPUs 908 (e.g., a memory thereof).I / O ports 912 may enable computing device 900 to be logically coupled to other devices, including I / O components 914, one or more presentation components 918, and / or other components, some of which may be incorporated (e.g., integrated) into computing device 900. Illustrative I / O components 914 include a microphone, a mouse, a keyboard, a joystick, a game pad, a game controller, a satellite dish, a scanner, a printer, a wireless device, etc. The I / O components 914 may provide a natural user interface (NUI) that processes user-generated air gestures, speech, or other physiological inputs. In some cases, the inputs may be transmitted to a suitable network element for further processing. A NUI may implement any combination of voice sensing, pen sensing, face sensing, biometric sensing, gesture sensing on both the screen and adjacent to the screen, air gestures, head and eye tracking, and touch sensing (as described in more detail below) associated with a display of computing device 900. Computing device 900 may include depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, touch screen technology, and combinations thereof, for gesture sensing and recognition. Additionally, computing device 900 may include accelerometers or gyroscopes (e.g., as part of an inertial measurement unit (IMU)) that enable motion detection. In some examples, the output of the accelerometers or gyroscopes may be used by computing device 900 to represent immersive augmented reality or virtual reality.The power supply 916 may include a hard-wired power supply, a battery power supply, or a combination thereof. Power supply 916 may provide power to computing device 900 to enable operation of the components of computing device 900.The one or more presentation components 918 may include a display (e.g., a monitor, a touch screen, a television screen, a heads-up display (HUD), other types of display, or a combination thereof), speakers, and / or other presentation components. The one or more presentation components 918 may receive data from other components (e.g., the one or more GPUs 908, the one or more CPUs 906, DPUs, etc.) and output the data (e.g., as an image, video, sound, etc.).EXAMPLE DATA CENTERFIG. 10 illustrates an example data center 1000 that may be used in at least one embodiment of the present disclosure. Data center 1000 may include a data center infrastructure layer 1010, a framework layer 1020, a software layer 1030, and / or an application layer 1040.As shown in FIG. 10, data center infrastructure layer 1010 may include resource orchestrator 1012, grouped computing resources 1014, and node computing resources ("node C.R.s") 1016( 1)- 1016(N), where "N" represents any integer, positive number. In at least one embodiment, node C.R.s 1016( 1)- 1016(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including DPUs, accelerators, field programmable gate arrays (FPGAs), graphics processors or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read only memory), memory devices (e.g., solid state or disk drives), network input / output devices (NW I / O), network switches, virtual machines (VMs), Power modules and / or cooling modules, etc. In some embodiments, one or more node C.R.s among the node C.R.s 1016( 1)- 1016(N) may correspond to a server having one or more of the aforementioned computing resources. Moreover, in some embodiments, the node C.R.s 1016( 1)- 10161(N) may include one or more virtual components, such as vPUs, vCPUs, and / or the like, and / or one or more of the node C.R.s 1016( 1)- 1016(N) may correspond to a virtual machine (VM).In at least one embodiment, grouped computing resources 1014 may include separate groupings of node C.R.s 1016 residing in one or more racks (not shown) or many racks in data centers at different geographic locations (also not shown). Separate groupings of node C.R.s 1016 within grouped computing resources 1014 may include grouped computing, network, memory, or storage resources that may be configured or allocated to support one or more workloads. In at least one embodiment, multiple node C.R.s 1016, including CPUs, GPUs, DPUs, and / or other processors, may be grouped into one or more racks to provide computing resources to support one or more workloads. The one or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.The resource orchestrator 1012 may configure or otherwise control one or more node C.R.s 1016( 1)- 1016(N) and / or grouped computing resources 1014. In at least one embodiment, resource orchestrator 1012 may include an entity for managing software design infrastructure (SDI) for data center 1000. The resource orchestrator 1012 may include hardware, software, or a combination thereof.In at least one embodiment, as shown in FIG. 10, framework layer 1020 may include a job scheduler 1033, a configuration manager 1034, a resource manager 1036, and / or a distributed file system 1038. The framework layer 1020 may include a framework that supports software 1032 of software layer 1030 and / or one or more applications 1042 of application layer 1040. The software 1032 or the one or more applications 1042 may each include web-based service software or applications as provided by Amazon Web Services, Google Cloud, and Microsoft Azure. The framework layer 1020 may be any type of free and open source software web application framework, such as, but not limited to, Apache SparkTM (hereafter "Spark"), which may utilize a distributed file system 1038 to process large amounts of data (e.g., "Big Data"). In at least one embodiment, job scheduler 1033 may include a spark driver to facilitate scheduling workloads supported by different layers of data center 1000. The configuration manager 1034 may be capable of configuring various layers, such as the software layer 1030 and the framework layer 1020 including spark and the distributed file system 1038, to support large volume data processing. The resource manager 1036 may be capable of managing clustered or grouped computing resources allocated or allocated to support the distributed file system 1038 and the job scheduler 1033. In at least one embodiment, clustered or grouped computing resources may include grouped computing resource 1014 at data center infrastructure layer 1010. The resource manager 1036 may coordinate with the resource orchestrator 1012 to manage these allocated or allocated computing resources.In at least one embodiment, software 1032 included in software layer 1030 may include software used by at least portions of node C.R.s 1016( 1)- 1016(N), grouped computing resources 1014, and / or distributed file system 1038 of framework layer 1020. One or more types of software may include, but are not limited to, website search software, e-mail virus scanning software, database software, and streaming video content software.In at least one embodiment, applications 1042 included in application layer 1040 may include one or more types of applications used by at least portions of node C.R.s 1016( 1)- 1016(N), grouped computing resources 1014, and / or distributed file system 1038 of framework layer 1020. One or more types of applications may include, but are not limited to, any number of a genomic application, a cognitive computing and a machine learning application, including training or inferencing software, machine learning framework software (e.g., PyTor, TensorFlow, Coffee, etc.), and / or other machine learning applications used in connection with one or more embodiments.In at least one embodiment, configuration manager 1034, resource manager 1036, and / or resource orchestrator 1012 may implement any number and type of self-modifying actions based on any amount and type of data captured in any technically feasible manner. Self-modifying actions may offload a data center operator of data center 1000 from making potentially bad configuration decisions and may avoid unutilized and / or malfunctioning portions of a data center.Data center 1000 may include tools, services, software, or other resources to train one or more machine learning models or predict or derive information using one or more machine learning models, according to one or more embodiments described herein. For example, one or more machine learning models may be trained by calculating weight parameters according to a neural network architecture using software and / or computing resources described above with respect to data center 1000. In at least one embodiment, trained or deployed machine learning models corresponding to one or more neural networks may be used to derive or predict information using resources described above with respect to data center 1000 by using weighting parameters calculated by one or more training techniques, such as, but not limited to, those described herein.In at least one embodiment, data center 1000 may use CPUs, application specific integrated circuits (ASICs), GPUs, FPGAs, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inferencing using resources described above. Moreover, one or more of the software and / or hardware resources described above may be configured as a service to enable users to train or inferencing information such as image capture, voice capture, or other artificial intelligence services.EXAMPLE NETWORK ENVIRONMENTSNetwork environments suitable for implementing embodiments of the disclosure may include one or more client devices, servers, network attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may be implemented on one or more instances of the one or more computing devices 900 of FIG. 9-e.g., each device may include similar components, features, and / or functionality of the one or more computing devices 900. When backend devices (e.g., servers, NAS, etc.) are implemented, the backend devices may also be included as part of a data center 1000, an example of which is described in more detail herein with reference to FIG. 10.The components of a network environment may communicate with each other over one or more networks, which may be wired, wireless, or both. The network may include multiple networks or a network of networks. The network may include, for example, one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks such as the Internet and / or a public switched telephone network (PSTN), and / or one or more private networks. If the network includes a wireless telecommunications network, components such as a base station, a communication tower, or even access points (as well as other components) may provide wireless connectivity.Compatible network environments may include one or more peer-to-peer network environments - in this case, a server may not be included in a network environment - and one or more client-server network environments - in this case, one or more servers may be included in a network environment. In peer-to-peer network environments, the functionality described herein with respect to one or more servers may be implemented on any number of client devices.In at least one embodiment, a network environment may include one or more cloud-based network environments, a distributed computing environment, a combination thereof, etc. A cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. A framework layer may include a framework for supporting software of a software layer and / or one or more applications of an application layer. The software or one or more applications may each include web-based service software or applications. In embodiments, one or more of the client devices may utilize the web-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be any type of free and open source software web application framework that uses, for example, a distributed file system for large volume data processing (e.g., "big data"), without limitation.A cloud-based networking environment may provide cloud computing and / or cloud storage that performs any combination of the computer and / or data storage functions described herein (or one or more portions thereof). Each of these various functions may be distributed over multiple locations from central or core servers (e.g., from one or more data centers that may be distributed over a state, region, country, globe, etc.). If a connection to a user (e.g., a client device) is relatively close to one or more edge servers, one or more core servers may transmit at least a portion of the functionality to the one or more edge servers. A cloud-based network environment may be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).The one or more client devices may include at least some of the components, features, and functions of the one or more example computing devices 900 described herein with reference to FIG. 9. A client device may be, for example, a personal computer (PC), a laptop, a mobile device, a smartphone, a tablet computer, a smart watch, a portable computer, a personal digital assistant (PDA), an MP3 player, a virtual reality headset, a global positioning system (GPS), or a global positioning device, a video player, a video camera, a monitoring device or system, a vehicle, a boat, a airport boat, a virtual machine, a drone, a robot, a hand-held communication device, a hospital device, a gaming device or gaming system, an entertainment system, a vehicle computer system, an embedded system controller, a remote controller, a device, an entertainment electronic device, a workstation, an edge device, any combination of these described devices, or any other suitable device.The disclosure of this application also includes the following numbered clause:Clause 1. A processor comprising:one or more processing units for:generating infrared image data; andgenerating RGB image data corresponding to the infrared image data based at least on applying a representation of the infrared image data to a generative opposing network generator.Clause 2. The processor of Clause 1, wherein the one or more processing units further generate, for each frame of one or more frames of a video feed of an occupant monitoring system, a corresponding frame of RGB image data using the generator.Clause 3. The processor of Clause 1 or Clause 2, wherein the one or more processing units further train the generative opposing network using ground truth infrared training images paired with ground truth RGB training images.Clause 4th The processor of any preceding clause, wherein the one or more processing units further train the generative opposing network using ground truth infrared training images paired with ground truth RGB training images.Clause 5. The processor of any preceding clause, wherein the one or more processing units further train the generative opposing network based on at least training a first branch that concatenates the generator configured to generate a synthesized RGB image based on at least one input infrared image with a second generator configured to generate a synthesized infrared image from an input RGB image and train a second branch that concatenates the second generator with the generator.Clause 6. The processor of any preceding clause, wherein the one or more processing units further train the generative opposing network based at least on the emphasis of loss for detected edge pixels using higher weights than for detected non-edge pixels.Clause 7. the processor of any preceding clause, wherein applying the representation of the infrared image data to the generative opposing network generator comprises applying to the generator an edge map extracted from the infrared image data.Clause 8th The processor of any preceding clause, wherein the one or more processing units extract an edge map from the infrared image data and pass the edge map to the generator over a wireless communication channel.The clause 9. processor of any preceding clause, wherein the one or more processing units generate the RGB image data based at least on application of a predetermined color to a segmented region known to have the predetermined color.The clause 10th processor of any preceding clause, wherein the processor comprises at least one of:a control system for an autonomous or semi-autonomous machine;a perception system for an autonomous or semi-autonomous machine;a system for performing simulation operations;a system for performing digital twining operations;a system for performing light transport simulations;a system for performing collaborative content creation for 3D assets;a system for performing deep learning operations;a system for performing real-time streaming;a system implemented using an edge device;a system implemented using a robot;a system for performing conversational AI operations;a system for generating synthetic data;a system including one or more virtual machines (VMs);a system implemented at least partially in a data center; ora system implemented at least in part using cloud computing resources.Clause 11th System, comprising one or more processing units to: generate RGB image data corresponding to infrared image data based at least on applying a representation of the infrared image data to a generative opposing network generator.Clause 12th System of Clause 11, wherein the one or more processing units further generate, for each frame of one or more frames of a video feed of an occupant monitoring system, a corresponding frame of RGB image data using the generator.Clause 13th The system of any of Clause 11-12, wherein the one or more processing units further train the generative opposing network using ground truth infrared training images paired with ground truth RGB training images.Clause 14th The system of any of Clause 11-13, wherein the one or more processing units further train the generative opposing network using ground truth infrared training images paired with ground truth RGB training images.Clause 15. The system of any of Clause 11-14, wherein the one or more processing units further train the generative opposing network based on at least training a first branch that concatenates the generator configured to generate a synthesized RGB image based on at least one input infrared image with a second generator configured to generate a synthesized infrared image from an input RGB image and train a second branch that concatenates the second generator with the generator. The clause 16th system of any of clause 11-15, wherein the one or more processing units further train the generative opposing network based at least on emphasizing loss for detected edge pixels using higher weights than for detected non-edge pixels.The clause 17th system of any of clause 11-16, wherein applying the representation of the infrared image data to the generative opposing network generator comprises applying to the generator an edge map extracted from the infrared image data.The clause 18th system of any of clause 11-17, wherein the system comprises at least one of:a control system for an autonomous or semi-autonomous machine;a perception system for an autonomous or semi-autonomous machine;a system for performing simulation operations;a system for performing digital twining operations;a system for performing light transport simulations;a system for performing collaborative content creation for 3D assets;a system for performing deep learning operations;a system for performing real-time streaming;a system for creating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;a system implemented using an edge device;a system implemented using a robot;a system for performing conversational AI operations;a system for generating synthetic data;a system including one or more virtual machines (VMs);a system implemented at least partially in a data center; ora system implemented at least in part using cloud computing resources.Clause 19: Method comprising:generating, for each frame of one or more frames of a video feed of an occupant monitoring system, RGB image data corresponding to the infrared image data based at least on application of a representation of infrared image data associated with the frame to a generative opposing network generator.Clause 20th The method of Clause 19, wherein the method is performed by at least one of:a control system for an autonomous or semi-autonomous machine;a perception system for an autonomous or semi-autonomous machine;a system for performing simulation operations;a system for performing real-time streaming;a system for creating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system for performing digital twinning operations;a system for performing deep learning operations;a system implemented using an edge device;a system implemented using a robot;a system including one or more virtual machines (VMs);a system implemented at least partially in a data center; or a system for performing light transport simulations;a system for performing collaborative content creation for 3D assets; a system for generating synthetic data;a system implemented at least partially using cloud computing resources.Clause 21 Processor comprising:one or more processing units for:generating a frame of infrared image data and a frame of RGB image data having at least partially overlapping fields of view; andgenerating synthesized RGB image data based at least on coloring the frame of IR image data using the frame of RGB image data.Clause 22 The processor of Clause 21, wherein the one or more processing units further color, for each successive frame of one or more frames of a video feed of a monitoring system, a corresponding successive frame of IR image data using a corresponding frame of RGB image data.Clause 23. The processor of any of Clause 21-22, wherein one or more processing units transmit at least one of one or more colors or one or more color statistics of one or more segmented regions of the frame of RGB image data to one or more corresponding segmented regions of the IR image data.Clause 24. The processor of any of Clause 21-23, wherein the one or more processing units color at least a portion of the IR image data based on at least the spread of one or more colors of a segmented region of the IR image data as a starting color to one or more corresponding segmented regions of the IR image data and using the starting color as an input that determines a target region of the IR image data for colorization.Clause 25: The processor of any of Clause 21-24, wherein one or more processing units transmit at least one of one or more colors or one or more color statistics of one or more segmented body parts of the frame of RGB image data to one or more corresponding segmented body parts of the IR image data.Clause 26. The processor of any of Clause 21-25, wherein the one or more processing units transmit at least one of one or more colors or one or more color statistics of one or more segmented regions of the frame of RGB image data to one or more corresponding segmented regions of the synthesized RGB image data based on at least determining that a difference between one or more segmented regions of the frame of the RGB image data and the one or more corresponding segmented regions of the synthesized RGB image data exceeds a threshold.Clause 27. The processor of any of Clause 21-26, wherein the one or more processing units further encode the synthesized RGB image data into a latent representation, modify one or more dimensions of the latent representation to produce a modified latent representation, and decode the modified latent representation.The clause 28th processor of any of clause 21-27, wherein the one or more processing units further determine to modify one or more dimensions of a latent representation of the synthesized RGB image data based on at least the determination that a difference between one or more segmented regions of the frame of RGB image data and one or more corresponding segmented regions of the synthesized RGB image data exceeds a threshold.The clause 29th processor of any of the clause 21-28, wherein the processor comprises at least one of:a control system for an autonomous or semi-autonomous machine;a perception system for an autonomous or semi-autonomous machine;a system for performing simulation operations;a system for performing digital twining operations;a system for performing light transport simulations;a system for performing collaborative content creation for 3D assets;a system for performing deep learning operations;a system for performing real-time streaming;a system implemented using an edge device;a system implemented using a robot;a system for performing conversational AI operations;a system for generating synthetic data;a system including one or more virtual machines (VMs);a system implemented at least partially in a data center; ora system implemented at least in part using cloud computing resources.Clause 30th System, comprising one or more processing units to generate a synthesized RGB image based at least on colorizing an infrared (IR) image using an RGB image, wherein the IR image and the RGB image have at least partially overlapping fields of view.Clause 31st System of Clause 30, wherein one or more processing units are further to color a corresponding IR image using a corresponding RGB image for each successive frame of one or more frames of a video feed of a monitoring system.The clause 32th system of any of clause 30 to clause 31, wherein one or more processing units transmit at least one of one or more colors or one or more color statistics of one or more segmented regions of the RGB image to one or more corresponding segmented regions of the IR image.The clause 33th system of any of clause 30 to clause 32, wherein the one or more processing units color at least a portion of the IR image based on at least one of spread of one or more colors of a segmented region of the IR image as a starting color to one or more corresponding segmented regions of the IR image and use the starting color as an input that determines a target region of the IR image for coloring.The clause 34th system of any of clause 30 to clause 33, wherein one or more processing units transmit at least one of one or more colors or one or more color statistics of one or more segmented body parts of the RGB image to one or more corresponding segmented body parts of the IR image.The clause 35. system of any of clause 30 to clause 34, wherein the one or more processing units transmit at least one of one or more colors or one or more color statistics of one or more segmented regions of the RGB image to one or more corresponding segmented regions of the synthesized RGB image based on at least the determination that a difference between one or more segmented regions of the RGB image and the one or more corresponding segmented regions of the synthesized RGB image exceeds a threshold.The clause 36th system of any of clause 30 to clause 35, wherein the one or more processing units further encode the synthesized RGB image into a latent representation, modify one or more dimensions of the latent representation to produce a modified latent representation, and decode the modified latent representation.Clause 37th The system of any of Clause 30 to 36, wherein the one or more processing units further determine to modify one or more dimensions of a latent representation of the synthesized RGB image based on at least the determination that a difference between one or more segmented regions of the RGB image and one or more corresponding segmented regions of the synthesized RGB image exceeds a threshold.The clause 38th system of any of the clause 30 to 37, wherein the system comprises at least one of:a control system for an autonomous or semi-autonomous machine;a perception system for an autonomous or semi-autonomous machine;a system for performing simulation operations;a system for performing digital twining operations;a system for performing light transport simulations;a system for performing collaborative content creation for 3D assets;a system for performing deep learning operations;a system for performing real-time streaming;a system for creating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;a system implemented using an edge device;a system implemented using a robot;a system for performing conversational AI operations;a system for generating synthetic data;a system including one or more virtual machines (VMs);a system implemented at least partially in a data center; ora system implemented at least in part using cloud computing resources.Clause 39: A method comprising: generating a frame of infrared (IR) image data and a frame of RGB image data; sharing a partially overlapping field of view with the frame of IR image data; and generating synthesized RGB image data for each frame of one or more frames of a video feed of an occupant monitoring system based on at least colorizing the frame of IR image data using the frame of RGB image data.Clause 40th The method of Clause 39, wherein the method is performed by at least one of:a control system for an autonomous or semi-autonomous machine;a perception system for an autonomous or semi-autonomous machine;a system for performing simulation operations;a system for performing real-time streaming;a system for creating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;a system for performing digital twining operations;a system for performing deep learning operations;a system implemented using an edge device;a system implemented using a robot;a system including one or more virtual machines (VMs);a system implemented at least partially in a data center;a system for performing light transport simulations;a system for performing collaborative content creation for 3D assets;a system for generating synthetic data; ora system implemented at least partially using cloud computing resources.The disclosure may be described in the general context of computer code or machine-usable instructions, including computer-executable instructions, such as program modules, executed by a computer or other machine, such as a personal data assistant or other portable device. Generally, program modules include routines, programs, objects, components, data structures, etc., and refer to code that performs particular tasks or implements particular abstract data types. The disclosure may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general purpose computers, more specific computing devices, etc. The disclosure may also be practiced in distributed computing environments where tasks are performed by remote controlled devices that are interconnected for communication via a network.As used herein, a phrase "and / or" with respect to two or more elements should be interpreted to mean only one element or a combination of elements. For example, "element A, element B, and / or element C" may include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or elements A, B, and C. Moreover, "at least one of element A or element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, "at least one of element A and element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.The subject matter of the present disclosure is specifically described herein to meet the legal requirements. However, the description itself is not intended to limit the scope of the present disclosure. Rather, the inventors have contemplated that the claimed subject matter may be embodied in other ways to include various steps or combinations of steps similar to those described herein in connection with other current or future technologies. Although the terms "step" and / or "block" may be used herein to refer to various elements of the methods employed, the terms should not be interpreted as implying a particular order among or between the various steps disclosed herein unless and except in the case where the order of the individual steps is expressly described.It is to be understood that the above-described aspects and embodiments are merely exemplary and that changes may be made in detail within the scope of the claims.Each device, method and feature disclosed in the specification and (optionally) in the claims and drawings may be presented independently or in any suitable combination.Reference numerals in the claims are intended for illustration purposes only and have no limiting effect on the scope of the claims.References included in the specificationThis list of documents cited by the applicant has been produced in an automated manner and is only included for the better information of the reader. The list is not part of the German patent application or utility model application. The DPMA does not take any adhesion for any faults or omissions.Patent Literature citedUS 16 / 101,232

[0118] Cited Non-Patent LiteratureLuan et al., Natural Image Colorization, Eurographics Symposium on Rendering (2007

[0053] SAE) "Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (

[0075] Taxonomy and definitions of terms related to automated driving systems for road vehicles)" (Standard No. J3016-201806 published June 15, 2018, Standard No. J3016-20169 published September 30, 2016

[0075]

Claims

A processor comprising: one or more processing units to: generate infrared image data; and generate RGB image data corresponding to the infrared image data based at least on applying a representation of the infrared image data to a generative opposing network generator.The processor of claim 1, wherein the one or more processing units further generate, for each frame of one or more frames of a video feed of an occupant monitoring system, a corresponding frame of RGB image data using the generator.The processor of claim 1 or claim 2, wherein the one or more processing units further train the generative opposing network using ground truth infrared training images paired with ground truth RGB training images.The processor of any preceding claim, wherein the one or more processing units further train the generative opposing network using ground truth infrared training images paired with ground truth RGB training images.The processor of any preceding claim, wherein the one or more processing units further train the generative opposing network based on at least training a first branch that concatenates the generator configured to generate a synthesized RGB image based on at least one input infrared image with a second generator configured to generate a synthesized infrared image from an input RGB image and train a second branch that concatenates the second generator with the generator.The processor of any preceding claim, wherein the one or more processing units further train the generative opposing network based at least on the emphasis of the loss for detected edge pixels using higher weights than for detected non-edge pixels.The processor of any preceding claim, wherein applying the representation of the infrared image data to the generative opposing network generator comprises applying an edge map extracted from the infrared image data to the generator.The processor of any preceding claim, wherein the one or more processing units extract an edge map from the infrared image data and pass the edge map to the generator over a wireless communication channel.The processor of any preceding claim, wherein the one or more processing units generate the RGB image data based at least on application of a predetermined color to a segmented region known to have the predetermined color.The processor of any preceding claim, wherein the processor comprises at least one of: an autonomous or semi-autonomous machine control system; an autonomous or semi-autonomous machine perception system; a system for performing simulation operations; a system for performing digital twinning operations; a system for performing light transport simulations; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing real-time streaming; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system including one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.A system comprising one or more processing units to: generate RGB image data corresponding to infrared image data based at least on applying a representation of the infrared image data to a generative opposing network generator.The system of claim 11, wherein the one or more processing units further generate, for each frame of one or more frames of a video feed of an occupant monitoring system, a corresponding frame of RGB image data using the generator.The system of claim 11 or 12, wherein the one or more processing units further train the generative opposing network using ground truth infrared training images paired with ground truth RGB training images.The system of any of claims 11 to 13, wherein the one or more processing units further train the generative opposing network using ground truth infrared training images paired with ground truth RGB training images.The system of any of claims 11 to 14, wherein the one or more processing units further train the generative opposing network based on at least training a first branch that concatenates the generator configured to generate a synthesized RGB image based on at least one input infrared image with a second generator configured to generate a synthesized infrared image from an input RGB image and train a second branch that concatenates the second generator with the generator.The system of any of claims 11 to 15, wherein the one or more processing units further train the generative opposing network based at least on the emphasis of the loss for detected edge pixels using higher weights than for detected non-edge pixels.The system of any of claims 11 to 16, wherein applying the representation of the infrared image data to the generative opposing network generator comprises applying an edge map extracted from the infrared image data to the generator.The system of any of claims 11 to 17, wherein the system comprises at least one of: an autonomous or semi-autonomous machine control system; an autonomous or semi-autonomous machine perception system; a system for performing simulation operations; a system for performing digital twinning operations; a system for performing light transport simulations; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system including one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.A method comprising: generating, for each frame of one or more frames of a video feed of an occupant monitoring system, based at least on application of a representation of infrared image data associated with the frame to a generative opposing network generator, RGB image data corresponding to the infrared image data.The method of claim 19, wherein the method is performed by at least one of: an autonomous or semi-autonomous machine control system; an autonomous or semi-autonomous machine perception system; a system for performing simulation operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system for performing digital twining operations; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system including one or more virtual machines (VMs); a system implemented at least partially in a data center; a system for performing light transport simulations; a system for performing collaborative content creation for 3D assets; a system for generating synthetic data; or a system implemented at least partially using cloud computing resources.

Images