Efficiently performing computations of a multi-input multi-output fully convolutional network

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating outputs for inputs of different sizes using a multi-input, multi-output fully convolutional network. One of the methods includes receiving new inputs at an invocation by a fully convolutional network deployed for processing inputs with a fixed size. For each of the received new inputs, a group of fixed-size input tiles are determined, and each of the groups of fixed-size input tiles are provided to a hardware accelerator for generating respective fixed-size outputs of the invocation using the deployed fully convolutional network. From the respective fixed-size outputs, a respective final output of the invocation is generated for the output layer that is equivalent to an output that would be generated from the output layer by processing the new inputs at the invocation using the fully convolutional network deployed for processing the corresponding inputs with the respective sizes.

Description

[0001]Attorney Docket No.56113-0510WO1 EFFICIENTLY PERFORMING COMPUTATIONS OF A MULTI-INPUT MULTI-OUTPUT FULLY CONVOLUTIONAL NETWORK BACKGROUND This specification generally relates to efficiently performing inference computations of a multi-input, multi-output, fully convolutional network. Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from a received input in accordance with current values of a respective set of network parameters. A fully convolutional network is a neural network that includes only convolutional neural network layers and, optionally, other layers that are made up solely of components that only operate on local input regions, e.g., pooling layers and element- wise layers, e.g., those that apply an element-wise non-linear activation function. Specifically, unlike other types of convolutional neural networks, a fully convolutional network does not have any fully connected layers. A fully convolutional network can be configured to make pixel-wise predictions of an input (e.g., an image with multiple pixels). In other words, the fully convolutional network can be used to make a respective prediction for each pixel of the input. An example of a task that requires making pixel- wise prediction is image segmentation, in which a neural network is configured to generate, for each pixel of the input image, a respective score for each of multiple classes. SUMMARY This specification generally describes techniques for performing inference computations of a neural network for processing multiple inputs at one time step or invocation with sizes different from particular sizes that the neural network is, once deployed, natively capable of processing. The term “invocation” used in this specification generally refers to a one-time call to execute operations represented by the neural network. For simplicity, the description below refers to each time step and / or each call to the operations represented by the FCN model as each invocation of the FCN model. More specifically, the neural network can be a fully convolutional network (FCN) having multiple input layers and multiple output layers. After the FCN is deployed on hardware (e.g., one or more computers), it is statically and natively capable of processing Attorney Docket No.56113-0510WO1 multiple inputs with particular sizes for each invocation of the deployed FCN model. The deployed FCN represents operations that, once executed, can generate one or more outputs for the multiple inputs at each invocation. The described techniques accordingly generally relate to techniques for empowering the deployed FCN model to process multiple inputs at each invocation with sizes different from the particular sizes that the FCN is deployed for. To achieve these perspectives, the described techniques relate to operations performed offline or at a “deployment stage” for generating tiling and stitching data for tiling multiple inputs at each invocation with different sizes into the particular sizes that the FCN model can natively process, and aggregating corresponding outputs to generate final outputs for the invocation as if the FCN model is capable of processing the inputs with different sizes at the invocation. Once the tiling and stitching data is obtained, the described techniques can incorporate the data with the deployed FCN model to process multiple inputs with various sizes at each invocation of the deployed FCN model. Note that an FCN can sometimes also refer to a fully convolutional neural network. An FCN can include at least one or more convolutional neural network layers and, optionally, pooling layers, transposed convolution layers, and element-wise layers (e.g., layers applying element-size activation functions). The described techniques can further process other types of layers in an FCN by handling them in the same or similar fashion as convolution layers or transposed convolution layers. An FCN can be deployed on a hardware accelerator to generate pixel-wise predictions of an input (e.g., an input image with multiple pixels). In general, an FCN includes a sequence of network layers with at least one input layer configured to receive an input, and at least one output layer configured to generate an output for the input. In some situations, an FCN can include an input layer and multiple output layers such that the FCN is configured to generate multiple outputs for a single input to the FCN. An FCN can also include multiple input layers each configured to receive a respective input and multiple output layers, where at least each input layer is topologically connected to a subset of the multiple output layers. This type of FCN is configured to generate multiple outputs via the multiple output layers for processing multiple inputs simultaneously received by the multiple input layers. In some situations, an FCN (e.g., a multi-input, and multi-output FCN or a single-input, multi-output FCN) can include tree structures or directed acyclic graph (DAG) structures. A tree structure generally refers to a layer whose output is received by more than one succeeding layer, and the tree structure is also referred to as a branch point. A DAG structure generally Attorney Docket No.56113-0510WO1 refers to a layer receiving, as input, output from more than one preceding layer. A DAG structure is also referred to as a “re-convergence” point. Note that for a multi-input, multi- output FCN, an intermediate layer or an output layer can also receive output generated by paths starting from multiple input layers, and such an intermediate layer or output layer is referred to as a “convergence point.” That said, the techniques described herein do not have to distinguish between a tree structure and a DAG structure. Nor do the techniques have to distinguish between a re-convergence point and a convergence point. This is described in more detail below. For purposes of this document, this type of FCN is also referred to as a multi- input, multi-output FCN throughout. For simplification and ease of illustration, a multi- input, multi-output FCN in the description below is by default referred to as an FCN, and other types of FCN (such as single-input, single-output FCNs or single-input, multiple- output FCNs) are generally described by full names. According to one aspect, a method performed by one or more computers to implement a neural network model for processing multiple inputs at an invocation to generate multiple outputs, the method including receiving two or more new inputs to be processed at the invocation by a fully convolutional network deployed on a hardware accelerator. Each of two or more new inputs has a respective size such that at least one of the respective sizes is different from a fixed size that the fully convolutional network is configured to process when deployed on the hardware accelerator. The fully convolutional network includes multiple network layers. The multiple network layers include two or more input layers and multiple output layers. The two or more input layers are each for one of the two or more new inputs received at the invocation . Each of the multiple output layers is configured to generate a respective output of the invocation for corresponding inputs received by corresponding input layers that are connected to the output layer. The method also includes, for each of the two or more new inputs received at the invocation, determining a group of fixed-size input tiles for the new input. Each fixed- size input tile has the fixed size. The method then includes providing each of the groups of fixed-size input tiles to the hardware accelerator for performing inference computations to generate respective outputs of the invocation using the deployed fully convolutional network. This step further includes: for each of the multiple output layers, generating respective fixed-size output tiles for corresponding fixed-size input tiles processed by at least one of the two Attorney Docket No.56113-0510WO1 input layers that are connected with the output layer according to the deployed fully convolutional network. The respective fixed-size outputs comprise one or more inaccurate pixel-wise values. The method further includes, for each output layer of the multiple output layers, generating, from the respective fixed-size outputs, a respective final output of the invocation for the output layer that is equivalent to an output that would otherwise be generated from the output layer by processing corresponding inputs of the two or more new inputs at the invocation through layers of the two or more input layers that correspond to the output layer using the fully convolutional network deployed for processing the corresponding inputs with the respective sizes. The subject matter described in this specification can be implemented in particular embodiments so as to realize one or more of the following advantages. The described techniques allow a statically-compiled, multi-input, multi-output FCN deployed on a hardware accelerator to efficiently process, for each invocation, multiple inputs with sizes different from the particular sizes that the FCN model, once deployed, is natively capable of processing and generate multiple outputs for the multiple inputs as if the deployed FCN can process the multiple inputs with sizes different from the particular sizes. This way, the deployed FCN model can process multiple inputs with varying sizes for each invocation without the need to recompile the FCN for each invocation. As an example, for a multi-input, multi-output FCN with M input layers deployed to process multiple inputs at each invocation with particular sizes {S1, S2, ... SM}, the described techniques can obtain data according to the characteristics of the deployed FCN model and use the obtained data to allow the deployed FCN model to process inputs of sizes {X1, X2, ... XM} different from the particular sizes of {S1, S2, ... SM} at one invocation. For another invocation, the FCN model can further process inputs of sizes {Y1, Y2, ... YM}, which can also be different from the sizes of {S1, S2, ... SM}. In addition, in contrast to a single-input, single-output FCN or a single-input, multiple-output FCN, the deployed FCN has multiple input layers, each configured to, for each invocation of multiple invocations, receive a respective input for the invocation with a respective size that may be different from a fixed size that the deployed FCN is configured to process. The described techniques are also robust to different network structures of a multi- input, multi-output FCN. Each of the input layers can be associated with a respective Attorney Docket No.56113-0510WO1 subset of layers according to different task requirements. For example, one or more layers in a subset of layers starting from a respective input layer can include a convolution layer, a transposed convolution layer, a pooling layer, or other suitable layers, In addition, the deployed FCN can include multiple branch points, convergence points, and re- convergence points according to various task requirements. The described techniques can further improve the accuracy of outputs generated for corresponding inputs from the deployed FCN. More specifically, the described techniques can determine a tiling pattern for each of the inputs according to alignment requirements determined by the characteristics of the deployed FCN. The described techniques are further capable of tracking accurate pixel values from all pixels in the output tiles and performing a particular stitching algorithm to combine accurate pixel values to reach a final output for the corresponding multiple inputs. Furthermore, the described techniques are robust to different hardware accelerator architectures. The described techniques can automatically identify hardware constraints or requirements, such as system memory bandwidth. The described techniques can efficiently tile arbitrary large-size inputs to fit the hardware accelerator where the FCN is deployed based on the identified hardware constraints or requirements. For example, for accelerators with advanced memory addressing capabilities (e.g., accelerators including direct memory access (DMA) engines), the described techniques can reduce or eliminate overhead time related to data manipulation for tiling inputs and stitching fixed-size outputs. For accelerators with a simpler architecture or less memory bandwidth, the described techniques can determine instructions for preparatory work and distribute tasks to a supporting host processor. Once the instructions are executed, the supporting host processor can perform operations such as generating fixed-size input tiles from an input or packing up fixed-size output tiles to generate a final output. Note that the techniques described in this specification are distinct and advantageous over conventional data parallelization techniques. In general, data parallelization techniques can divide input data (e.g., an input image) into multiple disjoint portions (e.g., segments of the input image) and assign the multiple portions to multiple hardware components (e.g., hardware accelerators) to process the portions independently and in parallel to generate partial outputs. After all of the portions are processed by the hardware components, a system configured to perform the data parallelization techniques can generate a final output by aggregating the partial outputs. As long as the operations are correctly performed by each hardware component for Attorney Docket No.56113-0510WO1 respectively designated portions, the system does not need to consider whether any parts of the partial outputs are not suitable or inaccurate for generating the final output. In contrast, an FCN generally does not take advantage of data parallelization techniques because an output generated by the deployed FCN processing an input tile (e.g., a tile of an input image) can include one or more incorrect or inaccurate pixel-wise values. The described techniques are capable of distinguishing accurate pixel-wise values from inaccurate ones and combining accurate ones to generate a final output for multiple inputs. The details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims. BRIEF DESCRIPTION OF THE DRAWINGS FIG.1 shows an example inference system for performing inference computations in a statically compiled, multi-input, multi-output FCN configured to simultaneously process two or more inputs with respective sizes. FIG.2 illustrates an example inference process using an example inference system. FIG.3 illustrates an example structure of multi-input, multi-output FCN. FIG.4 illustrates an example of backtracking stack in connection with another example multi-input, multi-output FCN. FIG.5 illustrates an example fixed-size input with a neighbor pixel region and an example of fixed-size output with a dummy region. FIG.6 illustrates an example process for performing inference computations in a statically deployed, multi-input, multi-output FCN for multiple inputs with respective sizes. DETAILED DESCRIPTION The techniques described below can allow a statically compiled, multi-input, multi-output FCN deployed (or to be deployed) on a hardware accelerator to effectively and dynamically process, for each invocation of multiple invocations of the FCN model, multiple inputs of respective sizes different from the particular sizes that the deployed Attorney Docket No.56113-0510WO1 FCN is natively capable of, and generate multiple outputs for the multiple inputs at the invocation. More specifically, the deployed FCN described in this document generally includes multiple input layers each configured to simultaneously receive and process a respective input for an invocation of the FCN model, and each of the respective inputs at the invocation has a respective size that may be different from a fixed size that the deployed FCN is natively configured to process. The FCN model also includes multiple output layers each configured to generate a respective output for processing the multiple inputs at the invocation. Since each output layer can be connected to a respective subset of input layers according to the topology of the FCN model, the output layer is configured to generate an output at an invocation for corresponding inputs received by corresponding input layers at the invocation. Similarly, an input layer of the FCN model can be viewed as being connected to a subset of output layers according to the topology of the FCN model. The implementation of the deployed FCN model is advantageous over conventional techniques since the deployed FCN model can, for each invocation of multiple invocations, process inputs with sizes different from the sizes that the FCN model is configured to when deployed. The described techniques accordingly improve the efficiency of performing inference operations of the FCN model since there is no need to re-compile the FCN model for differently sized inputs for each invocation. In addition, the implementation of the deployed FCN model is configured to process multiple inputs simultaneously at an invocation and generate multiple outputs for the multiple inputs at the invocation, which could further improve the efficiency of performing inference operations over models that could only process a single input at a time step, or generate a single output at a time step, or both. FIG.1 shows an example inference system 100 for performing inference computations in a statically compiled, multi-input, multi-output FCN 115 configured to simultaneously process two or more inputs with respective sizes. The inference system 100 can be implemented by one or more computers at one or more locations, in which systems, components, and techniques described below can be implemented. The one or more computers can be connected through wires or wirelessly coupled. Some of the components of the inference system 100 can be implemented as computer programs configured to run on one or more computers. Attorney Docket No.56113-0510WO1 As shown in FIG.1, the inference system 100 can be configured to process input data 150 to generate output data 170 using a multi-input, multi-output FCN 115 compiled and deployed for processing fixed-size inputs for each invocation of multiple invocations, where the fixed sizes can be different from sizes of all of the input data 150. More specifically, the multi-input FCN 115 is deployed from data 155, which represents such a model, and the inference system 100 is configured to receive and process data 155 to generate corresponding instructions for compiling the FCN model onto one or more hardware accelerators 110. Once the FCN model is compiled, it is configured to process multiple inputs at each invocation with fixed sizes. The described techniques allow the deployed FCN 115 to process multiple inputs at each invocation with respective sizes that might be different from the sizes that the FCN is natively compiled for and generate multiple outputs for the invocation as if the multiple inputs are processed by a dynamically deployed multi-input, multi-output FCN model. Additional details of processing data 155 for compilation and implementation of the compiled FCN 115 are described below. In general, a multi-input, multi-output FCN represented by data 115 can include two or more input layers and multiple output layers. Each of the two or more input layers is configured to receive and process a respective input with a respective size for each invocation of the FCN model. The multi-input, multi-output FCN can accordingly process multiple inputs simultaneously for each invocation and generate outputs for the multiple inputs through the multiple output layers for the invocation. Example FCNs described in this document are illustrated in FIGS.3 and 4, and additional details of an example multi- input, multi-output FCN model are described below in connection with FIG.3. A multi-input, multi-output FCN can further include one or more “paths” or “branches.” Different from a branch point that generally represents a network layer, the output of which is received and processed by more than one succeeding layer, the term “path” or “branch” used throughout the specification broadly represents a sequence of network layers. In general, a multi-input, multi-output FCN can include different structures, for example, a tree structure (also referred to as a branch point), a directed acyclic graph (DAG) structure (also referred to as a reconvergence point), and a convergence point that receives outputs along two or more branches (e.g., two or more preceding layers) as input. Note that in some implementations, a convergence point can be followed by a layer Attorney Docket No.56113-0510WO1 serving as an output layer, e.g., output layer 490 of FIG.4. More details are described in connection with FIG.3. FCN 115 can be implemented for performing tasks related to object classification and detection (e.g., human face detection), image segmentation, color-image generation, image super-resolution, image completion, or image colorization, to name just a few examples. For example, FCN 115 can be used to process a grayscale video that captures multiple persons, and an audio recording accompanying the video and representing speeches made by one or more of the captured persons. The first output can represent a colored video corresponding to the grayscale video, a second output can include data recognizing each person of the captured multiple persons, and a third output can include data that can represent a respective audio recording for each speaker in the video. In some implementations, FCN 115 can be implemented for background detection, boundary detection, and depth image generation. One example task for using a multi-input and multi-output FCN model can include processing data measured by stereo sensors located at different positions with respective orientations. The stereo sensors can include image sensors such as cameras positioned at different locations to capture a scene from a respective perspective. The input to the FCN model can include images from stereo sensors and, optionally, data representing depth information about a scene. Another example task can relate to splitting a large model into multiple smaller submodels for efficiency and practicality reasons. More specifically, a large model can be divided into multiple smaller models with sizes suitable for particular accelerators. These sub-models can be manipulated into a multi- input multi-output structure based on the characteristics of the original model. The input data 150 can have multiple inputs of the respective types and sizes for each invocation of multiple invocations. For example, input data 150 can have multiple images of respective sizes for a particular invocation, e.g., a size of 50 by 50 pixels, 100 by 100 pixels, 100 by 150 pixels, 200 by 100 pixels, 400 by 400 pixels, or other suitable sizes. Note that these input sizes for an invocation can be different from corresponding sizes that the FCN is deployed for, as described above. In some implementations, input data 150 can include other types of data in a tensor or matrix form. The generated output data 170 are outputs generated for a corresponding invocation by the system 100 performing inference operations of FCN model 115 for processing input data 150 for the corresponding invocation. In particular, the output data 170 can include multiple outputs for processing multiple inputs using the compiled FCN Attorney Docket No.56113-0510WO1 model 115. In some implementations, the multiple outputs from respective output layers can have different sizes. For example, for input sizes of 30 by 25 pixels, 50 by 65 pixels, and 100 by 100 pixels at an invocation, the outputs can include different sizes such as 20 by 20 pixels, 50 by 50 pixels, 25 by 100 pixels, or other suitable sizes for the invocation. Please note this example is made for ease of explanation, and one should appreciate that other possible sizes for inputs and outputs at an invocation can be processed for different task requirements. The inference system 100 can compile, using a compiling engine 160, a multi- input, multi-output FCN. To compile FCN 115, the host 130 can receive data 155 representing a multi-input, multi-output FCN, compile the FCN model, and generate instructions (e.g., binary data) to deploy the FCN model on one or more hardware accelerators 110. Host 130 can include a compile engine 160 for statically compiling FCN 155. In general, the compiling engine 160 can decode program codes written in any suitable high- order language and encoded with data representing characteristics of an FCN into machine-readable binary codes on a hardware accelerator. The data representing characteristics of an FCN can include hyper-parameters defining the structure of the FCN (e.g., input size, number of layers, number of input layers, number of output layers, overall topology of the model, number of nodes in each layer, layer types and positions, and padding, stride, filter size, and scale factor for one or more layers), and layer weights obtained from a training process. During compiling, the system 100 allocates respective computing resources based on characteristics of an FCN. For example, the system 100 can allocate respective data structures to accommodate respective calculations for performing inference computations, e.g., data structures allocated for layer weight matrices, activation inputs, and outputs, that are dependent on the input size. As another example, the system can allocate respective memories for storing respective data structures and associated computation results during performing inference operations for the deployed FCN. The inference system 100 can perform a “tiling and stitching” analysis for an input according to the characteristics of FCN 155. The system 100 can further determine a tiling pattern for each input of multiple inputs. The tiling pattern can include data representing a tiling size for compiling a multi-input, multi-output FCN 155 (e.g., the fixed size that the FCN is natively deployed to process). The tiling pattern can further include how different fixed-size tiles are generated from an input, e.g., overlapping Attorney Docket No.56113-0510WO1 between tiles, arrangements of input tiles, and stitching processes for output tiles based on the tiling pattern. The tiling pattern is generated under different constraints or requirements. For example, the described techniques can generate multiple output tiles for each output layer after processing corresponding input tiles at corresponding fixed sizes. Each of the output tiles can include valid pixels computed using accurate pixel values, and dummy regions computed with one or more dummy values. Given that, the constraint or requirement here is that an output tile of multiple output tiles for an output layer should have at least one valid pixel value. Note that the final output of an output layer can get, for each pixel in the final output, a valid pixel value from at least one output tile of the output tiles. As another example requirement, the coordinate shifts in both input and output tiles should satisfy particular alignment requirements inherent in an FCN according to the characteristics of the FCN. The details of performing the “tiling and stitching” analysis are described below. After performing the “tiling and stitching analysis,” system 100 can obtain data stored in a data structure (e.g., a data tuple) for performing the inference processes using the statically-compiled, multi-input, multi-output FCN 115. More specifically, system 100 (or host 130) can include a tiling engine 135 and a stitching engine 140 to assist the process. For each input of multiple inputs at each invocation, the tiling engine 135 can be configured to tile the input into multiple fixed-size inputs 138 based on a tiling pattern, and the fixed-size inputs 138 are provided to a corresponding input layer of multiple input layers. This way, FCN 115 is configured to simultaneously process multiple sets of fixed- size inputs 138 at the invocation such that each set of fixed-size inputs 138 is received and processed by a respective input layer. In cases where an input to the tiling engine 135 has a smaller size than the fixed size, system 100 can pad zeros around the input to reach the fixed size before processing the input through a corresponding input layer. System 100 can generate multiple fixed-size outputs 133 from different output layers for processing respective groups of fixed-size input 138 and provide the fixed-size outputs 133 to the stitching engine 140 to obtain output data 170. To generate the output data, the stitching engine 140 is configured to “stitch” multiple fixed-size outputs generated by each output layer to generate a respective output for corresponding input(s). Note again that an input layer is connected with a subset of output layers and an output layer is connected with a subset of input layers according to the topology of the deployed FCN. Attorney Docket No.56113-0510WO1 For each output layer of multiple output layers, system 100 obtains, after determining a tiling pattern, coordinate information for all valid pixels in each fixed-size output of the corresponding set of fixed-size outputs. The system can adopt a particular algorithm for the stitching process, which will be described in more detail below. In some implementations, the tiling and stitching process does not have to be performed on a host 130. For example, any suitable hardware accelerators having GPUs and CPUs can perform the tiling and stitching process. Moreover, the tiling and stitching process can be performed at different physical locations off the host. For example, the tiling can be performed by a first set of accelerators at a first location, the stitching process can be performed by a second set of accelerators at a second location, and the host 130 can be located at a third location and configured to receive outputs from the second set of accelerators. The accelerators and hosts can be communicatively connected, either physically or wireless at one or more locations. In some implementations, one or more hosts (e.g., offline analysis / compilation hosts) different from host 130 can perform the tiling-pattern analysis offline, at a “deployment” stage, or ahead of time off host 130, and compile and deploy the multi- input, multi-output FCN 155 on host 130 (e.g., one or more communicatively coupled computers), or deploy the multi-input, multi-output FCN 155 as an “application” on one or more edge devices (e.g., cell phones or tablets) for processing inputs of unknown sizes. System 100 can include any suitable type of hardware accelerator 110 for performing inference computations of an FCN model. For example, the hardware accelerator 110 can be a CPU, GPU, TPU, or any other suitable accelerator. The hardware accelerator 110 can include components such as memory for storing parameters of the FCN model. Moreover, the hardware accelerator 110 can include one or more computing units for parallel computation. Host 130 can communicate with hardware accelerator 110 by transferring data, instructions, or both. Host 130 and hardware accelerator 110 can communicate through wired or wireless connections and, in some cases, can be located remotely from each other. For example, the host 130 can be a server at a different physical location from where the accelerator 110 is located. FIG.2 illustrates an example inference process 200 using an example inference system 245. The example inference system 245 can be equivalent to the inference system 100 of FIG.1, and when appropriately programmed, inference system 245 can perform the inference process 200. Attorney Docket No.56113-0510WO1 In general, the described techniques can dynamically process sets of inputs with respective sizes at each invocation of multiple invocations. Each set of inputs can have respective sizes different from the particular sizes that the FCN is originally deployed for. This way, by using the techniques described herein, the statically deployed multi-input, multi-output FCN model no longer needs to be recompiled to process sets of inputs at each invocation of multiple invocations. As shown in FIG.2, FCN 235 can be a statically compiled, multi-input, multi- output FCN that is natively configured to process, for each invocation of multiple invocations (e.g., invocations 205a, 205b, and 205c), a set of inputs for the invocation with respective sizes to generate an output set (e.g., output set 220a, 220b, or 220c) including multiple outputs for the invocation. More specifically, inference system 245 can process, for invocation 205a, inputs including first input 210a, second input 210b, …, and mth input 210m, and generate the output set 220a including multiple outputs for the invocation, e.g., a first output 225a, a second output 225b, … , and the Nth output 225n. Each input (e.g., 210a, 210b, …, or 210m) for an invocation (e.g., invocation 205a) is processed by a corresponding input layer of multiple input layers of FCN 210. Each output (e.g., 225a, 225b, …, or 225n) of the output set (e.g., 220a) are generated by a corresponding output layer of multiple output layers of the FCN 210. Note that the techniques described herein are different from using multiple statically-compiled single-output FCNs where each of the single-output FCNs can process a respective input to generate a respective output. The statically-compiled, multi-input, multi-output FCN is a single FCN model having (i) two or more input layers for receiving and processing respective inputs at an invocation and (ii) multiple output layers for generating multiple outputs for corresponding inputs at the invocation. An example structure of a multi-input, multi-output FCN is described in greater detail in connection with FIGS.3 and 4. In addition, the general process of performing a “tiling and stitching” analysis includes, by system 100, selecting an input tile size for each input of multiple inputs processed by the multiple input layers, and generating a tiling pattern based on alignment information determined for the FCN and valid region information associated with output tiles for the multiple output layers. More details of the “tiling and stitching” analysis are described in connection with FIGS.3 and 5. FIG.3 illustrates an example structure of multi-input, multi-output FCN 300. In general, the described multi-input, multi-output FCN can include multiple input layers Attorney Docket No.56113-0510WO1 each configured to receive an input with a respective size. As shown in FIG.3, FCN 300 includes a first input layer 301 configured to receive and process a first input with a first size (and optionally of a first data type, e.g., a tensor or matrix). FCN 300 further includes a second input layer 303 configured to receive and process a second input with a second size (and optionally of a second type). The first size or the second size (or both) may be different from the fixed size that FCN 300 is configured to process when deployed. In other words, without implementing the techniques described in this document, a statically deployed FCN can only natively process inputs with the fixed size, and a conventionally deployed FCN needs to be re-compiled to process inputs with sizes different from the fixed size. Note that although FCN 300 includes two input layers for ease of illustration, a multi-input, multi-output FCN can include more than two input layers, e.g., three, five, six, eight, ten, or more input layers. A multi-input, multi-output FCN can further include multiple output layers. Each output layer of multiple output layers can be reachable by one or more input layers according to the topology of an FCN. Equivalently, each input layer is connected with at least a subset of output layers according to the topology of the FCN. As shown in FIG.3, FCN 300 includes a first output layer 351 reachable by input layer 301 through a first path, a second output layer 355 reachable by input layer 301 from a second path and input layer 303 from a third path, and a third output layer 359 reachable by input layer 303 through a fourth path. Specifically, the first path includes layers 301 and 351; the second path includes layers 301 and 355; the third path includes layers 303, 325, 330, 335, 339, and 355 or layers 303, 325, 333, 337, 339, and 355. The fourth path includes layers 303 and 359. Note that the topology of FCN 300 is illustrated for ease of description and FCN 300 can have other suitable topologies. Each output layer is configured to generate a respective output for corresponding input(s). In general, input layers 301 and 303 are configured to receive a respective input for an invocation, and the output layers 351, 355, and 359 are configured to generate a respective output for corresponding input(s) at the invocation. For example, the inputs received by layers 301 and 303 can have sizes of{40x40, 50x50}, respectively, and the outputs produced by output layers 351, 355, and 359 can have sizes of {100x100, 200x200, 300x300}, respectively. FCN 300 can, at a next invocation, process two inputs with sizes of {80x80, 100x100}, respectively, and generate three outputs with sizes of {200x200, 400x400, 600x600}, respectively. FCN 300 can repeatedly perform the above- described operations for a sequence of invocation. Attorney Docket No.56113-0510WO1 In other words and as shown in FIG.3, for a first invocation, output layer 351 is configured to generate a first output for processing a first input received and processed by input layer 301 at the first invocation. Output layer 355 is configured to generate a second output for processing the first input received by input layer 301 and a second input received by input layer 303 at the invocation. Output layer 359 is configured to generate a third output for processing the second input received and processed by input layer 303 at the first invocation. In some implementations, layer 335 can be referred to as a convergence point since it receives outputs from more than one branch. The outputs can include semantic classification of a human face, background objects, and depth images for processing an input image, as described above. Note that although layer 335 serves as an output layer in FCN 300, one should appreciate that not every convergence point is an output layer of a multi-input, multi-output FCN model. For example, layer 460 in FIG.4 is a convergence point but it is not an output layer. FCN 300 can include other network layers between input layers and the output layers. For example, FCN 300 can have a second layer 325 succeeding layer 303, layers 330 and 333 succeeding layer 325, layer 335 succeeding layer 330, layer 337 succeeding layer 333, and layer 339 succeeding layers 335 and 337. In some situations, layer 325 can be referred to as a branch point since the output from layer 325 is received as input by more than one succeeding layer (e.g., layers 330 and 333). Layer 339 can also be referred to as a reconvergence point since layer 339 receives outputs from more than one branch. Here, layer 339 receives an output from layer 335 from a branch including layers 325, 330, and 335, and receives another output from layer 337 along another branch, including layers 325, 333, and 337. A re-convergence point (e.g., layer 339) is generally different from a convergence point (e.g., layer 355) since a re-convergence point is generally coupled to a preceding branch point. That said, the described techniques do not need to distinguish between a re-convergence point and a convergence point on the implementation level. Nor do the described techniques need to distinguish between a tree structure (a branch point) and a DAG structure. More details of a tree structure and a DAG structure in an FCN model have been described in International Application No. PCT / US2023 / 012634, which is accordingly incorporated by reference herein. An FCN generally includes multiple paths or branches connecting the multiple input layers and multiple output layers according to the topology of the FCN. A branch or a path represents a sequence of network layers connecting each other in a predetermined order, as described above. As shown in FIG.3, FCN 300 includes a first path including Attorney Docket No.56113-0510WO1 layers 303, 325, 330, 335, 339, and 355, a second path including layers 303, 325, 333, 337, 339, and 355, a third path including 303 and 359, a fourth path 301 and 355, and a fifth path 301 and 351. The described techniques generally include operations to traverse back and forth along paths or branches of an FCN to determine the information needed for implementing the FCN on hardware. The described techniques further keep track of branches that have been traversed and that have not yet been traversed, and can sometimes restore traversed branches as untraveled ones under particular conditions. More details related to traversing along branches are described below in connection with projecting backward, projecting forward, and backtracking operations. Furthermore, the network layers included in an FCN can include different types. For example, the network layers can include one or more convolution layers, transposed convolution layers, pooling layers, or other suitable layers. The described techniques can handle different types of layers if they can be manipulated into structures similar to a convolution layer or a transposed convolution layer. Each layer can include a different size and parameters, for example, a different number of nodes, different nodal operations (e.g., different activation functions, normalization functions, or different nodal weight values), and / or a different stride size, filter size, and zero-padding size. It should be appreciated that the structure of the multi-input, multi-output FCN 300 is illustrated for simplicity. A multi-input, multi-output FCN 300 for implementing the described techniques can include different numbers of output layers, branches, branch points, convergence points, re-convergence points, and different types and arrangements of network layers. For example, the multi-input, multi-output FCN 300 can include 5, 10, 20, or more branches, 4, 9, 15, or more branch points, or 10, 20, 50, or more layers, etc. First Valid Pixel Offset To perform the “tiling and stitching” analysis of a multi-input, multi-output FCN 300, the system, e.g., the inference system 100 of FIG.1, can, for each output layer, obtain a first- valid-pixel-offset ^^^for the ^^௧^output, the ^^௧^output being generated from the ^^௧^output layer of the FCN 300. The operations for determining a first-valid-pixel-offset ^^^a multi- input multi-output FCN can utilize a similar approach used for a single-input, multiple- output FCN. More specifically, the system can manipulate the multi-input multi-output FCN to topologically look like a single-input multi-output FCN. For example, and as shown in FIG.3, the system can add a dummy input layer 380 having edges connecting all of the multiple input layers (e.g., layers 301 and 303). The edges between the dummy Attorney Docket No.56113-0510WO1 input layer 380 and the multiple input layers are configured such that the first valid pixel offsets determined for the multiple input layers would not be affected or changed. For example, the edges from the dummy input node 380 to each actual model input layer (e.g., input layers 301 and 303) do not create invalid pixels. By adding the dummy input layer 380 and corresponding edges, the modified FCN 300 looks similar to a single-input multi-output FCN, and the first-valid-pixel-offset algorithm for a single-input multi- output FCN can be deployed for FCN 300. The first-valid-pixel-offset is generally related to a dummy region and a valid region of an output from a network layer. The first-valid-pixel-offset can, for example, in one-dimensional cases, represent a width of ^^^pixels away from an edge of output such that pixels in the output that are at least ^^^pixels away from the edge are valid and accurate. More details of a valid region and a dummy region in an output tile are described below in connection with FIG.5. To generate the first-valid-pixel-offset ^^^for the ^^௧^output, the system can use the FirstValidPixelOffset() algorithm encoded based on characteristics of the FCN model, e.g., layer types, stride, filter, and padding. This FirstValidPixelOffset() algorithm for the multi-input, multi-output FCN model can be substantially the same as that for a single- input multi-output FCN model after the system modifies the multi-input, multi-output FCN model to include a dummy node with edges connecting all model input layers. An example FirstValidPixelOffset() algorithm can be presented below: function FirstValidPixelOffset(layers): / / First pixel offset at which the prior layer produced a valid result. first_valid_offset = 0 for layer = input to output layers: if layer type is “conv”: s = stride of layer; p = padding of layer; f = filter size of layer first_valid_offset = ceil((first_valid_offset + p) / s) else if layer type is “trans_conv”: s = stride of layer; p = padding of layer; f = filter size of layer / / Offset of last invalid pixel on the input activation. Can be -1 or greater. last_invalid_offset = first_valid_offset + p - 1 / / Offset of last invalid pixel on the output activation. last_invalid_offset = last_invalid_offset * s + f - 1 first_valid_offset = last_invalid_offset + 1 return first_valid_offset b=FirstValidPixelOffset(layers) Note that, although the example FirstValidPixelOffset() algorithm includes operations to process convolution layers and transposed convolution layers, the example Attorney Docket No.56113-0510WO1 FirstValidPixelOffset() algorithm can be configured to process other types of layers (e.g., pooling layers or other suitable layers) that can typically be mapped to the algorithms used for processing the convolution layers and / or transposed convolution layers. Alternatively, or in addition, algorithms for processing the convolution layers and / or transposed convolution layers can be generalized to process other types of layers. In general, the criteria for the first valid pixel calculated from the left edge and the right edge of a fixed-size output is not entirely symmetrical – a few pixels may remain on the right side of the fixed-size input where the filter could not be applied, which keeps one more valid pixel on the right side of the fixed-size output than the left. The output (e.g., the first valid offset ^^^) of the FirstValidPixelOffset() algorithm is calculated from the left, and this value should also be conservatively correct for the right. Similarly, the above-described analysis should also apply for calculations from the top or the bottom of the fixed-size output. More details of the FirstValidPixelOffset() algorithm have been described in International Application No. PCT / US2023 / 012634, which is accordingly incorporated by reference herein. Validating a Candidate Tile Size The system can validate the candidate output tile size for a particular output layer in the multi-input, multi-output FCN 300. The particular output layer can be a driver output layer. A driver output layer can be arbitrarily selected from all of the multiple output layers, and the described techniques should still hold. In some implementations, a driver output layer can be determined according to different criteria. For example, a system can select an output layer that generates fixed-size outputs with the smallest size among the multiple output layers 351, 355, and 359. As another example, a system can select an output layer that the system has the most information about. For example, and as shown in FIG.3, the system can select output layer 355 as the driver output layer, and validate the candidate tile size from the driver output by projecting forward and backward along different branches of FCN 300. In general, the system can select one of the two branches (e.g., the left branch or the right branch from layer 355) to perform backward projection, and the backward projection operations are similar to a single-input multi-output FCN. The system adds the unselected branch into an “unexplored” set for further operations. Once the selected branch has been explored, the system then selects one of the branches from the unexplored set for backward projection. For example, at layer 339, the system selects one of the branches (e.g., the branch having Attorney Docket No.56113-0510WO1 layer 337) for backward projections, while the remaining or unselected branch including layer 335 is added to the unexplored set. Due to the subsequence forward projections 370, the branch having layers 335 and 339 will be explored and this branch will consequently be removed from the unexplored set, leaving the branch from layer 301 to layer 335 in the unexplored set. An example process for validating a candidate tile size is described in connection with FIG.3. The system first performs a project backward 1(a) step (365) starting from 355. After reaching layer 325 during the backward projection, the system performs a project forward 1(b) step (370) to traverse layers 330 and 335 and ends at layer 339. The shape or size of layer 339 projected forward along 1(b) is validated against the shape or size of layer 339 set during backwards projection 1(a). Validation fails if the shapes or sizes of layer 339 mismatch from both projections. After reaching 303 during the backward projection, the system performs a project forward 1(c) step (375) to reach layer 359. The validation process that projects backward and forward along the right branch of layer 355 is substantially similar to validation processes for a single-input multi-output FCN. Example algorithms for projecting backward and projecting forward are described in greater detail below. Once completing the validation process on the right branch of layer 355, the left branch of layer 355 remained in the unexplored set since the backwards projections along the right branch (and any resulting forward projections) of layer 355 could not reach the left branch of layer 355. Then, to start the backward projection, the system can select the left branch from layer 355 stored in the unexplored set, and remove the left branch from the unexplored branch to initiate the backward projection step. The system performs project backward 2(a) step (390) from layer 355 to layer 301 and then performs project forward 2(b) step (385) to layer 351. The project backward and forward operations along the left branch from layer 355 are similar to projecting operations performed on a single- input, multi-output FCN. The overall validation operations complete successfully when the unexplored set becomes empty and the last project backward operation succeeds. At this point, all layers in the multi-input, multi-output FCN have been traversed, and the tile sizes are validated for layers. Referring back to the project forward and project backward operations in an FCN model, for convolution layers having stride sizes greater than one, the system can explore and store one or more possible tile sizes for these layers in a backtracking stack. This is Attorney Docket No.56113-0510WO1 because the input tile size to the output tile size in a convolution layer with a stride size greater than one is not guaranteed a one-to-one size mapping, and one output size of a network layer can be mapped to one or more different input tile sizes. For simplicity, layers (or more specifically, strided convolution layers) having stride sizes greater than one are also referred to as strided layers in this document. For both multi-input multi output FCNs and single-input multi-output FCNs, the system can deploy the same backward projection algorithm and forward projection algorithm. One example ProjectBackward() algorithm is presented immediately below. Note that this algorithm is called at multiple steps, for example, one or more steps in the project backwards 1(a) 365 of FIG.3. function ProjectBackward((ht, wt, hb, wb), layers): for layer = output to input layers: if ht == hb or wt == wb: THROW EXCEPTION; / / layer is vanished if layer type is “conv”: / / Conv layer: n = floor((m + 2 p - f) / s) + 1, n is output size, m = input size s = stride of layer; p = padding of layer; f = filter size of layer n_h = hb - ht; n_w = wb - wt / / output tile sizes in the h and w dims / / input tile size m_h = (n_h - 1) * s + f - 2 p; m_w = (n_w - 1) * s + f – 2p / / Note: m_h x m_w is the smallest input tile size, but any size up to / / (m_h + s - 1) x (m_w + s - 1) will work to produce n_h x n_w output. / / Selecting a larger size may matter if a trans_conv layer / / precedes the conv layer in model order and disallows certain sizes. / / Sizes can be explored as back-tracking, not shown here for simplicity. ht = ht * s; hb = ht + m_h wt = wt * s; wb = wt + m_w + 2 p - 1) * s + f s = stride of layer; p = padding of layer; f = filter size of layer n_h = hb - ht; n_w = wb - wt / / output tile sizes in the h and w dims / / input tile size m_h = Validate( (n_h - f) / s - 2 p + 1 ) m_w = Validate( (n_w - f) / s - 2 p + 1 ) ht = Validate( ht / s ); hb = ht + m_h wt = Validate( wt / s ); wb = wt + m_w return (ht, wt, hb, wb) where: function Validate(value): if value is integral: return value else: THROW EXCEPTION / / Value cannot be used for coordinates of a fixed-size input in an FCN. The terms (ht, wt, hb, wb) can represent the location and size of an output tile. For example and for a two-dimension output tile, “ht” represents a vertical coordinate of a top Attorney Docket No.56113-0510WO1 left corner pixel of the output tile, “wt” represents a horizontal coordinate of the top left corner pixel of the output tile, “hb” represents a vertical coordinate of a bottom right corner pixel of the output tile, and “wb” represents a horizontal coordinate of the bottom right corner pixel of the output tile. The returned term (ht, wt, hb, wb) can generally represent a location and size of an input tile projected from the output tile. In some implementations, the system can use an algorithm ProjectBackwardTL (), which projects backward only the top-left corner coordinate of an output tile –– not both the top-left and bottom right corner coordinates in the ProjectBackward() algorithm. An example ProjectBackwardTL () is presented below. function ProjectBackwardsTL((ht, wt), layers): for layer = output to input layers: if layer_type is “element wise op”: / / Any element-wise op layer, e.g., add two tensors continue / / Nothing to do else if layer type is “conv”: / / Conv layer s = stride of layer ht = ht * s wt = wt * s else if layer type is “trans_conv”: / / TransConv layer s = stride of layer ht = Validate( ht / s ) wt = Validate( wt / s ) return (ht, wt) The input to the ProjectBackwardTL() algorithm is the top left corner coordinate for the ^^௧^output, e.g., (ht, wt), and the output of the algorithm is the top left corner coordinate, e.g., for the height dimension in “ht” and the width dimension in “wt,” of 2- dimension FCN input data. Similar to the ProjectBackward() algorithm, the system can also implement an example ProjectForward() algorithm to determine or validate a selection of a fixed-size input by determining that a forward-projected output that corresponds to the fixed-size input has all integer coordinates. The example ProjectForward() algorithm is presented as follows: function ProjectForward((ht, wt, hb, wb), layers): for layer = input to output layers: if ht == hb or wt == wb: THROW EXCEPTION; / / layer is vanished if layer type is “conv”: / / Conv layer: n = floor((m + 2 p - f) / s) + 1, n is output size, m = input size s = stride of layer; p = padding of layer; f = filter size of layer m_h = hb - ht; m_w = wb - wt / / input tile sizes in the h and w dims / / output tile size n_h = floor((m_h + 2 p - f) / s) + 1; Attorney Docket No.56113-0510WO1 n_w = floor((m_w + 2 p - f) / s) + 1 ht = Validate( ht / s ); hb = ht + n_h wt = Validate( wt / s ); wb = wt + else if layer type is / / TransConv layer: f s = stride of layer; p = filter size of layer m_h = hb - ht; sizes in the h and w dims / / output tile size n_h = (m_h + 2 p - 1) *s + f; n_w = (m_w + 2 p - 1) *s + f; ht = ht * s; hb = ht + n_h = * = Similar to the ProjectBackwardTL () algorithm, another example forward projection algorithm can be ProjectForwardTL () algorithm, which projects forward only the top-left corner coordinate of the input tile. The example ProjectForwardTL () algorithm is presented as follows: function ProjectForwardTL((ht, wt), layers): for layer = input to output layers: if layer_type is “element wise op”: / / Any element-wise op layer, e.g., add two tensors continue / / Nothing to do else if layer type is “conv”: / / Conv layer s = stride of layer ht = Validate( ht / s ) wt = Validate( wt / s ) else if layer type is “trans_conv”: / / TransConv layer s = stride of layer ht = ht * s wt = wt * s return (ht, wt) Since the ProjectBackwardTL() and ProjectForwardTL () algorithms do not receive output tile sizes and input tile sizes as input, respectively, the process of determining alignment information is independent of input or output tile sizes. In other words, the process of validating a candidate output tile size is independent of the process of generating alignment values, so the system can perform the two processes in any suitable order or in parallel. The system further includes mechanisms to resolve failures or mismatches when exploring possible input or output tile sizes in the backward projection, the forward projection, or both. The system can store all possible input tile sizes for a strided convolution layer for a given output tile size in a backtracking data structure (e.g., a backtracking stack, or for simplicity, a stack, or other suitable data structures). The stack can include a sequence of cells, and each of the cells includes one or more possible input Attorney Docket No.56113-0510WO1 tile sizes for a strided convolution layer. The sequence of cells is stacked on one another according to the order of layers being traversed in the backward projection. When the system determines a first failure or mismatch at a layer, the system tracks back to an immediately preceding strided convolution layer associated with the top cell of the backtracking stack. All projected tile sizes along the path from the failure layer to the particular strided convolution layer are removed from the memory (also referred to as undone). The system then selects another input tile size from the top cell different from the previously chosen one and resumes the backward projection, the forward projection, or both from the particular strided convolution layer. If all other possible input tiles for the top cell have been tried and failed, the system can pop out the top cell from the backtracking stack, and backtracks to another strided convolution layer associated with the new top cell (the previously second top cell) in the backtracking stack. The system can repeatedly backtrack in the stack until a solution is found or all cells are popped out due to failures. The differences in backtracking between a multi-input, multi-output FCN and a single-input multi-output generally relate to situations where the backtracking step reverts from a branch connected to a different input. In these situations, the system will re-add the reverted branch in the unexplored set for future operations. More details related to backtracking are described in connection with FIG.4. The system can repeatedly perform the back-tracking search, together with the forward projection and the backward projection, at different branches along the path until reaching each of the multiple input layers, and determine compatible / valid input tile sizes for all layers in FCN 300. The valid input tile sizes are stored in a data structure, e.g., a tuple. A tuple can store valid input tile sizes and, optionally, associated network layers. For example, a tuple can include a set of data indicating the output tile size for the driver output layer 355, the input tile sizes for input layers 301 and 303, and the projected output tile sizes for output layers 351 and 359. Since tile sizes for the input layers, 301 and 303 and tile sizes for the output layers 351, 355, and 359 are generally not unique, the system can collect more than one data tuple for implementing a multi-input, multi-output FCN. The system can select any tuple of the data tuples as parameters used to tile inputs with various sizes for FCN 300. Referring back to the backtracking operations as described above, FIG.4 illustrates an example of backtracking stack 405 in connection with another example of multi-input, multi-output FCN 400. Backtracking stack 405 is an example data structure Attorney Docket No.56113-0510WO1 for storing all possible input tile sizes for strided convolution layers in an example FCN model 400. The FCN model 400 is similar to the FCN model 300 in FIG.3. FCN model 400 includes two input layers 480 and 430, and at least two output layers 440 and 490. As shown in FIG.4, the FCN model 400 can include multiple convolution layers with strides greater than one. For example, a first convolution layer 410 has a stride of 2, a second convolution layer 420 has a stride of 3, and a third convolution layer 470 has a stride of 2. The FCN model 400 can include additional strided convolution layers, which are omitted from FIG.4 for ease of illustration. In general, the system traverses one or more sequences of network layers in the FCN to determine whether a candidate tile size for a driver output layer is valid to use as a fixed tile size for deploying the FCN. During the validation process, the system also determines tile sizes for all other input layers and output layers using the candidate tile size. If the system can’t obtain tile sizes for all other input layers and output layers using the candidate tile size, the system rejects the candidate tile size for the driver output layer and chooses a different candidate tile size to start over with. In addition, the sequence of network layers can form a path for validating a candidate output tile size selected for one of the multiple output layers. To address potential failures or mismatches in the validation process, the system can perform a particular backtracking technique to (i) undo failure validation operations and (ii) select a different candidate input tile size stored for a preceding layer for resuming the validation process. The backtracking technique described in this document is associated with a particular type of network layer, e.g., a strided convolution layer or a network layer with a stride greater than one. As described above, the system can determine all possible input tile sizes for strided convolution layers along a path for the backward projection or the forward projection, or both. As shown in FIG.4, the system can perform project backward operation starting at output layer 440 and traversing backward to layer 410. When the system reaches layer 410 from a previous layer (not shown in FIG.4) during the backward projection, the system can determine a set of possible input tile sizes for layer 410. For example, the set of possible input tile sizes can be 11 and 12. The system can store these tile sizes in a first cell 413 (Cell #0) in the backtracking stack 405. Before proceeding to layer 420, the system selects one of the possible input tile sizes in the first cell 413 as the candidate input tile size for performing the rest of the backward and forward projections. For example, the system can select input tile size 11 Attorney Docket No.56113-0510WO1 for layer 410, as shown in a dashed box 415, for the rest of the backward and forward projections. Along the backward projection path, the system can determine another set of possible input tile sizes for layer 420 based on the output tile size for layer 420 (which is implicitly based on the selected tile size 11 for layer 410). For example, the other set of possible input tile sizes can be 24, 25, and 26. The system can store the other set of input tile sizes in a second cell 423 (Cell #1) stacked on top of the first cell 413. Similarly, the system can select one of these possible input tile sizes (e.g., tile size 24) for layer 420, as represented by the dashed box 425, for downstream operations. The system then performs corresponding project forward and backward operations to reach all possible input layers (here, e.g., layer 430) and all other output layers (e.g., layer 490). In cases where the projection operations encounter a failure, the system reverts all operations from the current failure layer to the layer associated with the top cell in the backtracking stack 405. For example, if a failure occurs at layer 450, the system reverts all operations performed for layers 450 and 430 and returns to layer 420 and cell 423 associated with layer 420. The system selects one of the unselected possible input tile sizes in cell 423 and resumes the projection operations from layer 420 using the newly selected tile size (e.g., tile size 25). In general, if the projection operations have been completed for backtracking, the system can select a branch from the unexplored set to continue performing the backtracking operations along that branch. For example, the system can determine that another input layer reaches the output layer 490, and perform the project backward operations along the path to input layer 480. The new branch extracted from the unexplored set will allow the system to reach an additional model input layer and corresponding output layers that are reachable by forward traversal from this additional model input layer. This allows the candidate tile size of the driver output to be projected to the additional model input and corresponding additional outputs, to determine their corresponding projected tile sizes. The system determines another set of possible input tile sizes for layer 470 based on the selected tile size 11 for layer 410 and the selected tile size 24 for layer 420. The other set of possible input tile sizes for layer 470 can include 50 and 51. The system can store these tile sizes in a third cell 473 (Cell #2) stacked on top of the second cell 423. The system can select input tile size 50 for layer 470 for the rest of the backward and forward projections, as shown in the dashed box 475. Attorney Docket No.56113-0510WO1 When a mismatch or failure occurs in layer 480, the system can backtrack to cell 473 and select another input tile size and resume the operations. However, if a mismatch or failure occurs in layer 465 before traversing to layer 470, the system will revert back to layer 420 and obtain a different tile size from box 423. Note the system will add the left branch of layer 490 back into the unexplored set. If the system determines that all possible input tile sizes for a cell have been tried and failed, the system can pop out the cell from the backtracking stack 405. For example, if tile sizes 50 and 51 of cell 473 have failed, the system can pop out cell 473 from the backtracking stack 405. The previously second top cell (e.g., cell 423) now becomes the top cell in the backtracking stack 405. In this case, the system will again add the left branch of layer 490 in the unexplored set. The system selects another possible value in the current top cell and resumes corresponding backward and forward projections, and re- calculates a new set of possible input tile sizes for layer 470 based on the newly-selected input tile size for layer 420. The re-calculated possible input tile sizes are then stored in a new cell stacked on top of cell 473. Using the above-described backtracking stack and the corresponding mechanism, the system can efficiently explore the solution space to find a solution (e.g., a set of tile sizes) for the multi-input, multi-output FCN model according to different requirements. Alignment To determine alignment information, the system can determine an alignment value for each input layer of multiple input layers (e.g., layers 301 and 303) such that the alignment value is compatible with each output from corresponding output layers 351, 355, and 359. Alignment information is important to generate reasonable tiling patterns when an FCN includes one or more transposed convolution layers. A transposed convolution layer, in general, is configured to increase the special resolution of an input to generate an output that has a larger size. The ratio of increase is based on the stride size of the transpose convolution layer. For example, FCN 300 can include a transposed convolution layer appended to a convolution layer with a stride size of 2. The transposed convolution layer can produce an output of 51 by 51 pixels by processing a 24 by 24 pixel output from the convolution layer. Since the stride is 2, adding one pixel to the input adds two pixels to the output of the transposed convolution layer. Due to the stride of 2, an output tile for the transposed convolution layer must be aligned to the final output of the output layer Attorney Docket No.56113-0510WO1 since an in-between location of the output tile does not correspond to values produced by the transposed convolution layer from the input tile. In some implementations, such alignment value can be multiples-of-2 (regarding the top-left coordinate of the output tile). The system can obtain alignment information for fixed-size outputs according to the computation requirements set forth by transposed convolution layers in FCN 300. For example, the requirement can include that indices for one or more pixels in the fixed-size inputs and corresponding fixed-size outputs should be integers. One example process performed by the system for the multi-input, multi-output FCN 300 is described below. In general, the system can compute, for each output layer of multiple output layers, an output tile alignment value ^^^ைfor the ^^௧^output from the ^^௧^output layer. Different from a single-input multi-output FCN model, operations for obtaining alignment information for a multi-input, multi-output FCN model need to compute theoutput tile alignment values ^^ ை^ for the ^^௧^ output from all possible paths arriving at the^^௧^ output layer from corresponding input layer(s). In addition, the operations furtherinclude computing common input alignments using multiple input layers since, generally, a single input layer may reach only a subset of output layers. For ease of illustration, an example alignment algorithm for a multi-input, multi- output FCN model extended from a single-input multi-output FCN is described immediately below, followed by an example alignment algorithm for a single-input multi- output FCN model. First, the system performs a depth-first-search by projecting backward along paths or branches from each output layer of multiple output layers. This way, the system can reach every input layer that is connected to the output layer. The algorithm can be similar to the one used for single-input multiple-output FCN model. However, in some cases, the algorithm now takes into consideration the contributions to the output tile alignment values ^^^ைfrom all input layers that are connected to the output layer. More specifically, for each input layer j of multiple input layers, the system can implement an example alignment algorithm for a single-input multi-output FCN to generate a common input alignment value ^^^ூ^using all output layers that are reachable from the input layer j. The details of an example alignment algorithm and how a common input alignment value is determined are described below. Attorney Docket No.56113-0510WO1 Next, the system projects forward the common input alignment value ^^^ூ^from the input layer j to all reachable output layers, such that for the ^^௧^output layer, a corresponding alignment value ^^ை^,^^is computed. Since more than one input layer can reach the same output layer to the topology of the FCN model, one output layer of multiple output layers more than one alignment value e.g., for example, layer 355 might have a first alignment value ^^ைଷହ^ହ,ଷ^^obtained from the input layer 301 along the left branch, and a second alignment value ^^ைଷହ^ହ,ଷ^ଷobtained from the input layer 303 along the right branch. The system determines, for those output layers having more than one alignment value that is obtained along branches coupled to different input layers, whether these alignment values match with each other. In response to determining that these alignment values at the ^^௧^output layer from different input values do not match each other, the system can calculate a reconciled output alignment value for the ^^௧^output layer. For example, the system can determine a least common multiple of all alignment values determined for the ^^௧^output layer reached by all possible input layers. The system then repeatedly projects backward the reconciled alignment value from the ^^௧^output layer to respective input layers to update the common input alignment values for each input layer. This is because an input layer can be connected with a subset of output layers. After updating the common input alignment values for each input layer, the system projects forward the updated common input alignment values to respective output layers. The system iteratively performs the above-noted operations until a stopping point. The stopping point can be represented by a threshold number of iterations. Note that since the described algorithm is based on the least common multiples (LCMs), the described techniques theoretically can always find the alignment value suitable for a particular multi-input, multi-output FCN model. We refer to the final alignment valueobtained at the stopping point as ^^ ைி^ for the kth output layer. This final alignment value^^ ைி^ is globally consistent with the alignment requirements of all model output layers.One example algorithm for obtaining output tile alignment values can be based on at least one of the local search or analytical methods. The output tile alignment values generally represent coordinate shifts for shifting the “unaligned” fixed-size output tiles leftward and upward in the corresponding outputs. An example AlignOutputTile() algorithm for obtaining output tile alignment values are presented as follows: Attorney Docket No.56113-0510WO1 function AlignOutputTile( (ht, wt, hb, wb), layers): if approach == “local search”: for (hs, ws) = try all values in some pattern from 0 to max_shift: try: return ProjectBackward( (ht - hs, wt-ws, hb-hs, wb-ws), layers ) except: / / Failed to project, keep trying with other shift values. THROW EXCEPTION / / failed to find valid alignment for tile else if approach == “analytical”: hts = int (ht / alignment) * alignment wts = int (wt / alignment) * alignment hbs =hb - (ht - hts) wbs =wb - (wt - wts) return (hts, wts, hbs, wbs) where: alignment = CalculateAnalyticalAlignment(layers) One example CalculateAnalyticalAlignment() algorithm called by the AlignOutputTile () algorithm is presented as follows: function CalculateAnalyticalAlignment( layers ): / / Algo to find the smallest correct alignment: presence of conv layers before trans_conv / / layers eases the alignment needed by the trans_conv layers. conv_stride_product = 1 / / product of strides of back-to-back conv layers trans_conv_stride_product = 1 / / product of strides of back-to-back trans conv layers alignment = 1 / / required tile alignment at the FCN output layer for layer = output to input layers: if layer type is “conv”: s = stride of layer conv_stride_product *= s else if layer_type is “trans_conv”: s = stride of layer trans_conv_stride_product *= s prev_layer = previous layer / / prev_layer produces the input for layer / / prev_layer == null if layer is the input layer for the whole FCN. if prev_layer == null OR prev_layer type != “trans_conv”: / / utilize the subsequent stack of conv layers to ease the / / alignment requirement imposed by a stack of trans_conv layers by using the greatest common divisor gcd = GCD (conv_stride_product, trans_conv_stride_product) alignment_for_stack = trans_conv_stride_product / gcd alignment *= alignment_for_stack / / Reset stacks conv_stride_product, trans_conv_stride_product = 1, 1 return alignment As shown in the example CalculateAnalyticalAlignment()” algorithm, the system determines a constant alignment value based on the characteristics of each layer of the FCN model. For example, the characteristics can be a layer type (e.g., convolutional, transpose convolution layer, or other layer such as pooling layer), or a size for padding, Attorney Docket No.56113-0510WO1 filter, and stride for the layer. In some implementations, other types of layers in the FCN model, e.g., pooling layers, are treated as a convolution layer or transposed convolution layer throughout the specification. Additional types of layers are also supported with algorithms or steps similar to those used for processing convolution layers and transposed convolution layers. The system can coordinate the arrangement of output tiles by ensuring valid pixels are generated at each coordinate in each output. For example, for each output in the multiple outputs other than the driver, the system can determine that an overlapping pattern for the input tiles can ensure that at least one of the output tiles for the output has a valid pixel value that corresponds to a pixel value in the final output of the output layer. If the forward-projected output tiles for an output ^^ has a valid region that does not abut or overlap with the valid region of a neighboring output tile to the left or the top, thesystem can shift the current output tile left or up to a pixel that aligns with the value of^^ ைி^ for the output layer k. In other words, the pixel coordinate of the top left corner ofthe current output tile within output k is shifted to become a multiple of the value of^^ ைி^ .The value of ^^^ைிis obtained for each of output layer k. To obtain the final alignment value ^^^ைி, the system checks whether the intermediate values ^^^,^ை^are required to be updated across different input and output layers. For example, the system projects backward the alignment value ^^^,^ை^for each output layer i that can reach input layer j along respective paths. The system then updates the common alignment values ^^^,^ூ^at input layer j, if there is any mismatched value at input layer j from different output layers i. The system then projects forward the updated input alignments ^^^,^ூ^to all output layers i that are reachable by input layer j. Upon the completion of the projection forward, the system updates the common alignments ^^^,^ை^for the output layer i. The system can repeatedly perform the above-noted process for all input layers j and output layers i until a stopping point is reached, e.g., finding the values of ^^^ைிfor output layers k that are consistent for all input layers and output layers. After determining the tiling pattern, the system can compile FCN 300 on one or more hardware accelerators, and perform inference operations for processing inputs with different sizes. Coordinated Tiling Pattern Attorney Docket No.56113-0510WO1 The tiling pattern algorithm is substantially similar to that for a single-input multi- output FCN. In general, the system determines a plurality of input tiles for each input according to the respective tiling pattern determined for the input layer (e.g., offset values, alignment values, etc.). The system processes the input tiles using corresponding input layers simultaneously to generate multiple outputs from multiple output layers. In general, each output layer generates a group of output tiles for corresponding inputs coupled to the output layer. Each of the output tiles includes accurate regions and inaccurate regions. The system is configured to combine accurate regions from the group of output tiles to generate an output for the output layer. More details related to accurate regions and inaccurate regions are described in connection with FIG.5. That said, the algorithm described in this document for coordinating the tiling pattern is modified to perform additional operations. For example, the algorithm further performs operations to check inconsistency due to a change in the initial projecting layer. When all outputs reachable from the driver output layer can be properly tiled, the system chooses a different output layer that is coupled to additional input layers as the new driver output layer and repeats the tiling pattern algorithm. If the system determines any inconsistency or failure in tiling the inputs, the system restarts the entire operations as described above starting from the new driver output layer. The consistency check operations terminate when all inputs and outputs can be properly and consistently tiled. Note that only forward and backward projections are associated with the tiling pattern. Since candidate tile sizes for inputs and outputs have been validated as described above, now the described techniques need to “coordinate shift” the driver output tile to have the minimum overlap of valid regions with the previous driver output tile, subject to alignment constraint at the driver output ^^^ைி. The described algorithm projects the shifted tile backward to a jth input layer, and then projects it forward to other output layers reachable from the jth input layer. If the valid regions of the previous tile and current tile do not overlap for an output layer k, the described techniques choose output layer k as the new driver output and restart the above-described process from the new driver output. Once tiling pattern operations succeed, the described techniques project the tile to another input layer repeatedly until finding a driver output layer that allows a consistent tiling pattern for all input layers and output layers of the FCN model. Attorney Docket No.56113-0510WO1 Additional descriptions of example algorithms for coordinating tiling patterns are described in International Application No. PCT / US2023 / 012634, which is accordingly incorporated by reference herein. FIG.5 illustrates an example fixed-size input 538 with a neighbor pixel region 510 and an example fixed-size output 533 with a dummy region 520. The fixed-size input 538 is equivalent to the fixed-size input 138 of FIG.1, and the fixed-size output 533 is equivalent to the fixed-size output 133 of FIG.1. As described above, the system can determine multiple input tiles 538 with the fixed-size after tiling a full input 550. The fixed-size input 538 can be surrounded by pixels in the neighboring pixel region 510. The term “neighbor pixel region 510” represents a region including neighbor pixels generated by replacing non-zero values in the original input with zero values. For example, the neighbor pixel region 510 can include one or more pixels in the full input data (e.g., 150 of FIG.1) surrounding the fixed-size input 538. The width 515 of the neighbor pixel region 510 can represent a number of pixels included in the neighbor pixel region 510. The system can determine the width 515 for the neighbor pixel region 510 based on the characteristics of FCN 115. The system can, as described above, generate multiple fixed-size outputs 533 for corresponding fixed-size inputs 538 using the statically-compiled FCN model 115. Due to the zero pixel values in the neighboring pixel region 510, the fixed-size outputs 533 can include pixel-wise values that are inaccurate. The system can determine, for each fixed- size output 533, a valid region 530 and a dummy region 520. In general, the valid region 530 can be located at the center of the fixed-size output 533, and the dummy region 520 can surround the periphery of the valid region 530 at a width of 535. The width 535 can represent a particular number of pixels that are invalid. The width 535 can be determined using, for example, the example FirstValidPixelOffset() algorithm as described above. The valid region 530 includes pixel-wise values for pixels computed using the valid pixel-wise values in the corresponding fixed-size input (e.g., 138 of FIG.1), and the dummy region 520 includes pixel-wise values computed using at least one or more neighbor pixels 510. The pixel-wise values in the valid region 530 contribute at least a portion to the final output 170, while the dummy pixels will be eliminated or discarded during the stitching process. The system can ensure each pixel value in one of the outputs (e.g., a full output 570) should be generated from at least one output tile for the output. To determine the coordinates of an output tile (e.g., fixed-size output 533) and a valid region 530 of the fixed-size output 533, the system can perform different Attorney Docket No.56113-0510WO1 algorithms. For two-dimensional data such as two-dimensional images, the system can use indices i and j to indicate grid positions of the fixed-size input tiles and corresponding fixed-size output tiles over the final inputs and final outputs of the multi-input, multi- output FCN model. Note that the same (i, j) indices identify corresponding fixed-size tiles over all model final inputs and final outputs. One example algorithm that considers the above-noted alignment requirement is presented below: Initialize:^^^0, 0^ = ^0, 0,^^ை ,^^ை^; ^^^0, 0^ = ^0, 0,^^ை − ^^,^^ை − ^^^Left boundary tiles:^^ை = ^ ^^^^^ − 1, 0^. ℎ^^ − ^^, 0,^^^^^ − 1, 0^. ℎ^^ − ^^ + ^^ை ,^^ை^^^^^^, 0^= ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^ை ^^^^^^, 0^= ^^^^^^, 0^. ℎ^^ + ^^, 0,^^^^^, 0^. ℎ^^ − ^^,^^ை − ^^^Top boundary tiles:^^ை = ^ 0,^^^0, ^^ − 1^.^^^^ − ^^,^^ை ,^^^0, ^^ − 1^.^^^^ − ^^ + ^^ை^^^^0, ^^^= ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^ை ^^^^0, ^^^= ^0,^^^0, ^^^.^^^^ + ^^,^^ை − ^^,^^^0, ^^^.^^^^ − ^^^Internal tiles:^^ை = ^ ^^^^^ − 1, ^^^. ℎ^^ − ^^,^^^^^, ^^ − 1^.^^^^ − ^^,^^^^^ − 1, ^^^. ℎ^^ − ^^ + ^^ை,^^^^^, ^^ − 1^.^^^^ − ^^ + ^^ை^^^^^^, ^^^ = ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^ை ^^^^^^, ^^^^^^ 520 as ^^ (which can be obtained from, for example, the example FirstValidPixelOffset()algorithm as above-described), and mapping functions ^^^^^, ^^^ = (ℎ^^ூ ூ ூ ூ^ ,^^^^^ , ℎ^^^ ,^^^^^),^^(^^, ^^) = (ℎ^^ை,^^^^ை ,ℎ^^ை,^^^^ை), and ^^( ) ^ ^ ^ ^^ ^ ^ ^ ^^, ^^ = (ℎ^^^ ,^^^^^ , ℎ^^^ ,^^^^^ ) for coordinates of a of the fixed-size output, respectively. The fixed-size output has both a dummy region and valid regionwith the left and top dummy regions being omitted, denoted as ^^^^. A fixed-size output^^(^^, ^^) is calculated using the above-noted ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^() algorithm for acorresponding ^^^^. Note the term ^^^^ represents fixed-size output coordinates notsubjected to the above-described alignment constraints, and the term ^^(^^, ^^) representsoutput tile coordinates after considering the alignment constraints. Each mapping functioncan return a particular coordinate in a particular direction (e.g., ^^(^^, ^^). ℎ^^ =ℎ^^ூ represents a coordinate in a vertical or height direction). the100 assumes the fixed-size inputs and fixed-size outputs are squares or rectangles in two- dimensional space, and denotes the size of the fixed-size inputs as ^^ூ, and the fixed-size outputs as ^^ை. The system 100 denotes the size for the initial input data as ^^ூand ^^ூ, andwithout losing generality, it is assumed that ^^ூ >= ^^ூ and ^^ூ >= ^^ூ. Attorney Docket No.56113-0510WO1 After computing all the fixed-size outputs for each output of the multiple outputsthrough the multi-input, multi-output FCN model, the system can apply the ^^(^^, ^^)and^^(^^, ^^) mappings to construct a full output for each output layer as if the input wereentirely processed by the multi-input, multi-output FCN model. An example StitchOutputImage() algorithm for “stitching” fixed-size outputs for an output is presented below: function StitchOutputImage(): for tile indices (i,j)in a top-to-bottom, left-to-right scan of the tiles: (ht_O, wt_O, hb_O, wb_O) = O(i,j) (ht_V, wt_V, hb_V, wb_V) = V(i,j) Output(ht_V:hb_V, wt_V:wb_V) = OutputTile( (ht_V-ht_O):(hb_V-ht_O), (wt_O- wt_V):(wb_O-wt_V) ) The ^^^^^^^^^^^^^^^^^^^^(^^, ^^) represents the fixed-size output of size ^^^ corresponding tothe (^^, ^^)௧^ fixed-size input. For example, a fixed-size input in the ^^௧^ column and ^^௧^ rowof a tiling grid. FIG.6 illustrates an example process 600 for performing inference computations in a multi-input, multi-output FCN for inputs with different sizes. For convenience, the process 600 is described as being performed by a system of one or more computers located in one or more locations. For example, an inference system, e.g., the system 100 of FIG.1, appropriately programmed, can perform the process 600. The system receives two or more new inputs for an invocation to be processed by an FCN deployed on a hardware accelerator (610). The FCN is a multi-input multi output FCN configured to process multi-inputs for each invocation of multiple invocations and generate multiple outputs for each of the corresponding invocations. Here, each of two or more new inputs received at the invocation has a respective size such that at least one of the respective sizes is different from a fixed size that the FCN is configured to process using a corresponding input layer when deployed on the hardware accelerator. In general, the multi-input, multi-output FCN includes multiple network layers. For example, the FCN includes two or more input layers each for receiving and processing one of the two or more new inputs received for an invocation. The FCN also includes multiple output layers. Each of the multiple output layers can be configured to generate a respective output of the invocation for inputs processed by corresponding input layers that are connected to the output layer. The multi-input, multi-output FCN can include a tree structure or branch point where the output of the branch point is processed by two or more succeeding layers in the Attorney Docket No.56113-0510WO1 FCN. The multi-input, multi-output FCN can further include a re-convergence point in a DAG structure, where two or more outputs from preceding layers are received as input at the re-convergence point. In addition, one or more output layers receive output from two or more branches, and thus are also referred to as convergence points. For example, one or more convergence points (e.g., output layers) can correspond to two of the two or more input layers through the topology of the FCN. Each of the two corresponding input layers is configured to process one of the two or more new inputs. As described above, the techniques described in this document do not have to distinguish between a tree structure and a DAG structure. Nor do the described techniques have to distinguish between a convergence point and a convergence point. More details are described above and in International Application No. PCT / US2023 / 012634 incorporated by reference herein. For each of the two or more new inputs at an invocation, the system determines a group of fixed-size input tiles for the new input (620). Each fixed-size input tile has the fixed size that the deployed FCN model is natively capable of processing. As described above, the system can determine the group of fixed-size input tiles by “tiling and stitching data” obtained following the operations described above. The “tiling and stitching data” represent tiling and stitching operations for processing any given input(s) at an invocation using the deployed multi-input, multi-output FCN model. The system incorporates the “tiling and stitching data” with the deployed multi-input, multi-output FCN to process inputs with varying sizes at each of multiple invocations. To determine fixed-size input tiles for the two or more new inputs, the system can perform operations offline or at a “deployment” stage prior to the runtime of processing the two or more new inputs. More specifically, the system determines a first pixel offset value for the deployed FCN. The system can manipulate the FCN by adding a dummy node and corresponding edges connecting each of the two or more input layers. The operations for obtaining the first pixel offset value for the modified FCN are similar to that for a single-input multi-output FCN. In general, the system computes, for each of the multiple output layers, a candidate pixel offset value traversing each path from the output layer to the dummy node according to the deployed fully convolutional neural network. The system ranks these candidate pixel offset values and sets the first pixel offset value to the maximum of the candidate pixel offset values. More details related to obtaining the first pixel offset value are described above. In addition, the system, at the “deployment” stage or offline, determines the fixed size for the fully convolutional neural network by validating one or more tile sizes using Attorney Docket No.56113-0510WO1 backward projections and forward projections. More specifically, the system selects an output layer from the multiple output layers as a driver output layer. The system then projects backward from the output layer to one of the two or more input layers. To project backward, the system first selects one branch that has not been traversed at a layer where two or more branches converge. The system projects backward from the selected branch at the layer and adds the rest of two or more branches that are unselected in an unexplored set. After reaching one of the two or more input layers along the path during the backward projection, the system then projects forward from the one of the two or more input layers to a corresponding subset of output layers. After successfully completing the projection backward and forward from the selected branch, the system starts to project backward along another branch from branches stored in the unexplored set and remove the other branch from the unexplored set. The system can repeatedly perform the projection operations along different branches from the unexplored set by selecting and removing branches from the unexplored set. The validation process completes when the unexplored set becomes empty. More details related to the validation process are described above. While projecting backward in a path in the FCN, the system can perform a backtracking search step for each convolution layer having a stride value greater than one in the path. Note that when the backtracking step triggers and the immediately preceding cell corresponds to a layer in a previous branch of a convergence point, the system adds back the current branch into the unexplored set. When the system determines that an input tile size of a particular layer is incompatible with one of the multiple output layers, the system can backtrack and obtain another input tile size stored in a backtracking stack. More specifically, the system accesses a set of possible input tile sizes stored in a first cell in the backtracking stack. The backtracking stack is generated during a backward projection process and includes multiple cells stacked on one another according to a sequence defined by a path in the backward projection. Each cell is associated with a respective layer and includes a respective set of possible input tile sizes for the respective layer. The first cell is the top cell in the backtracking stack. The system then selects, from the first cell, one of the set of possible input tile sizes that is different from the input tile size of the particular layer as the other input tile size. The system resumes the projection operations by projecting forward or backward from the corresponding layer according to the path using the other input tile size. More details related to the backtracking operations are described above. Attorney Docket No.56113-0510WO1 To determine groups of fixed-size inputs, the system further performs operations, at the “deployment” stage or offline, to determine alignment information for the FCN according to the characteristics of the FCN. More specifically, for each of the two or more input layers, the system determines a common input alignment value for the input layer by traversing from a subset of output layers connected to the input layer to the input layer. The system then determines a common output alignment value for each of the subset of output layers by projecting forward from the input to the subset of output layers using the determined common input alignment value. For each of the multiple output layers associated with different common output alignment values, the system updates the common output alignment value for the output layer based on the different output alignment values. For each of the two or more input layers, the system updates the common input alignment value for the input layer based at least on the updated common output alignments by traversing from the subset of output layers connected to the input layer to the input layer. The system further updates the common output alignment value for each of the subset of output layers by projecting forward from the input to the subset of the output layers using the updated common input alignment value. To update the common output alignment value, the system can determine the least common multiple of the different output alignment values for the output layer and set the updated common output alignment value to the least common multiple for the output layer. The system can repeatedly perform the above-noted operations until a stopping point. As an example, the stopping point can relate to one iteration where output layers associated with multiple input layers do not have unmatched common output alignment values. After obtaining and incorporating the “tiling and stitching data” with the deployed FCN model, the system provides each of the groups of fixed-size input tiles to the hardware accelerator for performing inference computations to generate respective outputs using the deployed FCN (630). To generate the respective outputs for the invocation, the system can generate, for each of the multiple output layers, respective fixed-size output tiles for corresponding fixed-size input tiles processed by at least one of the two input layers that are connected with the output layer according to the deployed FCN. The respective fixed-size outputs comprise one or more inaccurate pixel-wise values. More specifically, each of the respective fixed-size outputs comprises a central Attorney Docket No.56113-0510WO1 valid region, and a peripheral dummy region at a width of a first number of pixels. The central valid region includes at least a portion of the final output, The peripheral dummy region comprises one or more inaccurate pixel-wise values. More details regarding the inaccurate pixels or regions are described above. The system generates, for each output layer of the multiple output layers and from the respective fixed-size outputs, a respective final output of the invocation for the output layer (640). Each respective final output is equivalent to an output that would be generated from the output layer by processing corresponding inputs of the two or more new inputs through layers of the two or more input layers that correspond to the output layer using the FCN configured for processing the corresponding inputs of the two or more new inputs with the respective sizes. In some implementations, the fixed size for compiling the multi-input, multi- output FCN does not have to be a scalar. Instead, the fixed size can be a vector representing a rectangle in two-dimensional space, or a block in three-dimensional space. More specifically, the fixed size can include a respective value in a respective dimension. For example, if the input image is two-dimensional, the system can determine a fixed size vector with a first size for the first dimension (e.g., horizontal dimension) and a second size for a second dimension (e.g., vertical dimension) different from the first dimension. The system can generate multiple fixed-size inputs of 30 by 10 pixels from an input image with a size of 300 by 100 pixels. In addition, the multi-input, multi-output FCN can be adapted to process each of the multiple dimensions of an input as long as the dimension is fully convolutional. Forexample, a multi-input, multi-output FCN can process an image input with^^ × ^^ × ^^ × ^^ dimensions to generate multiple outputs. Assuming that the batchdimension and channel dimension are not fully convolutional, the multi-input, multi- output FCN can process the input only in the height and width dimensions, where the process can be generally considered a two-dimension problem. As another example, a multi-input, multi-output FCN can process an audio input with multiple dimensions by processing only a single dimension of the audio input if the rest of the dimensions are not fully convolutional. Alternatively, the multi-input, multi-output FCN model can process higher dimensions, e.g., higher than two dimensions, if these dimensions are fully convolutional. Implementations of the subject matter and the actions and operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied Attorney Docket No.56113-0510WO1 computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer programs, e.g., one or more modules of computer program instructions, encoded on a computer program carrier, for execution by, or to control the operation of, data processing apparatus. The carrier may be a tangible non- transitory computer storage medium. Alternatively or in addition, the carrier may be an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be or be part of a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. A computer storage medium is not a propagated signal. The term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. Data processing apparatus can include special-purpose logic circuitry, e.g., an FPGA (field programmable gate array), an ASIC (application-specific integrated circuit), or a GPU (graphics processing unit). The apparatus can also include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A computer program, which may also be referred to or described as a program, software, a software application, an app, a module, a software module, an engine, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program or as a module, component, engine, subroutine, or other unit suitable for executing in a computing environment, which environment may include one or more computers interconnected by a data communication network in one or more locations. A computer program may, but need not, correspond to a file in a file system. A computer program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated Attorney Docket No.56113-0510WO1 to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code. The processes and logic flows described in this specification can be performed by one or more computers executing one or more computer programs to perform operations by operating on input data and generating output. The processes and logic flows can also be performed by special-purpose logic circuitry, e.g., an FPGA, an ASIC, or a GPU, or by a combination of special-purpose logic circuitry and one or more programmed computers. Computers suitable for the execution of a computer program can be based on general or special-purpose microprocessors or both, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special-purpose logic circuitry. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to one or more mass storage devices. The mass storage devices can be, for example, magnetic, magneto-optical, or optical disks, or solid state drives. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few. To provide for interaction with a user, implementations of the subject matter described in this specification can be implemented on, or configured to communicate with, a computer having a display device, e.g., a LCD (liquid crystal display) monitor, for displaying information to the user, and an input device by which the user can provide input to the computer, e.g., a keyboard and a pointing device, e.g., a mouse, a trackball or touchpad. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser, or by interacting Attorney Docket No.56113-0510WO1 with an app running on a user device, e.g., a smartphone or electronic tablet. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone that is running a messaging application, and receiving responsive messages from the user in return. This specification uses the term “configured to” in connection with systems, apparatus, and computer program components. For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. For one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by a data processing apparatus, cause the apparatus to perform the operations or actions. For special-purpose logic circuitry to be configured to perform particular operations or actions means that the circuitry has electronic logic that performs the operations or actions. Implementations of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet. The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some implementations, a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, e.g., a result of the user interaction, can be received at the server from the device. While this specification contains many specific implementation details, these should not be construed as limitations on the scope of what is being or may be claimed, Attorney Docket No.56113-0510WO1 but rather as descriptions of features that may be specific to particular implementations of particular inventions. Certain features that are described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially be claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claim may be directed to a subcombination or variation of a subcombination. In addition to the embodiments described above, the following embodiments are also innovative: Embodiment 1 is a method performed by one or more computers to implement a neural network model for processing multiple inputs at an invocation to generate multiple outputs, the method comprising: receiving two or more new inputs to be processed at the invocation by a fully convolutional network deployed on a hardware accelerator, wherein each of two or more new inputs has a respective size such that at least one of the respective sizes is different from a fixed size that the fully convolutional network is configured to process when deployed on the hardware accelerator, wherein the fully convolutional network comprises a plurality of network layers, wherein the plurality of network layers comprise two or more input layers each for one of the two or more new inputs received at the invocation and a plurality of output layers, and wherein each of the plurality of output layers are configured to generate a respective output of the invocation for corresponding inputs received by corresponding input layers that are connected to the output layer; for each of the two or more new inputs received at the invocation, determining a group of fixed-size input tiles for the new input, each fixed-size input tile having the fixed size; providing each of the groups of fixed-size input tiles to the hardware accelerator for performing inference computations to generate respective outputs of the invocation using the deployed fully convolutional network, comprising: for each of the plurality of output layers, generating respective fixed-size output tiles for corresponding fixed-size input tiles processed by at least one of the two input layers that are connected with the output layer according to the deployed fully convolutional network, wherein the respective fixed-size outputs comprise one or more inaccurate pixel- wise values; and for each output layer of the plurality of output layers, generating, from the respective fixed-size outputs, a respective final output of the invocation for the output Attorney Docket No.56113-0510WO1 layer that is equivalent to an output that would be generated from the output layer by processing corresponding inputs of the two or more new inputs at the invocation through layers of the two or more input layers that correspond to the output layer using the fully convolutional network deployed for processing the corresponding inputs with the respective sizes. Embodiment 2 is the method of Embodiment 1, where the fully convolutional network comprises at least one convergence layer that corresponds to two of the two or more input layers, each configured to process one of the two or more new inputs. Embodiment 3 is the method of Embodiment 1 or 2, further comprising: determining a first pixel offset value for the deployed FCN, wherein determining the first pixel offset comprises: generating a dummy node connecting each of the two or more input layers, for each of the plurality of output layers, computing a candidate pixel offset value traversing each path from the output layer to the dummy node according to the deployed fully convolutional network; and setting the first pixel offset value to the maximum of the candidate pixel offset values. Embodiment 3 is the method of any of Embodiment 1-3, further comprising: determining the fixed size for the fully convolutional network, wherein determining the fixed size comprises: selecting an output layer from the plurality of output layers as a driver output layer, projecting backward from the output layer to one of the two or more input layers, comprising: selecting one branch that has not been traversed at a layer, wherein two or more branches converge at the layer; projecting backward from the selected branch at the layer, and adding the rest of two or more branches in an unexplored set; projecting forward from the one of the two or more input layers to a corresponding subset of output layers, and removing one or more branches that have been traversed during projecting forward from the unexplored set. Embodiment 5 is the method of Embodiment 4, further comprising: after successfully projecting backward for the driver output layer to reach the one of the two or more input layers, repeatedly selecting and removing a branch from the unexplored set and projecting backward from the selected branch until the unexplored set is empty. Embodiment 6 is the method of Embodiment 5, further comprising: during projecting backward in a path in the fully convolutional network, performing a backtracking search step for each convolution layer having a stride value greater than one in the path. Attorney Docket No.56113-0510WO1 Embodiment 7 is the method of any Embodiment 1-6, wherein determining the groups of fixed-size input tiles comprises determining alignment information for the fully convolutional network, wherein determining the alignment information comprises: for each of the two or more input layers, determining a common input alignment value for the input layer by traversing from a subset of output layers connected to the input layer to the input layer; and determine a common output alignment value for each of the subset of output layers by projecting forward from the input to the subset of output layers using the determined common input alignment value; for each of the plurality of output layers associated with different common output alignment values, updating the common output alignment value for the output layer based on the different output alignment values; and for each of the two or more input layers, updating the common input alignment value for the input layer based at least on the updated common output alignments by traversing from the subset of output layers connected to the input layer to the input layer; and updating the common output alignment value for each of the subset of output layers by projecting forward from the input to the subset of the output layers using the updated common input alignment value. Embodiment 8 is the method of Embodiment 7, wherein updating the common output alignment value for the output layer comprises: determining a least common multiple of the different output alignment values for the output layer, and setting the updated common output alignment value to the least common multiple for the output layer. Embodiment 9 is the method of any Embodiment 1-8, further comprising: determining that an input tile size of a particular layer is incompatible with one of the plurality of output layers; in response, obtaining another input tile size stored in a backtracking stack, wherein the obtaining process comprises: accessing a set of possible input tile sizes stored in a first cell in the backtracking stack, wherein the backtracking stack is generated during a backward projection process and includes multiple cells stacked on one another according to a sequence defined by a path in the backward projection, wherein each cell is associated with a respective layer and includes a respective set of possible input tile sizes for the respective layer, wherein the first cell is the top cell in the backtracking stack; and selecting, from the first cell, one of the set of possible input tile sizes that is different from the input tile size of the particular layer as the other input tile size; and projecting forward or backward from the corresponding layer according to the path using the other input tile size. Attorney Docket No.56113-0510WO1 Embodiment 10 is the method of any Embodiment 1-9, wherein each of the respective fixed-size outputs comprises a central valid region, and a peripheral dummy region at a width of a first number of pixels, wherein the central valid region includes at least a portion of the final output, wherein the peripheral dummy region comprises one or more inaccurate pixel-wise values. Embodiment 11 is a system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the method of any one of Embodiments 1 to 10. Embodiment 12 is one or more computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations of the method of any one of Embodiments 1 to 10. Similarly, while operations are depicted in the drawings and recited in the claims in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system modules and components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products. Particular implementations of the subject matter have been described. Other implementations are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. What is claimed is:

Claims

Attorney Docket No.56113-0510WO1 CLAIMS 1. A method for processing multiple inputs at an invocation to generate multiple outputs using a fully convolutional neural network implemented on a hardware accelerator, the method comprising: receiving two or more new inputs to be processed by the fully convolutional network at the invocation, wherein each of two or more new inputs has a respective size such that at least one of the respective sizes is different from a fixed size that the fully convolutional network is configured to process, for each of the two or more new inputs received at the invocation, determining a group of fixed-size input tiles for the new input, each fixed-size input tile having the fixed size; providing each of the groups of fixed-size input tiles to the hardware accelerator; in response to providing, generating respective outputs of the invocation, comprising: for each of a plurality of output layers of the fully convolutional network, generating respective fixed-size output tiles for corresponding fixed-size input tiles processed by at least one of two or more input layers of the fully convolutional network that are connected with the output layer; and for each of the plurality of output layers, generating, based on the respective fixed-size output tiles, a respective final output of the invocation for the output layer.

2. The method of claim 1, wherein generating a respective final output comprises: generating a respective final output that is equivalent to an output that would otherwise be generated from the output layer if at least one input that corresponds to the output layer was sized consistent with the fixed size that the fully convolutional network is configured to process.

3. The method of claim 1, where the fully convolutional network comprises at least one convergence layer that corresponds to two of the two or more input layers, each configured to process one of the two or more new inputs.

4. The method of claim 1, further comprising: determining a first pixel offset value for the fully convolutional network, wherein determining the first pixel offset comprises: generating a dummy node connecting each of the two or more input layers,Attorney Docket No.56113-0510WO1 for each of the plurality of output layers, computing a candidate pixel offset value traversing each path from the output layer to the dummy node according to the fully convolutional network; and setting the first pixel offset value to the maximum of the candidate pixel offset values.

5. The method of claim 1, further comprising: determining the fixed size that the fully convolutional network is configured to process, wherein determining the fixed size comprises: selecting an output layer from the plurality of output layers as a driver output layer, projecting backward from the output layer to one of the two or more input layers, comprising: selecting one branch that has not been traversed at a layer, wherein two or more branches converge at the layer; projecting backward from the selected branch at the layer, and adding the rest of two or more branches in an unexplored set; projecting forward from the one of the two or more input layers to a corresponding subset of output layers, and removing one or more branches that have been traversed during projecting forward from the unexplored set.

6. The method of claim 5, further comprising: after successfully projecting backward for the driver output layer to reach the one of the two or more input layers, repeatedly selecting and removing a branch from the unexplored set and projecting backward from the selected branch until the unexplored set is empty.

7. The method of claim 6, further comprising: during projecting backward in a path in the fully convolutional network, performing a backtracking search step for each convolution layer having a stride value greater than one in the path.

8. The method of claim 1, wherein determining the groups of fixed-size input tiles comprises determining alignment information for the fully convolutional network,Attorney Docket No.56113-0510WO1 comprising: for each of the two or more input layers, determining a common input alignment value for the input layer by traversing from a subset of output layers connected to the input layer; and determining a common output alignment value for each of the subset of output layers by projecting forward from the input to the subset of output layers using the determined common input alignment value.

9. The method of claim 8, wherein determining alignment information for the fully convolutional network comprises: for each of the plurality of output layers associated with different common output alignment values, updating the common output alignment value for the output layer based on the different output alignment values; and for each of the two or more input layers, updating the common input alignment value for the input layer based at least on the updated common output alignments by traversing from the subset of output layers connected to the input layer to the input layer; and updating the common output alignment value for each of the subset of output layers by projecting forward from the input to the subset of the output layers using the updated common input alignment value.

10. The method of claim 9, wherein updating the common output alignment value for the output layer comprises: determining a least common multiple of the different output alignment values for the output layer, and setting the updated common output alignment value to the least common multiple for the output layer.

11. The method of claim 1, further comprising: determining that an input tile size of a particular layer is incompatible with one of the plurality of output layers; in response, obtaining another input tile size stored in a backtracking stack, wherein the obtaining process comprises: accessing a set of possible input tile sizes stored in a first cell in theAttorney Docket No.56113-0510WO1 backtracking stack, wherein the backtracking stack is generated during a backward projection process and includes multiple cells stacked on one another according to a sequence defined by a path in the backward projection, wherein each cell is associated with a respective layer and includes a respective set of possible input tile sizes for the respective layer, wherein the first cell is the top cell in the backtracking stack; and selecting, from the first cell, one of the set of possible input tile sizes that is different from the input tile size of the particular layer as the other input tile size; and projecting forward or backward from the corresponding layer according to the path using the other input tile size.

12. The method of claim 1, wherein each of the respective fixed-size outputs comprises a central valid region, and a peripheral dummy region at a width of a first number of pixels, wherein the central valid region includes at least a portion of the final output, wherein the peripheral dummy region comprises one or more inaccurate pixel- wise values.

13. A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations of the method of any one of claims 1 to 10.

14. One or more computer storage media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations of the method of any one of claims 1 to 10.

Images