ERROR DETECTION IN OBJECT RECOGNITION INFERENCES USING NEURAL NETWORK-GENERATED TAGS

Abstract

Devices, systems, and techniques for detecting errors in content that are recognized by neural networks. In at least one embodiment, respective document transcriptions of one or more document images are generated using one or more neural networks. The respective document transcriptions can include document content and descriptive information of the document content. An error in the document content of at least one document transcription can be detected, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription.

Description

TECHNICAL AREA

[0001] At least one embodiment relates to processing resources used to perform and facilitate artificial intelligence for text identification. For example, at least one embodiment relates to processors or computing systems that use neural networks to identify text. BACKGROUND

[0002] Object recognition techniques are used to implement computer vision and other data processing. Generating the information required to perform object recognition techniques can consume significant storage, time, or computing resources. The amount of storage, time, or computing resources used to perform object recognition techniques can be improved. SUMMARY

[0003] The invention is defined by the claims. To illustrate the invention, aspects and embodiments that may or may not fall within the scope of the claims are described here.

[0004] Devices, systems, and techniques for detecting errors in content, which are recognized by neural networks, are disclosed. In at least one embodiment, respective document transcriptions of one or more document images are generated using one or more neural networks. The respective document transcriptions can include document content and descriptive information of the document content. An error in the document content of at least one document transcription can be detected, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription.

[0005] Further features of the disclosure are characterized by the independent and dependent claims.

[0006] Any feature of one aspect of the disclosure can be applied to other aspects of the disclosure in any suitable combination. In particular, procedural aspects can be applied to device or system aspects, and vice versa.

[0007] Furthermore, functions implemented in hardware can also be implemented in software, and vice versa. Any reference to software and hardware characteristics here should be interpreted accordingly.

[0008] Each system or device feature described here can also be provided as a process feature, and vice versa. System and / or device aspects that are described functionally (including resources plus functional features) can alternatively be expressed in terms of their corresponding structure, such as a suitably programmed processor and associated memory.

[0009] It should also be noted that certain combinations of the various features described and defined in all aspects of the revelation can be implemented and / or delivered and / or used independently of one another.

[0010] The disclosure also provides computer programs and computer program products comprising software code which, when executed on a data processing device, is capable of performing any of the procedures described herein and / or embodying any of the device and system features described herein, including any or all component steps of a procedure.

[0011] The disclosure also provides a computer or computer system (including networked or distributed systems) with an operating system that supports a computer program for performing one of the procedures described herein and / or for embodying one of the device or system features described herein.

[0012] The revelation also provides a computer-readable medium on which one or more of the aforementioned computer programs are stored.

[0013] The revelation also provides a signal that carries one or more of the aforementioned computer programs.

[0014] The disclosure extends to methods and / or devices and / or systems as described herein with reference to the accompanying drawings.

[0015] Aspects and embodiments of the disclosure will now be described purely by way of example with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 illustrates an example of a logical block diagram of error detection in object recognition in neural networks based on derived descriptive information according to at least one embodiment; Fig. Figure 2 illustrates an example of a descriptive token evaluation as part of an error detection in object recognition in neural networks according to at least one embodiment; Fig. Figure 3 illustrates an example of a batch transcription system that implements error detection in object recognition in neural networks based on derived descriptive information according to at least one embodiment; Fig. 4 illustrates an example of a method or technique that implements error detection in object recognition in neural networks based on derived descriptive information according to at least one embodiment; Fig. 5 illustrates an example of a method or technique that implements a descriptive token evaluation as part of an error detection in object recognition in neural networks according to at least one embodiment; Fig. Figure 6 illustrates an exemplary data center system according to at least one embodiment; Fig. Figure 7 illustrates a system-on-a-chip (SOC) according to at least one embodiment; Fig. Figure 8 illustrates a parallel processor according to at least one embodiment; Fig. Figure 9 illustrates an accelerator processor according to at least one embodiment; Fig. Figure 10A illustrates a central processing unit and a core of the central processing unit according to at least one embodiment; Fig. 10B illustrates a core of the central processing unit in Fig. 10A according to at least one embodiment; Fig. Figure 11 illustrates a neuromorphic processor according to at least one embodiment; Fig. 12 illustrates a supercomputer according to at least one embodiment; Fig. Figure 13 illustrates another accelerator processor according to at least one embodiment; Fig. 14 illustrates another processor according to at least one embodiment; Fig. Figure 15 illustrates another accelerator processor according to at least one embodiment; Fig. Figure 16 illustrates a tensor processor according to at least one embodiment; Fig. Figure 17 illustrates a RISC-V compatible processor according to at least one embodiment; Fig. 18A and Fig. Figure 18B illustrates a speech processing unit according to at least one embodiment; Fig. 19 illustrates a software stack of a programming platform according to at least one embodiment; Fig. 20 illustrates software supported by a programming platform according to at least one embodiment; Fig. Figure 21 illustrates how to compile code for execution on programming platforms of Fig. 18 according to at least one embodiment; Fig. 22A illustrates an inference and / or training logic according to at least one embodiment; Fig. 22B illustrates the training and use of a neural network according to at least one embodiment; DETAILED DESCRIPTION

[0016] Neural networks, such as multimodal LLMs (MLLMs), used for document transcription, can take an input image of a document, analyze the text within the input image, and generate a corresponding text output. However, these models can introduce errors in document transcription, sometimes referred to as "hallucinations," by erroneously inserting text or other content that is not actually present in the original document. Current error detection methods assess the reliability of the transcribed text and other content using confidence scores generated by the neural network. If the confidence scores fall below a certain threshold, the transcription is identified as erroneous.For example, when the neural network chooses between different possible words to select the word represented in a portion of the input image, it can use a confidence score of the different possible words (e.g., the normalized logits of different words corresponding to different probabilities) to select the word with the highest (or near-highest) confidence score. These same confidence scores can be monitored to detect transcription errors. However, hallucinated text can still achieve a high confidence score, making it difficult to effectively detect and correct transcription errors.

[0017] Several implementations describe techniques for error detection in object recognition within neural networks using derived descriptive information. These techniques can detect errors in the document content of a document transcription generated by a neural network using additional descriptive information about the document content, which is also included in the document transcription (e.g., the position of the bounding box for a paragraph, diagram, or other part of a document, or content labels such as "paragraph" or "heading"). Because descriptive information can conform to strict syntax patterns (e.g., bracket pairs and other symbols required to represent bounding box coordinates) or features (e.g., there is a limited set of possible semantic classes for labeling content), errors in this information (e.g.,Incorrect or missing characters can be easily identified (e.g., using pattern matching techniques). Hallucination errors in one part of the neural network's output, such as the descriptive information, can correlate strongly with errors in another part of the neural network's output, namely the content of the document transcription. Therefore, errors in the descriptive information of a document transcription can accurately indicate the presence of errors in the text of the document transcription, thus improving the accuracy of hallucination detection compared to techniques that use confidence scores.

[0018] As any sufficiently knowledgeable professional in the field would know, document processing techniques that employ computer vision and similar technologies to process large numbers of documents for various downstream tasks rely on the ability to accurately transcribe documents in order to make correct downstream decisions. For example, if a document is transcribed incorrectly, indexing techniques such as database systems may not store the document in the correct location, so that the document is effectively lost if it cannot be found because it is stored in a location based on erroneous content detected in the document transcription.Accordingly, the ability to accurately detect errors in document transcriptions generated by neural networks can provide the opportunity to correct errors and prevent these errors from causing further problems in a variety of downstream systems that rely on the accuracy of document transcriptions.

[0019] Fig. Figure 1 illustrates an example of a logical block diagram for error detection in object recognition in neural networks based on derived descriptive information according to at least one embodiment. In at least one embodiment, the one or more content recognition neural networks 110 can be trained to predict one or more recognized contents (e.g., of objects) in input data with the contents 102 together with the one or more content descriptions 106. Neural networks, such as the one or more content recognition neural networks 110 in Fig. 1 and the one or more neural networks 320 for document extraction in Fig. 3. There can be various types of neural network models for machine learning that use layers with one or more nodes (sometimes called neurons). Each node represents a computation, the output of which is passed to another node (e.g., in the next layer) connected to that node by an edge. This node then uses that value to perform a computation (e.g., typically in combination with outputs from other nodes also received from a previous layer). The computations performed at the nodes of different layers can be used to represent or perform various AI or other machine learning tasks (e.g.,Computer vision tasks such as image recognition, object recognition, and image classification; natural language processing such as machine translation, sentiment analysis, and text analysis; audio processing such as speech recognition; automation such as autonomous vehicle navigation; and generative tasks such as speech, image, audio, and video generation. For example, neural network calculations can be performed to recognize patterns (e.g., in input data), make predictions based on input data, or learn a task from input data.

[0020] In various embodiments, computations performed by a neural network at nodes can utilize different pieces of information, such as weights, biases, input data, and operations, such as an activation function, to compute a node's output. Weights and biases can be considered parameters of the neural network, learned as part of the network's training. Weights can be used to determine the importance of an input feature for performing a task by multiplying each input by an appropriate weight. Bias can be used to allow the neural network to delay an activation function, such as by adding a constant value to a weighted sum (e.g., weight multiplied by input) before the activation function is executed. An activation function can be used to determine the output (e.g.,The activation of a node is calculated according to a specific activation function (which can be selected and described as part of the neural network's architecture). Examples of common activation functions include, but are not limited to, Rectified Linear Unit (ReLU), Sigmoid, Tanh, and SoftMax, each providing distinct properties useful for different tasks. If a node has multiple inputs (e.g., from several other nodes in a previous layer), a weighted sum can be calculated using the weight before the bias is added and the activation function is applied.

[0021] The architecture of a neural network can describe the number of nodes, layers, connections between nodes / layers, and the computations performed by the neural network. Layers can sometimes be described as either an input layer, comprising one or more nodes to accept input data of various formats; a hidden layer(s), comprising one or more nodes that perform computations to carry out an AI or other machine learning task; or an output layer, comprising one or more nodes that can provide a result (e.g., an inference or other output) of the AI ​​or other machine learning task. Some neural networks may be referred to as deep neural networks (DNNs) if they contain multiple hidden layers (e.g., three or more) between the input and output layers.

[0022] Various neural network architectures can be implemented. One example is a feedforward neural network (FNN), where data flows through the network in one direction. Another example is a convolutional neural network (CNN), which can process input data according to a grid topology (e.g., different parts of an image) using convolutional computations (e.g., applying a filter to create or update a feature map) on different layers to detect features such as edges, textures, or other aspects of the input data. A further example is a recurrent neural network (RNN), which can have edges that are fed back to be used recursively in the computation.Another example of a neural network architecture is a Long Short-Term Memory (LTSM) network, which can store computational results for reuse over a period of time. A further example is a Generative Adversarial Network (GAN), which uses a generator network and a discriminator network to compete with each other. The generator attempts to generate new data from the input data, while the discriminator tries to distinguish between the generated new data and the real data samples. Another example of a neural network architecture is an autoencoder, which includes an encoder to compress or otherwise represent input data in coded form and a decoder to then reconstruct the input data from the coded form, with or without augmentation or analysis.

[0023] Another example of a neural network architecture is a transformer network, which can use an attention mechanism to process sequential data. Transformer-based neural networks, such as MLLMs, Large Language Models (LLMs), Vision Language Models (VLMs), or various other types of artificial intelligence or machine learning models, can use a transformer as part of generating inferences and / or other outputs. A transformer can be implemented to capture relationships between different data elements in a sequence (e.g., data tokens, such as tokens representing natural language, image data, time-series data, audio data, and / or various other types of data). In various implementations, tokens can provide a coded representation of data (e.g.,(as an embedding of one or more vectors, where different dimensions of the vector correspond to numerical representations of data attributes). These captured relationships can be used to iteratively predict and / or generate new data elements in the sequence using autoregressive techniques. Because transformers include functions such as self-awareness, which can be efficiently performed in parallel, transformer-based neural networks have gained popularity for performing a variety of artificial intelligence tasks as processors. This is because graphics processing units (GPUs), which can efficiently perform parallel operations, can be used to accelerate execution time and increase the predictive accuracy of artificial intelligence tasks.

[0024] In at least one embodiment, captured relationships can cover large amounts of input data, providing context for generating and / or predicting new data. For example, captured relationships can encompass contexts ranging from a small number of previously entered or generated words to a large number of documents or other data content. To avoid recalculating the relationships between data in contexts, such as large contexts, at least one embodiment implements a key-value (KV) cache to store the captured relationships as KV values. These values ​​can be accessed by a transformer and used to capture new relationship information when new input data is received (e.g., when...).When a query or other request is received for an LLM, the KV cache, which provides the context for the query or request, can be retrieved to generate a response or other reaction to the request.

[0025] In at least one embodiment, a transformer can be implemented in various ways, which may include a self-attention stage and a forward network stage. In at least one embodiment, a self-attention stage can be implemented as multi-head attention, which can use different attention heads to capture different types of relationships corresponding to different aspects of a sequence of tokens (e.g., syntax, semantics, and coreference for a sequence of tokens representing words or parts of words in natural language). Each attention head can be computed independently of other attention heads in the same multi-head attention stage, providing the possibility of multi-head attention execution in parallel.

[0026] In at least one embodiment, one (or more) feed-forward networks (FFNs) can be used to enrich the attention scores output by a self-attention stage to provide a hidden dimension representation (e.g., of a token) that resides in a larger feature space than an initial hidden dimension representation of a token. For example, an FFN can expand a feature space to four times the size of an initial hidden dimension, such as from an initial hidden dimension representation with 4096 parameters to a hidden dimension representation with 16384 parameters. In this way, an FFN can improve the accuracy of a transformer-based neural network by capturing additional features to distinguish between different possible data elements that might be predicted next in an input sequence.

[0027] Similar to the examples mentioned above regarding transformer-based neural networks, different neural network architectures can be combined in various ways to perform different tasks. For example, one or more convolutional layers can be combined with transformers in other neural network architectures.

[0028] Different neural network architectures can be designed, specialized, or otherwise used for specific tasks. For example, transformer-based neural networks can be used to implement MLLMs, LLMs, VLMs, or various other types of artificial intelligence or machine learning models to generate inferences and / or other outputs. Because a transformer can capture relationships between different data elements in a sequence (e.g., data tokens such as tokens representing natural language, image data, time-series data, audio data, and / or various other types of data), these captured relationships can be used to iteratively predict and / or generate new data elements in the sequence using autoregressive techniques.In another example, CNN-based neural networks can be used for various computer vision tasks, as image data (or data that can be converted into image data, such as heatmaps or spectrograms (for audio data)) can be processed in a grid-like manner to identify areas within the larger image for different purposes. LSTM or RNNs can be used in yet another example for time-series data analysis and / or forecasting. GANs can be used in generative tasks (e.g., generating images, audio, video, or other data) to create entirely new or augmented data.

[0029] As in Fig. As shown in Figure 1, the one or more neural networks for content recognition 110 can generate one or more content descriptions 106 that can correspond to a specific one or more of the recognized content 104 (e.g., bounding boxes and class labels can be specified for specific objects such as tables, text, or other objects recognized in an input document). For example, the one or more content descriptions 106 can be information that describes the structure of text information in the input data with content 102 (e.g., where it is located, as indicated by a bounding box), and / or can be information that describes what kind of text information it is (e.g., a block of text, a section heading, etc.).In at least one embodiment, the one or more neural networks for content recognition 110 can be trained to output the one or more corresponding descriptions 106 as tokens in a sequence together with content tokens, using their positions within a sequence to identify which content token is described by which description token (as, for example, in . Fig. As illustrated in Figure 2, bounding box tokens can surround a content token along with a subsequent content tag token, before a new bounding box token follows, which is followed by a new content token, and so on. In at least one embodiment, missing or erroneous information in a sequence of tokens (e.g., in description tokens) can indicate that a corresponding content token has a fault (since a fault for a description token is strongly correlated with a fault in a content token, including faults that repeat or block the generation of content tokens).

[0030] The content error detection 120 can use the one or more content descriptions 106 to evaluate whether the one or more detected content items 104 contain an error. In at least one embodiment, for example, an error can occur in the neural networks for content detection 110, which are implemented using a transformer-style architecture (such as an MLLM, LLM, or VLM) that generate the one or more detected content items 104 as a sequence of tokens. In at least one embodiment, the content error detection 120 can use pattern recognition or other techniques to analyze the one or more content descriptions 106 with respect to grammar, syntax, or another expected format.In at least one embodiment, an expected format of content descriptions can be a small combination of possible values ​​that allows for fast and highly accurate error detection, which is to be performed on the one or more corresponding detected content items 104. In at least one embodiment, the content error detection 120 can implement various error detection techniques, which are described below with respect to . Fig. 2 - 5 will be discussed. In at least one embodiment, the one or more content errors 108 can be recorded as markings or other indicators by the content error detection 120, thereby identifying and removing the one or more detected erroneous contents 104, or by means of various other error handling techniques, such as those described below in relation to Fig. 3 can be discussed and dealt with.

[0031] Fig. Figure 2 illustrates an example of descriptive token evaluation as part of error detection in object recognition in neural networks according to at least one embodiment. In at least one embodiment, the content error detection 120 can implement the description token evaluation 230. In at least one embodiment, the transcription token evaluation 230 for the document 210 can evaluate tokens generated. In at least one embodiment, tokens can be content tokens, such as the extracted content tokens 224a, 224b, 224c, and 224d (which contain textual information such as words, images, graphs, tables, charts, or various other types of content that may be present in a document), or descriptive tokens, such as the bounding box tokens 222a, 222b, 222c, 222d, 226a, 226b, and 226d, and content tagging tokens (e.g.,The content token may be a semantic class identifier token, such as tokens 228a, 228b, and 228d, or any other information describing the content. In some embodiments, a content token may provide a coded representation of the document content (e.g., an embedding of one or more vectors, where the dimensions in the vector correspond to different features of a portion of the document content represented by the token).

[0032] In at least one embodiment, the evaluation of the description token 230 can apply a syntax or other set of expected grammar or rules to a structure or other descriptive information found in descriptive tokens or in expected but missing descriptive tokens within a sequence of tokens (which may contain one or more descriptive and content tokens). Similar to the content tokens discussed above, descriptive tokens in various embodiments can provide a coded representation of descriptive information (e.g., an embedding of one or more vectors, the dimensions in the vector corresponding to different features of a portion of the descriptive information, such as a bounding box or semantic class tag represented by the token).For example, in at least one embodiment, one or more extracted content tokens 224c may be expected to be accompanied by both the bounding box tokens 222c and the one or more bounding box tokens that follow the one or more extracted content tokens 224c. In at least one embodiment, missing tokens (e.g., missing tokens for the one or more extracted content tokens 224c, such as missing bounding box tokens) may cause an error result 229c, indicating an error. Alternatively, although not illustrated, description tokens (e.g., bounding box tokens or content tag tokens) may have incorrect values, content, or formatting that violates the syntax for description tokens, which could also trigger error detection. As in . Fig. As shown in Figure 2, other extracted content (e.g., 224a, 224b, and 224d) may produce error-free results (e.g., as shown in Figures 229a, 229b, and 229d). In at least one embodiment, by individually evaluating extracted content tokens, error evaluation and content recognition for input data can continue even if an error is found for a portion of the extracted content, allowing content recognition and error evaluation to continue without interruptions caused by errors. This can improve processing for systems, services, or applications because errors can be identified without interrupting error checks for the remainder of a document transcription after an error is found (e.g., a large document may contain only one small error, and the rest of the document transcription may be correct). Furthermore, errors can be indicated as occurring in a specific part of the document (e.g.,B. in a specific paragraph, heading or sentence), which allows the application of corrections or other editing techniques only to the erroneous part of the document transcription.

[0033] Batch processing systems are an example of a system where techniques such as those described above in relation to Fig. 1 and Fig. 2 described, which can improve performance (e.g., throughput, utilization of computing resources, and accuracy). Fig. Figure 3 illustrates an example of a batch transcription system that implements error detection in object recognition using neural networks based on derived descriptive information according to at least one embodiment. In at least one embodiment, the batch transcription system 310 can implement one or more document extraction neural networks 320, document error detection 330, and document error handling 340. In at least one embodiment, the one or more document extraction neural networks 320 can implement a transformer-based vision encoder-decoder architecture. In at least one embodiment, a vision transformer encoder, designated ε, can be a neural network model of type Vision Transformer (ViT) that takes an input image of a document, I ∈ ℝ 3×H×W, maps to a latent representation, Z ∈ ℝN×d, where H and W are the image height and width respectively, d is a hidden dimension, and N is a sequence length. In at least one embodiment, a compressor can be implemented as part of the one or more neural networks for document extraction 320, which can also be called a “neck” and is connected with N characterized by a compression of the dimensionality and sequence length of a latent space into text. In at least one embodiment, data within lines can be more strongly correlated than within blocks, allowing the compressor to use horizontal kernel convolutions instead of square or rectangular convolutions, resulting in a reduced sequence length. In at least one embodiment, a decoder can be implemented as part of one or more neural networks for document extraction 320, which are described as D can be designated, and can use mBart to represent text tokens, T = {t P+1 , t P+2 ,..., t L}, based on a latent encoder representation, Z, and the context P(ti|N(Z),t <i) to predict, where Z = ε (I) and {t1, t2, ..., t P} the prompt tokens and where L is the sequence length augmented by the prompt. In at least one embodiment, the one or more neural networks for document extraction 320 can be implemented as autoregressive architectures that scale linearly during inference with respect to the decoder and the sequence length, using an encoder with a larger number of parameters than the decoder (e.g., a high-performance encoder with a resource-efficient decoder).

[0034] In at least one embodiment, a document can be transcribed using one or more neural networks for document extraction 320 according to a prompt that includes one or more options. In at least one embodiment, options can be specified in the form of an M-dimensional tuple. For example, in at least one embodiment, options can include an output format option with the options "structured text" and "plain text", a bounding box option with the options "enabled" and "disabled", and a class option also with the options "enabled" and "disabled". It is understood that this is merely an example and is not intended to be a limitation, since any number of options and option values ​​can be provided.In at least one embodiment, a request can then, for example, comprise a three-dimensional tuple of options, with eleven possible combinations existing, which can be described as follows: (<structured_text> |<plain_text> |<no_text> )(. <bbox>|<no_bbox>)( <classes>|<no_classes> )

[0035] In at least one embodiment, the information to be predicted can be reduced within each group as the options progress. In at least one embodiment, a maximally informative prompt can be specified as follows:<structured_text> <bbox> <classes>

[0036] In at least one embodiment, the one or more document extraction neural networks can process 320 transcriptions according to any combination of possible task prompts. In at least one embodiment, the one or more document extraction neural networks can be pre-trained on a user-defined dataset containing labels for a maximum informative setting and then, with a certain probability, reduce the information required for each group. In at least one embodiment, fine-tuning on datasets with varying information density enables a dataset with partial annotations, with an encoder being trained and improved as the dataset becomes visually rich.

[0037] In at least one embodiment, the one or more neural networks for document extraction 320 can implement a token stream generated by a decoder. In at least one embodiment, the one or more neural networks for document extraction 320 can predict bounding boxes in the form of discrete coordinates. In at least one embodiment, an example of a regular expression shows a prediction format for each box:

[0038] In at least one embodiment, a first set of coordinates designates an upper left corner and a second set a lower right corner. In at least one embodiment, bounding frame coordinates can be optimized using cross-entropy loss in the same way as regular text tokens, so that it is the task of a token embedding layer to approximate the spatial similarity. In at least one embodiment, H + W tokens for bounding frames, C tokens for semantic classes, and special tokens for the vocabulary prompts can be added.

[0039] In at least one embodiment, the document error detection 330 (which may resemble the content error detection 120) can evaluate descriptive information of transcribed or extracted content from the one or more neural networks for document extraction 320 according to various techniques described above in relation to Fig. 1 - 2 and below with reference to Fig. 3 - 4 will be discussed. In at least one embodiment, the document error detection can evaluate descriptive information for missing or erroneous content (e.g., missing bounding box information or erroneous class / semantic labels).

[0040] In at least one embodiment, the batch transcription system 310 can implement a document error handling system 340. In at least one embodiment, the document error handling system 340 can forward or process errors detected during document error detection 330. For example, in at least one embodiment, the document error handling system 340 can perform operations to redact, filter, or otherwise remove detected errors from transcriptions. In at least one embodiment, the document error handling system can store an error for analysis or correction by another system (e.g., including a human analysis interface). In at least one embodiment, the document error handling system 340 can halt the transcription of a document or a batch of documents when the number of errors exceeds a threshold value.

[0041] In at least one embodiment, the batch transcription system 310 can implement an interface (e.g., an application programming interface (API), a graphical user interface (GUI), or a command line) that supports transcription requests, such as the transcription request 302. In at least one embodiment, the transcription request 302 can contain various parameters, features, or other information. In at least one embodiment, the transcription request 302 can contain an identifier for documents or a group of documents to be transcribed according to the storage location, the storage object, the data store, or other information (e.g., an identifier for a storage location or container in the data store 350 in which the documents 352 are stored).In at least one embodiment, the batch transcription request 302 can contain a transcription configuration parameter that can specify, for example, which descriptive information is to be included (e.g., bounding boxes, class labels, etc.), with further examples discussed above. In at least one embodiment, the batch transcription request 302 can contain an error handling configuration or information that can control the execution of document error handling 340 for a particular batch.

[0042] In at least one embodiment, as specified in 312, the batch transcription system 310 can retrieve a batch of documents 312 from the data storage 350, generate transcribed documents using one or more neural networks for document extraction 320, and perform error detection 330. In at least one embodiment, the batch transcription system 310 can store the transcribed documents 354 in the data storage 350, as specified in 314. In at least one embodiment, the transcribed documents 354 can be provided to one or more downstream systems 360 for further processing (e.g., analysis, display, indexing, etc.), as specified in 316.

[0043] The above in relation to Fig. The examples discussed in sections 1-3 of error detection in object recognition in neural networks using derived descriptive information were presented in relation to various exemplary systems, services, or applications. Various other types of systems can implement these techniques, which aim to perform error detection in document transcriptions generated by neural networks. Fig. Figure 4 illustrates an example of a method or technique that implements error detection in object recognition in neural networks using derived descriptive information according to at least one embodiment. These techniques can also be implemented using various components, as described above with respect to Fig. 1 - 3 described.

[0044] In at least one embodiment, one or more neural networks can be instructed to generate respective document transcriptions of one or more document images, as described in 410. The document transcriptions can include both document content and descriptive information about the document content. In at least one embodiment, for example, one or more neural networks can be assigned to the one or more neural networks for content recognition 110 in Fig. 1 and / or the one or more neural networks for document extraction 320 in Fig. 3. In at least one embodiment, a document transcription can be generated using an input document (e.g., an image of a document in an image file format). In at least one embodiment, a document transcription can include many different objects or other content recognized from an input document, including, but not limited to, section headings, footnotes, text, tables, list items, page headings, images, formulas, figure captions, page footers, tables of contents, and bibliographies. In at least one embodiment, a document transcription can contain one or more parts of the recognized content (e.g., represented as tokens, text statements, image descriptions, formulas, or other descriptions in markup languages ​​such as LaTeX).In at least one embodiment, a document transcription can contain descriptive information that is separate from the recognized content (e.g., represented as tokens, text strings, or other descriptive formats) according to a defined syntax (e.g., a set of rules describing how a description can be formatted, in what order, and using specific words or characters). In at least one embodiment, descriptive information can be linked, mapped, or otherwise associated with specific content that it describes (e.g., as tokens surrounding or adjacent to content tokens, as in [reference]). Fig. 4 shown).

[0045] In at least one embodiment, one or more errors in the document content of at least one of the document transcriptions can be detected, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, as described in 420. In at least one embodiment, an error evaluation can identify syntax errors based on incorrect or missing information in the descriptive information that may indicate an error in corresponding content detected in a document transcription. In at least one embodiment, a regular expression or other pattern specification can be applied by a character parser or other pattern specification interpreter to search for the presence of a particular pattern in certain characters (e.g., in descriptive information and / or detected content).In at least one embodiment, as above with regard to . Fig. 2 and below with regard to Fig. As discussed in section 5, a regular expression pattern or other pattern specification can be applied to a sequence of tokens corresponding to a recognized object in a document transcription (e.g., a paragraph, a heading, a diagram, a table, a formula, etc.). In at least one embodiment, a heuristic or rule-based unit (engine) can apply or search for correct syntax with respect to a sequence of tokens corresponding to a recognized object in a document transcription. In at least one embodiment, a resource-efficient machine learning model or a finite automaton can be used to apply or search for correct syntax.In at least one embodiment, a list of correct semantic class identifiers, such as, but not limited to, section headings, footnotes, text, tables, list items, page headings, images, formulas, captions, page footers, tables of contents, and bibliographies, can be checked against the descriptive information for semantic class identifiers. If a new or unknown semantic class identifier is found (e.g., one that does not match any of the identifiers in the list), a syntax error in the semantic class identifier can be identified.

[0046] In at least one embodiment, the occurrence of an error with respect to an object or part of the content detected in a document transcription does not prevent the transcription or error analysis for another object or part of the content. In at least one embodiment, partially transcribed documents containing content or objects identified as erroneous based on an evaluation of the descriptive information are filtered, removed, or redacted from the document. In at least one embodiment, error handling techniques, including thresholds for handling a certain number of detected errors, can be implemented.

[0047] The techniques described above can be applied in some embodiments to a document transcription while it is generated as a stream of tokens output by one or more neural networks, or as a set or stack of tokens in the document transcription. Fig. Figure 5 illustrates an example of a method or technique that implements descriptive token evaluation as part of error detection in object recognition in neural networks according to at least one embodiment. In at least one embodiment, one or more descriptive tokens corresponding to one or more content tokens of the recognized content can be obtained, as described in Figure 510. In at least one embodiment, an LLM or other transformer-based neural network can be used to recognize content in input data as a sequence of tokens. In at least one embodiment, for certain input data (e.g., a document), individual objects can be recognized as a sequence of tokens that may comprise a pattern or set of one or more tokens, including one or more descriptive tokens, as described in Figure 510. Fig. 2 illustrated bounding box tokens and content tagging tokens, as well as one or more content tokens, as shown in Fig. Figure 2 illustrates extracted content tokens. In at least one embodiment, different tokens for recognized content can be of different data types (e.g., text tokens, including descriptive information formatted according to a specific syntax for labels or bounding boxes), which can be mixed with tokens that are image, audio, or other data types.

[0048] In at least one embodiment, one or more description tokens can be evaluated for missing or erroneous information, as described in 520. For example, one or more description tokens can indicate a number of subsequent tokens to be expected (e.g., a bounding box token can indicate that a second bounding box token can be described, which describes an opposite corner of a bounding box). In at least one embodiment, the one or more description tokens can indicate an error in one or more content tokens if an expected number of tokens are missing (e.g., a second bounding box token). In at least one embodiment, if a description token contains an error (e.g.,in form or content, such as in a correct content tag or a missing separator, character or other formatting error), an error is displayed in the one or more content tokens described by the one or more description tokens.

[0049] In at least one embodiment, as indicated by the positive outcome of 530, for missing or erroneous information determined by evaluation 520, the one or more content tokens described by the one or more description tokens can be identified as erroneous, as shown in 540 in at least one embodiment. In at least one embodiment, identifying a content token as erroneous can include adding an additional token, an additional label, or another indicator that identifies the one or more content tokens as erroneous. In at least one embodiment, one or more erroneous content tokens and one or more corresponding description tokens describing one or more content tokens can be stored separately from a transcription or other content recognition output.In at least one embodiment, one or more erroneous content tokens and one or more corresponding description tokens describing one or more content tokens can be filtered or removed from a transcription or other content recognition output. In at least one embodiment, in... Fig. The five described techniques can be repeated multiple times for separate sets of descriptive tokens and content tokens generated for input data (e.g., for the same document, recognizing the content of different parts of a document). In at least one embodiment, the individual evaluation of one or more content tokens using descriptive tokens specific to one or more particular content tokens can ensure that errors found in one part of the input data (e.g., a document) do not affect content recognition in another part of the input data (e.g., a partially or mostly correct document with correctly recognized content can be stored, with incorrectly recognized content removed, redacted, or displayed as erroneous). DATA CENTER

[0050] Fig. Figure 6 illustrates an exemplary data center 600 in which at least one embodiment can be used. The data center 600 can comprise one or more rooms with racks 602 and ancillary equipment used to house one or more racks 602 and one or more mainboards 604. The rack 602 can contain one or more mainboards 604. The rack 602 can include an enclosure that accommodates and supports individual mainboards 604. Operational aspects of the rack 602 can be controlled, among other things, at the rack level, according to a group of mainboards 604, or at the mainboard level, according to individual baseboards 604. The rack 602 or the mainboards 604 can have specifically selected maximum operating parameters, such as, but not limited to, power consumption, operating frequencies, and others.The Data Center 600 can be supported by various cooling systems, such as, but not limited to, cooling towers, cooling circuits, pumps, and other support systems. Cooling systems can include sensors and controllers to monitor and manage the cooling characteristics for the Racks 602. The Motherboards 604 within the Racks 602 can obtain their operating power from one or more Power Distribution Units (PDUs; not shown). PDUs can be located within the Racks 602, for example, between the Racks 602 containing the Motherboards 604, or within the Racks 602 that also contain the Motherboards 604.

[0051] The racks 602 and the mainboards 604 can include subsystems, modules, expansion cards, and other semiconductor components. The mainboards 604 can contain one or more arithmetic units 606, which can contain one or more processors 608, one or more memory units 610, and an interface controller 612. The arithmetic units 606 can contain any number of processors, such as, but not limited to, central processing units (“CPUs”), graphics processing units (“GPUs”), or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, etc.), including all processors described herein, such as, but not limited to, the processors in Fig. 7-18. The Computing Units 606 can contain one or more Storage Devices 610 (e.g., dynamic read-only storage, solid-state storage, or hard disk drives), as well as Network I / O Devices, Network Switches, Virtual Machines (VMs), Power Supply Modules, and Cooling Modules, etc. One or more Computing Units 606 can constitute a server with one or more of the above-mentioned computing resources.

[0052] The Compute Units 606 can contain separate groupings of compute units housed in one or more racks (not shown) or in many racks in data centers at different geographic locations (also not shown). Separate groupings of compute units can contain grouped compute, network, memory, or storage resources that can be configured or allocated to support one or more workloads. Multiple compute units (including, for example, CPUs and / or other processors) can be grouped in one or more racks to provide compute resources to support one or more workloads. A Resource Orchestrator 614 can configure or otherwise control one or more Compute Units 606 or groups of compute units.The Resource Orchestrator 614 can include a Software Design Infrastructure (“SDI”) management unit for the Data Center 600. The Resource Orchestrator 614 can include hardware, software, or a combination thereof.

[0053] The data center 600 can contain any framework layer 620, a software layer 630, and an application layer 6340, or any combination thereof. As in Fig. As shown in Figure 6, the framework layer 620 contains a job scheduler 622, a configuration manager 624, a resource manager 626, and a distributed file system 628. The framework layer 620 can contain a framework to support the software 632 of the software layer 630 and / or one or more applications 642 of the application layer 640. The software 632 or the one or more applications 642 can each be, but are not limited to, web-based service software or applications such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. The framework layer 620 can also be, but is not limited to, a type of free and open-source software for a web application framework such as Apache Spark™ (hereafter referred to as "Spark"), which can utilize a distributed file system 628 for handling large amounts of data (e.g., "big data").The Job Scheduler 622 can include a Spark driver to facilitate the scheduling of workloads supported by various layers of the data center 600. The Configuration Manager 624 can configure various layers, such as, but not limited to, the Software Layer 630 and the Framework Layer 620, which includes Spark and the Distributed File System 628 to support the processing of large amounts of data. The Resource Manager 626 can manage clustered or grouped compute units 606 that are allocated or assigned to support the Distributed File System 628 and the Job Scheduler 622. The Resource Manager 626 can coordinate with the Resource Orchestrator 614 to manage these allocated or assigned compute resources.

[0054] The software 632 can be contained in software layer 630 and can include software used by at least parts of a computing unit 606, one or more computing units 606, groups of computing units 606, and / or the distributed file system 628 of framework layer 620. One or more types of software can include, but are not limited to, web page search software, email virus scanning software, database software, and streaming video content software.

[0055] The one or more applications 642 may be contained in the application layer 640 and may include one or more types of applications used by at least parts of a compute unit 606, one or more compute units 606, groups of compute units 606, and / or the distributed file system 628 of the framework layer 620. One or more types of applications may include, but are not limited to, any number of genome applications, cognitive computations, applications, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0056] Each of the Configuration Manager 624, Resource Manager 626, and Resource Orchestrator 614 can implement any number and type of self-modifying actions based on any amount and type of data collected in any technically feasible way. Self-modifying actions can relieve a Data Center 600 operator of potentially making poor configuration decisions and potentially avoiding underutilized and / or poorly functioning parts of a data center.

[0057] The Data Center 600 may include tools, services, software, or other resources for training one or more machine learning models or for predicting or deriving information using one or more machine learning models according to one or more of the embodiments described herein. For example, a machine learning model may be trained by calculating weighting parameters according to a neural network architecture using software and computing resources described above in relation to the Data Center 600. Trained machine learning models corresponding to one or more neural networks may be used to derive or predict information using the resources described above in relation to the Data Center 600 by using weighting parameters calculated by one or more of the training techniques described herein.

[0058] The Data Center 600 can support CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs and / or other hardware (e.g., implementations in Fig. 7-18) to perform some or all of the processes and techniques described elsewhere herein, such as, but not limited to, training and / or inference using the resources described above. In addition, one or more of the software and / or hardware resources described above may be configured as a service to allow users to train or infer information, such as, but not limited to, image capture, speech capture, or other artificial intelligence services.

[0059] In at least one embodiment, the processor 608 may comprise one of the processors listed below and / or it may comprise one or more circuits for generating respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions comprise document content and descriptive information of the document content; and detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise perform any of the operations described above or elsewhere herein.In at least one embodiment, the processor 608 is configured by the software 632 to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions comprise document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein. The data center 600 may include logic, CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs, or other hardware (e.g., embodiments in ). Fig. 7-18) to perform each of the operations described above and elsewhere herein. PROCESSORS

[0060] The following figures represent, without limitation, exemplary processors and processing systems that can be used to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, based on a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform some or all of the processes, operations, and / or techniques described elsewhere herein.Exemplary processors and processing systems can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, based on a syntax error identified in the descriptive information of the at least one document transcription; or otherwise perform any of the operations described above or elsewhere herein. Processors and processing systems can include logic, central processing units (CPUs), application-specific integrated circuits (ASICs), graphics processing units (GPUs), field-programmable arrays (FPGAs), XPUs (i.e.,any computer architecture that best meets the requirements of an application) or other hardware (e.g., implementations in . Fig. 7-18) to perform any of the operations described above, below, or elsewhere herein. Processors and / or processing systems described herein may include one or more circuits that can be used to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.As used here, one or more circuits of software can be configured to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein. Fig. 22A and Fig. Figure 22B illustrates the logic 2215, which, as described elsewhere herein, can be used in one or more devices to perform operations such as, but not limited to, those discussed herein according to at least one embodiment. Logic may, for example, refer to any combination of software logic, hardware logic, and / or firmware logic to provide the functionality and / or operations described herein, the logic being collectively or individually embodied as a circuit that is part of a larger system, such as an integrated circuit (IC), an application-specific integrated circuit (ASIC), a field-programmable array (FPGA), a system-on-a-chip (SoC), or one or more processors (e.g., CPU, GPU).

[0061] Fig. Figure 7 illustrates a processor, which is a system-on-a-chip (SoC) 700 (which may be called a system-on-a-chip, superchip, or by another name), according to at least one embodiment. The SOC 700 may contain a processor complex 710 and a processor complex 740. The SOC 700 may contain any number of processor complexes 710 and / or processor complexes 740, which may contain any number of processors described herein, such as, but not limited to, those in Figure 7. Fig. 7-18, in any combination. For example, the 710 processor may contain a central processing unit (CPU), and the 740 processor may contain a graphics processing unit (GPU). Alternatively, the 710 processor may contain a GPU, and the 740 processor may contain a GPU. The 700 SOC may contain any number of 792 display controllers, any number of 794 multimedia units, any number of 770 I / O interfaces, any number of 780 memory controllers, and any number of 760 fabrics, in any combination. For clarification, multiple instances of similar objects are indicated here with reference numbers that identify the object and with numbers in parentheses that identify the instance where necessary. The 700 SOC may contain a Broadcom processor made in Palo Alto, California.

[0062] The 710 processor complex can contain a CPU, the 740 processor complex can contain a GPU, and the 700 SOC can be a processing unit that integrates the 710 and 740 onto a single chip. Some tasks can be assigned to the 710 processor complex, and other tasks can be assigned to the 740 processor complex. The 710 processor complex can be configured to run the main control software associated with the 700 SOC, such as, but not limited to, an operating system. The 710 processor complex can be the main processor of the 700 SOC, controlling and coordinating the operations of other processors. The 710 processor complex can issue instructions that control the operation of the 740 processor complex to perform some or all of the operations described herein. The 710 processor complex can be configured to receive commands from CUDA or other source code (e.g.,The 740 processor complex can execute host-executable code derived from HIP source code, and it can be configured to execute device-executable code derived from CUDA or other source code to perform any of the operations described herein.

[0063] The 710 processor complex can contain the 720(1)-720(4) cores and a 730 cache (e.g., L3 cache) for storing information to perform the operations described here. The 710 processor complex can contain any number of 720 cores and any number and type of caches in any combination. The 720 cores can be configured to execute instructions of a specific instruction set architecture (ISA) to perform some or all of the operations described here. Each 720 core can comprise one CPU core. The 720(1)-720(4) cores can be referred to as arithmetic units. The 700 SOC can contain any number of 710 processor complexes, 760 fabrics, 770 I / O interfaces, and 780 memory controllers.

[0064] Each Core 720 can contain a Fetch / Decode Unit 722, an Integer Execution Unit 724, a Floating-Point Execution Unit 726, and an L2 Cache 728. The Fetch / Decode Unit 722 can fetch instructions to perform some or all of the operations described here (such as, but not limited to, an API compiled to instructions), decode such instructions, generate microoperations, and send separate microinstructions to the Integer Execution Unit 724 and / or the Floating-Point Execution Unit 726. The Fetch / Decode Unit 722 can simultaneously send one microinstruction to the Integer Execution Unit 724 and another microinstruction to the Floating-Point Execution Unit 726. The Integer Execution Unit 724 can perform integer and memory operations. The 726 floating-point unit can perform floating-point and vector operations.The 722 retrieval / decoding unit can send micro-instructions to one or more execution units, replacing both the 724 integer execution unit and the 726 floating-point execution unit.

[0065] Each core 720(i), where i is an integer representing a specific instance of core 720, can access the L2 cache 728(i) contained within core 720(i). Each core 720 contained within core complex 710(j), where j is an integer representing a specific instance of core complex 710, can be connected to other core 720s contained within core complex 710(j) via an L3 cache 730(j) contained within core complex 710(j). The core 720s contained within core complex 710(j), where j is an integer representing a specific instance of core complex 710, can access the entire L3 cache 730(j) contained within core complex 710(j). The L3 cache 730 can contain any number of slices.

[0066] The 740 processor complex can be a graphics complex configured to perform computational operations (such as those involved in the operations described here) in a highly parallel manner. The 740 processor complex can be configured to execute graphics pipeline operations, such as, but not limited to, drawing commands, pixel operations, geometric calculations, and other operations related to rendering an image on a display. The 740 processor complex can be configured to perform non-graphics-related operations, such as, but not limited to, neural network training and / or simulations. The 740 processor complex can be configured to perform both graphics-related and non-graphics-related operations.

[0067] The 740 processor complex can contain any number of 750(1)-750(N) processing units, where N is any integer greater than 1, and an L2 cache 742. The 750 processing units can share the L2 cache 742, which can store information used to perform some or all of the operations described herein. The L2 cache 742 can be partitioned. The 740 processor complex can contain any number of 750 processing units and any number (including zero) and type of caches. The 740 processor complex can include any amount of dedicated graphics hardware.

[0068] Each compute unit 750 can contain any number of SIMD units 752(1)-752(N), where N is any integer greater than 1, and a shared memory 754. Each SIMD unit 752 can implement a SIMD architecture and be configured in parallel for some or all of the operations described here. Each compute unit 750 can execute any number of thread blocks, but each thread block can be executed on a single compute unit 750, although in some embodiments a thread block can be executed on multiple compute units. A thread block can contain any number of execution threads. A workgroup can be a thread block. Each SIMD unit 752 can execute a thread group. A thread group (e.g., 16 threads), also called a warp, subgroup, or wavefront (e.g.,(as used by AMD and Intel) where each thread in the warp, wave, subgroup, or wavefront can belong to a single thread block and is configured to process a different data set using a single instruction set. Predication can disable one or more threads in a warp, subgroup, or wavefront. A lane can be a thread. A worker can be a thread, as with OpenCL, but is not limited to this. Different warps, subgroups, or wavefronts in a thread block can be synchronized and communicate with each other via shared memory 754.Each compute unit 750 can contain one or more thread block clusters, wherein a thread block cluster can enable programmatic control of locality with a granularity that may exceed that of a single thread block of a single streaming multiprocessor (SM). Thread block clusters (also referred to as "clusters") can enable multiple thread blocks running concurrently on streaming multiprocessors to synchronize and jointly retrieve, exchange, or otherwise use data. In at least one embodiment, streaming multiprocessors ("SMs") can be referred to as streaming microprocessors, stream processors ("SPs"), stream processing units ("SPUs"), compute units ("CUs"), execution units ("EUs"), and / or slices, wherein a slice in this context can denote a portion of the processing resources in a processing unit (e.g.,16 cores, a ray tracing unit, a thread director or scheduler).

[0069] The Fabric 760 can be a system interconnect that facilitates data and control transfers across the 710 processor complex, the 740 processor complex, the 770 I / O interfaces, the 780 memory controllers, the 792 display controller, and the 794 multimedia unit to perform some or all of the operations described herein. The SOC 700 can include, in addition to or instead of the Fabric 760, any number and type of system interconnects that enable data and control transfers across any number and type of directly or indirectly connected components, which may be located inside or outside the SOC 700. The 770 I / O interfaces can represent any number and type of I / O interface (e.g., PCI, PCI-Extended (“PCI-X”), PCIe, Gigabit Ethernet (“GBE”), USB, etc.). Various types of peripheral devices can be connected to the 770 I / O interfaces.Peripheral devices that can be connected to the I / O interfaces 770 include keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network cards, etc.

[0070] The display controller 792 can display images on one or more display devices, such as a liquid crystal display (“LCD”), but is not limited to this. The multimedia unit 794 can include any number and type of circuitry related to multimedia, such as, but not limited to, a video decoder, a video encoder, an image signal processor, etc. The memory controllers 780 can facilitate data transfers between the SOC 700 and a unified system memory 790. The processor complex 710 and the processor complex 740 can share the unified system memory 790.The Unified System Memory 790 can include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as, but not limited to, synchronous graphics random access memory (SGRAM), including double data rate graphics memory (GDDR). The Unified System Memory 790 can include 3D stacked memory, including, but not limited to, high-bandwidth memory (HBM), HBM2e, or HDM3.

[0071] The SOC 700 can implement a memory subsystem comprising any number and type of memory controllers 780 and memory devices (e.g., shared memory 754) that can be dedicated to a component or shared by multiple components to perform any of the operations described herein. The SOC 700 can implement a cache subsystem that can include one or more cache memories (e.g., L2 caches 728, L3 cache 730, and L2 cache 742), each of which can be dedicated to or shared by any number of components (e.g., cores 720, core complex 710, SIMD units 752, compute units 750, and processor complex 740).

[0072] In at least one embodiment, the SOC 700 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0073] Fig. Figure 8 illustrates a parallel processor 800 according to at least one embodiment. The parallel processor 800 can be implemented using one or more circuits and can be a programmable processor (e.g., a CPU and / or GPU), logic, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other hardware (e.g., embodiments in Figure 8). Fig. 7-17) are designated to perform each of the operations described above and elsewhere herein.

[0074] The Parallel Processor 800 may include a Parallel Processing Unit 802 to perform any of the operations described above or elsewhere herein. The Parallel Processing Unit 802 may include an I / O Unit 804, which enables communication with other devices, including other instances of the Parallel Processing Unit 802. The I / O Unit 804 may be directly connected to other devices. The I / O Unit 804 may be connected to other devices via a hub or switch interface, such as, but not limited to, a Memory Hub 805. Connections between the Memory Hub 805 and the I / O Unit 804 may form a Communication Link 813.The I / O unit 804 can be connected to a host interface 806 and a memory crossbar 816, with the host interface 806 receiving commands to perform processing operations and the memory crossbar 816 receiving commands to perform memory operations.

[0075] When the host interface 806 receives a command buffer via the I / O unit 804, the host interface 806 can forward work operations to a frontend 808 to execute these commands. The frontend 808 can be coupled to a scheduler 810 (which can also be called a sequencer) configured to distribute commands or other work items to a processing cluster array 812. The scheduler 810 can ensure that the processing cluster array 812 is properly configured and in a valid state before tasks can be distributed to a cluster of the processing cluster array 812. The scheduler 810 can be implemented using firmware logic running on a microcontroller.The Scheduler 810 implemented on the microcontroller can be configured to perform complex scheduling and workload distribution operations with coarse and fine granularity, enabling fast preemption and context switching of threads running on the 812 processing array. Host software can check workloads for scheduling on the 812 processing cluster array via one of several graphics processing paths. The workloads can then be automatically distributed across the 812 processing array cluster by the Scheduler 810 logic within a microcontroller, including the Scheduler 810 itself.

[0076] The processing cluster array 812 can perform any of the operations described above or elsewhere herein and can contain up to "N" processing clusters (for example, cluster 85A, cluster 85B to cluster 85N), where "N" is a positive integer (which may differ from the integer "N" used in other figures). Each cluster 85A-85N of the processing cluster array 812 can execute a large number of concurrent threads. The scheduler 810 can allocate work to the clusters 814A-814N of the processing cluster array 812 using various scheduling and / or workload allocation algorithms, which can vary depending on the workload for each type of program or calculation. Scheduling can be dynamically handled by the Scheduler 810 or partially supported by the compiler logic during the compilation of the program logic, which is configured for execution by the Processing Cluster Array 812.Different clusters 814A-814N of the processing cluster array 812 can be assigned for processing different types of programs or for performing different types of calculations.

[0077] The 812 processing cluster array can be configured to perform various types of parallel processing operations, such as, but not limited to, any of the operations described above or elsewhere herein. The 812 processing cluster array can be configured to perform general parallel computing operations. For example, the 812 processing cluster array can contain logic for performing processing tasks, including filtering video and / or audio data, performing modeling operations, including physical operations, and performing data transformations.

[0078] The 812 processing cluster array can be configured to perform parallel graphics processing operations. The 812 processing cluster array can include additional logic to support the execution of such graphics processing operations, including, but not limited to, texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. The 812 processing cluster array can be configured to execute graphics processing-related shader programs, such as, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. The 802 parallel processing unit can transfer data from system memory for processing via the 804 I / O unit. During processing, the transferred data can be stored in on-chip memory (e.g., the 822 parallel processor memory) and subsequently written back to system memory.

[0079] When the 802 parallel processing unit is used for graphics processing, the 810 scheduler can be configured to divide a processing load into approximately equal tasks to improve the distribution of graphics processing operations across multiple 814A-814N clusters of the 812 processing cluster array. Parts of the 812 processing cluster array can be configured to perform different types of processing. For example, a first part can be configured to perform vertex shading and topology generation, a second part can be configured to perform tessellation and geometry shading, and a third part can be configured to perform pixel shading or other screen-space operations to produce a rendered image for display.Intermediate data generated by one or more 814A-814N clusters can be stored in buffers so that intermediate data can be transferred between the 814A-814N clusters for further processing.

[0080] The 812 processing cluster array can receive processing tasks to be executed by the 810 scheduler, which receives instructions defining these tasks from the 808 front end. Processing tasks can contain indices of data to be processed, such as surface data (patch data), primitive data, vertex data, and / or pixel data, as well as state parameters and instructions defining how the data is to be processed (e.g., which program is to be executed). The 810 scheduler can be configured to retrieve indices according to the tasks, or it can receive indices from the 808 front end. The 808 front end can be configured to ensure that the 812 processing cluster array is brought into a valid state before initiating a workload specified by incoming instruction buffers (e.g., batch buffers, push buffers, etc.).

[0081] Each of one or more instances of the Parallel Processing Unit 802 can be coupled to a Parallel Processing Memory 822 to perform any of the operations described above or elsewhere herein. The Parallel Processing Memory 822 can be accessed via a Memory Crossbar 816, which can receive memory requests from both the Processing Cluster Array 812 and the I / O Unit 804. The Memory Crossbar 816 can access the Parallel Processing Memory 822 via a Memory Interface 818. The Memory Interface 818 can include multiple partitioning units (e.g., Partitioning Unit 820A, Partitioning Unit 820B through Partitioning Unit 820N), each of which can be coupled to a portion (e.g., a memory unit) of the Parallel Processing Memory 822.A number of partitioning units 820A-820N can be configured to correspond to a number of storage units, such that a first partitioning unit 820A has a corresponding first storage unit 824A, a second partitioning unit 820B has a corresponding storage unit 824B, and an Nth partitioning unit 820N has a corresponding Nth storage unit 824N. A number of partitioning units 820A-820N need not be equal to a number of storage units.

[0082] The 824A-824N memory units can include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as, but not limited to, synchronous graphics random access memory (SGRAM), including double data rate graphics memory (GDDR). The 824A-824N memory units can also include 3D stacked memory, including, but not limited to, high-bandwidth memory (HBM), HBM2e, or HDM3. Render targets, such as frame stores or texture maps, can be distributed across the 824A-824N memory units, allowing the 820A-820N partition units to write portions of each render target in parallel to efficiently utilize the available bandwidth of the 822 parallel processor memory.A local instance of the Parallel Processor Memory 822 can be excluded in favor of a unified memory design that utilizes system memory in conjunction with local cache memory.

[0083] Each of the 814A-814N clusters of the 812 processing cluster array can process data written to any of the 824A-824N memory units within the 822 parallel processor memory. The 816 memory crossbar can be configured to transfer an output from each 814A-814N cluster to any 820A-820N partition unit or to another 814A-814N cluster that can perform additional processing operations on the output. Each 814A-814N cluster can communicate with the 818 memory interface via the 816 memory crossbar to read from and write to various external storage devices.The Memory Crossbar 816 can communicate with the I / O Unit 804 via a connection to the Memory Interface 818, as well as with a connection to a local instance of the Parallel Processing Unit 822. This allows processing units within different processing clusters 814A-814N to communicate with system memory or other memory not belonging to the Parallel Processing Unit 802. The Memory Crossbar 816 can use virtual channels to separate data streams between the clusters 814A-814N and the partitioning units 820A-820N.

[0084] Multiple instances of the Parallel Processing Unit 802 can be deployed on a single expansion card, or multiple expansion cards can be interconnected. Different instances of the Parallel Processing Unit 802 can be configured to work together, even if they have different numbers of processor cores, different amounts of local parallel processor memory, and / or other configuration differences. For example, some instances of the Parallel Processing Unit 802 can include higher-precision floating-point units compared to other instances.Systems incorporating one or more instances of the Parallel Processing Unit 802 or the Parallel Processor 800 can be implemented in a variety of configurations and form factors, including, but not limited to, desktop, laptop or handheld PCs, servers, workstations, game consoles and / or embedded systems.

[0085] Fig. Figure 8 further contains a block diagram of a partitioning unit 820 according to at least one embodiment. The partitioning unit 820 is an instance of one of the partitioning units 820A-820N from Fig. 8. The partitioning unit 820 can include an L2 cache 821, an image storage interface 825, and a ROP 826 (Raster Operations Unit). The L2 cache 821 can be a read / write cache configured to perform load and store operations received from the memory crossbar 816 and the ROP 826. Read errors and urgent write-back requests can be sent from the L2 cache 821 to the image storage interface 825 for processing. Updates can also be sent to an image storage device via the image storage interface 825 for processing. The image storage interface 825 can be connected to any of the memory units in the parallel processor memory, such as, but not limited to, the memory units 824A-824N from Fig. 8 (e.g., within the parallel processor memory 822).

[0086] The ROP 826 can be a processing unit that performs raster operations such as, but not limited to, stenciling, Z-testing, blending, etc. The ROP 826 can then output processed graphics data stored in graphics memory. The ROP 826 can include compression logic to compress depth or color data written to memory and decompress depth or color data read from memory. The compression logic can be lossless and use one or more of several compression algorithms. The type of compression performed by the ROP 826 can vary depending on the statistical properties of the data being compressed. For example, delta color compression is performed on depth and color data on each tile.

[0087] The ROP 826 can be used in any processing cluster (e.g., clusters 814A-814N in Fig. 8) instead of being contained within the partitioning unit 820. Read and write requests for pixel data can be transferred via the memory crossbar 816 instead of pixel fragment data. Processed graphics data can be displayed on a screen, which can then be forwarded for further processing by one or more processors 1502, or for further processing by one of the processing units within the parallel processor 800. Fig. 8 will be forwarded.

[0088] In at least one embodiment, the parallel processor 800 can include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0089] Fig. Figure 9 shows a Processor 900 according to at least one embodiment. The Processor 900 may comprise a hybrid-architecture processor (e.g., Lunar Lake or Meteor Lake) from Intel Corporation in Santa Clara, California, or any other processor incorporating at least some of the components described herein. The Processor 900 may include one or more central processing units (CPU 902), one or more graphics processing units (GPU 906), and / or one or more neural processing units (NPU 908), which may, for example, be a dedicated AI accelerator that offloads AI workloads from the CPU and GPU. The Processor 900 may use instructions that, when executed, cause the Processor 900 and / or one of its components to perform some or all of the processes and techniques described elsewhere herein.The Processor 900 can contain any number of Memory and Cache Units 910 to facilitate interprocessing between the various components. The Memory and Cache 910 on the Processor 900 can include one or more cache levels (e.g., L1, L2, L3, and / or last-level cache) and high-bandwidth memory (e.g., HBM2e or HBM3) in any combination. With respect to the Processor 900 and each of its components described above or elsewhere herein, one or more of the APIs described herein can, for example, be compiled into instructions that can be fetched by an instruction call logic or equivalent, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by an execution logic or equivalent, reordered, and subsequently terminated by a termination logic or equivalent.One or more APIs (and / or compiled instructions including one or more APIs) can be stored in any memory location outside or inside the processor (e.g., in the cache and / or memory). A result of one or more APIs can then be stored in memory location inside or outside the processor, including registers, DRAM, flash, SRAM, cache, or other memory. One or more of the APIs described here can contain a call.

[0090] The 900 series processor can contain computing units such as the 902 CPU and any number of cores, such as, but not limited to, up to 16 cores / 22 threads. The cores in the 902 CPU can be classified as P cores (Performance), E cores (Efficiency), and LP-E cores (Low-Power Efficient). Performance cores can be used for low-latency, single-threaded, and computationally intensive workloads, while Efficiency cores can be used for multithreaded, less computationally intensive workloads. Low-power Efficient cores can be used for scalable multithreaded performance and offloading background tasks. P cores can be used for single-threaded and limited-thread performance, while E and LP-E cores can be used for multithreaded throughput and energy efficiency.

[0091] The GPU 906 can contain any number of graphics units, such as, but not limited to, Intel® Arc™ Graphics Units (Xe LPG) with 8 Xe cores (up to 128 execution units (EUs)). As described in Fig. As shown in Figure 9, the GPU 906 can include vector units 910 and matrix units 912, which can, for example, perform FP, INT, and matrix operations concurrently, separately, or in batches. The GPU 906 can include a load / store unit 914, as well as other memory, such as, but not limited to, an instruction cache (I$) 916 and a subsystem local memory (SLM) L1 cache 918, which can, for example, store instructions for performing any of the operations described above or elsewhere herein.

[0092] The NPU 904 can contain one or more integrated Neural Processing Units (NPUs) with Intel® AI Boost. The NPU 904 can be mapped to the host processor as an integrated PCIe device. The NPU 904 can contain one or more (e.g., two) Neural Compute Engine (NCE) 930 tiles. Each tile can be configured with any combination of (e.g., 2000) Multiplication-Accumulation Units (MAC) 934, a Post-Processing Unit (not shown), an AI DSP Processor (not shown), and memory (2 MB dedicated SRAM) per tile, as shown in Fig. 9 shown. For general-purpose computing needs, the neural computing units 930 can include streaming hybrid architecture vector engines (SHAVE) 928 for high-performance parallel computing, which can include direct memory access (DMA) units 924 for transferring data between the system memory DRAM (Dynamic Random Access Memory) 926 and a software-controlled cache. The device's integrated memory management unit (MMU) 922 and input-output memory management unit (IOMMU) (not shown) can support multiple concurrent hardware contexts and provide security isolation between execution contexts according to the Microsoft Compute Driver Model (MCDM) architecture.The Processor 900 may also include a media unit (not shown) that is contained within the XCDs or other components of the processor, or exists separately, to enable video playback and video processing of compressed or uncompressed data, for example, using HEVC, AV1, VP9 and AVC-HW accelerated decoding support and HEVC, VP9 and AVC-HW accelerated encoding support.

[0093] An Intel® Thread Director, which includes firmware integrated into the processor, can prioritize and manage workload distribution and send tasks to optimized cores. For example, the Thread Director can connect P cores, E cores, and / or LP-E cores (as described above) with task scheduling capabilities and the ability to send less demanding tasks to the E cores or LP-E cores. Intel® Deep Learning Boost (Intel® DL Boost) (not shown) can provide built-in AI acceleration for training and inference workloads and may include VNNI (for CPU) and DP4a (for GPU) instruction set support. This instruction set can be optimized with the OpenVINO™ Toolkit and oneAPI to accelerate INT8 inference. A software stack, such as the one described elsewhere herein, can be used to enable AI inference with the OpenVINO™ Toolkit.The 900 processor can be configured to run an application program, such as, but not limited to, a CUDA program.

[0094] In at least one embodiment, the processor 900 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0095] The Processor 900 may alternatively comprise a processor based on Qualcomm Corporation's AI Engine Direct architecture in Santa Clara, California, or any other processor incorporating at least some of the components described herein, comprising any number of NPUs, GPUs, CPUs, and other associated components, such as, but not limited to, the NPU 904 as a Hexagon NPU, the GPU 906 as an Adreno GPU, the CPU 902 as a Qualcomm Kryo or Oryon CPU, a Qualcomm Sensing Hub (not shown), and a Memory Subsystem 910 in any combination. The Hexagon NPU 904 may include a power rail, a micro-tile inference unit, a hardware acceleration unit, a tensor unit, a scalar unit, and a vector unit (all not shown), which may have dedicated memory or share memory (e.g.,Cache or memory (such as HBM3) to store instructions for performing any of the operations described above or elsewhere herein. The Adreno GPU 906 can provide graphics and parallel processing for AI in formats such as, but not limited to, 32-bit floating-point (FP32), 16-bit floating-point (FP16), and 8-bit integer (INT8). Qualcomm Kryo or Oryon CPUs 902 can execute AI workloads and provide contextualization for ubiquitous generative AI applications. The CPU 902 can also include an instruction fetch unit, a rename and finish unit, a memory management unit, a vector execution unit, an integer execution unit, and a load and store unit for processing and instruction management.With respect to the Processor 900 and each of its components described above or elsewhere herein, one or more of the APIs described herein may, for example, be compiled into instructions that may be fetched by the Instruction Fetcher, decoded by a Processor Decoder or equivalent, scheduled for execution by a Scheduler or equivalent (e.g., sequentially or out of sequence), executed by Execution Logic or equivalent, reordered, and subsequently terminated by the Rename and Terminate Unit. One or more APIs (and / or compiled instructions including one or more APIs) may be stored in any memory outside or inside the Processor (e.g., in the cache and / or memory).Any number of CPU 902 cores can be contained in any number of CPU clusters, which can be coupled to memory and / or cache, such as, but not limited to, a shared L2 cache. Memory can be used separately or shared; for example, CPU clusters of CPU 902 cores can be coupled to a Memory Subsystem 910, which can contain a fabric, a system-level cache, and any number of memory management units capable of reading and writing to memory (e.g., DRAM). Qualcomm's Sensing Hub (not shown) includes micro-NPUs, a power rail, and conventional sensors (a gyroscope, an accelerometer, even a barometer) carrying voice and data streams. The Memory Subsystem 910 can contain memory and cache on the Processor 900, which can have one or more cache levels (e.g.,The memory subsystem 910 can include L1, L2, L3, and / or last-level cache) and high-bandwidth memory (e.g., HBM2e or HBM3) in any combination, for example, to store information and / or instructions for performing any of the operations described above or elsewhere herein. All or part of the memory and / or cache in the memory subsystem 910 can be shared or used individually by one or more components (e.g., GPU 906, NPU 904, and CPU 902) on the processor 900.

[0096] The Qualcomm 900 AI unit can be programmed and controlled with a software stack to perform some or all of the operations described here. This includes, for example, a Qualcomm® Neural Processing SDK for inference, with versions for Android, Linux, and Windows. Developer libraries and services support the latest programming languages, virtual platforms, and compilers. At a lower level of the software stack, the system software includes the core real-time operating system (RTOS), system interfaces, and drivers. The software stack supports various operating systems, including Android, Windows, Linux, and QNX, as well as deployment and monitoring infrastructures such as Prometheus, Kubernetes, and Docker. OpenCL and DirectML are supported for direct cross-platform access to the GPU.For the CPU, optimizations to the LLVM compiler infrastructure enable accelerated and efficient AI inference. With respect to the Qualcomm 900 AI unit and each of its components described above or elsewhere herein, one or more of the APIs described herein can, for example, be compiled into instructions that can be fetched by instruction-fetch logic or equivalent, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by execution logic or equivalent, reordered, and subsequently terminated by termination logic or equivalent. One or more APIs (and / or compiled instructions including one or more APIs) can be stored in any memory location outside or inside the processor (e.g., in the cache and / or memory).A result from one or more APIs can then be stored in memory inside or outside the processor, including registers, DRAM, flash, SRAM, cache or other memory.

[0097] In at least one embodiment, the Processor 900 or the AI ​​Unit of Qualcomm 900 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0098] Fig. Figure 10A illustrates a processor 1000 according to at least one embodiment. The processor 1000 may comprise a processor from the Intel Corporation Scalable Processor Family of Santa Clara, California, or any other processor incorporating at least some of the components described herein. The processor 1000 may contain one or more cores 1012(1)-1012(N), where N is any integer greater than 1 capable of performing the operations described elsewhere herein. The cores 1012(1)-1012(N) may be interconnected via ring and / or mesh connections. In the mesh connection architecture, an arrangement of vertical and horizontal communication paths may enable communication from one core to another 1012(1)-1012(N) via a shortest path (hop along the vertical path to the correct row and hop along the horizontal path to the correct column).In mesh connections, a die can accommodate cores 1012(1)-1012(N) and contain a grid of converged mesh stops (CMS) that can be mapped to cores 1012(1)-1012(N) (e.g., 1:1). Each core can be mapped to a slice of the lower-level cache (LLC) 1014(1)-1014(N), or cores 1012(1)-1012(N) can share a cache, such as a lower-level cache. The LLCs 1014(1)-1014(N) can be inclusive, containing blocks in a higher-level cache (e.g., L2 cache), or non-inclusive (containing blocks that may not be present in the higher-level cache). Each core and each LLC slice can contain a Caching and Home Agent (CHA) (not shown) that can maintain cache coherence by providing resource scalability over mesh connections for the cache coherence functionality of the Intel® Ultra Path Interconnect System (Intel® UPI 1016).The UPI 1016 can provide a coherent connection for scalable systems and allow multiple processors to share a single common address space via connections, such as, but not limited to, two or three UPI connections per processor.

[0099] The Processor 1000 may also include the System Agent 1010, which can host and / or perform various functions, such as, but not limited to, memory management, display functions, and / or input / output (I / O) functions. For example, the Processor 1000 may include one or more Integrated Memory Controllers (IMCs) 1008. The IMC 1008 can control and manage memory, such as, but not limited to, various memory types, e.g., DDR RAM, such as DDR4, or others described elsewhere herein. The System Agent 1010 may include a display controller (not shown) to support one or more displays. The System Agent 1010 can also contain PCIe 1004 (e.g., up to 20 PCIe lanes), which can be connected to an external dedicated graphics connector via a DMI bus (e.g., Intel's DMI 3.0 bus) 1006.The System Agent 1010 can include an Image Processing Unit (IPU) (not shown) that contains an Image Signal Processor (ISP) on the die. The Fabric 702 can provide scalability for the connection.

[0100] Fig. Figure 10B illustrates components within the core 1012 according to at least one embodiment. The core 1012 can comprise a frontend 1018, a backend or execution unit 1032, and a memory subsystem 1042. The frontend 1018 can provide the execution unit 1032 with operations (e.g., operations described elsewhere herein) by decoding instructions stored in memory. For example, the frontend 1018 can include a cache path for micro-operations (µOps) and / or a legacy path, as well as a branch prediction unit 1020 that can determine path instructions. A legacy path for instructions can include retrieving variable-length instructions (e.g., x86) from the L1 instruction cache, queuing the instructions into instruction queue 1024, and decoding the instructions using decoder 1026 into µOps that can be provided to assignment queue 1028.Alternatively, a µOPs cache path can include a cache containing pre-decoded µOps (µOps 1030) that can be sent to the allocation queue 1028. The allocation queue 1028 can act as an interface between the frontend 1018 and the execution unit 1032, providing instructions to the execution unit 1032. For example, one or more of the APIs described here can be compiled into instructions that can be stored, processed, and executed by the frontend 1018 and the execution unit 1032, and stored in the storage subsystem 1042.

[0101] The execution unit 1032 can receive micro-operations in the reorder buffer 1034, which can register, rename, and complete these micro-operations. From the reorder buffer, micro-operations can be sent to the scheduler 1036, which can be connected to one or more different execution units 1038. The execution units 1038 can perform, for example, basic arithmetic logic unit (ALU) operations, multiplications, divisions, and / or more complex operations, such as, but not limited to, various vector operations. The scheduler 1036 can queue the micro-operations for one or more execution units 1038, for example, depending on the operations to be performed.

[0102] The 1042 memory subsystem can handle load and store requests as well as order operations. For example, µOPs can relate to memory accesses (such as load and store), and these can be sent to dedicated scheduler ports that can perform these memory operations. Store and load operations, for example, can be sent to one or more 1044 load and store buffers. The 1042 memory subsystem can also include a shared or separate L1 data and instruction cache 1046, as well as an L2 cache 1048 that can be used and shared with the L1 data and instruction cache 1046. As above for Fig. As described in 10A, each 1012 core can be connected to a slice of a third-level cache (e.g., LLC 1014) that can be shared by all 1012 cores.

[0103] In at least one embodiment, the processor 1000 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0104] A neuromorphic computing system is described that uses a multi-core architecture, where each core contains the computing elements including neurons, synapses with on-chip learning capability, and local memory for storing synaptic weights and routing tables. Fig. Figure 11 is a simplified block diagram 1100 illustrating an example of at least one part of such a neuromorphic computing device 1105 according to at least one embodiment. The neuromorphic computing device 1105 may include a neuromorphic processor from Intel Corporation in Santa Clara, California, or another processor incorporating at least some of the components described herein. As shown in this example, a device 1105 may be provided with a network 1110 consisting of multiple neural network cores interconnected by an internal network, such that several different connections between the cores can potentially be defined.For example, a network 1110 consisting of pulsed neural network cores can be provided in the device 1105, with each core able to communicate via short packetized pulse messages (spike messages) sent from core to core across the network channels. Each core (e.g., 1115) can have processing and memory resources, as well as logic, to implement a certain number of primitive nonlinear temporal computational elements, such as, but not limited to, multiple (e.g., 1000+) distinct artificial neurons (referred to here as "neurons"). For example, each core can be capable of implementing multiple neurons simultaneously, so the collection of neuromorphic cores can implement many multiples of neurons using the device.With respect to the neuromorphic computing device 1105 and each of its components described above or elsewhere herein, one or more of the APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by an instruction call logic or equivalent, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by an execution logic or equivalent, reordered, and subsequently terminated by a termination logic or equivalent. One or more APIs (and / or compiled instructions including one or more APIs) may be stored in any memory outside or inside the processor (e.g., in the cache and / or memory).A result from one or more API(s) can then be stored in memory inside or outside the processor, including registers, DRAM, Flash, SRAM, cache or other memory equivalents.

[0105] Continuing the example from Fig. A neuromorphic computing device 1105 can additionally include a processor 1120 and system memory 1125 to implement one or more components for managing and providing the functionality of the device. For example, a system manager 1130 can be provided to manage global attributes and operations of the device (e.g., attributes affecting the network of cores 1110, multiple cores in the network, connections of the device 1105 with other devices, managing access to the global system memory 1125, among other possible examples). In one example, a system manager 1130 can define and provide specific routing tables for the various routers in the network 1110, orchestrate a network definition and attributes (e.g., weights, decay rates, etc.).), which are to be used in the network, manage core synchronization and time-division multiplexing, forward inputs to the appropriate cores and other possible functions.

[0106] As a further example, a neuromorphic computing device 1105 can additionally include a programming interface 1135 through which a user or a system can specify a neural network definition to be applied (e.g., via a routing table and individual neuron properties) and an application to be implemented by the network 1110 of neuromorphic cores. A software-based programming tool can be provided together with or separately from the neuromorphic computing device 1105, through which a user can provide a definition for a specific neural network to be implemented using the network 1110 of neuromorphic cores. The programming interface 1135 can accept the programmer's input in order to then generate corresponding routing tables and access the local memory of individual neuromorphic cores (e.g.,1115) to fill with the specified parameters in order to implement a corresponding, adapted network of artificial neurons, which is implemented by the neuromorphic nuclei.

[0107] In some cases, a neuromorphic computing device 1105 can be advantageously connected to and cooperate with other devices, including general-purpose computing devices, to realize specific applications and use cases. Accordingly, in some cases, an external interface logic 1140 may be provided to communicate with one or more other devices (e.g., via one or more defined communication protocols). An external interface 1140 can be used to accept input data from another device or an external storage controller acting as the input data source. An external interface 1140 can additionally or alternatively be used to transmit results or outputs of computations from a neural network implemented using the neuromorphic computing device 1105 to another device (e.g., a computer, a computer, or a storage controller).to provide another general-purpose processor that implements a machine learning algorithm) in order to realize, among other things, additional applications and improvements.

[0108] As in Fig. Figure 11 shows a network 1110 consisting of multiple neural network cores connected by an in-device network to illustrate part of a network structure that connects multiple neuromorphic cores (e.g., 1115 ad). For example, multiple neuromorphic cores (e.g., 1115 ad) can be deployed in a network, with each core connected by a network comprising multiple routers (e.g., 1150). In one implementation, each neuromorphic core (e.g., 1115 ad) can be connected to a single router (e.g., 1150), and each router can be connected to at least one other router (as shown in Figure 1110). Fig. (11 shown). In a particular implementation, for example, four neuromorphic cores (e.g., 1115 ad) can be connected to a single router (e.g., 1150), and each of the routers can be connected to two or more other routers to form a manycore network, allowing each of the neuromorphic cores to be connected to every other neuromorphic core in the device. Since each neuromorphic core can be configured to implement multiple different neurons, the device's router network can similarly define connections, or artificial synapses (or simply "synapses"), between any two of the potentially many (e.g., 30,000+) neurons defined by the network of neuromorphic cores provided in a neuromorphic computing device.

[0109] Fig. Figure 11 shows a block diagram illustrating the internal components of an exemplary implementation of a neuromorphic kernel 1115. In one example, a single neuromorphic kernel can implement a certain number of neurons (e.g., 1024) that share the neuromorphic kernel's architectural resources in a time-division multiplexed manner. In another example, each neuromorphic kernel 1115 can comprise a processor block 1155 capable of performing arithmetic functions and routing in connection with the realization of a digitally implemented artificial neuron, such as, but not limited to, what is described here.Each neuromorphic core 1115 can additionally provide local memory in which a routing table for a neural network can be stored and retrieved, the accumulated potential of each soma of each neuron implemented using the core can be tracked, parameters of each neuron implemented by the core can be recorded, among other data and uses. Components or architectural resources of a neuromorphic core 1115 can further include an input interface 1165 for accepting input pulse messages generated by other neurons on other neuromorphic cores, and an output interface 1170 for sending pulse messages to other neuromorphic cores via the mesh network. In some cases, the routing logic for the neuromorphic core 1115 can be implemented at least partially using the output interface 1170. Furthermore, in some cases, a core (e.g.,1115) implement several neurons within an exemplary SNN, and some of these neurons may be interconnected. In such cases, pulse messages sent between the neurons hosted on the specific nucleus can bypass communication via the routing structure of the neuromorphic computing device and instead be managed locally on the specific neuromorphic nucleus.

[0110] Each neuromorphic core can additionally contain logic to implement an artificial dendrite 1180 and an artificial soma 1185 (referred to here simply as "dendrite" and "soma," respectively) for each neuron 1175. The dendrite 1180 can be a hardware-implemented process that receives pulses (spikes) from the network. The soma 1185 can be a hardware-implemented process that receives the accumulated neurotransmitter amounts of each dendrite for the current time and further develops the potential state of each dendrite and soma to generate outgoing pulse messages at appropriate times. A dendrite 1180 can be defined for each connection that receives input from another source (e.g., another neuron). In one implementation, the dendrite process 1180 can receive and process pulse messages as they arrive serially from the network in a time-division multiplexed manner.When impulses are received, the activation of the neuron (tracked by Soma 1185 and local memory 1160) can increase. If the neuron's activation exceeds a threshold set for neuron 1175, the neuron can generate a impulse message that is relayed via output interface 1170 to a fixed set of fanout neurons. The network distributes the impulse messages to all target neurons, and in response, these neurons can update their activations in a transient, time-dependent manner, and so on. This may cause the activation of some of these target neurons to also exceed the corresponding thresholds and trigger further impulse messages, as in real biological neural networks.

[0111] As mentioned above, a neuromorphic computing device can reliably implement an impulse-based model of neural computation. Such models can also be referred to as spiking neural networks (SNNs). In addition to neuronal and synaptic state, SNNs also integrate the concept of time. In an SNN, communication occurs, for example, via event-driven action potentials or pulses (spikes) that convey no explicit information other than the pulse time and an implicit source-target neuron pair corresponding to the transmission of the pulse. Computation takes place in each neuron as a result of the dynamic, nonlinear integration of the weighted pulse input. In some implementations, recurrence and dynamic feedback can be integrated into a computational SNN model.Furthermore, various network connectivity models can be used to model different real-world networks or relationships, including fully connected (all-to-all) networks, feedforward trees, completely random projections, small-world networks, and other examples. A homogeneous, two-dimensional network of neuromorphic kernels, such as in the example in [reference missing], is one such example. Fig. As shown in Figure 11, but not limited to, the device can advantageously support all these network models. Since all cores of the device can be interconnected, all neurons defined in the cores can also be fully interconnected via a specified number of router hops. Furthermore, the device can include fully configurable routing tables to define a variety of different neural networks by allowing the neurons of each core to distribute their impulses to any number of cores in the network, thus realizing completely arbitrary connectivity graphs.

[0112] In an improved implementation of a system that can support SNNs, such as, but not limited to, those in the example of Fig. Figure 9 illustrates how very large-scale integration (VLSI) hardware devices can provide fast and reliable circuits to implement SNNs (sequencing neural networks) to model the information processing algorithms used by the brain, but in a more programmable way. For example, while a biological brain can only implement a specific set of defined behaviors conditioned by years of development, a neuromorphic processor device can provide the ability to rapidly reprogram all neural parameters. Accordingly, a single neuromorphic processor can be used to realize a wider range of behaviors than what a single piece of biological brain tissue can offer.This distinction can be achieved by using a neuromorphic processor with neuromorphic design implementations that differ significantly from those of naturally occurring neural circuits.

[0113] For example, a neuromorphic processor can utilize time-division multiplexing (TDM) in both the pulse communication network and the device's neural machinery to implement SNNs. Accordingly, the same physical circuitry of the processor device can be shared by many neurons to achieve a higher neuron density. With TDM, the network can connect N cores with a total wiring length of O(N), whereas discrete point-to-point wiring would scale as O(N²), resulting in a significant reduction in wiring resources and enabling, among other things, planar and non-plastic VLSI wiring technologies.In neuromorphic nuclei, time-division multiplexing can be implemented through dense memory allocation, for example, using static random access memory (SRAM) with shared buses, address decoding logic, and other multiplexed logic elements. The state of each neuron can be stored in the processor's memory, with the data describing the state of each neuron, including, but not limited to, configuration and other information, the state of each neuron's collective synapses, all currents and voltages across its membrane, and other information.

[0114] A neuromorphic processor can use a "digital" implementation, which differs from other processors that employ more "analog" or "isomorphic" neuromorphic approaches. For example, a digital implementation can integrate synaptic current using digital adder and multiply circuits, in contrast to analog isomorphic neuromorphic approaches, which accumulate charge on capacitors in an electrically analogous way to how neurons accumulate synaptic charge on their lipid membranes. The accumulated synaptic charge can, for example, be stored for each neuron in the local memory of the corresponding nucleus.Furthermore, at the architectural level of an exemplary digital neuromorphic processor, reliable and deterministic operation can be achieved by synchronizing time across the network of cores, ensuring that any two implementations of the design, under identical initial conditions and configurations, produce identical results. Asynchronicity can be maintained at the circuit level to allow individual cores to operate as quickly and freely as possible, while preserving determinism at the system level. Accordingly, the concept of time as a temporal variable in neural computations can be abstracted by separating it from the "wall clock" time used by the hardware to perform the computation. Consequently, some implementations may include a time synchronization mechanism that globally synchronizes the neuromorphic cores at discrete time intervals.The synchronization mechanism allows the system to perform a neural computation as quickly as possible, with a deviation between the runtime and the biological time that the neuromorphic system models.

[0115] In operation, the neuromorphic mesh device can begin in a resting state where all neuromorphic cores are inactive. As each core asynchronously traverses its neurons, it generates pulse messages that the mesh connection network forwards to the corresponding target cores, which contain all the target neurons. Since the implementation of multiple neurons on a single neuromorphic core can be temporally multiplexed, a time step can be defined in which all pulses involving the multiple neurons can be processed and accounted for using the shared resources of a corresponding core. Once each core has completed processing its neurons for a given time step, the cores can communicate with neighboring cores in some implementations (e.g.,(via handshake), by using synchronization messages to clear the network of all pulse messages currently in transmission, so that the cores can reliably determine that all pulses for the time step have been processed. At this point, all cores can be considered synchronized, allowing them to advance their time step, return to the initial state, and begin the next time step.

[0116] Against this background, and as presented above, a device (e.g., 1105) can be provided that implements a network 1110 of interconnected neuromorphic nuclei, with the nucleus potentially implementing multiple artificial neurons that can be interconnected to implement a SNN. Each neuromorphic nucleus (e.g., 1115) can provide two loosely coupled asynchronous processes: an input dendrite process (e.g., 1180) that receives pulses from the network and applies them to the appropriate target dendrite compartments at the appropriate future time points, and an output soma process (e.g., 1185) that receives the accumulated neurotransmitter amounts of each dendrite compartment for the current time point and further develops the membrane potential state of each dendrite and soma, generating outgoing pulse messages at the appropriate time points (e.g., when a threshold potential of the soma has been reached).It should be noted that from a biological point of view, the terms "dendrite" and "soma" used here only approximate the role of these functions and should not be interpreted too literally.

[0117] In at least one embodiment, the neuromorphic computing device 1105 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0118] Fig. Figure 12 is a block diagram of an embodiment of a multi-node network in which remote memory computation can be implemented according to any embodiment. System 1200 can represent a network of nodes described herein, which can be used, for example, to perform some or all of the operations described herein. System 1200 can represent a data center. System 1200 can represent a server farm. System 1200 can represent a data cloud or a processing cloud. System 1200 can represent a supercomputer. System 12 can comprise dozens, hundreds, or thousands of nodes. The nodes of System 1200 can contain processors, such as, but not limited to, central processing units (CPUs), graphics processing units (GPUs), or any combination of processors described herein, such as, but not limited to, other processors in Fig. 7-18. With respect to any of the processors in the System 1200 and any of its components described above or elsewhere herein, one or more of the APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by an instruction call logic or equivalent, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by an execution logic or equivalent, reordered, and subsequently terminated by a termination logic or equivalent. One or more APIs (and / or compiled instructions including one or more APIs) may be stored in any memory outside or inside the processor (e.g., in the cache and / or memory).A result from one or more APIs can then be stored in memory inside or outside the processor, including registers, DRAM, flash, SRAM, cache, or other memory equivalents. The System 1200 can comprise over 9000 nodes, with each node comprising two Intel Xeon Max processors, six Intel Max series GPUs, and a uniform memory architecture, such as, but not limited to, that used in Intel Corporation's Intel Aurora supercomputer in Santa Clara, California, or any other supercomputer incorporating at least some of the components described herein.

[0119] One or more clients 1202 make requests to system 1200 via network 1204. Network 1204 represents one or more local area networks, wide area networks, or a combination thereof. The clients 1202 can be human or machine clients that generate requests for system 1200 to perform operations. System 1200 executes applications or data processing tasks requested by the clients 1202.

[0120] The System 1200 can comprise one or more racks, which provide structural and interconnect resources for housing and connecting multiple compute nodes. The Rack 1210 can contain multiple nodes 1230. The Rack 1210 can host multiple blade components 1220. Hosting can refer to the provision of power, structural or mechanical support, and interconnects. The Blades 1220 can refer to compute resources on printed circuit boards (PCBs), with each PCB carrying the hardware components for one or more nodes 1230. The Blades 1220 may or may not include a rack, enclosure, or other "box" not provided by the Rack 1210. The Blades 1220 may include an enclosure with an exposed connector for connection to the Rack 1210.The System 1200 may or may not include a Rack 1210, and each Blade 1220 may include a rack or enclosure that can be stacked or otherwise arranged in close proximity to other Blades, enabling the connection of Nodes 1230. The System 1200 can contain 10,624 compute Blades, housing 63,744 Intel Max Series GPUs and 21,248 Intel Xeon Max CPUs in 166 racks.

[0121] System 1200 can include a Fabric 1270, which represents one or more connections for nodes 1230. Fabric 1270 can include multiple switches 1272, routers, or other hardware to forward signals between the nodes 1230. Additionally, Fabric 1270 can connect System 1200 to the network 1204 to enable access by clients 1202. Besides routing equipment, Fabric 1270 can also include the cables, ports, or other hardware that connect the nodes 1230. Fabric 1270 can have one or more associated protocols to manage the forwarding of signals through System 1200. The protocol(s) depend, at least in part, on the hardware used in System 1200.

[0122] As illustrated, Rack 1210 can contain N Blades 1220. In addition to Rack 1210, System 1200 can contain Rack 1250. As illustrated, Rack 1250 can contain M Blades 1260. M is not necessarily equal to N; therefore, it is understood that different hardware components can be used and coupled together via Fabric 1270 to form System 1200. The Blades 1260 can be identical to or similar to the Blades 1220. The Nodes 1230 can be any node type described here and do not necessarily all have to be of the same type. System 1200 is neither restricted to homogeneity nor to non-homogeneity.

[0123] A node in Blade 1220(0) is illustrated in detail. However, other nodes in System 1200 may be the same or similar. At least some nodes 1230 may be compute nodes with one processor 1232 and one memory 1240. A compute node refers to a node with processing resources (e.g., one or more processors) that runs an operating system and can receive and process one or more tasks. At least some nodes 1230 may include storage server nodes with a server as processing resources 1232 and memory 1240. A storage server refers to a node with more storage resources than a compute node, and instead of having processors to perform tasks, a storage server includes processing resources to manage access to the storage nodes within the storage server.

[0124] The node 1230 can include an interface controller 1234, which can represent logic for controlling the node 1230's access to the fabric 1270. The logic can include hardware resources for connecting to the physical connection hardware. The logic can also include software or firmware logic for managing the connection. The interface controller 1234 can be a host fabric interface or include one that can be a fabric interface according to an embodiment described herein.

[0125] Node 1230 can include a storage subsystem 1240. Storage 1240 can include storage computation resources (comp) 1242, which represent one or more capabilities of storage 1240 for performing storage computations. System 1200 enables remote storage operations, such as, but not limited to, those described elsewhere herein. Thus, nodes 1230 can request storage computations from remote nodes, with the data for the computation remaining locally on the executing node instead of being sent via Fabric 1270 or from storage to the Fabric interface. In response to the execution of the storage computation, the executing node can deliver a result to the requesting node.

[0126] The 1232 processor can comprise one or more separate processors. Each separate processor can comprise a single processing unit, a multi-core processing unit, or a combination thereof. The processing unit can be a main processor, such as, but not limited to, a CPU (central processing unit), a peripheral processor, such as, but not limited to, a GPU (graphics processing unit), or a combination of these. The 1240 memory can be or include memory devices and a memory controller.

[0127] The term "storage devices" can refer to various types of storage. Generally, "storage devices" refers to volatile memory technologies. Volatile memory is memory whose state (and therefore the data stored on it) is indeterminate when the device's power supply is interrupted. Non-volatile memory refers to memory whose state remains definite even when the device's power supply is interrupted. Dynamic volatile memory requires the data stored in the device to be refreshed to maintain its state. Examples of dynamic volatile memory include DRAM (Dynamic Random Access Memory) or a variant thereof, such as, but not limited to, synchronous DRAM (SDRAM).A memory subsystem described here may be compatible with a number of memory technologies, such as, but not limited to, DDR3 (Dual Data Rate Version 3, originally published by JEDEC (Joint Electronic Device Engineering Council) on 27.June 2007 (currently in version 21), DDR4 (DDR version 4, first specification published in September 2012 by JEDEC), DDR4E (DDR version 4, extended, currently under discussion by JEDEC), LPDDR3 (Low Power DDR version 3, JESD209-3B, August 2013 by JEDEC), LPDDR4 (LOW POWER DOUBLE DATA RATE (LPDDR) version 4, JESD209-4, originally published by JEDEC in August 2014), WIO2 (Wide I / O 2 (WideI02), JESD229-2, originally published by JEDEC in August 2014), HBM (HIGH BANDWIDTH MEMORY DRAM, JESD235, originally published by JEDEC in October 2013), DDR5 (DDR version 5, currently under discussion by JEDEC), LPDDR5 (currently under discussion by JEDEC), HBM2 (HBM Version 2, currently under discussion by JEDEC) or other or combinations of memory technologies and technologies based on derivatives or extensions of such specifications.

[0128] In addition to or as an alternative to volatile memories, in one embodiment the reference to memory devices may refer to a non-volatile memory device whose state is determined even when the device's power supply is interrupted. In one embodiment, the non-volatile memory device is a block-addressable memory device, such as, but not limited to, NAND or NOR technologies. Thus, a memory device may also include future-generation non-volatile devices, such as, but not limited to, a three-dimensional crosspoint memory device (3DXP), other byte-addressable non-volatile memory devices, or memory devices that use a chalcogenide phase-change material (e.g., chalcogenide glass).In one embodiment, the storage device may be or include a multi-threshold NAND flash memory, a NOR flash memory, a single- or multi-stage phase change memory (PCM) or a single-switch phase change memory (PCMS), a resistive memory, a nanowire memory, a ferroelectric transistor random access memory (FeTRAM), a magnetoresistive random access memory (MRAM) using memristor technology, or a spin transfer torque (STT) MRAM, or a combination of any of the above, or any other memory.

[0129] In at least one embodiment, the system 1200 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0130] Fig. Figure 13 illustrates the accelerated processing unit 1300 according to at least one embodiment. The accelerated processing unit 1300 may comprise a processor based on a CDNA architecture from AMD Corporation in Santa Clara, California, or any other processor incorporating at least some of the components described herein. The accelerated processing unit 1300 may include one or more Accelerator Complex Dies (XCDs) 1304 for performing operations described elsewhere herein, such as, but not limited to, graphics processing and / or parallel processing, as well as instruction-level parallel computations, including support for a wide range of precisions (INT8, FP8, BF16, FP16, TF32, FP32, and FP64) and sparse matrix data. XCDs may, in some cases, be referred to as graphics compute dies (GCDs).The Accelerated Processing Unit 1300 can include one or more Complex Compute Dies (CCDs) 1306 to perform operations described elsewhere herein, such as, but not limited to, those operations performed by host processors. CCDs may in some cases be referred to as Core Complexes or CCXs, such as, but not limited to, CCXs used in AMD Ryzen processors. XCDs and CCDs can share any type of cache or memory (e.g., one or more Memory Units 1302) or have cache or memory allocated to each XCD or CCD or group of XCDs or CCDs. On the package, for example, the AMD Infinity Fabric connects the XCDs and CCDs to the shared AMD Infinity Cache 1308 and, in some embodiments, to high-bandwidth memory (e.g., HMB3).The Accelerated Processing Unit 1300 can be an AMD MI300a processor, which contains three CPU chiplets (or CCDs) and six accelerator chiplets (XCDs) arranged on four input-output dies (IODs) stacked on a single silicon die. These IODs can be interconnected (e.g., via AMD Infinity Fabric) to form eight stacks of high-bandwidth DRAM surrounding the superchip. An AMD MI300x processor replaces the CCDs with two additional XCDs to create a pure accelerator system.

[0131] The Accelerated Processing Unit 1300 can include one or more input / output (I / O) interfaces. For example, the XCDs 1304 and the CCDs 1306 can be arranged together on one or more Input-Output Dies (IODs) 1310, which can include one or more I / O interfaces. The IODs 1310 can include any number and type of I / O interfaces (e.g., PCI, PCI-Extended ("PCI-X"), PCIe, Gigabit Ethernet ("GBE"), USB, etc.). Various types of peripheral devices can be connected to the I / O interfaces 770. The I / O interfaces of the IODs 1310 can also be used to connect one or more Accelerated Processing Units 1300, for example, in a server architecture.

[0132] The Accelerated Processing Unit 1300 can include one or more Memory Units 1302 for storing instructions and other information used to perform operations described elsewhere herein. The Memory Units 1302 can contain any volatile memory, such as, but not limited to, the types of memory described elsewhere herein, and can include, for example, high-bandwidth memory (e.g., HMB3) or high-bandwidth DRAM. The memory associated with the Accelerated Processing Unit 1300 (e.g., the Memory Units 1302) can include system memory, which can be used, for example, for commands, instructions, and constants, as well as input and output.The 1302 memory units can also include device memory that can be used as storage for, for example, instructions, directives, constants, input / output, return buffers, and private data. The 1302 memory units can be connected to one or more 1310 IODs. In at least one embodiment, the 1320 L1 cache initiates a memory hierarchy that includes a shared 1328 L2 cache, for example, within the XCDs. AMD Infinity Cache™ is a last-level cache (LLC) located on an active I / O die (IOD). The 1306 CCDs and the 1304 XCDs can have separate or shared memory. The AMD Infinity architecture and AMD Infinity Fabric™ technology enable a coherent, high-throughput unification of GPU and CPU chiplet technologies (e.g., XCDs, CCDs, and / or CCXs) with memory (e.g.,stacked HBM3 memory) in individual devices and across multiple device platforms.

[0133] As in Fig. As shown in Figure 13, an XCD 1304 can include a shared set of global resources 1330, which may include a hardware scheduler 1312 and asynchronous compute engines (ACEs) 1324 that send tasks (e.g., compute shader workgroups) to compute units (CUs or cores) 1330. The ACEs 1324 (e.g., four) can each be connected to the CUs 1330 (e.g., 40 CUs), and some of the CUs can be disabled for yield management. The CUs 1330 can have a dedicated cache or share a cache (e.g., L2 cache) 1328 that can be used to aggregate all memory traffic for the die. The CUs 1330 can handle threaded and parallel processor cores, including instruction retrieval and scheduling with the Scheduler (S) 1312, the Matrix Core Unit (MCU) 1316 and the Shader Core (SC) 1318 (e.g.The execution units (for scalar, vector, and matrix data types) as well as load / store pipelines with an L1 cache 1320 and local data sharing (LDS) 1314 may include, for example, a scratch RAM with integrated computational functions, enabling data sharing between threads within a workgroup. An instruction cache 1340 (for example, for storing and providing instructions to perform the operations described elsewhere herein) may be associated with one or more CUs and shared by two CUs. The matrix cores 1316 can process a variety of data types, such as, but not limited to, INT8, FP8, FP16, BF16, and TF32 data types. The accelerated processing unit 1300 may include the compute units 1330, which may be arranged in an array format, for example, as a data-parallel processor (DPP) array.The Ultra-Threaded Dispatch Processor 1342 can communicate with the ACU 1330, and the Instruction Processor 1344 can read instructions written by the host to memory-addressed registers in a system memory address space (not shown). The Instruction Processor 1344 can send hardware-generated interrupts to a host processor (such as a CCD) when the instruction completes. The Memory Controller 1336 can also have direct access to all device memory and the host-specified areas of system memory. To meet read and write requests, the Memory Controller 1336 can perform Direct Memory Access (DMA) functions, including calculating memory address offsets based on the format of the requested data in memory.For example, one or more of the APIs described here can be compiled into instructions that are stored in the 1340 instruction cache and subsequently retrieved by the instruction call logic in the 1340 processor, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by execution logic or equivalent, reordered, and then terminated by termination logic or equivalent. One or more APIs (and / or compiled instructions including one or more APIs) can be stored in any memory outside or inside the 1300 processor (e.g., in the cache and / or memory).A result from one or more API(s) can then be stored in memory inside or outside the processor, including registers, DRAM, Flash, SRAM, cache or other memory equivalents.

[0134] An application can consist of a program running on a host processor (e.g., a CCD) and programs, called kernels, running on one or more XCDs. Programs can be controlled by host instructions that specify internal base address and other configuration registers, specify a data range in which the Accelerated Processing Unit 1300 can operate, invalidate and flush caches on the Accelerated Processing Unit 1300, and cause the Accelerated Processing Unit 1300 to begin executing a program. Kernels can be referred to as programs executed by the Accelerated Processing Unit 1300. A kernel can be executed independently on each worker or as a group of workers, which can be called a wavefront, and which can execute the kernel on all workers in the group (e.g., 64) in a single pass.The 1330 compute units can include a scalar arithmetic logic unit (ALU) capable of handling one value per wavefront (common to all work elements), a vector ALU capable of handling unique values ​​per work element, a local data share 1314 enabling work elements within a workgroup to communicate and share data, a scalar memory (not shown) capable of transferring data between scalar general-purpose registers (SGPRs) and memory via a cache, and a vector memory capable of transferring data between vector general-purpose registers (VGPRs) and memory, including texture map sampling. The control flow of a kernel can be managed via scalar ALU instructions, which may include if / else statements, branches, and loops.Scalar ALU (SALU) and memory instructions can be applied to an entire wavefront and operate on one or more SGPRs. Vector memory and ALU instructions can be applied to all work elements in the wavefront simultaneously.

[0135] In at least one embodiment, the accelerated processing unit 1300 can include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0136] Fig. Figure 14 illustrates a Processor 1400, such as, but not limited to, a processor based on a Zen architecture (such as Zen 1, 2, 3, 4, 5, or others) from AMD Corporation in Santa Clara, California, or any other processor incorporating at least some of the components described herein. The Processor 1400 includes one or more CPU dies 1402(1)-1402(N), where N is any integer greater than 1. The CPU die 1402 can include any number of processor cores 1416 (for example, to perform any of the operations described herein) and any number of cache memories (for example, to store instructions and other information for performing any of the operations described herein) in any combination.For example, the L2 cache units 1418 can be coupled to one or more processor cores 1416, which can share the L2 cache units 1418 and / or be individually coupled to them. The processor cores 1416 can be individually coupled to the L3 cache 1422 and / or share the L3 cache, which can be a lowest-level cache (LLC) 1422 for accessing data and other information used by the processor cores 1416. One or more processor cores 1416 and one or more L2 cache units 1418 can be contained in a core complex (CCX) 1420, which can contain a shared cache (e.g., L3 cache 1422) of 32 MB. The Core Complex 1420 can be manufactured on a Die (CCD or CPU Die) 1402. For example, up to 12 Core Complexes 1420 together with 8 CPU Dies 1402 can be configured in a processor to provide up to 96 Processor Cores 1416 for the processor.A “Zen 4c” Core Complex 1420, for example, can comprise up to eight cores 1416 and a shared 16 MB L3 cache 1422. Two of these Core Complexes 1420 can be combined on a single CPU die 1402 to provide 16 cores per die and a total of 32 MB L3 cache 1422 per die. Up to eight CPU dies 1402 can be combined with an I / O unit 1404 to provide CPUs with up to 128 processor cores 1416. Up to four of the “Zen 4c” dies described above can be combined to provide CPUs with up to 64 processor cores 1416.

[0137] The 1400 processor can incorporate a variety of input / output configurations, which are described in more detail below. The 1404 I / O unit can include one or more 1406 memory controllers, which manage memory usage (e.g., DDR5 memory) for the 1400 processor. The 1404 I / O unit can include one or more 1412 SATA hard disk controllers for managing memory and one or more 1414 Compute Express Link (CXL™) 1.1+ memory controllers, which provide CPU-to-device and CPU-to-memory connections and can be flexibly assigned to specific functions during server design. The 1404 I / O unit can also include a 1408 PCIe controller for connecting peripherals and other components connected to the 1400 processor. The I / O unit 1404 can include USB ports 1410 for connecting to other components that are separate from the processor 1400.The CPU dies 1402 can support any number of connections, such as one or two, to the I / O unit 1404. As shown, the I / O unit 1404 comprises the components described further here, and the I / O unit 1404 can be an I / O die that carries several different components. The memory controller 1406, the PCIe controller 1408, the USB ports 1410, the SATA controller 1412, and / or the CXL controller 1414 can be integrated at any location within the processor 1400, either separately or in any groups or combinations thereof.

[0138] The 1400 processor can include Infinity Fabric 1424 links (which may resemble or be based on PCIe architectures) that provide connections between CPUs (e.g., CPU dies 1402(1)-1402(N)), one or more GPUs 1426, one or more inference units 1432, and other components in the multi-chip architecture, such as the one or more secure processors 1428 and the I / O unit 1404. One or more AMD Infinity Fabric™ 1410 links can be connected to the CPU dies 1402(1)-1402(N) and serve as a connection between CPUs. One or more Infinity Fabric 1410 links can connect each CPU die 1402 to the I / O unit 1410.

[0139] In at least one embodiment, the Processor 1400 may comprise central processing units (CPUs) and other associated hardware and software, as described above and herein. The Processor 1400 may also include one or more Graphics Processors 1426. The Graphics Processor 1426 may be used for image generation and processing, as well as for other computations and operations described herein. The Graphics Processor 1426 may be based on AMD's RDNA 3 or 3.5 architecture in Santa Clara, California. The Graphics Processor 1426 may include Graphics Compute Dies (GCDs) and Memory Cache Dies (MCDs). GCDs may include any number of Compute Units (CUs) for graphics or other processing purposes, such as operations performed by Arithmetic Logic Units (ALUs), which are described herein.The 1426 graphics processor can contain an L2 cache that can be used by compute units. MCDs (not shown) can contain any number of memory units and can include a cache, such as an L3 cache, as well as memory interfaces for coupling to memory, such as the 1442(1)-(N) memory, where N is an integer. Components within the 1426 graphics processor can be interconnected using various approaches, such as the Infinity Fabric 1424 connections outside or inside the 1426 graphics processor.

[0140] The inference unit 1432 can provide neural processing functions to the processor 1400 for computational processes used for neural networks, deep learning, and other operations related to artificial intelligence, which are further described herein. The processor 1400 can include one or more secure processors 1428 for managing processor security, a display controller 1430 for controlling displays, a system management unit 1434 for managing and operating some or all of the components on the processor 1400, multimedia units 1436 for audio and video operations, a fusion controller hub 1438 for managing USB, SATA, and PCIe connections to the processor, and a sensor fusion hub 1440 for managing sensors, such as accelerometers. The processor 1400 can also include the memory units 1442(1)-(N), where N is any integer.The memory can include various memory types, such as LPDDR5 and / or DDR5 or others, which are described elsewhere here.

[0141] To perform the operations described below, the 1400 processor can include an execution pipeline containing a front end, which may include a cache (e.g., an L1 cache) for storing instructions (not shown). The instruction flow can be modified by a branch predictor. Instructions can be decoded by a decoder, forwarded to a back end for execution, and renamed. Instruction fetch and decode pipelines can, for example, output to integer or floating-point execution operations, which can be scheduled by a scheduler and transferred to vector and / or universal registers. Floating-point multiplication and / or addition operations can be processed, and arithmetic logic units (ALUs) can also be used to perform calculations such as arithmetic and logical operations.Outputs from the compute units can be coupled with a load / store queue, which can be connected to a cache, such as an L1 cache and / or an L2 cache.

[0142] With respect to the 1400 processor and each of its components described above or elsewhere herein, one or more of the APIs or equivalents described herein may, for example, be compiled into instructions or equivalents (e.g., AVX-512 instructions based on a SIMD model) that may be fetched by instruction call logic or equivalent, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by execution logic or equivalent, reordered, and subsequently terminated by termination logic or equivalent. One or more APIs (and / or compiled instructions including one or more APIs) may be stored in any memory outside or inside the processor (e.g., in the cache and / or memory).A result from one or more API(s) can then be stored in memory inside or outside the processor, including registers, DRAM, Flash, SRAM, cache or other memory equivalents.

[0143] In at least one embodiment, the processor 1400 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0144] Fig. Figure 15 illustrates an example of a 1500 processor core that can implement an Arm architecture (e.g., v9.0-A) or another processor that incorporates at least some of the components described herein. The Neoverse™ V2 1500 core can be implemented within a DynamIQ Shared Unit (DSU) cluster via a DSU-110 connection 1554 for one or more connected cores, for example, for parallel processing. The Neoverse™ V2 core can be implemented as a single core in a DSU cluster configured for Direct Connect, with or without L3 cache, snoop filtering, or snoop control unit (SCU) logic (not shown). The Neoverse™ V2 core can include a CPU bridge 1552, which connects the core 1500 to the DSU-110 link, which can also connect the core 1500 to an external storage system and the rest of a system on a chip.The L1 instruction storage system 1502 can retrieve instructions from an instruction cache 1504 and transmit the instructions (e.g., one or more APIs described herein, which can be compiled into instructions) to an instruction decoding unit 1510 to perform, for example, some or all of the operations described above or elsewhere herein. The L1 instruction storage system 1502 can access the L1 instruction cache 1504, e.g., with 64-byte cache lines, the L1 translation lookaside buffer (TLB) 1506, etc. B. with native support for pages of 4 KB, 16 KB, 64 KB and 2 MB, the Macro Operation Cache (MOP) 1508 (e.g. 4-way skew-associative L0 MOP cache with 1536 entries), which can contain decoded and optimized instructions for higher performance. The Instruction Decoding Unit 1510 can decode AArch64 instructions into an internal format.The Register Rename Unit 1512 can perform register renaming to facilitate unordered execution and output decoded instructions to various output queues. The Instruction Output Unit 1514 can control when decoded instructions are output to the execution pipelines and can contain output queues for storing instructions waiting to be output to the execution pipelines. The Integer Execution Pipeline 1516 can be included in an execution pipeline and may include an Integer Execution Unit 1518, which can perform arithmetic and logical data processing operations. The Vector Execution Unit 1520 can be included in an execution pipeline and can perform Advanced SIMD and Floating Point Operations (FPU) 1522, which execute SVE and SVE2 (Scalable Vector Extension) instructions, respectively.Scalable Vector Extension 2) 1524 and optionally the cryptographic instructions (Crypto) 1526. Advanced SIMD can include a media and signal processing architecture, adding instructions primarily for audio, video, 3D graphics, image, and speech processing. A floating-point architecture provides support for single- and double-precision floating-point operations. The L1 data storage system 1530 can execute load and store instructions and handle memory coherence requirements. The L1 data storage system 1530 can include an L1 data cache 1532 and a fully associative L1 data TLB 1534 with native support for page sizes of 4 KB, 16 KB, and 64 KB, and block sizes of 2 MB and 512 MB.The Memory Management Unit (MMU) 1528 can provide fine-grained memory system control through a series of mappings from a virtual address to a physical address and memory attributes that can be stored in translation tables, which can be saved to the TLB 1534 when translating an address. The L2 storage system 1536 can include an L2 cache 1538 and be connected to the DSU-110 1554 via an asynchronous CPU bridge 1552. The Neoverse™ V2 core 1500 can support a range of debugging, testing, and tracing options, including a Trace Unit 1542 and a Trace Buffer 1540, as well as an Embedded Logic Analyzer (ELA) 1548.The Neoverse™ V2 Core 1500 can implement the Statistical Profiling Extension (SPE) 1544 to provide a statistical view of the performance characteristics of executed instructions, which software developers can use to optimize their code for improved performance. The Performance Monitoring Unit (PMU) 1546 can provide performance monitors that can be configured to collect statistics on the operation of each core and the memory system. This information can be used for debugging and code profiling. The Generic Interrupt Controller (GIC) CPU interface 1550, when integrated with an external distributor component, can serve as a resource for supporting and managing interrupts in a cluster system. In a cluster, there can be a CPU bridge 1552 between each Neoverse™ V2 Core 1500 and the DSU-110 1554.The CPU Bridge 1552 can manage buffering and synchronization between the Core 1500 and the DSU-1101554. The CPU Bridge 1552 can operate asynchronously to allow different frequency, power, and area implementation points for each Core 1500. The CPU Bridge 1552 can also operate synchronously without affecting other interfaces, such as, but not limited to, debugging and tracing, which may operate asynchronously.

[0145] In at least one embodiment, the core 1500 can include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0146] Fig. Figure 16 illustrates one or more chips with one or more Tensor Processing Units (TPUs) 1600 according to at least one embodiment. The TPUs 1600 in Fig. 16 TPUs may include application-specific integrated circuits (ASICs) to perform, for example, some or all of the operations described above or elsewhere herein, such as, but not limited to, accelerating machine learning workloads that perform matrix operations. The TPUs 1600 may be ASICs from Alphabet Corporation in Mountain View, California. A Cloud TPU comprises a cloud service that provides TPUs as a scalable resource for processing tasks such as, but not limited to, machine learning workloads that can run on frameworks such as, but not limited to, TensorFlow, PyTorch, and JAX.

[0147] The Chip 1600 can contain any number of TPUs, which can contain the Tensor Cores 1606. The Tensor Core 1606 can comprise one or more Core Sequencers 1608, a Vector Processing Unit (VPU) 1610, a Matrix Multiply Unit (MXU) 1612(A)-1614(N), where N is any integer greater than 1, and a Transposition and Permutation Unit 1616. The 1608 core sequencer can retrieve instructions (e.g., VLIW (Very Long Instruction Word)) from the 1606 core's instruction memory (Imem), perform scalar operations using a scalar data memory (Smem) and scalar registers (Sregs) (not shown), and forward vector instructions to the 1610 vector processing unit (VPU).The instructions can, for example, trigger eight operations: two scalar operations, two vector ALU operations, vector load and store operations, and a pair of buffer slots that pass data to and receive data from the matrix multiplication and transposition units. The VPU 1610 can perform vector operations using a large on-chip vector memory (Vmem) and vector registers (Vregs). The VPU 1610 can stream data to and from the MXU via decoupling FIFOs. The VPU 1610 can collect and distribute data to Vmem through data-level parallelism (2D matrix and vector functional units) and instruction-level parallelism (8 operations per instruction). A large two-dimensional matrix multiplication unit (MXU) 1612(A)-1612(N), for example, can use a systolic array to reduce area and power consumption, as well as large, software-controlled on-chip memories instead of caches. The transposition reduction permutation unit 1616 can (e.g.128x128) matrix transpositions, reductions, and permutations of the VPU 1610 lanes can be performed. The high-bandwidth memory 1604 can be used for on-chip applications. One or more chips 1600 can be interconnected for computations. For example, one or more chips 1600 can be connected as a torus, such as a 2D torus. The chip 1600 can also contain any number (e.g., four) of Inter-Core Interconnect (ICI) links 1618, which allow direct connections between chips to form a supercomputer.

[0148] With respect to any of the processors in the Chip 1600 and any of its components described above or elsewhere herein, one or more of the APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by an instruction call logic or equivalent, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by an execution logic or equivalent, reordered, and subsequently terminated by a termination logic or equivalent. One or more APIs (and / or compiled instructions including one or more APIs) may be stored in any memory outside or inside the processor (e.g., in the cache and / or memory).A result from one or more API(s) can then be stored in memory inside or outside the processor, including registers, DRAM, Flash, SRAM, cache or other memory equivalents.

[0149] In at least one embodiment, the chip 1600 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0150] Fig. Figure 17 illustrates a vector processor according to at least one embodiment. The Vector Processor 1700 can support a RISC-V standard. The Vector Processor 1700 can include one or more cores 1710 (e.g., scalar units) with one or more vector processing units (VPUs) 1742 (e.g., vector units) that can perform, for example, some or all of the operations described above or elsewhere herein. The core 1710 can include an Andes Custom Extension (ACE) 1716, which can be used for communicating user-defined instructions to the processor 1700. The core 1710 can include a 1-cycle multiplier and a 1-cycle instruction / local data store (ILM / DLM) to increase parallelism by concurrently fetching instructions and accessing data.The Memory Management Unit (MMU) 1724 can manage system memory and the cache, and handle branch execution, instruction-pair output, L1 instruction / data caches, and local memory. The 1710 core can include a Physical Memory Protection and Programmable Physical Memory Attribute Unit (PMP / PPMA) 1722. The 1710 core can also include a Digital Signal Processor (DSP) 1728, a Floating Point Unit (FPU) 1726, and a Load-Store Unit (LSU) 1732 for interface to the memory hierarchy (D$ 1734 and I$ 1730). Finally, the 1710 core can include a Branch Prediction Unit (BPU) 1718 and a Multiplier Unit (MPU) 1720.

[0151] The Vector Processing Unit (VPU) 1742 can include one or more Vector Functional Units (FUs) 1746(A)-1746(N) which can be chained together for parallel processing, independent memory paths for RISC-V vector (RVV) load / store accesses via ACE-RVV 1748 and load / store accesses via Andes Streaming Port (ASP) 1744, and a Vector Load / Store Unit (VLSU) 1750.

[0152] The Vector Processor 1700 can include bus interfaces such as, but not limited to, an L2 cache memory port 1756 for cacheable access, an MMIO port 1754 for non-cacheable access, an Input-Output Coherence Port (IOCP) 1758 for a cacheless bus master, local memory access ports for ILM / DLM 1712 and high-bandwidth vector memory (HVM) 1736, and a shared peripheral port (SPP) 1752 for external peripherals. Additional memory ports include the LM slave port AXI 1702 and the HVM subordinate port AXI 1704.

[0153] With respect to any of the processors in the Processor 1700 and any of its components described above or elsewhere herein, one or more of the APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by an instruction call logic or equivalent, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by an execution logic or equivalent, reordered, and subsequently terminated by a termination logic or equivalent. One or more APIs (and / or compiled instructions including one or more APIs) may be stored in any memory outside or inside the processor (e.g., in the cache and / or memory).A result from one or more API(s) can then be stored in memory inside or outside the processor, including registers, DRAM, Flash, SRAM, cache or other memory equivalents.

[0154] In at least one embodiment, the vector processor 1700 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0155] Fig. Figure 18A illustrates a diagram of an exemplary microarchitecture of a tiled multi-core processor. The tiled multi-core processor in Fig. 18 can include a speech processing processor. As in Fig. As illustrated in Figure 18A, each "tile" of the processor architecture is a processing element interconnected via a network-on-chip (NoC) that can be used, for example, to perform some or all of the operations described above or elsewhere herein. For instance, each tile may include an instruction output unit 1804, an integer (INT) 1806 and floating-point (FP) 1808, a load-store unit (LSU) 1812 for connecting to the memory hierarchy (data cache (D$) 1810 and instruction cache (I$) 1814), and a network (NET) interface 1816 for communicating with other tiles of the architecture. Some tiles in the 1800 processor may include the memory controller 1802 for managing and controlling memory, as further described herein. The 1800 processor may have a functional slice architecture.The 1800 processor can be located on an application-specific integrated circuit (ASIC), and . Fig. 18A can represent the ASIC layout. The 1800 processor can include a coprocessor designed to execute instructions for a prediction model. The prediction model is any model configured to make a prediction from input data. The prediction model can use a classifier to make a classification prediction. The prediction model can be a machine learning model, such as, but not limited to, a tensor flow model, and the 1800 processor is a tensor streaming processor.

[0156] The 1800 processor can use different microarchitectures, which are represented in each tile in Fig. The functional units shown in Figure 18B are divided. Instead, the function tiles of the 1800 processor can be grouped into a multitude of functional process units (hereafter referred to as "slices") 1804, each corresponding to a specific function type (e.g., FP / INT, NET, MEM). As shown in Figure 18B, the functional tiles of the 1800 processor can be grouped into a variety of functional process units (hereafter referred to as "slices") 1804, each corresponding to a specific function type (e.g., FP / INT, NET, MEM). Fig. As illustrated in Figure 18B, each slice can correspond to a column of function tiles extending in a north-south direction. Furthermore, the processor also includes communication channels for transferring data between the tiles of different slices, each running horizontally in an east-west direction. Each communication channel can be connected to any of the slices of the 1800 processor.

[0157] The 1804 slices of the 1800 processor can each correspond to a different function and can include arithmetic-logic slices (e.g., FP / INT), channel-switching slices (e.g., NET), and memory slices (e.g., MEM). The arithmetic-logic units perform one or more arithmetic and / or logical operations on the data received over the communication channels to generate output data. Examples of arithmetic-logic units include matrix multiplication units and vector multiplication units. The memory slices comprise memory cells that store data. The memory slices can provide data to other slices over the communication channels. The memory slices can also receive data from other slices over the communication channels. The channel-switching slices can forward data from one communication channel to any other communication channel in a configurable manner.For example, data from a first channel can be provided to a second channel via a channel-switching slice. In some embodiments, the channel-switching slice can be implemented as a crossbar switch. Each 1804 slice also includes its own instruction queue (not shown) that stores instructions, and an instruction control unit (ICU) for controlling the execution of those instructions. The instructions in a given instruction queue can only be executed by tiles in its associated functional slice and cannot be executed by the other slice of the processor.

[0158] By arranging the tiles of the 1800 processor into different functional slices 1804, the instruction and control flow on the 1800 processor chip can be decoupled from the data flow. For example, an arrow in Fig. 18B the flow of instructions within the processor architecture according to some embodiments. Another arrow in Fig. Figure 18B illustrates the data flow within the processor architecture according to at least one embodiment. As illustrated, the instruction and control flow in a first direction across the tiles of the 1800 processor (e.g., from north to south along the length of the functional slices, as shown by the first arrow), while the data streams flow in a second direction across the tiles of the 1800 processor (e.g., from east to west, across the functional slices, as shown by the second arrow), perpendicular to the first direction.

[0159] Different functional slices of the processor can correspond to MEM (Memory), VXM (Vector Execution Module), MXM (Matrix Execution Module), NIM (Numerical Interpretation Module), and SXM (Switching and Permutation Module). Each slice can contain N tiles, all of which can be controlled by the same Instruction Control Unit (ICU) (not shown). Each of the slices can operate completely independently and can only be coordinated using barrier-like synchronization primitives or by the compiler exploiting tractable determinism. Each tile of the processor can correspond to an execution unit organized as an ×M SIMD tile. For example, each tile of the processor's on-chip memory can be organized to atomically store an L-element vector.Thus, a MEM slice can work together with N tiles to store or process a large vector (e.g., with a total of N×M elements).

[0160] Tiles within the same slice can execute instructions in a staggered manner, with instructions within the slice being issued tile by tile over a period of N cycles. Functional slices can be physically arranged on the chip to enable efficient data flow for pipeline execution over hundreds of cycles for common patterns. Data flows can perform a single reversal (change of direction) corresponding to a single matrix operation before being written back to memory. In some embodiments, a given data flow can change its direction multiple times (due to multiple matrix and vector operations) before the resulting data is written back to memory.

[0161] To achieve good single-threaded performance, a conventional multi-core processor design (e.g., as in Fig. (Figure 18A illustrates this) typically reserves a significant portion of the silicon area for uncovering and exploiting instruction-level parallelism (ILP). This usually involves register renaming schemes and large instruction windows through which instructions have no explicit knowledge of the hardware on which they are executed, while maintaining the illusion of orderly program execution. In contrast, when using a processor (e.g., TSP) with a functional slice architecture, the TSP compiler generates an explicit plan for how the processor executes the microprogram. The compiler determines when each operation is performed, which functional slices perform the work, and which STREAM registers contain the operands.The compiler maintains a highly accurate (cycle-accurate) model of the TSP's hardware state so that the microprogram can coordinate the data flow.

[0162] The Processor 1800 (e.g., TSP) can use a web-based compiler that takes a model as input (e.g., a machine learning model such as, but not limited to, a TensorFlow model) and outputs a proprietary instruction stream targeted to the processor's TSP hardware. The compiler is responsible for coordinating the program's control and data flow, establishing any instruction-level parallelism by explicitly grouping instructions that can and should be executed concurrently so they can be sent together. The primary hardware structure is the architecturally visible streaming register file (STREAMs), described in more detail below, which serves as the conduit for operands to flow from memory slices (e.g., SRAM) to functional slices and vice versa.

[0163] The processor's MEM unit serves as: (1) memory for model parameters, microprograms, and the data on which they operate, and (2) network-on-chip (NoC) for communication of data operands from the MEM to the functional slices and of computed results back to the MEM. In some embodiments, the on-chip memory occupies approximately 75% of the processor's chip area. In some embodiments, the on-chip memory of the MEM tiles may consist of SRAM rather than DRAM due to the processor's bandwidth requirements. The processor's on-chip memory capacity determines (i) the number of machine learning models that can be stored on the chip simultaneously, (ii) the size of a given model, and (iii) the partitioning of large models to fit into multi-chip systems.In some embodiments, the processor's MEM system provides a multitude of memory slices organized into two different hemispheres (referred to as "MEM WEST" and "MEM EAST").

[0164] The memory slices of each hemisphere can be mirrored, so that the slices can be physically numbered {0, ... L} in the eastern hemisphere (410), and {L, ... 0} in the western hemisphere (405), such that memory slice 0 for each hemisphere corresponds to the slice closest to the VXM slices (415) between the hemispheres, with each hemisphere containing L slices. The direction of data transmission toward the center of the chip can be described as "inward," while data transmission toward the outer (easternmost or westernmost) edge of the chip can be described as "outward." Although the processor's memory hemispheres can be referred to as East and West, it is understood that in other embodiments, other designations may be used to denote the different memory hemispheres.

[0165] In some embodiments, a streaming register file called STREAMS transfers operands and results between the SRAM of the MEM slices and the functional slices of the processor. In some embodiments, multiple MEM slices (e.g., between 2 and 10 adjacent MEM slices) can be physically organized as a set. Each set of slices can be located between a pair of STREAM register files, allowing each slice to read from or write to the STREAM registers in both directions. Positioning STREAM register files between sets of MEM slices reduces the number of cycles required to transfer data operands across one hemisphere (e.g., by a factor equal to the number of slices per set). The number of slices per set can be configured based on the distance over which data can be transferred in a single clock cycle.

[0166] Regarding each of the processors in Fig. 18 and any components described above or elsewhere herein can, for example, compile one or more of the APIs or equivalents described herein into instructions or equivalents that can be fetched by an instruction call logic or equivalent, decoded by a processor decoder or equivalent, scheduled for execution by a scheduler or equivalent (e.g., sequentially or out of sequence), executed by an execution logic or equivalent, reordered, and subsequently terminated by a termination logic or equivalent. One or more APIs (and / or compiled instructions including one or more APIs) can be stored in any memory outside or inside the processor (e.g., in the cache and / or memory).A result from one or more API(s) can then be stored in memory inside or outside the processor, including registers, DRAM, Flash, SRAM, cache or other memory equivalents.

[0167] In at least one embodiment, the processor 1800 may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein. SOFTWARE DESIGNS

[0168] The following figures represent, without limitation, examples of software constructs for implementing at least one embodiment.

[0169] Fig. Figure 19 illustrates a software stack of a programming platform according to at least one embodiment. A programming platform may comprise a platform for utilizing hardware on a computing system to accelerate computational tasks. A programming platform may be accessible to software developers through libraries, compiler directives, and / or extensions to programming languages ​​in at least one embodiment. A programming platform may be CUDA, Radeon Open Compute Platform (“ROCm”), OpenCL (OpenCL™ was developed by the Khronos group), SYCL, or Intel oneAPI.

[0170] A Software Stack 1900 of a programming platform can provide an execution environment for an Application 1901. The Application 1901 can include any computer software that can be launched on the Software Stack 1900. The Application 1901 can include an artificial intelligence (AI) / machine learning (ML) application, a high-performance computing (HPC) application, a virtual desktop infrastructure (VDI), or a data center workload.

[0171] The application 1901 and the software stack 1900 run on the hardware 1908. The hardware 1908 can include one or more GPUs, CPUs, FPGAs, AI units, and / or other types of computing devices that support a programming platform. The software stack 1900 can be vendor-specific and compatible only with devices from certain manufacturers, such as CUDA, ROCm, OneAPI, OpenCL, or other implementations. The hardware 1908 can include a host connected to one or more devices that can be accessed via application programming interfaces (“APIs”) to perform computational tasks. A device within the hardware 1908 can include a GPU, an FPGA, an AI unit, or another computing device (but also a CPU) and its memory, as opposed to a host within the hardware 1908, which can include a CPU (but also a computing device) and its memory in at least one embodiment.With respect to any of the hardware described above or elsewhere herein, one or more of the APIs described herein may, for example, be compiled into instructions that may be fetched by the instruction call logic, decoded by a processor decoder, scheduled for execution by a scheduler (e.g., sequentially or out of sequence), executed by execution logic, reordered, and subsequently terminated by the termination logic. One or more APIs (and / or compiled instructions including one or more APIs) may be stored in any memory outside or inside the processor (e.g., in the cache and / or memory). A result of one or more APIs may then be stored in memory inside or outside the processor, including registers, DRAM, flash, SRAM, cache, or other memory. One or more of the APIs described herein may contain a call.One or more of the APIs described here may include a library or part of a library to perform a function described by the call. One or more of the APIs described here may include a call and a library or part of a library to perform a function described by the call.

[0172] The software stack 1900 of a programming platform can comprise a set of libraries 1903, a runtime environment 1905, an optional driver / interface 1907, and a device kernel driver 1908. Each of the libraries 1903 can contain data and program code that can be used by computer programs and leveraged in software development. The libraries 1903 can contain pre-written code and subroutines, classes, values, type specifications, configuration data, documentation, help data, and / or message templates. The libraries 1903 can contain functions optimized for execution on one or more device types. The libraries 1903 can contain functions for performing mathematical, deep learning, and / or other types of operations on devices.Libraries 1903 can be linked to corresponding APIs 1902, which may include one or more APIs that provide functionality implemented in libraries 1903. A processor (e.g., CPU, GPU) can execute, call, or otherwise use one or more APIs to prioritize kernels. For example, a first kernel (e.g., a parent kernel) can start a second kernel (e.g., a child kernel), and the second kernel can be used by a processor to start additional kernels (e.g., sub-kernels) independently of the first kernel. A processor can execute an API or call an API from memory to support dynamic stream priority (e.g., updating the priority while a stream is being used to perform operations).For example, when a processor executes the API, a programmer can copy the stream priority from one stream to one or more other streams.

[0173] Software Stack 1900 can include an API to support dynamic stream priority (e.g., updating the priority while a stream is being used to perform operations), which allows a programmer to set the priority of a stream at any time after its creation. Software Stack 1900 can include an API to support dynamic stream priority (e.g., updating the priority while the stream is being used to perform operations), which allows a programmer to determine the current priority of a stream, where the priority is one of several attributes of a stream. Software Stack 1900 can include an API to support dynamic stream priority (e.g.,Software Stack 1900 may include an API to support dynamic stream priority (e.g., updating the priority while the stream is used to perform operations), which allows a programmer to retrieve the current priority of a stream as a single attribute. Software Stack 1900 may include an API to support dynamic stream priority (e.g., updating the priority while the stream is used to perform operations), which allows a programmer to start a kernel to perform operations on a stream with a set priority that may differ from the stream priority. Software Stack 1900 may include an API to indicate whether an object (e.g.,A thread synchronization object (such as, but not limited to, a barrier) tracks whether all data movement operations for a group of threads running on a GPU have completed after a predetermined time period and are in a predetermined state, where a predetermined state can be a state indicating that data has been moved and is ready for use, and is specified using an expected parity value as input to the API.

[0174] The Software Stack 1900 can include one or more APIs for updating kernels. A processor can execute an API or call an API from memory to update an existing API to support context-free kernels, allowing a programmer to add a kernel node to a graph without a graphics context, so that a graphics context can be dynamically assigned to a kernel at runtime. The Software Stack 1900 can include one or more APIs that allow a programmer to obtain a kernel identifier and a graphics context as separate parameters from a kernel node, enabling parameters to be obtained from both kernels and context-free kernels. The Software Stack 1900 can include one or more APIs to utilize one or more parallel processors, such as one or more graphics processors, to manage task graphs (e.g.,to start task graphs) and execute one or more task graphs (e.g., including one or more programs).

[0175] The Software Stack 1900 can include one or more APIs to map one or more instructions to one or more memory order operations, such as, but not limited to, a fence or membar operation. Instructions can be mapped to one or more domains, so that a memory order operation is performed in conjunction with one or more specific domains without affecting instructions in other domains. An API can indicate that a thread has arrived (for example, at a thread synchronization barrier) or has completed a work phase with respect to asynchronous data movement operations on a GPU.The Software Stack 1900 can include one or more APIs that allow programmers to manually specify an expected transaction count when a thread has completed a work phase, which can be used to update an object that tracks whether all data movement operations have been completed for a group of threads.

[0176] The 1901 application can be written as source code, which is compiled into executable code, as described below in conjunction with Fig. 20 and Fig. 21 is discussed in more detail. The executable code of Application 1901 can be executed, at least partially, in an execution environment provided by Software Stack 1900. During the execution of Application 1901, code may be encountered that must be executed on a device rather than a host. In such a case, Runtime Environment 1905 can be invoked to load and start the required code on the device. Runtime Environment 1905 can include any technically feasible runtime system capable of supporting the execution of Application 1901.

[0177] The 1905 runtime environment can be implemented as one or more runtime libraries associated with corresponding APIs, represented as the one or more API(s) 1904. One or more of these runtime libraries can include, among other things, memory management, execution control, device management, error handling, and / or synchronization functions. Memory management functions can include functions for allocating, releasing, and copying device memory, as well as for transferring data between host memory and device memory.Execution control functions may include functions for starting a function (sometimes called a "kernel" if it is a global function that can be called from a host) on a device and for setting attribute values ​​in a buffer managed by a runtime library for a particular function to be executed on a device.

[0178] Runtime libraries and their one or more corresponding APIs (1904) can be implemented in any technically feasible way. One (or any number of) APIs can provide a set of subordinate functions for the fine-grained control of a device, while another (or any number of) APIs can provide a set of such superior functions. A superior runtime API can build upon a subordinate API. One or more runtime APIs can be language-specific APIs that can build upon a language-independent runtime API.

[0179] An optional driver or interface 1907 can be implemented, for example, for CUDA and ROCm implementations, which are described in more detail below. The optional driver / interface 1907 can be associated with optional driver or interface APIs, such as, but not limited to, CUDA and / or ROCm APIs.

[0180] One or more processors disclosed in "Processing Systems" can execute, access, or otherwise use Software Stack 1900. For example, the System on a Chip 700, the Parallel Processor 800, the Graphics Multiprocessor 834, the Processor 900, the Processor 1000, the Accelerator 1100, the Neuromorphic Processor 1105, the Supercomputer 1200, the Acceleration Processing Unit 1300, the Processor 1400, the Processor 1500, the Tensor Processor 1600, the Processor 1700, and the Speech Processing Unit 1800 can execute, use, call, or otherwise implement (e.g., by accessing memory) one or more APIs contained in Software Stack 1900.

[0181] The device's 1908 kernel driver can be configured to facilitate communication with an underlying device. The device's 1908 kernel driver can provide low-level functions upon which APIs, such as but not limited to, one or more 1904 APIs and / or other software, are based. The device's 1908 kernel driver can be configured to compile intermediate representation (IR) code into binary at runtime. For CUDA or other implementations such as, but not limited to, ROCm, OneAPI, or OpenCL, the device's 1908 kernel driver can compile non-hardware-specific Parallel Thread Execution (PTX) IR code into binary for a specific target device at runtime (caching the compiled binary), a process sometimes referred to as "finalizing" the code.This allows the finalized code to be executed on a target device that may not have existed when the source code was originally compiled into PTX code. Alternatively, the device's source code can be compiled offline into binary code without requiring the device's kernel driver 1908 to compile the IR code at runtime.

[0182] Processors described elsewhere here, such as, but not limited to, processors in Fig. 7-18, may contain one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein. One or more circuits may be implemented by software, e.g.the software stack 1900, to be configured to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0183] According to at least one embodiment, the software stack 1900 can consist of Fig. 19 in a CUDA implementation. A CUDA software stack 1900, on which an application 1901 can be started, can include CUDA libraries 1903, a CUDA runtime environment 1905, a CUDA driver 1907, and a kernel driver 1908 for the device. The CUDA software stack 1900 can run on hardware 2209, which can include a GPU that supports CUDA and was developed by NVIDIA Corporation in Santa Clara, California.

[0184] The device's 1901 application, 1905 CUDA runtime environment, and 1908 kernel driver can perform functions described above and elsewhere herein. The 1907 CUDA driver can include a library (libcuda.so) that can implement a 1906 CUDA driver API. Similar to a 1904 CUDA runtime API implemented by a CUDA runtime library (cudart), the 1906 CUDA driver API can provide, among other things, functions for memory management, execution control, device management, error handling, synchronization, and / or graphics interoperability. The CUDA driver API 1906 may differ from the CUDA runtime API 1904 in that the CUDA runtime API 1904 simplifies the device's code management through implicit initialization, context management (analogous to a process), and module management (analogous to dynamically loaded libraries).Unlike the parent CUDA Runtime API 1904, the CUDA Driver API 1906 can be a subordinate API, allowing finer control of the device, particularly regarding contexts and module loading. The CUDA Driver API 1906 can provide context management capabilities that the CUDA Runtime API 1904 might not offer. The CUDA Driver API 1906 can also be language-independent and, in addition to supporting languages ​​like OpenCL, the CUDA Runtime API 1904 supports other languages. Furthermore, development libraries, including the CUDA Runtime Environment 1905, can be considered separate from the driver components, including the user-mode CUDA Driver 1907 and the kernel-mode device driver 1908 (sometimes referred to as the "display driver").

[0185] The CUDA 1903 libraries can include mathematical libraries, deep learning libraries, parallel algorithm libraries, and / or signal / image / video processing libraries that can be used by parallel computing applications such as, but not limited to, the CUDA 1901 application. The CUDA 1903 libraries can include, among others, mathematical libraries such as, but not limited to, a cuBLAS library, which is an implementation of basic linear algebra subprograms (BLAS) for performing linear algebra operations; a cuFFT library for calculating fast Fourier transforms (FFTs); and a cuRAND library for generating random numbers.The CUDA libraries 1903 can include deep learning libraries, such as, but not limited to, a cuDNN library with primitives for deep neural networks and a TensorRT platform for high-performance deep learning inference.

[0186] In at least one embodiment, processors described elsewhere herein, such as, but not limited to, processors in Fig. 7-18, containing one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein. One or more circuits may be implemented by software, e.g.the software stack 1900, to be configured to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0187] According to at least one embodiment, the software stack 1900 can consist of Fig. 19 in an ROCm implementation. An ROCm software stack 1900, on which an application 1901 can be started, includes a language runtime environment 1903, a system runtime environment 1905, a thunk 1907, and an ROCm kernel driver 1908. The ROCm software stack 1900 can be run on hardware 1909, which may include a GPU that supports ROCm and was developed by AMD Corporation in Santa Clara, California.

[0188] Application 1901 can perform similar functions as described above in conjunction with Fig. 19 discussed. Furthermore, the Language Runtime Environment 1903 and the System Runtime Environment 1905 can perform similar functions to those discussed above in conjunction with Fig. 19. Runtime Environment 1905 was discussed. The Language Runtime Environment 1903 and the System Runtime Environment 1905 can differ in that the System Runtime Environment 1905 is a language-independent runtime that implements a ROCr System Runtime API 1904 and utilizes a Heterogeneous System Architecture (“HSA”) runtime API. The HSA runtime API can include a lean, user-mode API that provides interfaces for accessing and interacting with an AMD GPU, including functions for memory management, execution control via the designed disposition of kernels, error handling, system and agent information, and runtime environment initialization and shutdown. In contrast to the System Runtime Environment 1905, the Language Runtime Environment 1903 can be an implementation of a language-specific runtime API 1902 that builds upon the ROCr System Runtime API 1904.The language runtime API can include, among other things, a Heterogeneous Compute Interface for Portability (“HIP”) language runtime API, a Heterogeneous Compute Compiler (“HCC”) language runtime API, or an OpenCL API. In particular, the HIP language is an extension of the C++ programming language with functionally similar versions of CUDA mechanisms, and an HIP language runtime API can include functions similar to those of the above in conjunction with... Fig. 19 discussed CUDA runtime APIs may resemble, such as, but not limited to, functions for memory management, execution control, device management, error handling and / or synchronization.

[0189] The Thunk (ROCt) 1907 can be an interface 1906 that can be used to interact with the underlying ROCm driver 1908. The ROCm driver 1908 can be a ROCk driver, which is a combination of an AMDGPU driver and an HSA kernel driver (amdkfd). The AMDGPU driver can be a kernel driver developed by AMD for GPUs, providing similar functionality to the one described above in conjunction with Fig. 19 discussed kernel drivers of the device. The HSA kernel driver can be a driver that allows different types of processors to share system resources more effectively via hardware functions.

[0190] Various libraries (not shown) may be included in the ROCm software stack 1900 above the language runtime environment 1903 and provide functions that complement the above in conjunction with Fig. The 19 CUDA libraries discussed in 1903 are similar. Various libraries may include, among others, mathematical, deep learning, and / or other libraries, such as, but not limited to, a hipBLAS library that implements functions similar to those of CUDA cuBLAS, and a rocFFT library for calculating FFTs, similar to CUDA cuFFT.

[0191] Processors described elsewhere here, such as, but not limited to, processors in Fig. 7-18, may contain one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein. One or more circuits may be implemented by software, e.g.the software stack 1900, to be configured to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0192] According to at least one embodiment, the software stack 1900 can consist of Fig. 19 in an OpenCL implementation. An OpenCL software stack 1900, on which an application 1901 can be started, may include an OpenCL framework 1903, an OpenCL runtime environment 1905, and a driver 1908. The OpenCL software stack 1900 can run on hardware 1909, which is not vendor-specific. Because OpenCL is supported by devices from various manufacturers, special OpenCL drivers may be required to work with the hardware of these manufacturers.

[0193] The application 1901, the OpenCL runtime environment 1905, the device kernel driver 1908, and the hardware 1909 can perform similar functions to other implementations of the application 1901, the runtime environment 1905, the device kernel driver 1908, and the hardware 1909 described above in conjunction with Fig. 19 will be discussed. Application 1901 may further include an OpenCL kernel (not shown) with code to be executed on a device.

[0194] OpenCL can define a "platform" that allows a host to control devices connected to it. An OpenCL framework can provide a platform layer API and a runtime API, shown as Platform API 1902 and Runtime API 1904, respectively. Runtime API 1904 can use contexts to manage kernel execution on devices. Each identified device can be associated with a corresponding context, which Runtime API 1904 can use to manage, among other things, command queues, program objects, and kernel objects for that device, as well as to share memory objects. Platform API 1902 can provide functionality that allows the use of device contexts to, among other things, select and initialize devices, submit work to devices via command queues, and enable data transfer to and from devices.In addition, the OpenCL framework can provide various built-in functions (not shown), including mathematical functions, relational functions, and image processing functions.

[0195] A compiler (not shown) can also be integrated into the OpenCL 1903 framework. Source code can be compiled offline before an application is executed or online during application execution. Unlike CUDA and ROCm, OpenCL applications can be compiled online by a compiler that is representative of any number of compilers that can be used to compile source code and / or IR code, such as, but not limited to, standard portable intermediate representation (SPIR-V), into binary code. Alternatively, OpenCL applications can be compiled offline before such applications are executed.

[0196] In at least one embodiment, processors described elsewhere herein, such as, but not limited to, processors in Fig. 7-18, containing one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein. One or more circuits may be implemented by software, e.g.the software stack 1900, to be configured to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0197] According to at least one embodiment, software can be supported by a programming platform configured to support various programming models, middleware, and / or libraries and frameworks upon which an application can rely. The application can be an AI / ML application implemented, for example, using a deep learning framework such as, but not limited to, MXNet, PyTorch, or TensorFlow, which can rely on libraries such as cuDNN, NVIDIA Collective Communications Library (“NCCL”), and / or NVIDIA Developer Data Loading Library (“DALI”) CUDA libraries to enable accelerated computing on the underlying hardware.

[0198] The programming platform can be one of the above in conjunction with Fig. The 19 discussed platforms could be CUDA, ROCm, or OpenCL. The programming platform can support multiple programming models, which may be abstractions of an underlying computing system that allow for the expression of algorithms and data structures. Programming models can expose functions of the underlying hardware to improve performance. Examples of programming models include CUDA, HIP, OpenCL, C++ Accelerated Massive Parallelism (“C++AMP”), Open Multi-Processing (“OpenMP”), Open Accelerators (“OpenACC”), and / or Vulcan Compute.

[0199] Libraries and / or middleware can provide implementations of programming model abstractions. Such libraries can contain data and code that can be used by computer programs and leveraged in software development. This middleware can include software that provides services to applications beyond those available from the programming platform. Libraries and / or middleware can include cuBLAS, cuFFT, cuRAND, and other CUDA libraries, or rocBLAS, rocFFT, rocRAND, and other ROCm libraries. Additionally, libraries and / or middleware can include NCCL and ROCm Communication Collectives Library ("RCCL") libraries, which provide communication routines for GPUs, an MIOpen library for deep learning acceleration, and / or an Eigen library for linear algebra, matrix and vector operations, geometric transformations, numerical solvers, and related algorithms.

[0200] Application frameworks can depend on libraries and / or middleware. Each application framework can be a software framework used to implement a standard application software structure. Returning to the AI / ML example discussed earlier, an AI / ML application can be implemented using a framework such as, but not limited to, Caffe, Caffe2, TensorFlow, Keras, PyTorch, or MxNet deep learning frameworks.

[0201] In at least one embodiment, processors described elsewhere herein, such as, but not limited to, processors in Fig. 7-18, containing one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein. One or more circuits may be implemented by software, e.g.The programming platforms described herein are configured to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0202] Fig. Figure 20 illustrates how to compile code for execution on one of the programming platforms described above. Fig. 19 according to at least one embodiment. A Compiler 2001 is configured to receive Source Code 2000, compile Source Code 2000, and output an executable file 2010. The Compiler 2001 can be configured to convert Source Code 2000 into Host-executable Code 2007 and Device-executable Code 2008. Source Code 2000 can be compiled either offline before an application is executed or online during application execution. Source Code 2000 can contain code in any programming language supported by Compiler 2001, such as, but not limited to, C++, C, Fortran, etc. Source Code 2000 can be contained in a single source file that includes a mixture of host code and device code, specifying the positions of the device code within it. A single source file can be a .cu file containing CUDA code, or a .hip file.The source code can be a .cpp file containing HIP code, or a file in another format containing both host code and device code. Alternatively, instead of a single source file, the source code can comprise multiple source code files, in which the host code and device code can be separate. The 2001 compiler includes or has access to one or more libraries to detect a sequence of API calls to execute a single fused API, where a single fused API is a combined API for two or more APIs. In at least one embodiment, the 2001 compiler can be an NVIDIA CUDA compiler (“NVCC”) for compiling CUDA code into .cu files, an HCC compiler for compiling HIP code into .hip.cpp files, or other compilers.

[0203] The 2001 compiler can be configured to compile 2000 source code into 2007 host-executable code and 2008 device-executable code. The 2001 compiler performs operations including parsing the 2000 source code into an abstract system tree (AST), performing optimizations, and generating executable code. If the 2000 source code is contained in a single source file, the 2001 compiler can separate the device code from the host code in such a single source file, compile the device code and the host code separately into 2008 device-executable code and 2007 host-executable code, respectively, and then join the 2008 device-executable code and the 2007 host-executable code together in a single file.

[0204] The 2001 compiler can include a 2002 compiler frontend, a 2005 host compiler, a 2006 device compiler, and a 2009 linker. The 2002 compiler frontend can be configured to separate the 2004 device code from the 2003 host code in the 2000 source code. The 2004 device code can be compiled by the 2006 device compiler into a 2008 device execution code, which, as described, can contain binary code or IR code in at least one embodiment. Separately, the 2003 host code can be compiled by the 2005 host compiler into a 2007 host-executable code.For NVCC compilers, such as, but not limited to, those for oneAPI, ROCm, and OpenCL, the Host Compiler 2005 can be a general-purpose C / C++ compiler that outputs native object code, while the Device Compiler 2006 can be a low-level virtual machine (LLVM)-based compiler that branches an LLVM compiler infrastructure and outputs PTX code or binary code. For HCC, both the Host Compiler 2005 and the Device Compiler 2006 can be LLVM-based compilers that output target binary code.

[0205] After compiling the source code 2000 into host executable code 2007 and device executable code 2008, the linker 2009 can link the host executable code and the device executable code 2007 and 2008 together in a single executable file 2010. The native object code for a host and the PTX or binary code for a device can be linked in a single file in Executable and Linkable Format (ELF), a container format used to store object code. The host executable code 2007 and the device executable code 2008 can be in any suitable format, such as, but not limited to, binary code and / or IR code. In the case of CUDA, in at least one embodiment the host-executable code 2007 can contain native object code, and the device-executable code 2008 can contain code in PTX intermediate code.In the case of ROCm, both the host-executable code 2007 and the device-executable code 2008 can contain target binary code in at least one embodiment. Other implementations, such as, but not limited to, oneAPI and OpenCL, are provided and can be implemented similarly to the CUDA and ROCm implementations mentioned above.

[0206] Source code 2000 can be translated before compiling the source code. The source code is passed through a translation tool (not shown) that translates the source code 2000 into translated source code. A compiler 2001 can be used to compile the translated source code into host-executable code 2007 and device-executable code 2008 in a process similar to the compilation of source code 2000 by compiler 2001 into host-executable code 2007 and device-executable code 2008, as described above in conjunction with Fig. 20 is discussed.

[0207] A translation performed by the translation tool can be used to port the Source Code 2000 for execution in a different environment than originally intended. The translation tool may include a HIP translator, which is used to "HIPify" CUDA code intended for a CUDA platform into HIP code that can be compiled and executed on a ROCm platform. Source Code 2000 translation may involve parsing the Source Code 2000 and converting calls to one or more APIs provided by one programming model (e.g., CUDA) into corresponding calls to one or more APIs provided by another programming model (e.g., HIP), as described below in conjunction with Fig. This will be discussed in more detail in section 21. To return to the example of HIP conversion of CUDA code: Calls to the CUDA runtime API, the CUDA driver API, and / or the CUDA libraries can be converted into corresponding HIP API calls. The automated translations performed by the 2001 translation tool can sometimes be incomplete, so additional manual work is required to fully port the 2000 source code.

[0208] One or more of the techniques described here can utilize other methods for converting one code type to another to enable interchangeability between different device architectures. In at least one embodiment, an application for one platform (e.g., a CUDA application) can be compiled into code for implementation on another platform (e.g., an AMD processor, an Intel processor, or another processor). For example, Source Code 2000 can contain source code for one platform (e.g., CUDA). Compiler 2001 can compile the Source Code 2000 into an executable file 2010 that can be used by another platform (e.g., AMD or Intel). Programming toolkits can compile applications for one platform (e.g., CUDA) for implementation on another platform (e.g., AMD or Intel) (e.g., natively).A GPGPU programming toolkit can, for example, enable the native compilation of CUDA applications for AMD GPUs. Programs (e.g., CUDA programs) or their build system do not need to be modified or translated into another language before compilation to code for a different platform. A compiler can accept the same command-line options and programming dialect (e.g., CUDA dialect) as another compiler (e.g., nvcc for CUDA) and serves as a drop-in replacement to emulate an installation of a toolkit (e.g., NVIDIA's CUDA Toolkit), allowing existing build tools and scripts (e.g., such as cmake) to function without further modification. In at least one embodiment, an nvcc-compatible compiler can be used to compile nvcc-dialect CUDA for AMD GPUs, including PTX assembly. Implementations of the CUDA runtime and driver APIs for AMD GPUs can be used. Libraries (e.g.,Open-source wrapper libraries can provide APIs, such as CUDA-X APIs, by delegating to the corresponding ROCm libraries. One example implementation is SCALE from Spectral Compute in London, England. Instead of providing a new method for writing GPGPU software, SCALE allows the direct compilation of programs written in the widely used CUDA language for AMD GPUs. Other implementations may include a Clang compiler, providing a language front-end and tooling infrastructure for languages ​​in the C family of languages ​​(C, C++, Objective C / C++, OpenCL, CUDA, and RenderScript). In at least one embodiment, compilers described herein, such as Compiler 2001, Compiler 2005, and / or Compiler 2006, but not limited to, may include one or more circuits for compiling code (e.g.,CUDA, HIP, OpenCL, One API or others), to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0209] Fig. Figure 21 illustrates a system 2100 configured according to at least one embodiment to compile and execute CUDA source code 2110 using various types of processing units. The system 2100 comprises the CUDA source code 2110, a CUDA compiler 2150, a host-executable code 2170(1), a host-executable code 2170(2), a device-executable CUDA code 2184, a CPU 2190, a CUDA-enabled GPU 2194, a GPU 2192, a CUDA-to-HIP translation tool 2120, a HIP source code 2130, a HIP compiler driver 2140, an HCC 2160, and a device-executable HHC code 2182.

[0210] CUDA source code 2110 can be a collection of human-readable code written in a CUDA programming language. A CUDA programming language can be an extension of the C++ programming language that includes mechanisms for defining device code and distinguishing between device code and host code. Device code can contain source code that, after compilation, is executable in parallel on a device. A device can be a processor optimized for parallel instruction processing, such as, but not limited to, a CUDA-enabled GPU 2190, a GPU 2192, or another GPGPU, etc. Host code is source code that, after compilation, is executable on a host. A host is a processor optimized for sequential instruction processing, such as, but not limited to, a CPU 2190.

[0211] The CUDA source code 2110 can contain any number (including zero) of global functions 2112, any number (including zero) of device functions 2114, any number (including zero) of host functions 2116, and any number (including zero) of host / device functions 2118. The global functions 2112, device functions 2114, host functions 2116, and host / device functions 2118 can be mixed in the CUDA source code 2110. Each of the global functions 2112 can be executable on a device and callable by a host. One or more of the global functions 2112 can therefore serve as entry points to a device. Each of the global functions 2112 can be a kernel. In a technique known as dynamic parallelism, one or more of the global functions 2112 can define a kernel that is executable on and called by a device.A kernel can be executed N times (where N is any positive integer) in parallel by N different threads on a device during execution.

[0212] Each of the device functions 2114 can be executed on a device and can only be called by such a device. Each of the host functions 2116 can be executed on a host and can only be called by such a host. Each of the host / device functions 2116 can define both a host version of a function, which can be executed on a host and can only be called by such a host, and a device version of the function, which can be executed on a device and can only be called by such a device.

[0213] CUDA source code 2110 can also contain any number of calls to any number of functions that can be defined via a CUDA runtime API 2102. The CUDA runtime API 2102 can contain any number of functions that run on a host to allocate and release device memory, transfer data between host memory and device memory, manage multi-device systems, and so on. CUDA source code 2110 can also contain any number of calls to any number of functions that can be specified in any number of other CUDA APIs. A CUDA API can be any API designed for use by CUDA code. CUDA APIs can include the CUDA runtime API 2102, a CUDA driver API, APIs for any number of CUDA libraries, and so on, including any APIs described elsewhere herein.Compared to the CUDA runtime API 2102, a CUDA driver API can be a lower-level API, yet provide finer control over a device. Examples of CUDA libraries include cuBLAS, cuFFT, cuRAND, cuDNN, etc.

[0214] The CUDA compiler 2150 can compile the input CUDA code (e.g., the CUDA source code 2110) to generate the host-executable code 2170(1) and the device-executable CUDA code 2184. The CUDA compiler 2150 can be NVCC, but is not limited to this. The host-executable code 2170(1) can be a compiled version of the host code contained in the input source code and executable on the CPU 2190. The CPU 2190 can be any processor optimized for sequential instruction processing.

[0215] The device-executable CUDA code 2184 can be a compiled version of the device code contained in the input source code, executable on a CUDA-enabled GPU 2194. The device-executable CUDA code 2184 can contain binary code. The device-executable CUDA code 2184 can contain IR code, such as, but not limited to, PTX code, which is further compiled at runtime by a device driver into binary code for a specific target device (e.g., a CUDA-enabled GPU 2194). The CUDA-enabled GPU 2194 can include any processor optimized for parallel instruction processing that supports CUDA. The CUDA-enabled GPU 2194 may have been developed by NVIDIA Corporation of Santa Clara, California.

[0216] The CUDA-to-HIP translation tool 2120 can be configured to translate CUDA source code 2110 into functionally similar HIP source code 2130. The HIP source code 2130 can contain a collection of human-readable code in a HIP programming language. A HIP programming language can include an extension of the C++ programming language that contains functionally similar versions of CUDA mechanisms for defining device code and distinguishing between device code and host code. A HIP programming language can comprise a subset of the functionality of a CUDA programming language.For example, a HIP programming language may include one or more mechanisms for defining global functions 2112, but such a HIP programming language may not support dynamic concurrency, so global functions 2112 defined in HIP code may only be called by one host.

[0217] The HIP source code 2130 can contain any number (including zero) of global functions 2112, any number (including zero) of device functions 2114, any number (including zero) of host functions 2116, and any number (including zero) of host / device functions 2118. The HIP source code 2130 can also contain any number of calls to any number of functions that can be specified in a HIP runtime API 2132. The HIP runtime API 2132 can contain functionally similar versions of a subset of functions contained in the CUDA runtime API 2102. The HIP source code 2130 can also contain any number of calls to any number of functions that can be specified in any number of other HIP APIs. A HIP API can be any API designed for use by HIP code and / or ROCm.HIP APIs can include the HIP runtime API 2132, a HIP driver API, APIs for any number of HIP libraries, APIs for any number of ROCm libraries, etc.

[0218] The CUDA-to-HIP translation tool 2120 can convert any kernel call in CUDA code from CUDA syntax to HIP syntax and convert any number of other CUDA calls in CUDA code to any number of other functionally similar HIP calls. A CUDA call can include a call to a function specified in a CUDA API, and a HIP call can include a call to a function specified in a HIP API. The CUDA-to-HIP translation tool 2120 can convert any number of calls to functions specified in the CUDA runtime API 2102 to any number of calls to functions specified in the HIP runtime API 2132.

[0219] The CUDA-to-HIP translation tool 2120 may include a tool called hipify-perl, which performs a text-based translation process. Alternatively, the CUDA-to-HIP translation tool 2120 may include a tool called hipify-clang, which performs a more complex and robust translation process compared to hipify-perl. In hipify-clang, CUDA code is parsed using clang (a compiler frontend), and the resulting symbols are then translated. The conversion of CUDA code to HIP code may involve additional modifications beyond those performed by the CUDA-to-HIP translation tool 2120 (such as manual edits).

[0220] The HIP compiler driver 2140 can include a front end that determines a target device 2146 and then configures a compiler compatible with the target device 2146 to compile the HIP source code 2130. The target device 2146 can contain a processor optimized for parallel instruction processing. The HIP compiler driver 2140 can determine the target device 2146 in any technically feasible way.

[0221] If the target device 2146 is CUDA-compatible (e.g., a CUDA-enabled GPU 2194), the HIP compiler driver 2140 can generate a HIP / NVCC compilation instruction 2142. The HIP / NVCC compilation instruction 2142 can configure the CUDA compiler 2150 to compile the HIP source code 2130 using a HIP-to-CUDA translation header and a CUDA runtime library. In response to the HIP / NVCC compilation instruction 2142, the CUDA compiler 2150 can generate host-executable code 2170(1) and device-executable CUDA code 2184.

[0222] If the target device 2146 is not CUDA-compatible, the HIP compiler driver 2140 can generate a HIP / HCC compilation instruction 2144. The HIP / HCC compilation instruction 2144 can configure the HCC 2160 to compile the HIP source code 2130 using a HIP-to-HCC translation header and an HCC runtime library. In response to the HIP / HCC compilation instruction 2144, the HCC 2160 can generate the host-executable code 2170(2) and the device-executable HCC code 2182. The device-executable HCC code 2182 can be a compiled version of the device code contained in the HIP source code 2130, which is executable on the GPU 2192. The GPU 2192 could be any processor optimized for parallel instruction processing, not compatible with CUDA, but compatible with HCC. The GPU 2192 may have been developed by AMD Corporation of Santa Clara, California.The GPU 2192 may include a non-CUDA-capable GPU 2192.

[0223] For explanatory purposes only, in Fig. Figure 21 shows three different processes that can be implemented in at least one embodiment to compile the CUDA source code 2110 for execution on the CPU 2190 and various devices 21. A direct CUDA process can compile the CUDA source code 2110 for execution on the CPU 2190 and the CUDA-enabled GPU 2194 without translating the CUDA source code 2110 into the HIP source code 2130. An indirect CUDA process can translate the CUDA source code 2110 into the HIP source code 2130 and then compile the HIP source code 2130 for execution on the CPU 2190 and the CUDA-enabled GPU 2194. A CUDA / HCC process can translate the CUDA source code 2110 into the HIP source code 2130 and then compile the HIP source code 2130 for execution on the CPU 2190 and the GPU 2192.

[0224] A direct CUDA flow that can be implemented is represented by dashed lines and a series of circles labeled A1-A3. As shown in the circle labeled A1, the CUDA compiler 2150 can receive the CUDA source code 2110 and a CUDA compilation instruction 2148, which can configure the CUDA compiler 2150 to compile the CUDA source code 2110. The CUDA source code 2110, which can be used in a direct CUDA flow, can be written in a CUDA programming language based on a programming language other than C++ (e.g., C, Fortran, Python, Java, etc.). In response to the CUDA compilation instruction 2148, the CUDA compiler 2150 can generate a host-executable code 2170(1) and a device-executable CUDA code 2184 (in the circle labeled A2).As indicated by the circle labeled A3, the host-executable code 2170(1) and the device-executable CUDA code 2184 can each be executed on the CPU 2190 and the CUDA-enabled GPU 2194, respectively. The device-executable CUDA code 2184 can contain binary code. The device-executable CUDA code 2184 can contain PTX code and be further compiled into binary code at runtime for a specific target device.

[0225] An indirect CUDA flow that can be implemented is represented by dashed lines and a series of circles labeled B1-B6. As shown in the circle labeled B1, the CUDA-to-HIP translation tool 2120 can receive the CUDA source code 2110. As shown in the circle labeled B2, the CUDA-to-HIP translation tool 2120 can translate the CUDA source code 2110 into the HIP source code 2130. As shown in the circle labeled B3, the HIP compiler driver 2140 can receive the HIP source code 2130 and determine that the target device 2146 is CUDA-enabled.

[0226] As shown in the circle labeled B4, the HIP compiler driver 2140 can generate a HIP / NVCC compilation instruction 2142 and transfer both the HIP / NVCC compilation instruction 2142 and the HIP source code 2130 to the CUDA compiler 2150. The HIP / NVCC compilation instruction 2142 can configure the CUDA compiler 2150 to compile the HIP source code 2130 using a HIP-to-CUDA translation header and a CUDA runtime library. The HIP-to-CUDA translation header can translate any number of mechanisms (e.g., functions) specified in any number of HIP APIs into any number of mechanisms specified in any number of CUDA APIs.The CUDA compiler 2150 can use the HIP-to-CUDA translation header in conjunction with a CUDA runtime library that conforms to the CUDA runtime API 2102 to generate the host-executable code 2170(1) and the device-executable CUDA code 2184. In response to the HIP / NVCC compilation instruction 2142, the CUDA compiler 2150 can generate the host-executable code 2170(1) and the device-executable CUDA code 2184 (represented by the circle labeled B5). As shown by the circle labeled B6, the host-executable code 2170(1) and the device-executable CUDA code 2184 can be executed on the CPU 2190 and the CUDA-enabled GPU 2194, respectively. The CUDA code 2184 executable by the device can contain binary code. The CUDA code 2184 executable by the device can contain PTX code and be further compiled into binary code at runtime for a specific target device.

[0227] A CUDA / HCC workflow that can be implemented is represented by solid lines and a series of circles labeled C1-C6. As shown in the circle labeled C1, the CUDA-to-HIP translation tool 2120 can receive the CUDA source code 2110. As shown in the circle labeled C2, the CUDA-to-HIP translation tool 2120 can translate the CUDA source code 2110 into the HIP source code 2130. As shown in the circle labeled C3, the HIP compiler driver 2140 can receive the HIP source code 2130 and determine that the target device 2146 is not CUDA-enabled.

[0228] The HIP compiler driver 2140 can generate the HIP / HCC compilation command 2144 and transfer both the HIP / HCC compilation command 2144 and the HIP source code 2130 to the HCC 2160 (represented by the circle labeled C4). The HIP / HCC compilation command 2144 can configure the HCC 2160 to compile the HIP source code 2130 using an HCC translation header and an HCC runtime library. The HIP / HCC runtime library can correspond to the HIP runtime API 2132. The HCC header can contain any number and type of interoperability mechanisms for HIP and HCC. In response to the HIP / HCC compilation instruction 2144, the HCC 2160 can generate the host-executable code 2170(2) and the device-executable HCC code 2182 (represented by the circle labeled C5).As shown by the circle marked C6, the host-executable code 2170(2) and the device-executable HCC code 2182 can each be executed on the CPU 2190 and the GPU 2192, respectively.

[0229] After translating the CUDA source code 2110 into the HIP source code 2130, the HIP compiler driver 2140 can then be used to generate executable code for the CUDA-enabled GPU 2194 or the GPU 2192 without re-executing the CUDA-to-HIP translation tool 2120. The CUDA-to-HIP translation tool 2120 can translate the CUDA source code 2110 into the HIP source code 2130, which is then stored in memory. The HIP compiler driver 2140 can then configure the HCC 2160 to generate the host-executable code 2170(2) and the device-executable HCC code 2182 from the HIP source code 2130. In at least one embodiment, the HIP compiler driver 2140 then configures the CUDA compiler 2150 to generate the host-executable code 2170(1) and the device-executable CUDA code 2184 from the stored HIP source code 2130.

[0230] An example kernel can be generated using the CUDA-to-HIP translation tool 2120. Fig. 21 according to at least one embodiment. The CUDA source code 2110 divides an overall problem, for the solution of which a particular kernel is designed, into relatively coarse subproblems that can be solved independently of each other using thread blocks. Each thread block contains an arbitrary number of threads. Each subproblem can be divided into relatively fine parts that can be solved cooperatively in parallel by threads within a thread block. Threads within a thread block can cooperate by sharing data over a shared memory and synchronizing execution to coordinate memory accesses.

[0231] The CUDA source code 2110 can organize thread blocks associated with a given kernel into a one-dimensional, two-dimensional, or three-dimensional lattice of thread blocks. Each thread block can contain any number of threads, and a lattice can contain any number of thread blocks.

[0232] A kernel can be a function in the device code defined with a "__global__" declaration specifier. The dimension of a grid that executes a kernel for a specific kernel call and associated streams can be specified using a CUDA kernel start syntax. The CUDA kernel start syntax is specified as follows: "KernelName<<<GridSize, BlockSize, SharedMemorySize, Stream> >>(KernelArguments);". An execution configuration syntax can contain a "<<<...>>>" construct inserted between a kernel name ("KernelName") and a list of kernel arguments enclosed in parentheses ("KernelArguments"). The CUDA kernel startup syntax can contain a CUDA startup function syntax instead of an execution configuration syntax.

[0233] `GridSize` can be of type `dim3` and specify the dimension and size of a grid. The `dim3` type can be a CUDA-defined structure containing the unsigned integers `x`, `y`, and `z`. If `z` is not specified, it defaults to one. If `y` is not specified, it defaults to one. The number of thread blocks in a grid can be the product of `GridSize.x`, `GridSize.y`, and `GridSize.z`. `BlockSize` can be of type `dim3` and specify the dimension and size of each thread block. The number of threads per thread block can be the product of `BlockSize.x`, `BlockSize.y`, and `BlockSize.z`. Each thread executing a kernel can be assigned a unique thread ID, which can be accessed within the kernel via a built-in variable (e.g., `threadIdx`).

[0234] Regarding the CUDA kernel start syntax, `SharedMemorySize` can be an optional argument that specifies, in addition to the statically allocated memory, a number of bytes in shared memory that are dynamically allocated per thread block for a given kernel call. By default, `SharedMemorySize` can be zero. Similarly, `Stream` can be an optional argument that specifies an associated stream and is zero by default to indicate a standard stream. A stream can be a sequence of commands (possibly issued by different host threads) that are executed sequentially. Different streams can execute commands related to each other or simultaneously out of sequence.

[0235] The CUDA source code 2110 can contain a kernel definition for an example kernel "MatAdd" and a main function. The main function can be host code that runs on a host and contains a kernel call that causes the kernel MatAdd to execute on a device. The kernel MatAdd can add two matrices A and B of size NxN, where N is a positive integer, and store the result in a matrix C. The main function can define a variable threadsPerBlock as 16 by 16 and a variable numBlocks as N / 16 by N / 16. The main function can then make the kernel call "MatAdd<<<numBlocks, threadsPerBlock> >>(A, B, C);" is specified. According to the CUDA kernel start syntax, the kernel MatAdd can be executed using a grid of thread blocks with a dimension of N / 16 by N / 16, where each thread block has a dimension of 16 by 16.Each thread block can contain 256 threads, a grid can be created with enough blocks to have one thread per matrix element, and each thread in such a grid can execute the kernel MatAdd to perform pairwise addition.

[0236] When translating CUDA source code 2110 to HIP source code 2130, the CUDA-to-HIP translation tool 2120 can translate each kernel call in the CUDA source code 2110 from CUDA kernel start syntax to HIP kernel start syntax and convert any number of other CUDA calls in the source code 2110 to any number of other functionally similar HIP calls. The HIP kernel start syntax can be specified as follows: "hipLaunchKernelGGL(KernelName, GridSize, BlockSize, SharedMemorySize, Stream, KernelArguments);". Each of the `KernelName`, `GridSize`, `BlockSize`, `ShareMemorySize`, `Stream`, and `KernelArguments` can have the same meaning in the HIP kernel start syntax as in the CUDA kernel start syntax (as described previously here). The `ShareMemorySize` and `Stream` arguments can be required in the HIP kernel start syntax and optional in the CUDA kernel start syntax.

[0237] Part of the HIP source code 2130 can be identical to part of the CUDA source code 2110 shown, except for a kernel call that executes the MatAdd kernel on a device. The MatAdd kernel can be defined in the HIP source code 2130 with the same declaration specifier "__global__" as the MatAdd kernel in the CUDA source code 2110. A kernel call in the HIP source code 2130 might be "hipLaunchKernelGGL(MatAdd, numBlocks, threadsPerBlock, 0, 0, A, B, C);", while a corresponding kernel call in the CUDA source code 2110 might be "MatAdd<<<numBlocks, threadsPerBlock> >>(A, B, C);" reads.

[0238] Other implementations are conceivable and can be carried out similarly to the CUDA and HIP implementations mentioned above, such as oneAPI, OpenCL, and other programming platforms. Code can be translated in either direction. For example, CUDA can be translated to HIP, and CUDA can be translated to OpenCL. SnuCL-Tr and CUCL can be used to translate OpenCL to CUDA and CUDA to OpenCL, respectively. Compiled code or intermediate code (e.g., CUDA-PTX code) can also be translated to run on other processor platforms (e.g., AMD or Intel). For example, PTX code can be translated using a translation tool like ZLUDA to run on Intel or AMD processors.

[0239] One or more of the techniques described here can utilize a oneAPI programming model. A oneAPI programming model can refer to a programming model for interacting with various compute accelerator architectures. OneAPI can refer to an application programming interface (API) designed for interacting with various compute accelerator architectures. A oneAPI programming model can utilize a DPC++ programming language. A DPC++ programming language can refer to a high-level language for the productivity of data-parallel programming. A DPC++ programming language can be based, at least partially, on the C and / or C++ programming languages. A oneAPI programming model can be a programming model such as, but not limited to, those developed by Intel Corporation of Santa Clara, California.

[0240] OneAPI and / or the OneAPI programming model can be used to interact with various accelerator, GPU, and processor architectures, and / or variations thereof. OneAPI can include a number of libraries that implement different functions. OneAPI can include at least one oneAPI DPC++ library, oneAPI math kernel library, oneAPI data analysis library, oneAPI deep neural network library, oneAPI collective communication library, oneAPI threading building block library, oneAPI video processing library, and / or variations thereof.

[0241] A oneAPI DPC++ library, also known as oneDPL, can be a library that implements algorithms and functions to accelerate DPC++ kernel programming. OneDPL can implement one or more functions from the Standard Template Library (STL). OneDPL can implement one or more parallel STL functions. OneDPL can provide a number of library classes and functions, such as, but not limited to, parallel algorithms, iterators, function object classes, scope-based APIs, and / or variations thereof. OneDPL can implement one or more classes and / or functions from a C++ standard library. OneDPL can implement one or more random number generator functions.

[0242] A oneAPI mathematics kernel library, also known as oneMKL, can be a library that implements various optimized and parallelized routines for different mathematical functions and / or operations. OneMKL can implement one or more basic linear algebra subprograms (BLAS) and / or linear algebra package (LAPACK) routines for dense linear algebra. OneMKL can implement one or more sparse BLAS routines for linear algebra. OneMKL can implement one or more random number generators (RNGs). OneMKL can implement one or more vector mathematics (VM) routines for mathematical operations on vectors. OneMKL can implement one or more fast Fourier transform (FFT) functions.

[0243] A oneAPI data analysis library, also known as oneDAL, can contain a library that implements various data analysis applications and distributed computations. OneDAL can implement various algorithms for preprocessing, transformation, analysis, modeling, validation, and decision-making for data analysis in batch, online, and distributed processing modes. OneDAL can implement various C++ and / or Java APIs and various connectors to one or more data sources. OneDAL can implement DPC++ API extensions to a standard C++ interface and enables the use of GPUs for various algorithms.

[0244] A oneAPI library for deep neural networks, also known as oneDNN, can contain a library that implements various deep learning functions. OneDNN can implement different neural networks, machine learning and deep learning functions, algorithms, and / or variations thereof.

[0245] A oneAPI library for collective communication, also known as oneCCL, can contain a library that implements various applications for deep learning and machine learning workloads. OneCCL can be built on top of lower-level communication middleware, such as, but not limited to, the Message Passing Interface (MPI) and libfabrics. OneCCL can enable a number of deep learning-specific optimizations, such as, but not limited to, prioritization, persistent operations, out-of-order execution, and / or variations thereof. OneCCL can implement various CPU and GPU functions.

[0246] A oneAPI library for Threading Building Blocks, also known as oneTBB, can comprise a library that implements various parallelized processes for different applications. OneTBB can be used for task-based, shared parallel programming on a host. OneTBB can implement generic parallel algorithms. OneTBB can implement concurrent containers. OneTBB can implement a scalable memory allocator. OneTBB can implement a work-stealing task scheduler. OneTBB can implement low-level synchronization primitives. OneTBB can be compiler-independent and used on various processors, such as, but not limited to, GPUs, PPUs, CPUs, and / or variants thereof.

[0247] A oneAPI video processing library, also known as oneVPL, can comprise a library used to accelerate video processing in one or more applications. OneVPL can implement various video decoding, encoding, and processing functions. OneVPL can implement various functions for media pipelines on CPUs, GPUs, and other accelerators. OneVPL can implement device detection and selection in media-centric and video analytics workloads. OneVPL can implement API primitives for zero-copy buffer sharing.

[0248] A oneAPI programming model can utilize a DPC++ programming language. A DPC++ programming language can contain functionally similar versions of CUDA mechanisms for defining device code and distinguishing between device code and host code. A DPC++ programming language can encompass a subset of the functionality of a CUDA programming language. One or more CUDA programming model operations can be performed using a oneAPI programming model with a DPC++ programming language.

[0249] Each application programming interface (API) described here can be compiled by a compiler, interpreter, or other software tool into one or more instructions, operations, or other signals. Compilation can involve generating one or more machine-executable instructions, operations, or other signals from source code. An API compiled into one or more instructions, operations, or other signals can, when executed, utilize one or more processors, such as, but not limited to, those found, for example, in... Fig. 7-18 processors described herein, or any other logic circuit further described herein, cause one or more arithmetic operations to be performed.

[0250] In at least one embodiment, translation tools described elsewhere herein, such as, but not limited to, HIP, oneAPI, OpenCL, or any other language used to perform any of the operations described above or elsewhere herein, may include one or more circuits to translate CUDA code in order to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription.One or more circuits can be configured by software to translate CUDA code to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, HIP, oneAPI, OpenCL, or any other language used to perform each of the operations described above or elsewhere herein. CLOUD- AND WEB-BASED SERVICES

[0251] The following description presents, without limitation, cloud-based and / or web-based services and / or systems that can be used to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, based on a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform some or all of the processes, operations, and / or techniques described elsewhere herein.Cloud-based and / or web-based services and / or systems can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0252] Cloud computing can encompass a type of data processing where dynamically scalable and often virtualized resources are provided as a service over the internet. Users do not need any knowledge, expertise, or control over the technological infrastructure that can be described as "in the cloud" and supports them. Cloud computing can include Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), and other variations, all of which rely on the internet to meet users' computing needs. A typical cloud deployment, such as in a private cloud (e.g., a corporate network) or a data center (DC) in a public cloud (e.g., the internet), can include thousands of servers (or alternatively, virtual machines), hundreds of Ethernet, Fibre Channel, or Fibre Channel over Ethernet (FCoE) ports, switching and storage infrastructure, and so on.A cloud can also include network service infrastructure such as IPsec VPN hubs, firewalls, load balancers, wide area network (WAN) optimizers, etc. Remote participants can securely access cloud applications and services via a VPN connection, such as an IPsec VPN tunnel.

[0253] Cloud computing can encompass a model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or interaction with the service provider.

[0254] Cloud computing can be characterized by on-demand self-service, where a consumer can unilaterally and automatically provision computing resources such as server time and network storage as needed, without requiring human interaction with the respective service provider. Cloud computing can also be characterized by broad network access, where resources are available over a network and can be accessed via standard mechanisms that facilitate use by heterogeneous thin- or thick-client platforms (e.g., mobile phones, laptops, and PDAs).Cloud computing can be characterized by resource pooling, where a provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, dynamically allocating and reallocating various physical and virtual resources according to consumer demand. In at least one embodiment, there is a sense of location independence, as a customer generally has no control over or knowledge of the precise location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center). Examples of resources include storage, processing power, memory, network bandwidth, and virtual machines.Cloud computing can be characterized by rapid elasticity, where capacity can be provisioned quickly and elastically, in some cases automatically, to scale up rapidly and be released quickly to scale up rapidly. In at least one embodiment, the capacity available for provisioning to a consumer often appears unlimited and can be obtained in any quantity at any time. Cloud computing can also be characterized by a metered service, where cloud systems automatically control and optimize resource utilization by using a metering function at an abstraction level appropriate for a given service type (e.g., storage, processing, bandwidth, and active user accounts). Resource utilization can be monitored, controlled, and reported, providing transparency for both the provider and the consumer of a service.

[0255] Cloud computing can be associated with various services. Cloud Software as a Service (SaaS) can refer to a service where a consumer has the ability to use a provider's applications running on a cloud infrastructure. These applications can be accessed from various client devices via a thin-client interface such as a web browser (e.g., web-based email). In at least one embodiment, the consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, with a possible exception of limited user-specific application configuration settings.

[0256] Cloud Platform as a Service (PaaS) can refer to a service that allows consumers to deploy applications they have created or purchased, built using programming languages ​​and tools supported by the provider, on the cloud infrastructure. In at least one embodiment, a consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but has control over the deployed applications and potentially over configurations of the application hosting environment.

[0257] Cloud Infrastructure as a Service (IaaS) can refer to a service that allows consumers to provision computing power, storage, networking, and other basic computing resources on which they can deploy and run any software, including operating systems and applications. In at least one embodiment, the consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and potentially limited control over selected network components (e.g., host firewalls).

[0258] Cloud computing can be used in various ways. A private cloud can refer to cloud infrastructure operated exclusively for one organization. A private cloud can be managed by that organization or a third party and can be located on-premises or off-premises. A community cloud can refer to cloud infrastructure shared by multiple organizations and supporting a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). A community cloud can be managed by organizations or a third party and can be located on-premises or off-premises.A public cloud can refer to cloud infrastructure made available to the general public or a large industry association and owned by an organization that provides cloud services. A hybrid cloud can refer to a cloud infrastructure consisting of two or more clouds (private, community, or public) that remain independent entities but are connected through standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds). A cloud computing environment is service-oriented and focuses on statelessness, low coupling, modularity, and semantic interoperability. LOGIC AS WELL AS TRAINING AND USE OF NEURAL NETWORKS

[0259] The following figures represent, without limitation, examples of logic- and artificial intelligence-based systems that can be used to implement the functions and / or operations described here.

[0260] Fig. Figure 22A illustrates the logic 2215, which, as described elsewhere herein, can be used in one or more devices or systems (e.g., one of the processors, data centers, cloud-based, or web-based services described herein) to perform operations such as, but not limited to, those discussed herein according to at least one embodiment. Logic may refer to any combination of software logic, hardware logic, and / or firmware logic to provide the functionality and / or operations described herein, the logic being collectively or individually embodied as a circuit that is part of a larger system, such as an integrated circuit (IC), an application-specific integrated circuit (ASIC), a field-programmable array (FPGA), a system-on-a-chip (SoC), or one or more processors (e.g., CPU, GPU). The logic described in Fig. The logic illustrated in Figure 22A, 2215, can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as, but not limited to, a Google TensorFlow® processing unit, a Graphcore™ inference processing unit (IPU), or an Intel Corp. Nervana® processor (e.g., “Lake Crest”). Fig. 22A Illustrated logic 2215 can be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware or other hardware, such as, but not limited to, field-programmable gate arrays (“FPGAs”).

[0261] Fig. Figure 22A illustrates the inference and / or training logic 2215 according to at least one embodiment; The inference and / or training logic 2215 may include hardware logic in which computing resources can be dedicated or otherwise used exclusively in conjunction with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. The in Fig. 22A The illustrated inference and / or training logic 2215 can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as, but not limited to, a Google TensorFlow® processing unit, a Graphcore™ inference processing unit (IPU), or an Intel Corp. Nervana® processor (e.g., “Lake Crest”). The in Fig. 22A The illustrated inference and / or training logic 2215 can be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware, or other hardware, such as, but not limited to, field-programmable gate arrays (“FPGAs”). The inference and / or training logic 2215 can include a code and / or data store 2201 and a code and / or data store 2205, which can be used to store code (e.g., code for graphs), weight values, and / or other information, including distortion values, gradient information, momentum values, and / or other parameter or hyperparameter information. In Fig. For example, in 22A, the code and / or data memory 2201 and the code and / or data memory 2205 are each assigned to a dedicated computing resource, such as, but not limited to, the computing hardware 2202 and computing hardware 2206, respectively. The computing hardware 2202 and the computing hardware 2206 can each comprise one or more ALUs that perform mathematical functions, such as, but not limited to, linear algebraic functions, only on information stored in the code and / or data memory 2201 and the code and / or data memory 2205, respectively, with the result being stored in the activation memory 2220.

[0262] The code and / or data stores 2201 and 2205, as well as the corresponding computing hardware 2202 and 2206, respectively, each correspond to different layers of a neural network. The activation resulting from a memory / computing pair 2201 / 2202 of the code and / or data store 2201 and the computing hardware 2202 is provided as input to the next memory / computing pair 2205 / 2206 of the code and / or data store 2205 and the computing hardware 2206, thus representing a conceptual organization of a neural network. Each of the memory / computing pairs 2201 / 2202 and 2205 / 2206 can correspond to more than one neural network layer. Additional memory / computing pairs following or in parallel to the memory / computing pairs 2201 / 2202 and 2205 / 2206 (not shown) may be included in the inference and / or training logic 2215.

[0263] In at least one embodiment, the logic 2215 described elsewhere herein may include one or more circuits to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more circuits in the logic 2215 can be configured by software described herein to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0264] Fig. Figure 22B illustrates the training and deployment of a deep neural network according to at least one embodiment. An untrained neural network 2226 can be trained using a training dataset 2222. The training framework 2224 can be a PyTorch framework, and / or a training framework 2204 can include a TensorFlow, Boost, Caffe, Microsoft Cognitive Toolkit / CNTK, MXNet, Chainer, Keras, Deeplearning4j, or other training framework. The training framework 2224 can train an untrained neural network 2226 and allows it to be trained using the processing resources described herein to generate a trained neural network 2228. The weights can be selected randomly or by pretraining using a deep belief network. The training can be performed in a supervised, semi-supervised, or unsupervised manner.

[0265] The untrained neural network 2226 can be trained using supervised learning, where the training dataset 2222 contains an input paired with a desired output for that input, or where the training dataset 2222 contains inputs with a known output, and an output of the neural network 2226 is manually graded. The untrained neural network 2226 can be trained in a supervised manner, processing inputs from the training dataset 2222 and comparing the resulting outputs to a set of expected or desired outputs. Errors can then be propagated back through the untrained neural network 2226. The training framework 2224 can adjust weights that control the untrained neural network 2226.The training framework 2224 can include tools to monitor how well the untrained neural network 2226 converges to a model, such as, but not limited to, the trained neural network 2228, which is suitable for generating correct answers, such as in result 2232, but not limited to, input data, such as, but not limited to, a new dataset 2230. The training framework 2224 can repeatedly train the untrained neural network 2226 while adjusting the weights to refine an output of the untrained neural network 2226 using a loss function and an adjustment algorithm, such as stochastic gradient descent, but not limited to, the training framework 2224. The training framework 2224 can train the untrained neural network 2226 until the untrained neural network 2226 achieves a desired accuracy.The trained neural network 2228 can then be used to perform any number of machine learning operations.

[0266] The untrained neural network 2226 can be trained using unsupervised learning, whereby the untrained neural network 2226 attempts to train itself using unlabeled data. The training dataset 2222 for unsupervised learning can contain input data without associated output data or "ground truth" data. The untrained neural network 2226 can learn groupings within the training dataset 2222 and determine how individual inputs might relate to the untrained dataset 2222. Unsupervised training can be used to generate a self-organizing map in a trained neural network 2228 that is capable of performing operations useful for reducing the dimensionality of the new dataset 2230.Unsupervised training can also be used for anomaly detection, which can identify data points in the new dataset 2230 that deviate from the normal patterns of the new dataset 2230.

[0267] Semi-supervised learning can be used, a technique in which the training dataset 2222 contains a mixture of labeled and unlabeled data. The training framework 2224 can be used to perform incremental learning, such as through transferred learning techniques, but is not limited to this. Incremental learning can enable the trained neural network 2228 to adapt to the new dataset 2230 without forgetting the knowledge imparted to the trained neural network 2228 during the initial training.

[0268] The training framework 2224 can include a framework that is processed in conjunction with a software development toolkit, such as, but not limited to, an OpenVINO toolkit (Open Visual Inference and Neural Network Optimization). An OpenVINO toolkit can include a toolkit such as, but not limited to, those developed by Intel Corporation of Santa Clara, California.

[0269] OpenVINO can include a toolkit to facilitate the development of applications, particularly neural network applications, for various tasks and operations, such as, but not limited to, human vision emulation, speech recognition, natural language processing, recommendation systems, and / or variations thereof. OpenVINO can support neural networks, such as, but not limited to, convolutional neural networks (CNNs), recurrent and / or attention-based neural networks, and / or various other neural network models. OpenVINO can support various software libraries, such as, but not limited to, OpenCV, OpenCL, and / or variations thereof.

[0270] OpenVINO can support neural network models for various tasks and operations, such as, but not limited to, classification, segmentation, object recognition, face recognition, speech recognition, pose estimation (e.g., people and / or objects), monocular depth estimation, image retouching, style transfer, action recognition, colorization, and / or variations thereof.

[0271] OpenVINO can include one or more software tools and / or modules for model optimization, also known as model optimizers. A model optimizer may include a command-line tool that facilitates the transitions between training and deploying neural network models. A model optimizer can optimize neural network models for execution on various devices and / or processing units, such as, but not limited to, a GPU, CPU, PPU, GPGPU, and / or variations thereof. A model optimizer can generate an internal representation of a model and optimize this model to produce an intermediate format. A model optimizer can reduce the number of layers in a model. A model optimizer can remove layers from a model that can be used for training.A model optimizer can perform various neural network operations, such as, but not limited to, modifying inputs to a model (e.g., changing the size of inputs to a model), modifying the size of inputs to a model (e.g., modifying a batch size of a model), modifying a model structure (e.g., modifying layers of a model), normalization, standardization, quantization (e.g., converting the weights of a model from a first representation, such as, but not limited to, floating point, to a second representation, such as, but not limited to, integer), and / or variations thereof.

[0272] OpenVINO can include one or more software libraries for inference, also known as inference units. An inference unit can be a C++ library or any suitable programming language library. An inference unit can be used to derive input data. An inference unit can implement various classes to derive input data and generate one or more results. An inference unit can implement one or more API functions to process an intermediate format, specify input and / or output formats, and / or run a model on one or more devices.

[0273] OpenVINO can provide various functions for the heterogeneous execution of one or more neural network models. Heterogeneous execution or heterogeneous computing can refer to one or more computing processes and / or systems that utilize one or more types of processors and / or cores. OpenVINO can provide various software functions to run a program on one or more devices. OpenVINO can provide various software functions to run a program and / or parts of a program on different devices. For example, OpenVINO can provide various software functions to run one part of the code on a CPU and another part of the code on a GPU and / or an FPGA. OpenVINO can provide various software functions to run one or more layers of a neural network on one or more devices (e.g.,...a first set of layers on a first device, such as a GPU, but not limited to that, and a second set of layers on a second device, such as a CPU, but not limited to that).

[0274] OpenVINO can encompass various functions similar to those associated with a CUDA programming model, such as, but not limited to, various neural network model operations related to frameworks such as, but not limited to, TensorFlow, PyTorch, and / or variations thereof. One or more CUDA programming model operations can be performed using OpenVINO. Various systems, methods, and / or techniques described herein can be implemented using OpenVINO.

[0275] In at least one embodiment, one or more circuits can be used to cause one or more neural networks and training frameworks described elsewhere herein to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions at least partially based on a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.One or more neural networks and training frameworks can be configured by software to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, by means of a syntax error identified in the descriptive information of the at least one document transcription, or otherwise to perform any of the operations described above or elsewhere herein.

[0276] At least one embodiment of the disclosure can be described with regard to the following clauses: 1. Processor, which includes the following: one or more circuits to: to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, based on a syntax error that was identified in the descriptive information of the at least one document transcription. 2. Processor according to clause 1, wherein the syntax error is identified by at least one of the following: Structural information in the descriptive information; or a semantic class identifier in the descriptive information. 3. Processor according to one of clauses 1-2, wherein the at least one document transcription comprises a sequence of tokens, the sequence of tokens containing one or more content tokens comprising a coded representation of the document content and one or more description tokens comprising a coded representation of the descriptive information of the document content. 4. Processor according to clause 3, where the syntax error is identified by a missing description token in the sequence of tokens. 5. Processor according to clause 3, wherein the syntax error is found in one of the one or more description tokens. 6. Processor according to one of clauses 1-5, wherein the error evaluation of the at least one document transcription is continued after error detection. 7. Processor according to clause 1, wherein the one or more neural networks are a multimodal large language model (MLLM) comprising at least one encoder, compressor or decoder. 8. Procedure comprising the following: Generating, by one or more processors, respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and Detect, by one or more processors, an error in the document content of at least one document transcription of the respective document transcriptions, at least partially based on a syntax error identified in the descriptive information of the at least one document transcription. 9. Procedure according to clause 8, wherein the syntax error is identified by at least one of the following: Structural information in the descriptive information; or a semantic class identifier in the descriptive information. 10. Method according to any of clauses 8-9, wherein the at least one document transcription comprises a sequence of tokens, the sequence of tokens containing one or more content tokens comprising a coded representation of the document content and one or more description tokens comprising a coded representation of the descriptive information of the document content. 11. Procedure according to clause 10, wherein the syntax error is identified by a missing description token in the sequence of tokens. 12. Procedure according to clause 10, wherein the syntax error is found in one of the one or more description tokens. 13. Procedure according to one of clauses 8-12, wherein the error assessment of the at least one document transcription is continued after error detection. 14. Method according to any of clauses 8-13, wherein the one or more neural networks are a multimodal large language model (MLLM) comprising at least one encoder, compressor or decoder. 15. System comprising the following: one or more processors to: to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, based on a syntax error that was identified in the descriptive information of the at least one document transcription. 16. System according to clause 15, wherein the syntax error is identified by at least one of the following: Structural information in the descriptive information; or a semantic class identifier in the descriptive information. 17. System according to one of clauses 15-16, wherein the at least one document transcription comprises a sequence of tokens, the sequence of tokens containing one or more content tokens comprising a coded representation of the document content and one or more description tokens comprising a coded representation of the descriptive information of the document content. 18. System according to clause 17, wherein the syntax error is identified by a missing description token in the sequence of tokens. 19. System according to clause 17, wherein the syntax error is found in one of the one or more description tokens. 20. System according to one of clauses 15-19, wherein the error assessment of the at least one document transcription continues after error detection.

[0277] As will be apparent to a person sufficiently skilled in the art, other variations are within the meaning of the present disclosure. Thus, although the disclosed techniques permit various modifications and alternative constructions, certain embodiments are shown in the drawings and have been described in detail above. It is understood, however, that the disclosure is not intended to be limited to any particular disclosed form or forms, but rather, on the contrary, to cover all modifications, alternative constructions, and equivalents that fall within the spirit and scope of the disclosure, as defined in the attached claims.

[0278] The use of the terms "a", "an", "the", "a", and "a", as well as similar designations, in connection with the description of disclosed embodiments (particularly in connection with the following claims) is to be interpreted as covering both the singular and the plural unless otherwise specified herein or clearly contradicted by the context, and not as a definition of a term. The terms "comprising", "with", "including", and "containing" are to be understood as open terms (i.e., "including but not limited to") unless otherwise specified. The use of "may" is intended as an example without limiting any particular embodiment or component or other functions described above, below, or elsewhere herein.The term "connected," when it remains unchanged and refers to physical connections, is to be understood as "partially or wholly contained in, attached to, or connected with," even if something in between is involved. The specification of value ranges here is merely a method of abbreviation to refer individually to each value falling within the range, unless otherwise stated, and each individual value will be included in the description as if it were listed here separately. The use of the terms "set" (e.g., "a set of elements") or "subset" is, unless otherwise stated or contradicted by the context, to be understood as a non-empty collection comprising one or more elements.Unless otherwise noted or contradicted by the context, the term "subset" of a corresponding sentence does not necessarily refer to a true subset of the corresponding sentence; rather, subset and corresponding sentence may be the same.

[0279] Conjunctive formulations such as, but not limited to, sentences of the form "at least one of A, B, and C" or "at least one of A, B, and C" are, unless expressly stated otherwise or clearly contradicted by the context, generally understood to mean that an element, term, etc., can be either A, B, or C, or any non-empty subset of the sentence A, B, and C. In an illustrative example of a sentence with three elements, the conjunctive formulations "at least one of A, B, and C" and "at least one of A, B, and C" refer to one of the following sentences: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Such conjunctive formulations are therefore not generally intended to mean that in certain embodiments at least one of A, at least one of B, and at least one of C must be present.Unless otherwise noted or contradicted by the context, the term "plurality" also denotes a state of plurality (e.g., "a multitude of elements" means multiple elements). The number of items in a plurality can be at least two, but can also be more if this is either explicitly stated or indicated by the context. Unless otherwise stated or evident from the context, the phrase "based on" means "at least partly based on" and not "exclusively based on".

[0280] The operations of the processes described herein may be performed in any suitable order, unless otherwise specified or the context clearly indicates otherwise. A process such as, but not limited to, the processes described herein (or variations and / or combinations thereof) may be carried out under the control of one or more computer systems configured with executable instructions and is implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or combinations thereof. Code may be stored on a computer-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions that can be executed by one or more processors.A computer-readable storage medium can be a non-volatile, computer-readable storage medium that excludes volatile signals (e.g., a propagating transient electrical or electromagnetic transmission) but includes non-volatile data storage circuitry (e.g., buffers, caches, and queues) within transceivers for volatile signals. Code (e.g., executable code or source code) can be stored on a set of one or more non-volatile, computer-readable storage media containing executable instructions (or other memory for storing executable instructions) which, when executed by one or more processors of a computer system (i.e., as a result of execution), cause the computer system to perform the operations described herein.A set of non-volatile computer-readable storage media can comprise multiple non-volatile computer-readable storage media, and one or more of the individual non-volatile storage media within a set may lack all the code, while multiple non-volatile computer-readable storage media collectively store all the code. Executable instructions can be executed in such a way that different instructions are carried out by different processors—for example, a non-volatile computer-readable storage medium stores instructions, and a main processor (“CPU”) executes some of the instructions, while a graphics processing unit (“GPU”) executes other instructions. Different components of a computer system can have separate processors, and different processors can execute different subsets of instructions.

[0281] An arithmetic logic unit (ALU) can contain a set of combinational logic circuits that require one or more inputs to produce a result. An ALU can be used by a processor to implement mathematical operations such as, but not limited to, addition, subtraction, or multiplication. An ALU can also implement logical operations such as, but not limited to, AND / OR or XOR operations. An ALU can be stateless and consist of physical switching components, such as, but not limited to, semiconductor transistors arranged to form logic gates. An ALU can also operate internally as a stateful logic circuit with an associated clock.An arithmetic logic unit (ALU) can be implemented as an asynchronous logic circuit whose internal state is not stored in an associated register set. An ALU can be used by a processor to combine operands stored in one or more of the processor's registers and generate an output that the processor can store in another register or memory location.

[0282] As a result of processing an instruction received from the processor, the processor can pass one or more inputs or operands to an arithmetic logic unit (ALU), causing the ALU to produce a result based at least partially on an instruction code provided to the ALU's inputs. The instruction codes provided by the processor to the ALU can be based at least partially on the instruction executed by the processor. The combinational logic in the ALU can process the inputs and produce an output that is placed on a bus within the processor. A processor can select a destination register, memory location, output device, or output memory location on the output bus, so that the processor's clocking causes the results produced by the ALU to be sent to the desired location.

[0283] Within the scope of this application, the term arithmetic logic unit or ALU is used to refer to any computational logic circuit that processes operands to produce a result. In this document, the term "ALU" may, for example, refer to a floating-point unit, a DSP, a tensor core, a shader core, a coprocessor, or a CPU.

[0284] One or more components of the systems and / or processors disclosed above can communicate with one or more CPUs, ASICs, GPUs, FPGAs, or other hardware, circuit, or integrated circuit components, which may include, for example, an upscaler or upsampler for upscaling an image, an image mixer or image mixer component for blending, mixing, or combining images, a sampler for sampling an image (e.g., as part of a DSP), a neural network circuit configured to upscale an image (e.g.,from a low-resolution image to a high-resolution image), or include other hardware for modifying or generating an image, frame, or video to adjust its resolution, size, or pixels; one or more components of the systems and / or processors disclosed above may use components described in this disclosure to perform procedures, operations, or instructions that generate or modify an image.

[0285] Computer systems can be configured to implement one or more services that individually or collectively perform operations of the processes described herein, and such computer systems are configured with suitable hardware and / or software that enables the execution of operations. Furthermore, a computer system implementing at least one embodiment of the present disclosure is a single device, and in another embodiment, a distributed computer system comprising multiple devices that operate differently, such that a distributed computer system performs the operations described herein, and a single device does not perform all operations.

[0286] The use of any and all examples or illustrative phrases (e.g., "such as, but not limited to") serves only to better illustrate embodiments of the disclosure and does not constitute a limitation of the scope of the disclosure unless otherwise claimed. No wording in the description should be interpreted as implying that an unclaimed element is essential to the implementation of the disclosure.

[0287] All references cited herein, including publications, patent applications and patents, are hereby incorporated by reference to the same extent as if each reference were individually and expressly stated as being incorporated by reference and reproduced here in its entirety.

[0288] The terms "coupled" and "connected," as well as their derivatives, may be used in the description and claims. It is understood that these terms are not necessarily synonymous. Rather, in certain examples, "connected" or "coupled" may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" can also mean that two or more elements are not in direct contact with each other but nevertheless cooperate or interact.

[0289] Unless expressly stated otherwise, it can be understood that terms such as "processing", "calculating", "calculating", "determining", etc., throughout this description refer to actions and / or processes of a computer or computing system or similar electronic computing device that manipulate and / or convert data represented as physical, e.g., electronic, quantities in the registers and / or memories of the computing system into other data represented in a similar manner as physical quantities in the memories, registers, or other information storage, transmission, or display devices of the computing system.

[0290] Similarly, the term "processor" can refer to any device or part of a device that processes electronic data from registers and / or memories and converts that electronic data into other electronic data that can be stored in registers and / or memories. As non-restrictive examples, "processor" can be a CPU or a GPU. A "computing platform" can include one or more processors. The term "software" processes, as used here, can include, for example, software and / or hardware units that perform work over time, such as, but not limited to, tasks, threads, and intelligent agents. Each process can also refer to multiple processes for executing instructions sequentially or in parallel, continuously or intermittently.The terms "system" and "procedure" are used interchangeably here, since the system can comprise one or more procedures and the procedures can be considered as a system.

[0291] Reference can be made to the acquisition, acquisition, reception, or input of analog or digital data into a subsystem, computer system, or computer-implemented machine. Processes for acquiring, acquiring, receiving, or inputting analog and digital data can be performed in various ways, such as, but not limited to, receiving data as parameters of a function call or an application programming interface call. The processes for acquiring, acquiring, receiving, or inputting analog or digital data can be performed by transmitting data over a serial or parallel interface. The processes for acquiring, acquiring, receiving, or inputting analog or digital data can be performed by transmitting data over a computer network from the providing unit to the receiving unit.It can also refer to the provision, output, transmission, sending, or display of analog or digital data. In various examples, processes for providing, outputting, transmission, sending, or displaying analog or digital data can be carried out by transferring data as input or output parameters of a function call, as parameters of an application programming interface, or of an interprocess communication mechanism.

[0292] Although exemplary implementations of the described techniques are presented in the present descriptions, other architectures may be used to implement the described functions, which are intended to fall within the scope of this disclosure. While certain distributions of responsibilities may be defined above for descriptive purposes, various functions and responsibilities may also be distributed and divided in different ways depending on the circumstances.

[0293] Even if the subject matter is described in language relating to structural features and / or methodological actions, it is understood that the subject matter claimed in the attached claims is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as exemplary forms of implementing the claims.

[0294] It is understood that the aspects and embodiments described above are purely exemplary and that modifications of details may be made within the scope of protection of the claims.

[0295] Each device, each method and each feature disclosed in the description and (where applicable) the claims and drawings may be provided independently or in any suitable combination.

[0296] The reference numerals included in the claims are for illustrative purposes only and are not intended to have any limiting effect on the scope of the claims.< / classes> < / bbox> < / classes> < / bbox>

Claims

[1] Processor comprising the following: one or more circuits to: to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, based on a syntax error that was identified in the descriptive information of the at least one document transcription. [2] Processor according to claim 1, wherein the syntax error is identified by at least one of the following: Structural information in the descriptive information; or a semantic class identifier in the descriptive information. [3] Processor according to claim 1 or 2, wherein the at least one document transcription comprises a sequence of tokens, the sequence of tokens comprising one or more content tokens comprising a coded representation of the document content and one or more description tokens comprising a coded representation of the descriptive information of the document content. [4] Processor according to claim 3, wherein the syntax error is identified by a missing description token in the sequence of tokens. [5] Processor according to claim 3, wherein the syntax error is found in one of the one or more description tokens. [6] Processor according to any of the preceding claims, wherein the error evaluation of the at least one document transcription is continued after error detection. [7] Processor according to one of the preceding claims, wherein the one or more neural networks are a multimodal large language model (MLLM) comprising at least one encoder, compressor or decoder. [8] Procedure comprising the following: Generating, by one or more processors, respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and Detect, by one or more processors, an error in the document content of at least one document transcription of the respective document transcriptions, at least partially based on a syntax error identified in the descriptive information of the at least one document transcription. [9] Method according to claim 8, wherein the syntax error is identified by at least one of the following: Structural information in the descriptive information; or a semantic class identifier in the descriptive information. [10] Method according to claim 8 or 9, wherein the at least one document transcription comprises a sequence of tokens, the sequence of tokens comprising one or more content tokens comprising a coded representation of the document content and one or more description tokens comprising a coded representation of the descriptive information of the document content. [11] Method according to claim 10, wherein the syntax error is identified by reference to a missing description token in the sequence of tokens. [12] Method according to claim 10, wherein the syntax error is found in one of the one or more description tokens. [13] Method according to any one of claims 8 to 12, wherein the error evaluation of the at least one document transcription is continued after the error detection. [14] Method according to any one of claims 8 to 13, wherein the one or more neural networks are a multimodal large language model (MLLM) comprising at least one encoder, compressor or decoder. [15] System comprising the following: one or more processors to: to generate respective document transcriptions of one or more document images using one or more neural networks, wherein the respective document transcriptions include document content and descriptive information of the document content; and to detect an error in the document content of at least one document transcription of the respective document transcriptions, at least partially, based on a syntax error that was identified in the descriptive information of the at least one document transcription. [16] System according to claim 15, wherein the syntax error is identified by at least one of the following: Structural information in the descriptive information; or a semantic class identifier in the descriptive information. [17] System according to claim 15 or 16, wherein the at least one document transcription comprises a sequence of tokens, the sequence of tokens comprising one or more content tokens comprising a coded representation of the document content and one or more description tokens comprising a coded representation of the descriptive information of the document content. [18] System according to claim 17, wherein the syntax error is identified by a missing description token in the sequence of tokens. [19] System according to claim 17, wherein the syntax error is found in one of the one or more description tokens. [20] System according to any one of claims 15 to 19, wherein the error evaluation of the at least one document transcription is continued after the error detection.

Images