Methods and apparatus for document parsing for retrieval-augmented generation

Abstract

Systems, apparatus, articles of manufacture, and methods are disclosed for document parsing for retrieval-augmented generation. An example non-transitory machine readable storage medium comprises instructions to cause programmable circuitry to at least: extract a characteristic of a document to be parsed; generate a prompt for a natural language processing machine learning model engine based on the characteristic, the prompt to provide an instruction for the natural language processing machine learning model engine to parse the document into a plurality of segments based on document context; determine an output of the natural language processing machine learning model engine based on the prompt, the output indicating a segmentation of the document into the plurality of segments; and cause a plurality of embedding vectors for the plurality of segments to be stored in a vector database.

Images