Method for model training and document compression for implementing ai agent based on ai, rpa, and llm

Abstract

This application proposes a model training method and document compression method for implementing an AI Agent based on AI, RPA, and LLM, covering the fields of AI, RPA, LLM, and AI Agent. The method includes: acquiring sample document images and the original sample documents corresponding to the document content displayed in the sample document images; compressing the original sample documents using a compression representation grammar to obtain compressed sample text; and training a Visual Language Model (VLM) based on the sample document images and the compressed sample text, so that the trained VLM learns the compression representation grammar. Thus, the trained VLM can effectively filter redundant information corresponding to meaningless visual elements (such as blank rows in tables), output compressed text conforming to the compression representation grammar, reduce the data volume of the generated text, alleviate the resource burden of data transmission and storage, and eliminate the need for subsequent additional redundant information removal operations. In subsequent data processing, it also helps reduce computational overhead and improve the accuracy and reliability of model inference.

Description

Technical Field

[0001] This application relates to the fields of Artificial Intelligence (AI), Robotic Process Automation (RPA), and Artificial Intelligence Agent (AI Agent), and in particular to a model training method and document compression method for implementing an AI Agent based on AI, RPA, and LLM. Background Technology

[0002] Robotic Process Automation (RPA) uses specific "robot software" to simulate human operations on a computer and automatically execute process tasks according to rules.

[0003] Artificial intelligence (AI) is a technical science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence.

[0004] Large Language Models (LLMs) are models trained on massive amounts of text that can recognize human language, perform language-related tasks, and have a large number of parameters.

[0005] Artificial Intelligence Agents (AI Agents) are capable of perceiving their environment, making decisions, and executing actions. Unlike traditional artificial intelligence, they possess the ability to think and act independently, and can utilize tools to achieve given goals. Based on LLM (Limited Learning Model) as its core computing engine, AI Agents can engage in dialogue, perform tasks, reason, and exhibit a degree of autonomy. They possess the ability to autonomously understand, perceive, plan, remember, and use tools, enabling them to automate complex tasks. Specifically, LLM-driven AI Agents, comprised of various AI capabilities, can interact with employees using natural language, understand their instructions and needs, and provide feedback and responses; they can acquire domain-specific knowledge relevant to the business to complete complex professional tasks; they can break down complex tasks into several executable tasks and use data and tools to complete them; and they can collaborate with employees, and AI Agents can also collaborate to complete complex tasks, enabling digital employees to leap from automation to intelligence, helping employees complete their work more efficiently, and fully realizing human-machine collaboration.

[0006] In related technologies, the Vision Language Model (VLM) has been applied to multimodal document processing tasks. One typical requirement is to accurately convert the table structure in an image into a certain text format (such as HTML). VLM, through an end-to-end generation mechanism, strictly adheres to the visual presentation of tables. For example, when encountering consecutive empty rows, the system independently generates complete labels and nested structures for each row. While this "what you see is what you get" conversion logic completely preserves the visual features of the original table, ensuring that the generated result strictly corresponds to the input image in layout, it also introduces some problems when processing large and sparse document tables. On the one hand, it generates a large amount of redundant information, which not only increases the data transmission burden but also brings unnecessary computational overhead to subsequent storage, parsing, and secondary editing. On the other hand, because the VLM model uses a self-attention mechanism, when duplicate tokens are generated during inference, it is prone to causing infinite text loops, leading to document parsing failure and further inference failure. Summary of the Invention

[0007] This application provides a model training method and document compression method for implementing an AI Agent based on AI, RPA, and LLM, to solve one of the technical problems existing in related technologies. The technical solution is as follows:

[0008] In a first aspect, embodiments of this application provide a model training method for implementing an AI Agent based on AI, RPA, and LLM, including: acquiring sample document images and sample original documents corresponding to the document content displayed by the sample document images; compressing the sample original documents using a compressed representation grammar to obtain sample compressed text; and training a visual language model (VLM) based on the sample document images and sample compressed text, so that the trained VLM learns the compressed representation grammar.

[0009] Secondly, embodiments of this application provide a document compression method for document images, comprising: acquiring a target document image; using a visual language model (VLM) with a learned compression representation syntax to compress the document content displayed in the target document image to obtain target compressed text; wherein the VLM is trained using the method described in any of the embodiments of the first aspect.

[0010] Thirdly, embodiments of this application provide a document decompression method, comprising: obtaining target compressed text; wherein the target compressed text is obtained by compressing the document content displayed in a target document image using a Visual Language Model (VLM) with a learned compressed representation syntax, and the VLM is trained using the method described in any of the embodiments of the first aspect; identifying at least one compressed representation in the target compressed text; wherein the compressed representation includes a compression attribute; decompressing the compressed representation according to the compression attribute in the compressed representation to obtain a corresponding target document fragment; and generating a target restored text of the target compressed text based on the target document fragment.

[0011] Fourthly, embodiments of this application provide a model training device for implementing an AI Agent based on AI, RPA, and LLM, comprising: an acquisition module for acquiring sample document images and sample original documents corresponding to the document content displayed in the sample document images; a compression module for compressing the sample original documents using a compression representation syntax to obtain sample compressed text; and a training module for training a visual language model (VLM) based on the sample document images and sample compressed text, so that the trained VLM learns the compression representation syntax.

[0012] Fifthly, embodiments of this application provide a document compression apparatus for document images. The apparatus includes: an acquisition module for acquiring a target document image; and a compression module for compressing the document content displayed in the target document image using a Visual Language Model (VLM) with a learned compression representation syntax to obtain target compressed text. The VLM is trained using the method described in any of the embodiments of the first aspect.

[0013] Sixthly, embodiments of this application provide a document decompression apparatus, comprising: an acquisition module for acquiring target compressed text; wherein the target compressed text is obtained by compressing the document content displayed in a target document image using a Visual Language Model (VLM) with a learned compressed representation syntax, and the VLM is trained using the method described in any of the embodiments of the first aspect; an identification module for identifying at least one compressed representation in the target compressed text; wherein the compressed representation includes compression attributes; a decompression module for decompressing the compressed representation according to the compression attributes in the compressed representation to obtain a corresponding target document fragment; and a generation module for generating target restored text of the target compressed text according to the target document fragment.

[0014] In a seventh aspect, embodiments of this application provide an electronic device comprising a memory and a processor. The memory and the processor communicate with each other via an internal connection path. The memory stores instructions, and the processor executes the instructions stored in the memory. When the processor executes the instructions stored in the memory, it causes the processor to perform the method described in any of the above embodiments.

[0015] Eighthly, embodiments of this application provide a computer-readable storage medium storing a computer program, wherein when the computer program is run on a computer, the methods in any of the above-described embodiments are executed.

[0016] Ninthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the methods in any of the above-described embodiments.

[0017] The advantages or beneficial effects of the above technical solutions include at least the following:

[0018] Obtain the sample document image and the original sample document corresponding to the document content displayed in the sample document image; compress the original sample document using the compressed representation syntax to obtain the sample compressed text; train the visual language model (VLM) based on the sample document image and the sample compressed text so that the trained VLM learns the compressed representation syntax. By semantically refining the original sample documents using compressed representation grammar and then combining this with cross-modal training of the VLM (Visual Model), the model can break through the inherent "what you see is what you get" generation logic, learn the deep relationship between the visual form of the document and its core semantic content, effectively filter redundant information corresponding to meaningless visual elements (such as blank rows in tables), reduce the data volume of the generated text, and alleviate the resource burden of data transmission and storage. Secondly, the trained VLM can directly output a concise text format that conforms to the compressed representation grammar, without the need for subsequent additional redundant information removal operations, effectively reducing the computational overhead of text parsing and secondary editing, and improving the accuracy and reliability of model inference. In addition, users do not need to make large-scale modifications to the VLM's infrastructure; the model's capabilities can be optimized and upgraded simply by building specific training sample pairs. It has strong compatibility and scalability and can be quickly adapted to various practical business scenarios such as HTML table conversion and document information extraction. Furthermore, the accuracy and effectiveness of the model's output can be improved based on AI, RPA, LLM, and AI Agent technologies.

[0019] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0020] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0021] Figure 1 This is a flowchart of a model training method for implementing an AI Agent based on AI, RPA, and LLM, provided in one embodiment of this application;

[0022] Figure 2 This is a flowchart of a model training method for implementing an AI Agent based on AI, RPA, and LLM, provided in another embodiment of this application;

[0023] Figure 3 This is a flowchart of a model training method for implementing an AI Agent based on AI, RPA, and LLM, provided in another embodiment of this application;

[0024] Figure 4 This is a flowchart of a model training method for implementing an AI Agent based on AI, RPA, and LLM, provided in another embodiment of this application;

[0025] Figure 5 This is a flowchart of a model training method for implementing an AI Agent based on AI, RPA, and LLM, provided in another embodiment of this application;

[0026] Figure 6 This is a flowchart of a document compression method for document images provided in one embodiment of this application;

[0027] Figure 7 This is a flowchart of a document decompression method provided in one embodiment of this application;

[0028] Figure 8 This is a schematic diagram illustrating the implementation principle of the model training method for AI Agent based on AI, RPA, and LLM provided in this application.

[0029] Figure 9 This is a structural diagram of a model training device for implementing an AI Agent based on AI, RPA, and LLM, provided in one embodiment of this application;

[0030] Figure 10 This is a structural diagram of a document compression apparatus for document images provided in one embodiment of this application;

[0031] Figure 11This is a structural diagram of a document decompression apparatus provided in one embodiment of this application;

[0032] Figure 12 A structural block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0033] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0034] In related technologies, Visual Modeling (VLM) has been applied to multimodal document processing tasks. One typical requirement is to accurately convert table structures in images into a specific text format, such as HTML. VLM, through an end-to-end generation mechanism, strictly adheres to the visual representation of tables. For example, when encountering consecutive empty rows, the system independently generates complete labels and nested structures for each row. This "what you see is what you get" conversion logic completely preserves the visual features of the original table, ensuring that the generated result strictly corresponds to the input image in layout.

[0035] However, when dealing with large and sparse document tables, this conversion method exposes many problems, generating a large amount of redundant information, which in turn leads to a series of drawbacks that cannot be ignored:

[0036] Token inefficiency: Because the model generates complete HTML tags for consecutive blank lines, these redundant tags directly cause the length of the model's input and output sequences to increase linearly. Furthermore, the VLM model itself has a context window constraint, and excessively long sequences easily trigger this constraint, thus affecting the model's complete understanding and accurate processing of the input information, and reducing overall processing efficiency.

[0037] High computational cost: Increased sequence length means the model needs to process more data units during forward propagation. Each forward propagation operation consumes computational resources, and longer sequences will undoubtedly require the model to perform more forward propagation operations. This not only increases inference latency and prolongs the time it takes for the model to output results, but also significantly increases GPU resource utilization, resulting in a waste of computational resources.

[0038] Heavy economic burden: Currently, many commercial APIs, such as those provided by OpenAI, charge based on the number of tokens. Since the aforementioned redundant tags significantly increase the number of tokens, this directly translates into higher usage costs, which is undoubtedly a considerable expense for users or businesses that frequently use this technology for table transformations.

[0039] Model inference risks: The Transformer architecture is the core architecture of VLM, and its self-attention mechanism is quite sensitive to patterns in the input data. Highly repetitive HTML tag patterns may interfere with the normal operation of the self-attention mechanism, causing the model to fail to accurately capture key information during inference, leading to chaotic model output or even falling into a repetitive loop error state, seriously affecting the accuracy and reliability of the transformation results.

[0040] To address at least one of the problems existing in related technologies, embodiments of this application propose a model training method and a document compression method for implementing an AI Agent based on AI, RPA, and LLM. These and other aspects of the embodiments of this application will become clear with reference to the following description and accompanying drawings. In these descriptions and drawings, specific implementations of some embodiments of this application are specifically described to illustrate some ways of implementing the principles of the embodiments of this application; however, it should be understood that the scope of the embodiments of this application is not limited thereto. Rather, the embodiments of this application include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0041] Before describing the specific embodiments of this application, for ease of understanding, commonly used technical terms will first be introduced:

[0042] In the description of this application, the term "multiple" means two or more.

[0043] In the description of this application, the term "VLM" is a multimodal artificial intelligence model that combines computer vision and natural language processing technologies. Its core objective is to achieve cross-modal understanding, reasoning, and generation by integrating visual information, such as images and videos, and linguistic information, such as text and speech.

[0044] In this application / description, the term "OCR" refers to Optical Character Recognition, specifically the process by which an electronic device examines characters printed on paper, determines their shape by detecting dark and light patterns, and then translates the shape into computer text using character recognition methods; that is, for printed characters, it uses optical methods to convert the text in a paper document into a black and white dot matrix image file, and then uses recognition software to convert the text in the image into text format for further editing and processing by word processing software.

[0045] In the description of this application, the term "document image" is a digital representation of a physical document, which includes visual elements such as text, tables, charts, illustrations, stamps, and handwritten notes, and is usually stored in image formats such as JPEG, PNG, TIFF, and PDF.

[0046] In the description of this application, the term "empty table row" means that there is at least one data row in a table, and all cells in that row are empty.

[0047] In the description of this application, the term "duplicate table content row" refers to a table containing multiple rows of data where the cell content of each column is exactly the same or certain key fields are identical.

[0048] In the description of this application, the term "duplicate cell" refers to a row of multiple consecutive cells in a table where the content of each cell is the same.

[0049] In the description of this application, the term "consecutive empty cells" refers to consecutive empty cells, which means that in a row of table data, the contents of multiple consecutive cells are empty.

[0050] In the description of this application, the term "repeating text line" refers to a text line in a non-table area of ​​the corresponding document with the same text content.

[0051] These and other aspects of the embodiments of this application will become clear from the following description and accompanying drawings. In these descriptions and drawings, specific embodiments of the present application are specifically described to illustrate some ways of implementing the principles of the embodiments of the present application; however, it should be understood that the scope of the embodiments of the present application is not limited thereto. Rather, the embodiments of the present application include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0052] The following describes, with reference to the accompanying drawings, a model training method and a document compression method for implementing an AI Agent based on AI, RPA, and LLM according to embodiments of this application.

[0053] Figure 1 This is a flowchart of a model training method for implementing an AI Agent based on AI, RPA, and LLM, provided in one embodiment of this application.

[0054] In one possible implementation provided in this application, the model training method for implementing an AI Agent based on AI, RPA, and LLM is configured in a model training device for implementing an AI Agent based on AI, RPA, and LLM as an example. This model training device can be applied to any electronic device with computing capabilities.

[0055] The electronic device can be a personal computer, a mobile terminal, a server (or a server), etc. The mobile terminal can be a mobile phone, a tablet computer, a personal digital assistant, or other hardware device with various operating systems.

[0056] In another possible implementation of this application embodiment, the model training method for implementing an AI Agent based on AI, RPA, and LLM can be applied to an AI Agent, wherein the AI ​​Agent can run on any electronic device with computing capabilities.

[0057] In another possible implementation of this application's embodiments, the model training method for AI Agents based on AI, RPA, and LLM can be applied to intelligent automation platforms or digital employee platforms. The implementation of digital employee platforms involves three stages. The first stage is the automation stage: for RPA stages with low business complexity, software automation technology is used to automate rule-based, predefined procedural tasks. The second stage is the intelligence stage: leveraging AI to extend the boundaries of RPA, such as processing unstructured documents and making data-driven decisions. The third stage is the human-machine collaboration stage: utilizing the understanding, planning, and execution capabilities of large models to automate complex tasks end-to-end.

[0058] In the human-machine collaboration phase, the digital employee platform serves as a bridge connecting workers and systems, workers and data, and systems and data. It is capable of: operating complex systems, processing various types of data, and interacting and collaborating with employees. The digital employee platform helps industries build large-scale, model-enabled digital employees—intelligent agents—to automate, intelligently manage business processes and achieve human-machine collaboration.

[0059] The digital employee platform can seamlessly integrate multiple capabilities such as agent process automation, agent document processing (ADP), and agent business insight. It has five major functions: "business understanding", "process creation", "run anywhere", "centralized management and control" and "human-machine collaboration". It enables enterprises to achieve end-to-end intelligent automation of business processes, replace manual operations, further improve business efficiency, and accelerate digital transformation.

[0060] ADP is a next-generation platform based on LLM and VLM, combined with AI Agent technology, that enables end-to-end automated document processing. It represents a new generation of document processing solutions using LLM and AI Agent technologies. It is no longer a "tool" that requires configuring templates or annotating samples, but rather an "intelligent agent" capable of understanding business needs and autonomously planning and executing. Traditional document processing systems are "tools": users need to explicitly tell the system "how to do it." ADP, on the other hand, is an "intelligent agent": users only need to tell the system "what to do," and the system can autonomously understand, plan, and execute.

[0061] like Figure 1 As shown, the model training method for implementing an AI Agent based on AI, RPA, and LLM may include the following steps S101 to S103:

[0062] Step S101: Obtain the sample document image and the original sample document corresponding to the document content displayed in the sample document image.

[0063] The sample document images can display corresponding document content, which may include structured content such as tables and lists, or unstructured content such as text paragraphs, headings, footnotes, or endnotes. It should be noted that this application does not limit the number of sample document images; it may be, but is not limited to, one.

[0064] The original sample document can be uncompressed text data that is equivalent to the document content shown in the sample document image.

[0065] The original document in the sample can be in the following text formats, but are not limited to: HTML, Markdown, plain text, etc.

[0066] To acquire sample document images, in one example where the executing entity of this application is a digital employee platform, the AI ​​Agent of the digital employee platform, upon receiving the task instruction for model training, can invoke the ADP set in the digital employee platform and use the LLM in the ADP to parse the task instruction to determine the paper documents and quantity to be collected. Then, the digital employee platform can activate the RPA to establish communication with the relevant scanner. Subsequently, the RPA issues scanning task instructions through the scanner's control interface, which can carry scanning parameters to trigger the scanner to scan the placed paper documents. After scanning is completed, the RPA automatically retrieves the generated image files from the scanner and uploads them to the digital employee platform, which can then use these image files as sample document images.

[0067] In another example, the digital employee platform can provide a mobile data collection portal. When a user takes a picture of a document using the camera on their mobile device, the mobile device can automatically upload the captured document image to the cloud storage directory specified by the digital employee platform through the mobile data collection portal. The AI ​​Agent can schedule RPA to monitor the cloud storage directory in real time. When document images are detected, these document images can be retrieved and used as sample document images.

[0068] In another example, the AI ​​Agent of the digital employee platform can automatically obtain sample document images directly from a pre-set digital image library. These images include images obtained by converting PDF documents, webpage screenshots, etc.

[0069] To obtain the original sample document corresponding to the document content displayed by the sample document image, in one example, the AI ​​Agent of the digital employee platform can call the OCR engine to perform character recognition on the sample document image, obtain the initial text, and map the initial text into the original sample document in target formats such as HTML, Markdown, or plain text. Alternatively, if a preset digital image library contains sample document images and the original documents corresponding to the document content displayed by the sample document images, the AI ​​Agent can automatically retrieve the original document corresponding to the sample document image as the original sample document, and so on.

[0070] It should be noted that this application does not impose any restrictions on the method of obtaining the sample document image or the method of obtaining the original sample text.

[0071] Step S102: Compress the original sample document using the compression representation syntax to obtain the compressed sample text.

[0072] Compression representation syntax refers to a method that uses concise symbols or compression rules to represent complex data or structures.

[0073] As an example, when the implementing entity of this application is a digital employee platform, for blank rows in a table, the AI ​​Agent of the digital employee platform can use the corresponding compressed representation syntax to convert the blank rows of the table into corresponding tag representations, such as:

[0074] ;

[0075] Here, `compressed` indicates whether compression is enabled; `true` indicates compression. `type` indicates the compression type, with a value of `empty_row`, indicating that the compression type is empty rows. `cols` indicates the number of rows occupied, and `rows` indicates the number of columns occupied.

[0076] For rows with duplicate content, the corresponding compression syntax is used to convert the duplicated table row content into corresponding tag representations, such as:

[0077] <data>……< / data> ;

[0078] Duplicate rows refer to rows in a table where multiple rows contain identical cell content or share certain key fields. For example, suppose table 1 is as shown in Table 1:

[0079] Table 1 Data Table 1

[0080]

[0081] In Table 1, there are 3 rows of data where the content of each cell in each column is exactly the same. Therefore, the multiple rows of data in Table 1 with completely identical content are considered duplicate rows. For example, suppose Table 2 is as shown in Table 2:

[0082] Table 2 Data Table 2

[0083]

[0084] In Table 2, the first column contains the same keyword "project" and the second column contains the same keyword "cost: yuan". Therefore, the situation in Table 2 is identified as a duplicate table content row.

[0085] Among them, compressed indicates whether compression is used; when its value is true, it means compression is used. type indicates the compression type, and its value is repeated_text_line, which indicates that the compression type is repeated table content rows. count indicates the number of repetitions. template indicates the template format of repeated table content rows. <data>……< / data> Used to indicate the values ​​of each field in a row of duplicate table content;

[0086] For duplicate cells, the appropriate compression syntax is used to convert the content of the duplicated cells into corresponding tags, such as:

[0087] ;

[0088] Among them, "duplicate cells" refers to multiple consecutive cells in a row of table data with the same cell content value; "compressed" indicates whether compression is used, and when it is true, it means compression is used; "type" indicates the compression type, and its value is "repeated_cell", which indicates that the compression type is duplicate cells; "count" indicates the number of times it is repeated; and "value" indicates the cell content value.

[0089] For consecutive empty cells, the corresponding compression syntax is used to convert the content of consecutive empty cells into corresponding tag representations, such as:

[0090] ;

[0091] Among them, consecutive empty cells refer to multiple consecutive cells in a row of table data with empty cell content; compressed indicates whether compression is used, when its value is true, it means compression is used; type indicates the compression type, its value is empty_cell, which indicates that the compression type is consecutive empty cells; count indicates the number of repetitions;

[0092] For repeated text lines in non-table areas of a document, the corresponding compression rule is to use a general tagging method to convert the repeated text lines in non-table areas into corresponding tag representations, such as:

[0093] [COMPRESSED type=repeated_line count= template="..."];

[0094] Among them, repeated text lines refer to text lines with the same text content in non-table areas of the corresponding document; COMPRESSED indicates that it is a compressed form; type indicates the compression type, and its value is repeated_line, which indicates that the compression type is repeated text lines; count indicates the number of repetitions; template indicates the text content of the repeated text lines.

[0095] In the embodiments of this application, a compression representation syntax can be used to compress the original sample document to obtain compressed sample text.

[0096] Step S103: Train the VLM based on sample document images and sample compressed text so that the trained VLM learns the compressed representation syntax.

[0097] Among them, VLM has the function of recognizing and parsing the document content in the corresponding document image. For example, it can be an LLaVA (Large Language and Vision Assistant) model, a Qwen-VL (QwenVision-Language Model) model, etc. This application does not limit it.

[0098] In this embodiment of the application, when the executing entity of this application is a digital employee platform, the AI ​​Agent of the digital employee platform can combine sample document images and sample compressed text into image-compressed text pairs; then, the image-compressed text pairs are used as training samples, and the training samples are used to train the visual language model (VLM) in the ADP set in the digital employee platform, so that the trained visual language model learns the compressed representation syntax.

[0099] This application's embodiment presents a model training method for an AI Agent based on AI, RPA, and LLM. It performs semantic condensation processing on the original sample documents using compressed representation grammar, and then conducts cross-modal training of the VLM using sample document images. This allows the model to break through the inherent "what you see is what you get" generation logic, learning the deep correlation between the document's visual form and core semantic content. It effectively filters redundant information corresponding to meaningless visual elements such as blank rows in tables, reducing the data volume of generated text and alleviating the resource burden of data transmission and storage. Secondly, the trained VLM can directly output a simplified text format conforming to compressed representation grammar, eliminating the need for subsequent redundant information removal operations. This effectively reduces the computational overhead of text parsing and secondary editing, improving the model's inference accuracy and reliability. Furthermore, it requires no large-scale modification of the VLM's infrastructure; the model's capabilities can be optimized and upgraded simply by constructing specific training sample pairs. It possesses strong compatibility and scalability, and can be quickly adapted to various practical business scenarios such as HTML table conversion and document information extraction. In addition, the accuracy and effectiveness of the model's output can be improved based on AI, RPA, LLM, and AIAgent technologies.

[0100] To clearly illustrate how the compressed representation syntax is used in any embodiment of this application to compress the original sample document and obtain the compressed sample text, this application also proposes a model training method for implementing an AI Agent based on AI, RPA, and LLM.

[0101] Figure 2 This is a flowchart of a model training method for implementing an AI Agent based on AI, RPA, and LLM, provided in another embodiment of this application.

[0102] It should be noted that the model training method for implementing AI Agent based on AI, RPA and LLM can be executed alone, or it can be executed together with any embodiment of this application or possible implementation methods in the embodiments, or it can be executed together with any technical solution in related technologies. The embodiments of this application do not limit this.

[0103] like Figure 2 As shown, the model training method for implementing an AI Agent based on AI, RPA, and LLM may include the following steps S201 to S205:

[0104] Step S201: Obtain the sample document image and the original sample document corresponding to the document content displayed in the sample document image.

[0105] The explanation of step S201 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0106] Step S202: Parse the original sample document to identify at least one type of first document fragment and its corresponding compression attribute in the original sample document.

[0107] The first document fragment may contain document content that is repeated.

[0108] The fragment type can be used to indicate the type of the corresponding document fragment, including but not limited to: blank table rows, repeating table content rows, repeating cells, consecutive empty cells, repeating text lines in non-table areas, etc.

[0109] In any embodiment of this application, when the fragment type of the first document fragment is a table empty row, its corresponding compression attributes include at least one of the following: whether to compress, fragment type, number of rows occupied, and number of columns involved;

[0110] When the fragment type of the first document fragment is a repeating table content row, its corresponding compression attributes include at least one of the following: whether to compress, fragment type, number of repetitions, first content template, and cell content value;

[0111] When the fragment type of the first document fragment is a repeating cell, its corresponding compression attributes include at least one of the following: whether to compress, fragment type, cell content value, and number of times it appears repeatedly;

[0112] When the fragment type of the first document fragment is consecutive empty cells, its corresponding compression attributes include at least one of the following: whether to compress, fragment type, and number of consecutive empty cells;

[0113] When the fragment type of the first document fragment includes repeated text lines, its corresponding compression attributes include at least one of the following: fragment type, number of repetitions, and second content template.

[0114] The cell content value can be used to indicate the specific data or information stored in the corresponding cell.

[0115] The first content template can be used to indicate the template format for repeating table content rows. Taking the data in Table 1 as an example, its corresponding first content template is, for example: template="Project A: Cost: 150 yuan"; taking the data in Table 2 as an example, its corresponding first content template is, for example: template="Project {name}&& Cost: {amount} yuan", where {name} and {amount} are placeholders.

[0116] The second content template can be used to indicate the text content of the repeating text lines. As an example, suppose the content of the repeating text lines is as follows:

[0117] Clause 1: This is the content of the first clause.

[0118] Clause 1: This is the content of the first clause.

[0119] Clause 1: This is the content of the first clause.

[0120] For the aforementioned repeated text lines, the corresponding second content template would be, for example: template="Clause 1: This is the content of the first clause."

[0121] It should be noted that the examples of the first and second content templates above are merely illustrative. In practical applications, other forms are also possible, and this application does not impose any restrictions on them.

[0122] Therefore, differentiated compression attribute dimensions were defined for different types of first document fragments, enabling refined feature characterization of various repetitive and redundant fragments. This provides comprehensive and accurate data support for subsequent data processing and avoids compression decision bias caused by missing attribute dimensions.

[0123] It should also be noted that this application does not restrict the fragment type to which the first document fragment in the original sample document belongs.

[0124] Step S203: Based on the compression representation syntax and the compression attributes of at least one first document fragment, compress at least one first document fragment to obtain a compressed representation of at least one first document fragment.

[0125] In any embodiment of this application, when there are multiple compression representation syntaxes, for any first document fragment, a target compression representation syntax corresponding to the fragment type to which the first document fragment belongs is determined from the multiple compression representation syntaxes; the target compression representation syntax is used to compress the first document fragment based on the compression attributes corresponding to the first document fragment to obtain the corresponding compression representation.

[0126] As an example, in the case where the executing entity of this application is a digital employee platform, the AI ​​Agent of the digital employee platform can pre-establish the correspondence between compressed representation syntax and fragment types and save the correspondence. Then, in subsequent applications, after determining the fragment type to which the first document fragment belongs, the AI ​​Agent can query the above correspondence based on the fragment type to which the first document fragment belongs, so as to obtain the target compressed representation syntax corresponding to the fragment type to which the first document fragment belongs from multiple compressed representation syntaxes.

[0127] Therefore, by supporting the matching of exclusive target compression representation syntax for different types of first document fragments, precise adaptation of fragment type and compression rules is achieved. Compared with the "one-size-fits-all" approach of a single compression syntax for all repeating fragments, it can formulate differentiated compression logic based on the structural features and repetition patterns of different fragments. This avoids semantic loss caused by over-compression and eliminates the problem of redundant information left by under-compression, thus improving the quality and accuracy of sample compressed text. In addition, the multi-syntax matching mechanism allows the trained VLM to learn the ability to classify and differentiate document semantic compression. When processing document images, it can autonomously identify different types of repeating fragments and call the corresponding compression rules, flexibly cope with complex and diverse document structures, and effectively solve the pain point of inconsistent compression results when traditional VLM processes heterogeneous documents.

[0128] In any embodiment of this application, for any first document fragment, a corresponding reference threshold is determined based on the fragment type to which the first document fragment belongs; key attributes are extracted from the compression attributes corresponding to the first document fragment; based on the comparison result between the reference threshold and the target value of the key attribute, it is determined whether to compress the first document fragment; if it is determined that the first document fragment should be compressed, a target compression representation syntax corresponding to the fragment type to which the first document fragment belongs is determined from multiple compression representation syntaxes.

[0129] As an example, in order to determine the reference threshold corresponding to the first document fragment, a correspondence between the reference threshold and the fragment type can be established in advance and the correspondence can be saved. Then, in subsequent applications, after determining the fragment type to which the first document fragment belongs, the above correspondence can be queried based on the fragment type to which the first document fragment belongs in order to determine the reference threshold corresponding to the fragment type to which the first document fragment belongs.

[0130] In this embodiment, key attributes can be extracted from the compression attributes corresponding to the first document fragment. As an example, assuming the fragment type of the first document fragment includes empty table rows, its corresponding compression attributes include whether it is compressed, fragment type, number of rows occupied, and number of columns involved; the number of rows occupied can be used as the key attribute. Assuming the fragment type of the first document fragment includes repeating table content rows, its corresponding compression attributes include whether it is compressed, fragment type, number of repetitions, first content template, and cell content value; the number of repetitions can be used as the key attribute. Assuming the fragment type of the first document fragment includes repeating cells, its corresponding compression attributes include whether it is compressed, fragment type, cell content value, and number of repetitions; the number of repetitions can be used as the key attribute. Assuming the fragment type of the first document fragment includes consecutive empty cells, its corresponding compression attributes include whether it is compressed, fragment type, and number of consecutive empty cells; the number of consecutive empty cells can be used as the key attribute. Assuming the fragment type of the first document fragment includes repeating text lines, its corresponding compression attributes include fragment type, number of repetitions, and second content template; the number of repetitions can be used as the key attribute.

[0131] In this embodiment, a reference threshold and a target value of a key attribute can be compared, and based on the comparison result, it can be determined whether to compress the first document fragment. For example, when the executing entity of this application is a digital employee platform, if the comparison result indicates that the target value of the key attribute is greater than the reference threshold, the AI ​​Agent of the digital employee platform determines to compress the first document fragment; conversely, if the comparison result indicates that the target value of the key attribute is not greater than the reference threshold, the AI ​​Agent of the digital employee platform determines not to compress the first document fragment. For instance, assuming the key attribute is "number of lines occupied," the target value corresponding to the number of lines occupied is n, and the reference threshold is k, if n > k, then it is determined to compress the first document fragment, and if n ≤ k, then it is determined not to compress the first document fragment.

[0132] Furthermore, in this embodiment of the application, if it is determined that the first document fragment is to be compressed, a target compression representation syntax corresponding to the fragment type to which the first document fragment belongs can be determined from multiple compression representation syntaxes.

[0133] Therefore, by configuring dedicated reference thresholds for different types of first document fragments and combining them with the target values ​​of key attributes for quantitative judgment, the compression operation can be triggered in a refined and controllable manner. Compared with blind compression without threshold constraints, it can effectively distinguish between "redundant and repetitive fragments that need to be compressed" and "necessary repetitive fragments that need to be retained," effectively avoiding damage to the core semantic logic of the document due to excessive compression, and effectively ensuring the semantic integrity and accuracy of the sample compressed text. In addition, by combining threshold judgment with the syntax matching process, the trained VLM can learn the hierarchical processing logic of "attribute quantification analysis - compression necessity judgment - target syntax call," enabling it to efficiently filter meaningless repetitive and redundant information and properly retain repetitive content with business value in actual document image parsing tasks. This improves the model's adaptive processing capability for complex documents and enhances the flexibility and customizability of model data processing.

[0134] In this embodiment of the application, a target compression representation syntax can be used to compress the first document fragment based on the compression attributes corresponding to the first document fragment, thereby obtaining the corresponding compressed representation.

[0135] As an example, suppose the first document fragment belongs to the type of empty row in a table, and its corresponding compression tag in the target compression representation syntax is:; and the compression attributes of the first document fragment include the fragment type type as "empty_row", the number of rows occupied cols as 5, and the number of columns involved as 6. Then, the values ​​of the above compression attributes are filled into the compression tag shown in the target compression representation syntax to obtain the compression representation of the first document fragment:.

[0136] Step S204: Generate sample compressed text based on the compressed representation of the first document fragment and the second document fragment whose content is not repeated in the original sample document.

[0137] As an example, the compressed representation of the first document fragment and the second document fragment can be merged according to the order of their positions in the original sample document, so as to obtain the sample compressed text.

[0138] In any embodiment of this application, for any first document fragment, when it is determined, based on the comparison result between a reference threshold and the target value of a key attribute in the compression attributes corresponding to the first document fragment, that the first document fragment can be used as a first target document fragment; wherein, the reference threshold is determined based on the fragment type to which the first document fragment belongs; when it is determined, based on the corresponding comparison result, that the first document fragment can be used as a second target document fragment.

[0139] Furthermore, as one possible implementation, when there are multiple first document fragments, and each first document fragment includes a first target document fragment and a second target document fragment, sample compressed text can be generated based on the compressed representation corresponding to the first target document fragment among the multiple first document fragments, the second target document fragment among the multiple first document fragments, and the second document fragment. For example, the compressed representation corresponding to the first target document fragment among the multiple first document fragments, the second target document fragment among the multiple first document fragments, and the second document fragment can be fused according to the order of their positions in the original sample document to obtain sample compressed text.

[0140] Step S205: Train the VLM based on sample document images and sample compressed text so that the trained VLM learns the compressed representation syntax.

[0141] The explanation of step S205 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0142] In any embodiment of this application, a VLM can be used to compress the document content displayed by the sample document image to obtain predicted compressed text; based on the difference between the predicted compressed text and the sample compressed text, the VLM is trained so that the trained VLM learns the compressed representation syntax.

[0143] As an example, the AI ​​Agent of the digital employee platform can input sample document images into the VLM in the digital employee platform and obtain predicted compressed text in response to the output of the VLM. Then, the AI ​​Agent adopts a target loss function and determines the corresponding loss value based on the difference between the predicted compressed text and the sample compressed text. Finally, the AI ​​Agent can optimize and adjust the model parameters of the VLM according to the loss value using a target parameter adjustment method to minimize the loss value.

[0144] The target loss function can be, for example, the negative log-likelihood loss function, etc. This application does not impose any restrictions on it.

[0145] The target parameter adjustment method can be a full parameter fine-tuning method or a parameter-efficient fine-tuning method, such as the LoRA (Low-Rank Adaptation) algorithm, and this application does not limit it.

[0146] It should be noted that the above example only uses minimizing the loss value as the termination condition for training the visual language model. In actual applications, other termination conditions can also be set. For example, the termination condition can be that the number of training iterations reaches a set number, or the training duration reaches a set duration, etc. This application does not impose any restrictions on this.

[0147] Therefore, by using sample compressed text as a supervision benchmark and guiding model parameter updates through difference calculation, the trained VLM can master the compression logic of different types of redundant segments, enabling the compressed text generated by the model when processing actual document images to be highly consistent with the expected grammatical norms, thereby improving the accuracy and effectiveness of the model's output compression results.

[0148] As another example, the AI ​​Agent of the digital employee platform can acquire cue words and input the cue words and sample document images into the Visual Language Model (VLM). In response to the output of the Visual Language Model, it obtains the predicted compressed text. Then, the AI ​​Agent trains the Visual Language Model based on the difference between the predicted compressed text and the sample compressed text, so that the trained Visual Language Model learns the compressed representation syntax.

[0149] The model training method for AI Agent based on AI, RPA, and LLM in this application embodiment parses the original sample document to identify at least one type of first document fragment and its corresponding compression attribute; wherein the first document fragment contains document content with repeated content; based on the compression representation syntax and the compression attribute of at least one first document fragment, the at least one first document fragment is compressed to obtain a compressed representation of at least one first document fragment; and sample compressed text is generated based on the compressed representation of the first document fragment and a second document fragment with non-repeating content in the original sample document. Therefore, on the one hand, by pre-analyzing the original sample documents and identifying the first document fragment containing duplicate content and its compression attributes, it is possible to perform targeted structured compression on redundant and duplicate information. Compared with indiscriminate text compression methods, this not only ensures the semantic integrity of the non-duplicate second document fragments but also achieves efficient simplification of duplicate content, improving the semantic density and quality of the sample compressed text. On the other hand, targeted compression based on the compression representation syntax and the compression attributes of document fragments allows the trained VLM to learn the logical chain of "identifying duplicate fragments - matching compression rules - outputting simplified representations." When processing document images containing duplicate content, it can actively filter redundant information and generate structured compressed text, effectively solving the problem of a large number of redundant labels generated by the "what you see is what you get" approach in scenarios such as table conversion using traditional VLM, reducing the resource consumption of data transmission, storage, and subsequent parsing. In addition, the compression attributes and syntax rules can be adjusted according to the fragment characteristics of different types of documents, exhibiting strong flexibility and adaptability.

[0150] To clearly illustrate how the original sample document is parsed in any embodiment of this application to identify at least one type of first document fragment and corresponding compression attributes corresponding to the original sample document, this application also proposes a model training method for implementing an AI Agent based on AI, RPA, and LLM.

[0151] Figure 3 This is a flowchart of a model training method for implementing an AI Agent based on AI, RPA, and LLM, provided in another embodiment of this application.

[0152] It should be noted that the model training method for implementing AI Agent based on AI, RPA and LLM can be executed alone, or it can be executed together with any embodiment of this application or possible implementation methods in the embodiments, or it can be executed together with any technical solution in related technologies. The embodiments of this application do not limit this.

[0153] like Figure 3 As shown, the model training method for implementing an AI Agent based on AI, RPA, and LLM may include the following steps S301 to S307:

[0154] Step S301: Obtain the sample document image and the original sample document corresponding to the document content displayed in the sample document image.

[0155] The explanation of step S301 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0156] Step S302: Obtain the text format of the original sample document.

[0157] The original document in the sample can be in the following text formats, but are not limited to: HTML, Markdown, plain text, etc.

[0158] For example, the text format of the original sample document can be determined based on its file extension or metadata; or, relevant format detection algorithms or machine learning-based classifiers can be used to determine the text format of the original sample document, and so on.

[0159] Step S303: Based on the text format, the original sample document is segmented to obtain at least one document content block.

[0160] It should be noted that this application does not limit the number of document content blocks.

[0161] In any embodiment of this application, such as Figure 4As shown, the implementation process of step S303 may include the following steps S3031 to S3035:

[0162] Step S3031: Use a table detection strategy corresponding to the text format to detect the original sample document to determine whether the original sample document contains table content.

[0163] As an example, the AI ​​Agent of a digital employee platform can detect whether the original sample document contains... Tags and Tags; in determining the original sample document including Tags and In the case of tags, the AI ​​Agent can determine that the original sample document includes table content; if it determines that the original sample document does not contain... Tags and In the case of tags, the AI ​​Agent can determine that the original sample document does not include table content. Assuming that the original sample document is in Markdown format, the AI ​​Agent can detect whether the original sample document matches the target feature pattern "|.*|[\n\r]+|[-:| ]+|". If the original sample document matches the target feature pattern, it is determined that the original sample document includes table content; otherwise, it is determined that the original sample document does not include table content. The structure "|.*|[\n\r]+|[-:| ]+|" contains the .* part used to match any character, vertical bars separating columns, and a structure below the table header row consisting of hyphens (-) and vertical bars separating rows, etc.

[0164] It should be noted that the above-described method for detecting the original sample document is merely exemplary. In practical applications, other methods may also be used, and this application does not impose any restrictions on them.

[0165] It should be noted that there may be cases where the original sample document does not include table content.

[0166] Step S3032: If it is determined that the original sample document includes table content, extract the table content block from the original sample document and obtain the location information of the table content block.

[0167] In this embodiment of the application, if it is determined that the original sample document includes table content, the table content block can be extracted from the original sample document, and the location information of the table content can be obtained.

[0168] In one example, assuming the original sample document is in plain text format, the AI ​​Agent can first identify the starting row of the table. For example, by scanning text lines, it can find rows where the first row contains regular column separators such as spaces, tabs, and vertical lines, and the number of columns is fixed. Furthermore, this row is followed by matching separators such as "----", "====", or continuously aligned content lines. Then, the AI ​​Agent can identify the ending row of the table. For example, it can traverse downwards from the starting row until a row has a different column number or separator position than the main body of the table, or contains irregular text. The row preceding this row is the ending row of the table. Subsequently, the AI ​​Agent extracts all text from the starting row to the ending row from the original sample document, thus obtaining the table content block. It can then determine the position information of the table content block by including the row number of the starting row, the row number of the ending row, the total character offset of the first character of the table in the document, and the total character offset of the last character of the table in the document.

[0169] In another example, suppose the original text of the sample contains The AI ​​Agent can identify the outermost layer of tags. The starting and ending tag positions; the AI ​​Agent extracts the outermost tag from the original sample document. The entire HTML code between the start tag and the corresponding end tag of the table content block is used to generate the table content block. The starting character offset of this starting label, The offset of the closing character of this closing tag is used to determine the position information of the table content block.

[0170] In another example, assuming the original sample text is in Markdown format, the AI ​​Agent can first find the dividing lines in the original sample document, such as lines composed of |, -, :, etc. The line above this dividing line is the table header line, which is determined as the table start line. The AI ​​Agent traverses downwards from this dividing line until a line has no vertical line separating it or the number of columns is inconsistent with the table header. The line above this line is then determined as the table end line. The AI ​​Agent extracts the Markdown text between the table start line and the table end line from the original sample document to obtain the table content block. It can determine the position information of the table content block by the line number of the table start line, the line number of the table end line, the total character offset of the first character of the table (such as a vertical line), and the total character offset of the last character of the table.

[0171] It should be noted that the above examples of methods and location information for extracting table content blocks are merely illustrative. In practical applications, other methods and location information may be used for extracting table content blocks. This application does not impose any restrictions on the methods and location information for extracting table content blocks.

[0172] Step S3033: Analyze the original sample document based on the location information of the table content blocks to determine whether the original sample document includes non-table content.

[0173] As an example, suppose the position information of the table content block includes the starting character offset, which is the total character offset of the first character of the table, and the ending character offset, which is the total character offset of the last character of the table. When the starting character offset of the table content block is greater than 0, it indicates that there is non-table content before the table content block in the original document of the sample; if the ending character offset of the table content block is less than the total number of characters in the document, it indicates that there is non-table content after the table content block.

[0174] It should be noted that in practical applications, there may be cases where the original sample document does not include non-table content.

[0175] Step S3034: If the original sample document contains non-table content, extract the non-table content block from the original sample document.

[0176] As an example, text in a non-table area before a table content block, between any two table content blocks, or after a table content block can be extracted from the original sample document and used as non-table content.

[0177] Step S3035: Treat both table content blocks and non-table content blocks as document content blocks.

[0178] It should be noted that the above example only illustrates that document content blocks include table content blocks and non-table content blocks. In actual applications, after the original sample document is segmented, there may be only table content blocks, or only non-table content blocks, or both table content blocks and non-table content blocks. This application does not impose any restrictions on this.

[0179] Therefore, it is possible to accurately determine whether the original document contains table content, and then further identify and separate non-table content blocks by combining the position information of the table content blocks. This effectively realizes the structured separation of table and non-table content, effectively avoids the segment recognition deviation caused by the mixing of the two types of content, and helps to improve the accuracy of subsequent redundant segment detection and compressed attribute extraction.

[0180] Step S304: Detect any document content block to identify a first document fragment of at least one fragment type and its corresponding compression attribute.

[0181] In any embodiment of this application, such as Figure 5 As shown, the implementation process of step S304 may include the following steps S3041 to S3043:

[0182] Step S3041: For any document content block, determine the target type corresponding to the document content block from multiple fragment types.

[0183] It should be noted that this application does not limit the number of target types corresponding to document content blocks; there may be one or more.

[0184] As an example, for any document content block, assuming the document content block belongs to a table content block, the target types corresponding to the document content block are determined to include blank table rows, repeating table content rows, repeating cells, and consecutive empty cells; assuming the document content block belongs to a non-table content block, the target types corresponding to the document content block are determined to include repeating text lines.

[0185] It should be noted that a correspondence between content block types and fragment types can be established in advance and saved. Then, after determining the content block type to which a document content block belongs, the above correspondence can be queried to determine the corresponding target type.

[0186] Content block types can include table content blocks, non-table content blocks, etc.

[0187] Step S3042: Using duplicate content recognition rules that correspond to the target type, information recognition is performed on the document content block to determine whether there is first information in the document content block that matches the target type and has duplicate content.

[0188] As an example, assuming the target type of the document content block is a blank table row, for any row in the table within the document content block, the AI ​​Agent can examine the cell content value of each cell, remove leading and trailing whitespace characters from the corresponding cell content value, and determine whether it is an empty string. If all cells in the row meet the above condition, the AI ​​Agent marks this row as a blank row, and can identify the row in the table marked as a blank row as the first piece of information that matches the target type "blank table row" and has duplicate content, and can record the positions of consecutively occurring blank rows.

[0189] As another example, assuming the target type of the document content block is a duplicate table content row, for any cell in any column of any row in the table within the document content block, the AI ​​Agent compares the cell content value of that cell with the cell content value of the corresponding cell in the previous row to determine if they are exactly the same. If it is determined that the cell content values ​​of all columns in the current row are exactly the same as the cell content values ​​of the corresponding cells in the previous row, then the current row and the previous row are determined to be duplicate table content rows. Alternatively, for any cell in any column of any row in the table within the document content block, the AI ​​Agent compares the cell content value of that cell with the cell content value of the corresponding cell in the previous row to determine if the key fields are the same. If it is determined that the cell content values ​​of all columns in the current row are the same as the cell content values ​​of the corresponding cells in the previous row, then the AI ​​Agent can determine that the current row and the previous row are the first pieces of information matching the target type "duplicate table content row" and having duplicate content.

[0190] As another example, suppose the target type of the document content block is repeating text lines. For any text line in the text content block that belongs to a non-table content block, the AI ​​Agent compares the text content of the text line with the text content of the previous text line to determine whether they are the same. If they are the same, the AI ​​Agent determines that the text line and the previous text line are the first information that match the target type "repeating text lines" and have repeated content.

[0191] In any embodiment of this application, when the document content block is a table content block, the table content block can be formatted to obtain the target table information of the target data structure; the duplicate content identification rules corresponding to the target type are used to identify the target table information to determine whether there is first information in the target table information that matches the target type and has duplicate content.

[0192] The target data structure can be a standardized tabular data structure, such as a standardized JSON tabular data structure, and can include table-level information, row-level information, and cell-level information. Table-level information can include a unique table identifier, source format, and row list, etc. Row-level information can include blank row markers, row structure feature hash values, etc. Cell-level information can include text content, null value markers, and the row and column numbers to which they belong, etc.

[0193] As an example, assuming the original sample document's text format is HTML, the AI ​​Agent can use the BeautifulSoup library to parse the table content blocks and organize them into a standardized table data structure along the "row-column" dimension. Alternatively, assuming the original sample document's text format is HTML, the AI ​​Agent can first locate the header row, separator rows, and content rows of the table content blocks, split each line of text content using pipe commands, remove formatting characters such as hyphens and colons from the separator rows, and then extract the cell content values ​​from each column after cleaning. Simultaneously, it records the cell's positional attributes, such as row number, column number, and alignment, including at least one of the left, right, or center alignment corresponding to the colon position in the separator row, thus organizing the data into a standardized table data structure along the "row-column" dimension.

[0194] Step S3043: In response to the existence of first information in the document content block that matches the target type and has duplicate content, determine the first document fragment and the corresponding compression attribute based on the first information and the target type.

[0195] For example, when there is first information in the document content block that matches the target type and has duplicate content, AIAgent can extract the first information from the document content block based on the location information of the first information and identify the first information as the first document fragment; it can also determine the compression attribute corresponding to the first document fragment based on the first information and the target type. For example, assuming the target type is an empty table row, the number of rows and columns involved in the compression attribute can be determined based on the first information, and the target type can be identified as the fragment type in the compression attribute.

[0196] Therefore, by matching different document content blocks with corresponding target types and calling corresponding duplicate content recognition rules to carry out targeted detection, the typological and precise identification of duplicate fragments is achieved. Compared with generalized recognition logic, it can accurately capture the differences in duplicate features of different types of content blocks, effectively avoiding the problems of missed detection and false detection caused by the mismatch between recognition rules and content types, and improving the recognition accuracy of the first document fragment and compression attributes. In addition, the target type-based targeted recognition strategy can make the subsequent compression process deeply bound to content features, enhancing the targeting and effectiveness of compression processing.

[0197] It is understood that when there is at least one target type corresponding to a document content block, there may be a document content block that does not contain first information that matches any target type and whose content is duplicated. In this case, the document content block can be used as the second document fragment in this application.

[0198] Step S305: Based on the compression representation syntax and the compression attributes of at least one first document fragment, compress at least one first document fragment to obtain a compressed representation of at least one first document fragment.

[0199] Step S306: Generate sample compressed text based on the compressed representation of the first document fragment and the second document fragment whose content is not repeated in the original sample document.

[0200] Step S307: Train the VLM based on sample document images and sample compressed text so that the trained VLM learns the compressed representation syntax.

[0201] It should be noted that the explanations of steps S305 to S307 can be found in the relevant descriptions in any embodiment of this application, and will not be repeated here.

[0202] This application's embodiment of the AI ​​Agent model training method based on AI, RPA, and LLM involves: acquiring the text format of the original sample document; segmenting the original sample document based on the text format to obtain at least one document content block; and detecting any document content block to identify at least one first document fragment of at least one fragment type and its corresponding compression attribute. Thus, by first identifying the text format of the original sample document and then segmenting the document content block based on format features, a hierarchical processing method is achieved, enabling accurate deconstruction of the document structure. This avoids the misjudgment problems across formats and content blocks that occur when directly identifying fragments of the entire document, and helps improve the accuracy and efficiency of identifying the first document fragment and its corresponding compression attribute.

[0203] Corresponding to the above model training method, this application also proposes a model application method.

[0204] Figure 6 This is a flowchart of a document compression method for document images provided in one embodiment of this application.

[0205] like Figure 6 As shown, the document compression method for document images may include the following steps S601 to S602:

[0206] Step S601: Obtain the target document image.

[0207] The target document image can display corresponding document content, which may include structured content such as tables and lists, or unstructured content such as text paragraphs, headings, footnotes, or endnotes. It should be noted that this application does not limit the number of target document images; it may be, but is not limited to, one.

[0208] Step S602: Using the VLM with the learned compression representation syntax, compress the document content displayed by the target document image to obtain the target compressed text.

[0209] The VLM can be trained using the model training method based on AI, RPA, and LLM to implement the AI ​​Agent in any of the above embodiments of this application.

[0210] In the embodiments of this application, the target document image can be input into a VLM that has learned the compression representation syntax, and the target compressed text can be obtained in response to the output of the VLM that has learned the compression representation syntax.

[0211] The document compression method for document images in this application involves acquiring a target document image and then using a Virtual Model (VLM) with a learned compression representation syntax to compress the document content displayed in the target document image to obtain target compressed text. The VLM is trained using a model training method based on AI, RPA, and LLM to implement an AI Agent. Therefore, by leveraging the VLM with its mastered compression representation syntax, end-to-end compression processing of the target document image is performed directly, eliminating the need for additional intermediate modules such as text parsing and format conversion. This effectively simplifies the document image compression processing chain and reduces operational complexity. Simultaneously, the trained VLM can accurately identify various redundant segments in the target document image and execute standardized compression logic. The generated target compressed text not only removes meaningless information such as blank lines and duplicate content but also fully preserves the core semantics and business logic of the document. This effectively solves the pain point of excessive redundant information in traditional document image conversion schemes and reduces the costs of subsequent data transmission, storage, and secondary editing.

[0212] This application also proposes a method for decompressing documents. Figure 7 This is a flowchart of a document decompression method provided in one embodiment of this application.

[0213] like Figure 7 As shown, the document decompression method may include the following steps S701 to S704:

[0214] Step S701: Obtain the target compressed text.

[0215] The target compressed text can be obtained by compressing the document content displayed by the target document image using a VLM with a learned compression representation syntax.

[0216] The VLM can be trained using the model training method based on AI, RPA, and LLM to implement the AI ​​Agent in any of the above embodiments of this application.

[0217] Step S702: Identify at least one compression representation in the target compressed text.

[0218] The compression representation may include compression attributes. It should be noted that the explanation of compression attributes in any of the above embodiments also applies to this embodiment, and will not be repeated here.

[0219] As one possible implementation, the target compressed text can be traversed to determine whether the target compressed text includes a set first target field; if it is determined that the target compressed text includes the target field, the tag with the target field is determined as the compressed representation of the target compressed text.

[0220] The first target field can be pre-defined, or it can be a field used to indicate whether compression is required, such as "compressed", "COMPRESSED", etc.

[0221] For example, if the target compressed text includes the first target field "compressed", the AI ​​Agent will determine the tag having the first target field as a compressed representation if the first target field is true. For instance, suppose the target compressed text includes a tag where the tag includes the first target field "compressed" and its value is "true", then the tag can be determined as a compressed representation.

[0222] As another possible implementation, if the target compressed text is determined to include a tag with a set second target field, the tag with the set second target field is determined as the compressed representation.

[0223] The first target field can be pre-defined and can be a field used to indicate whether compression is required, such as "compressed" or "COMPRESSED". It should be noted that the first target field can be the same as or different from the second target field; this application does not impose any restrictions on this.

[0224] For example, suppose the target compressed text includes the tag [ COMPRESSED, type=repeated_linecount=3 template="Clause 1: This is the content of the first clause."], where the tag includes the second target field "COMPRESSED", then the tag with the set second target field can be identified as a compressed representation.

[0225] Step S703: Decompress the compressed representation according to the compression attributes in any compressed representation to obtain the corresponding target document fragment.

[0226] As an example, for any compressed representation, in response to the compression attributes in the compressed representation including fragment type, number of rows occupied, and number of columns involved, the AI ​​Agent can generate the corresponding target document fragment based on the fragment type, number of rows occupied, and number of columns involved. For example, assuming the compressed representation is as follows, the AI ​​Agent can generate the corresponding target document fragment based on the fragment type value of "empty_row" in the compressed representation, which indicates empty rows in the table, and then based on the number of rows occupied and the number of columns involved, for example:

[0227]

[0228] It should be noted that the above-described target document fragments are merely illustrative. In practical applications, the target document fragments can also be in other formats, and this application does not impose any restrictions on this.

[0229] Step S704: Generate the target compressed text's target restored text based on the target document fragment.

[0230] As an example, when the executing entity of this application is a digital employee platform, the digital employee platform AIAgent can merge the target document fragment and the uncompressed text fragments in the target compressed text, excluding the compressed representation, according to the order in which the corresponding compressed representation and uncompressed text fragments appear in the target compressed text, thereby obtaining the target restored text of the target compressed text.

[0231] The document decompression method of this application embodiment involves: obtaining target compressed text; wherein the target compressed text is obtained by compressing the document content displayed by the target document image using a VLM with a learned compression representation syntax, and the VLM is trained using a model training method based on AI, RPA, and LLM to implement an AI Agent; identifying at least one compression representation in the target compressed text; wherein the compression representation includes compression attributes; decompressing the compression representation according to the compression attributes in any compression representation to obtain the corresponding target document fragment; and generating the target restored text of the target compressed text based on the target document fragment. Therefore, by performing targeted decompression operations based on the compression attributes carried in the target compressed text, a complete closed-loop processing link of "compression-decompression" is constructed. This not only achieves lightweight processing of document image data through the compression stage, but also ensures lossless restoration of core document content through the decompression stage. This effectively solves the pain point of "difficulty in accurate restoration after compression" in traditional document compression solutions, balancing the efficiency of data processing and the integrity of content. In addition, the compression attributes embedded in the compression representation can provide clear restoration basis for the decompression process, allowing the system to accurately locate the original features of various compressed segments and execute differentiated decompression logic for different types of compressed content such as blank rows in tables and repeated cells, ensuring that the generated target restored text is highly consistent with the original document content, thus improving the accuracy and reliability of document content restoration. Furthermore, the lightweight compressed text is easy to store and transmit, while the on-demand decompression mechanism can meet the needs of document viewing and secondary editing in different scenarios. At the same time, combined with training models empowered by AI, RPA, LLM, and AI Agent, the intelligence and automation level of the entire process is further improved.

[0232] To clearly illustrate the method of this application, a detailed explanation is provided below with examples.

[0233] As an example, Figure 8 This is a schematic diagram illustrating the implementation principle of the model training method for an AI Agent based on AI, RPA, and LLM as described in this application. Figure 8 As shown in the diagram, the implementation of the schematic may include the following steps:

[0234] Step S810: Data Processing and Training Sample Generation

[0235] Step S811: Sample Data Collection

[0236] A large-scale, diverse dataset of sample table images (referred to as sample document images in this application) is constructed, along with standard, uncompressed sample table documents (referred to as sample original documents in this application). It should be noted that this data may come from the application dataset or business data.

[0237] The text format of the sample table document can be HTML, Markdown, plain text, etc.

[0238] It should also be noted that the sample table document may include both tabular and non-tabular text.

[0239] Step S812: Text Compression

[0240] Step S8121: Input Analysis and Format Recognition

[0241] Identify the text format of the sample table document; based on the text format, segment the sample table document to obtain table content blocks and non-table content blocks.

[0242] For example, when the text format of the sample table document is HTML, regular expressions can be used to extract table content blocks from the sample table document; and non-table content blocks can be determined based on document fragments other than table content blocks in the sample table document.

[0243] Step S8122: Content parsing and structuring

[0244] For table content blocks, the table content blocks are parsed into a unified data structure. For example, the table content blocks are parsed into target table information of a standardized table data structure. For instance, assuming the original sample document's text format is HTML, the BeautifulSoup library is used to parse the table content blocks and organize them into a regular standardized table data structure along the "row-column" dimension. Assuming the original sample document's text format is HTML, the header row, separator row, and content row of the table content block are first located. Each line of text content is split using pipe commands, and formatting characters such as hyphens and colons in the separator row are removed. After cleaning, the cell content values ​​of each column are extracted. At the same time, the position attributes of the cells are recorded, such as row number, column number, and alignment, such as at least one of the left, right, or center alignment corresponding to the colon position in the separator row. The data is then organized into a regular standardized table data structure along the "row-column" dimension.

[0245] Step S8123: Compression Mode Recognition

[0246] For any document content block, a target compression mode corresponding to the document content block is determined from multiple compression modes; a detection strategy corresponding to the target compression mode (referred to as the duplicate content identification rule in this application) is used to detect the document content block to determine whether there is first information in the document content block that matches the target compression mode and has duplicate content.

[0247] The document content block can include table content blocks and non-table content blocks.

[0248] Compression modes may include, for example, table blank row compression mode, repeated table content row compression mode, repeated cell compression mode, consecutive empty cell compression mode, and repeated text line compression mode.

[0249] It should be noted that each compression mode has a corresponding fragment type. For example, the fragment type corresponding to the table empty row compression mode is table empty row, and the fragment type corresponding to the repeated table content row compression mode is repeated table content row, and so on.

[0250] As an example, for any given table content block, the target compression mode corresponding to the table content block is determined from multiple compression modes, including table blank row compression mode, duplicate table content row compression mode, duplicate cell compression mode, and consecutive empty cell compression mode. The corresponding detection strategies are as follows:

[0251] A. Table blank line compression mode

[0252] The system detects consecutive rows of empty data in the table content block. When the number of rows occupied is not less than n1 (referred to as the reference threshold in this application, such as 1), it determines that there is first information in the table content block that matches the table empty row compression mode and has repeated content, and the consecutive rows of empty data can be identified as the first information.

[0253] B. Duplicate table content row compression mode

[0254] The system detects consecutive data rows with the same row structure in the table content block, extracts the text template, and identifies the variable part. When the number of repetitions is not less than n2 (referred to as the reference threshold in this application, such as 2), it determines that there is first information in the table content block that matches the compression pattern of the repeated table content rows and has repeated content, and consecutive data rows with the same row structure can be identified as the first information.

[0255] C. Repeating cell compression mode

[0256] The system detects consecutive cells with the same content value within a row of a table content block. When the number of consecutive cells with the same content value is not less than n3 (referred to as the reference threshold in this application, such as 3), it determines that there is first information in the table content block that matches the compression pattern of repeated cells and has repeated content. The system can then identify each consecutive cell with the same content value as the first information.

[0257] D. Consecutive empty cell compression mode

[0258] Detect consecutive empty cells within rows of a table content block; when the number of consecutive empty cells is not less than n4 (referred to as the reference threshold in this application, such as 3, etc.), determine that there is first information in the table content block that matches the compression pattern of consecutive empty cells and has duplicate content, and the consecutive empty cells can be identified as the first information.

[0259] For any non-table content block, the target compression mode corresponding to the non-table content block is determined from multiple compression modes, including the repeated text line compression mode. The corresponding detection strategy is as follows:

[0260] Repeated text line compression mode: For example, N-gram syntax can be used to detect consecutively repeated identical text; when the number of characters in the repeated text is not less than a set number and the number of times the text is repeated is not less than n5 (referred to as the reference threshold in this application, such as 3, etc.), it is determined that there is first information in the non-table content block that matches the repeated text line compression mode and has repeated content, and consecutively repeated identical text can be identified as the first information; where repeated text refers to the part of the text content that appears consecutively and identically in the non-table content block, and the set number is, for example, 3, etc.

[0261] Step S8124: Application of compression rules (referred to as compression representation syntax in this application)

[0262] If it is determined that there is first information in the document content block that matches the target compression mode and has duplicate content, the corresponding first document fragment is determined based on the first information, and the first document fragment can be compressed using the target compression rule corresponding to the target compression mode (referred to as the target compression representation syntax in this application) to obtain the corresponding compressed representation.

[0263] Each target compression mode can have a corresponding target compression rule. For example, the target compression rule corresponding to the table empty row compression mode is the table empty row compression rule, which is specifically as follows:

[0264] ;

[0265] Here, `compressed` indicates whether compression is enabled; `true` indicates compression. `type` indicates the compression type, with a value of `empty_row`, indicating that the compression type is empty rows. `cols` indicates the number of rows occupied, and `rows` indicates the number of columns occupied.

[0266] The target compression rule corresponding to the duplicate table content row compression mode is the duplicate table content row compression rule, which is specifically as follows:

[0267] <data>……< / data> ;

[0268] Among them, compressed indicates whether compression is used; when its value is true, it means compression is used. type indicates the compression type, and its value is repeated_text_line, which indicates that the compression type is repeated table content rows. count indicates the number of repetitions. template indicates the template format of repeated table content rows. <data>……< / data> Used to indicate the values ​​of each field in a row of duplicate table content;

[0269] The target compression rule for the repeated cell compression mode is the repeated cell compression rule, which is as follows:

[0270] ;

[0271] Here, compressed indicates whether compression is enabled; when it is true, it means compression is enabled; type indicates the compression type, and its value is repeated_cell, which indicates that the compression type is repeated cells; count indicates the number of repetitions; and value indicates the cell content value.

[0272] The target compression rule for the consecutive empty cell compression mode is the consecutive empty cell compression rule, which is specifically as follows:

[0273] ;

[0274] Here, compressed indicates whether compression is enabled; when it is true, it means compression is enabled; type indicates the compression type, and its value is empty_cell, which indicates that the compression type is consecutive empty cells; count indicates the number of repetitions.

[0275] The target compression rule corresponding to the repeated text line compression mode is the repeated text line compression rule, which is specifically as follows:

[0276] [COMPRESSED type=repeated_line count= template="..."];

[0277] Among them, COMPRESSED indicates that it is a compressed format; type indicates the compression type, which takes the value repeated_line, indicating that the compression type is repeated text lines; count indicates the number of repetitions; and template indicates the text content of the repeated text lines.

[0278] Step S8125: Compressed output generation

[0279] The compressed representation of the first document fragment and the second document fragment (whose content is not repeated in the original sample document) are recombined while maintaining the original arrangement order, thereby generating a standard sample compressed text with compression attributes.

[0280] Step S8126: Verification and Optimization

[0281] After obtaining the sample compressed text, the document content of the compressed sample text can be checked to detect the integrity, accuracy and reversible compressibility of the document content.

[0282] Optionally, the compression rate metric corresponding to the sample table document can also be calculated.

[0283] The compression ratio metric can be used to measure the effectiveness of document compression, indicating the degree to which the data volume is reduced after compression.

[0284] Step S813: Training sample pairing

[0285] For each sample table image, generate a pair of labeled data: (sample table image, sample table document) and (sample table image, sample compressed text). During model training, the (sample table image, sample compressed text) pair is used to fine-tune the model.

[0286] Step S820: Model Training

[0287] 1. Model Selection

[0288] For example, you can choose a powerful multimodal VLM as the base model, such as the LLaVA model or the Qwen-VL model.

[0289] 2. Fine-tuning method

[0290] The sample table image is used as input to the VLM and the predicted compressed text is obtained in response to the output of the VLM. The negative log-likelihood loss function is used to determine the negative log-likelihood loss value based on the difference between the predicted compressed text and the sample compressed text. Based on the negative log-likelihood loss value, the model parameters of the VLM are fine-tuned using full parameter fine-tuning or parameter-efficient fine-tuning methods to minimize the negative log-likelihood loss value.

[0291] Optionally, when fine-tuning, use clear system prompts such as: "You are a table analysis expert. Please convert the given table image into compact HTML code. For consecutive blank rows or merged cells, please use special compression tags to save space."

[0292] 3. Training Objectives: Through training, the model will learn two things: accurately understand the visual structure and semantic content of tables, and actively use predefined compressed representation syntax to efficiently express the identified structure.

[0293] Furthermore, the trained model is applied to the table image to be processed. Specifically, during the inference phase, the table image to be processed is input into the trained VLM, and the model directly outputs the target compressed text. Thus, because the VLM can learn to represent redundant or repetitive table and text data in a more concise form, it efficiently compresses repetitive structures in documents, such as consecutive blank lines and repeated cells, thereby reducing computational resource consumption and improving processing efficiency and economy. Consequently, in the subsequent inference phase, it can improve inference efficiency and success rate.

[0294] In the post-processing stage, a parser can be used to decompress the target compressed text. For example, the parser identifies at least one compression representation in the target compressed text, wherein the compression representation includes compression attributes. The parser decompresses the compression representation according to the compression attributes in the compression representation to obtain the corresponding target document fragment. The parser generates the target restored text of the target compressed text according to the target document fragment.

[0295] The method described above in this application,

[0296] 1. Efficiency optimization: For example, when processing sparse table data, the consumption of tokens is reduced from O(n) to O(1), thereby improving processing efficiency.

[0297] 2. End-to-end solutions: Providing practical technical solutions covering the entire process from data preparation, model training, to inference post-processing.

[0298] 3. High versatility and excellent scalability: Its core compression concept can be widely applied to various repetitive structures inside and outside the table, and its application protection scope is very wide.

[0299] 4. Perfect backward compatibility: With a lightweight post-processor parser, it can ensure seamless integration with the existing ecosystem, effectively reducing the difficulty and threshold of deployment.

[0300] 5. Reduced resource consumption: It can significantly reduce API call costs and internal computing resource consumption.

[0301] To implement the above embodiments, this application also provides a model training device for implementing an AI Agent based on AI, RPA, and LLM.

[0302] Figure 9This is a structural diagram of a model training device for implementing an AI Agent based on AI, RPA, and LLM, provided in one embodiment of this application.

[0303] like Figure 9 As shown, the model training device 900 for implementing an AI Agent based on AI, RPA, and LLM includes: an acquisition module 910, a compression module 920, and a training module 930.

[0304] The acquisition module 910 is used to acquire the sample document image and the original sample document corresponding to the document content displayed by the sample document image.

[0305] Compression module 920 is used to compress the original sample document using compression representation syntax to obtain compressed sample text.

[0306] Training module 930 is used to train the Visual Language Model (VLM) based on sample document images and sample compressed text, so that the trained VLM learns the compressed representation syntax.

[0307] In any embodiment of this application, the compression module 920 is configured to: parse the original sample document to identify at least one type of first document fragment and its corresponding compression attribute in the original sample document; wherein the first document fragment contains document content with repeated content; compress the at least one first document fragment based on the compression representation syntax and the compression attribute of the at least one first document fragment to obtain a compressed representation of the at least one first document fragment; and generate sample compressed text based on the compressed representation of the first document fragment and a second document fragment in the original sample document with non-repeating content.

[0308] In any embodiment of this application, in response to the fragment type to which the first document fragment belongs including empty table rows, the compression attributes include: whether to compress, fragment type, number of rows occupied, and number of columns involved; in response to the fragment type to which the first document fragment belongs including repeated table content rows, the compression attributes include: whether to compress, fragment type, number of repetitions, first content template, and cell content value; the first content template is used to indicate the template format of the repeated table content rows; in response to the fragment type to which the first document fragment belongs including repeated cells, the compression attributes include: whether to compress, fragment type, cell content value, and number of repetitions; in response to the fragment type to which the first document fragment belongs including consecutive empty cells, the compression attributes include: whether to compress, fragment type, and number of consecutive empty cells; in response to the fragment type to which the first document fragment belongs including repeated text lines, the compression attributes include: fragment type, number of repetitions, and second content template; the second content template is used to indicate the text content of the repeated text lines.

[0309] In any embodiment of this application, the compression module 920 is configured to: obtain the text format of the original sample document; segment the original sample document based on the text format to obtain at least one document content block; and detect any document content block to identify a first document fragment of at least one fragment type corresponding to the document content block and the corresponding compression attribute.

[0310] In any embodiment of this application, the compression module 920 is configured to: employ a table detection strategy corresponding to the text format to detect the original sample document to determine whether the original sample document includes table content; if the original sample document is determined to include table content, extract table content blocks from the original sample document and obtain the position information of the table content blocks; analyze the original sample document based on the position information of the table content blocks to determine whether the original sample document includes non-table content; if the original sample document includes non-table content, extract non-table content blocks from the original sample document; and treat both table content blocks and non-table content blocks as document content blocks.

[0311] In any embodiment of this application, the compression module 920 is configured to: for any document content block, determine a target type corresponding to the document content block from multiple fragment types; use a duplicate content recognition rule corresponding to the target type to identify information in the document content block to determine whether there is first information in the document content block that matches the target type and has duplicate content; in response to the existence of first information in the document content block that matches the target type and has duplicate content, determine a first document fragment and its corresponding compression attribute based on the first information and the target type.

[0312] In any embodiment of this application, the compression module 920 is configured to: for any first document fragment, determine a target compression representation syntax corresponding to the fragment type to which the first document fragment belongs from a plurality of compression representation syntaxes; and compress the first document fragment using the target compression representation syntax based on the compression attributes corresponding to the first document fragment to obtain a corresponding compressed representation.

[0313] In any embodiment of this application, the compression module 920 is configured to: for any first document fragment, determine a corresponding reference threshold based on the fragment type to which the first document fragment belongs; extract key attributes from the compression attributes corresponding to the first document fragment; determine whether to compress the first document fragment based on the comparison result between the reference threshold and the target value of the key attribute; and if it is determined that the first document fragment should be compressed, determine a target compression representation syntax corresponding to the fragment type to which the first document fragment belongs from multiple compression representation syntaxes.

[0314] In any embodiment of this application, the training module 930 is used to: compress the document content displayed by the sample document image using VLM to obtain predicted compressed text; and train the VLM based on the difference between the predicted compressed text and the sample compressed text so that the trained VLM learns the compressed representation syntax.

[0315] It should be noted that the model training apparatus for implementing AI Agent based on AI, RPA and LLM provided in this application embodiment can implement all the method steps implemented in any of the above-mentioned model training method embodiments for implementing AI Agent based on AI, RPA and LLM, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0316] This application also provides a document compression device for document images.

[0317] Figure 10 This is a structural diagram of a document compression apparatus for document images provided in one embodiment of this application.

[0318] like Figure 10 As shown, the document compression device 1000 for document images includes: an acquisition module 1010 and a compression module 1020.

[0319] The acquisition module 1010 is used to acquire the target document image.

[0320] Compression module 1020 is used to compress the document content displayed by the target document image using a visual language model (VLM) that has learned a compression representation syntax, to obtain the target compressed text; wherein, the VLM is trained using any possible implementation of the model training method based on AI, RPA and LLM to implement the AI ​​Agent.

[0321] It should be noted that the document compression device for document images provided in this application embodiment can implement all the method steps implemented in any of the above-described document compression method embodiments for document images, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0322] This application also provides a document decompression device.

[0323] Figure 11 This is a structural diagram of a document decompression device provided in one embodiment of this application.

[0324] like Figure 11As shown, the document decompression device 1100 includes: an acquisition module 1110, an identification module 1120, a decompression module 1130, and a generation module 1140.

[0325] The acquisition module 1110 is used to acquire the target compressed text. The target compressed text is obtained by compressing the document content displayed in the target document image using a visual language model (VLM) that has learned a compressed representation syntax. The VLM is trained using any possible implementation of the model training method based on AI, RPA, and LLM to implement the AI ​​Agent.

[0326] The recognition module 1120 is used to recognize at least one compression representation in the target compressed text; wherein the compression representation includes compression attributes.

[0327] The decompression module 1130 is used to decompress the compressed representation according to the compression attributes in any compressed representation to obtain the corresponding target document fragment.

[0328] The generation module 1140 is used to generate the target compressed text of the target document fragment.

[0329] It should be noted that the document decompression device provided in this application embodiment can implement all the method steps implemented in any of the above document decompression method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0330] Figure 12 A structural block diagram of an electronic device according to an embodiment of this application is shown. Figure 12 As shown, the electronic device includes a memory 1210 and a processor 1220. The memory 1210 stores a computer program that can run on the processor 1220. When the processor 1220 executes the computer program, it implements the model training method for AI Agent based on AI, RPA, and LLM as described in the above embodiments, or a document compression or decompression method for document images. The number of memories 1210 and processors 1220 can be one or more.

[0331] The electronic device also includes:

[0332] The communication interface 1230 is used to communicate with external devices and exchange and transmit data.

[0333] If the memory 1210, processor 1220, and communication interface 1230 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0334] Optionally, in a specific implementation, if the memory 1210, processor 1220 and communication interface 1230 are integrated on a single chip, the memory 1210, processor 1220 and communication interface 1230 can communicate with each other through an internal interface.

[0335] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a model training method for AI Agent based on AI, RPA, and LLM provided in any embodiment of this application, or a document compression or decompression method for document images.

[0336] This application also provides a chip, which includes a processor for calling and executing instructions stored in the memory, causing a communication device equipped with the chip to execute a model training method for AI Agent based on AI, RPA and LLM provided in any embodiment of this application, or a document compression or document decompression method for document images.

[0337] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the model training method for implementing AI Agent based on AI, RPA, and LLM provided in any embodiment of the application, or the document compression or decompression method for document images.

[0338] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Computing (RISC) machines (ARM) architecture.

[0339] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0340] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0341] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0342] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0343] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0344] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0345] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0346] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0347] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method for an AI agent based on artificial intelligence (AI), robotic process automation (RPA), and large language modeling (LLM), characterized in that, include: Obtain the sample document image and the original sample document corresponding to the document content displayed in the sample document image; The original sample document is compressed using a compression representation syntax to obtain compressed sample text. Based on the sample document images and the sample compressed text, the Visual Language Model (VLM) is trained so that the trained VLM learns the compressed representation syntax. The original sample document is compressed to obtain compressed sample text, including: The original sample document is parsed to identify at least one type of first document fragment and its corresponding compression attribute in the original sample document; wherein the first document fragment contains document content with repeated content; Based on the compression representation syntax and at least one compression attribute of the first document fragment, the at least one first document fragment is compressed to obtain a compressed representation of the at least one first document fragment; The sample compressed text is generated based on the compressed representation of the first document fragment and the second document fragment in the original sample document whose content is not repeated. In response to the fact that the fragment type to which the first document fragment belongs includes a table empty row, the compression attribute includes at least one of the following: whether to compress, fragment type, number of rows occupied, and number of columns involved; In response to the fact that the fragment type to which the first document fragment belongs includes repeating table content rows, the compression attribute includes at least one of the following: whether to compress, fragment type, number of repetitions, first content template, and cell content value; the first content template is used to indicate the template format of the repeating table content rows; In response to the fact that the fragment type to which the first document fragment belongs includes repeating cells, the compression attribute includes at least one of the following: whether to compress, fragment type, cell content value, and number of repetitions; In response to the fact that the fragment type to which the first document fragment belongs includes consecutive empty cells, the compression attribute includes at least one of the following: whether to compress, fragment type, and number of consecutive empty cells; The compression attribute is provided in response to the fact that the fragment type to which the first document fragment belongs includes repeating text lines.

2. The method according to claim 1, characterized in that, The step of parsing the original sample document to identify at least one first document fragment of at least one fragment type and its corresponding compression attribute includes: Obtain the text format of the original document of the sample; Based on the text format, the original sample document is segmented to obtain at least one document content block; Detect any of the document content blocks to identify the first document fragment of at least one of the fragment types corresponding to the document content block and the corresponding compression attributes.

3. The method according to claim 2, characterized in that, The step of segmenting the original sample document based on the text format to obtain at least one document content block includes: A table detection strategy corresponding to the text format is used to detect the original sample document to determine whether the original sample document contains table content. If it is determined that the original sample document includes table content, extract the table content block from the original sample document and obtain the location information of the table content block; Based on the location information of the table content blocks, the original sample document is analyzed to determine whether the original sample document includes non-table content; If the original sample document contains non-table content, extract the non-table content block from the original sample document; Both the table content block and the non-table content block are considered as the document content block.

4. The method according to claim 2, characterized in that, The step of detecting any of the document content blocks to identify at least one of the fragment types of the first document fragment and its corresponding compression attribute in the document content block includes: For any of the document content blocks, determine the target type corresponding to the document content block from among the various fragment types; Using a duplicate content identification rule corresponding to the target type, information identification is performed on the document content block to determine whether there is first information in the document content block that matches the target type and has duplicate content; In response to the presence of first information in the document content block that matches the target type and has duplicate content, the first document fragment and its corresponding compression attributes are determined based on the first information and the target type.

5. The method according to claim 1, characterized in that, The step of compressing the at least one first document fragment based on the compression representation syntax and at least one compression attribute of the first document fragment to obtain a compressed representation of the at least one first document fragment includes: For any of the first document fragments, a target compression representation syntax corresponding to the fragment type to which the first document fragment belongs is determined from a plurality of compression representation syntaxes; Using the target compression representation syntax, the first document fragment is compressed based on the compression attributes corresponding to the first document fragment to obtain the corresponding compressed representation.

6. The method according to claim 5, characterized in that, For any given first document fragment, determining the target compression representation syntax corresponding to the fragment type of the first document fragment from a plurality of compression representation syntaxes includes: For any of the first document fragments, a corresponding reference threshold is determined based on the fragment type to which the first document fragment belongs; Extract key attributes from the compression attributes corresponding to the first document fragment; Based on the comparison between the reference threshold and the target value of the key attribute, determine whether to compress the first document fragment; If it is determined that the first document fragment should be compressed, a target compression representation syntax corresponding to the fragment type to which the first document fragment belongs is determined from a plurality of compression representation syntaxes.

7. The method according to any one of claims 1-6, characterized in that, The step of training a Visual Language Model (VLM) based on the sample document images and the sample compressed text, so that the trained VLM learns the compressed representation syntax, includes: The VLM is used to compress the document content displayed in the sample document image to obtain predicted compressed text; The VLM is trained based on the difference between the predicted compressed text and the sample compressed text, so that the trained VLM learns the compressed representation syntax.

8. A document compression method for document images, characterized in that, The method includes: Obtain the target document image; A visual language model (VLM) with a learned compressed representation grammar is used to compress the document content displayed in the target document image to obtain the target compressed text; wherein the VLM is trained using any of the methods described in claims 1-7.

9. A document decompression method, characterized in that, The method includes: Obtain target compressed text; wherein the target compressed text is obtained by compressing the document content displayed in the target document image using a visual language model (VLM) that has learned a compressed representation syntax, and the VLM is trained using any of the methods described in claims 1-7; Identify at least one compression representation in the target compressed text; wherein the compression representation includes compression attributes; Based on the compression attributes in any of the compression representations, the compression representation is decompressed to obtain the corresponding target document fragment; Based on the target document fragment, generate the target restored text of the target compressed text.

10. A model training device for implementing an AI Agent based on AI, RPA, and LLM, characterized in that, include: The acquisition module is used to acquire a sample document image and the original sample document corresponding to the document content displayed by the sample document image; The compression module is used to compress the original sample document using a compression representation syntax to obtain compressed sample text; The training module is used to train the Visual Language Model (VLM) based on the sample document images and the sample compressed text, so that the trained VLM learns the compressed representation syntax. The compression module is used for: The original sample document is parsed to identify at least one type of first document fragment and its corresponding compression attribute in the original sample document; wherein the first document fragment contains document content with repeated content; Based on the compression representation syntax and at least one compression attribute of the first document fragment, the at least one first document fragment is compressed to obtain a compressed representation of the at least one first document fragment; The sample compressed text is generated based on the compressed representation of the first document fragment and the second document fragment in the original sample document whose content is not repeated. In response to the fact that the fragment type to which the first document fragment belongs includes empty table rows, the compression attributes include: whether to compress, fragment type, number of rows occupied, and number of columns involved; In response to the fact that the fragment type to which the first document fragment belongs includes repeating table content rows, the compression attributes include: whether to compress, fragment type, number of repetitions, first content template, and cell content value; the first content template is used to indicate the template format of the repeating table content rows; In response to the fact that the fragment type to which the first document fragment belongs includes repeating cells, the compression attributes include: whether to compress, fragment type, cell content value, and number of repetitions; In response to the fact that the fragment type to which the first document fragment belongs includes consecutive empty cells, the compression attributes include: whether to compress, fragment type, and number of consecutive empty cells; In response to the fact that the fragment type to which the first document fragment belongs includes repeated text lines, the compression attributes include: fragment type, number of repetitions, and a second content template; the second content template is used to indicate the text content of the repeated text lines.

11. The apparatus according to claim 10, characterized in that, The training module is used for: The VLM is used to compress the document content displayed in the sample document image to obtain predicted compressed text; The VLM is trained based on the difference between the predicted compressed text and the sample compressed text, so that the trained VLM learns the compressed representation syntax.

12. A document compression device for document images, characterized in that, The device includes: The acquisition module is used to acquire the target document image; A compression module is used to compress the document content displayed by the target document image using a visual language model (VLM) that has learned a compressed representation syntax, to obtain target compressed text; wherein the VLM is trained using any of the methods described in claims 1-7.

13. A document decompression device, characterized in that, The device includes: An acquisition module is used to acquire target compressed text; wherein the target compressed text is obtained by compressing the document content displayed by the target document image using a visual language model (VLM) with a learned compressed representation syntax, and the VLM is trained using any of the methods described in claims 1-7; A recognition module is configured to recognize at least one compression representation in the target compressed text; wherein the compression representation includes compression attributes; The decompression module is used to decompress the compressed representation according to the compression attributes in any of the compressed representations to obtain the corresponding target document fragments; The generation module is used to generate the target restored text of the target compressed text based on the target document fragment.

14. An electronic device, characterized in that, include: A processor and a memory, wherein instructions are stored in the memory and are loaded and executed by the processor to implement the method of any one of claims 1 to 7, or the method of claim 8, or the method of claim 9.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7, or the method as described in claim 8, or the method as described in claim 9.

Images