Document processing method and apparatus, and electronic device

Abstract

The application discloses a document processing method and device and electronic equipment. Relate to the field of artificial intelligence and document processing, the method comprises: carrying out content identification on a target document, obtaining document content, and splitting the document content according to element types to obtain M document elements; obtaining the attribute information and text content of each document element, and determining the association relationship between the M document elements; generating initial graph structure data of the target document according to the association relationship; determining the feature information of each document element according to the attribute information and text content of each document element, and updating the edges in the initial graph structure data according to the feature information to obtain target graph structure data of the target document. Through the application, the problem that the existing document recognition technology in the related art only outputs linear text, resulting in low content acquisition efficiency and low document use efficiency of the document, is solved.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and document processing, and more specifically, to a document processing method, apparatus, and electronic device. Background Technology

[0002] Currently, content recognition for electronic documents whose content cannot be directly copied (such as scanned image documents) mainly relies on optical character recognition technology or low-level parsing engines. The output is usually just a linear text sequence, lacking the ability to analyze the internal structure of the document. This results in the extracted information being presented as an unorganized stream of characters, which is difficult to support efficient retrieval, semantic reasoning and knowledge reuse, and seriously affects the user's efficiency in understanding complex documents and the user experience.

[0003] There is currently no effective solution to the problem that existing document recognition technologies only output linear text, resulting in low efficiency in document content acquisition and document usage. Summary of the Invention

[0004] The main objective of this application is to provide a document processing method, apparatus, and electronic device to solve the problem that existing document recognition technologies in the related art only output linear text, resulting in low efficiency in document content acquisition and document usage.

[0005] To achieve the above objectives, according to one aspect of this application, a document processing method is provided. The method includes: performing content recognition on a target document to obtain document content, and splitting the document content according to element type to obtain M document elements, where each document element belongs to an element type and M is a positive integer; obtaining attribute information and text content of each document element, and determining the association relationships between the M document elements based on the attribute information and text content; generating initial graph structure data of the target document based on the association relationships, wherein the initial graph structure data includes M nodes, each node corresponding to a document element, and the association relationships correspond to the edges between the nodes; determining feature information of each document element based on the attribute information and text content of each document element, and updating the edges in the initial graph structure data based on the feature information to obtain target graph structure data of the target document.

[0006] Optionally, the document content is split according to element type to obtain M document elements, including: splitting the document content to obtain multiple text blocks; inputting a preset set of element types and multiple text blocks into the target recognition model to obtain the element type of each text block, wherein the target recognition model is trained through a first training set, the first training set includes multiple first sample data, each first sample data includes a sample text block and the element type of the sample text block; and determining each element type and the corresponding text block as a document element to obtain M document elements.

[0007] Optionally, the attribute information includes the bounding box coordinates of the document elements in the target document. Determining the association between the M document elements based on the attribute information and text content includes: for any first document element, determining whether there is second text content in the text content of the M document elements, wherein the second text content contains the first text content of the first document element; if the second text content exists, determining that there is a reference relationship between the second document element to which the second text content belongs and the first document element, wherein the reference relationship is used to represent that the first document element references the second document element; and / or, for any third document element, determining whether there is a fourth document element containing the third document element based on the bounding box coordinates; if the fourth document element exists, determining that there is a hierarchical relationship between the third document element and the fourth document element, wherein the hierarchical relationship is used to represent that the third document element is contained in the fourth document element.

[0008] Optionally, the attribute information includes the bounding box coordinates of document elements in the target document. Determining the feature information of each document element based on its attribute information and text content includes: for any target document element, determining its font format feature vector based on the attribute information; inputting the bounding box coordinates and text content into the gaze probability recognition model to obtain the gaze probability value of the target document element, where the gaze probability value characterizes the probability that the content viewed by the user during their first gaze of the target document is the target document element. The gaze probability recognition model is trained using a second training set, which includes multiple second sample data sets, each containing a sample element and its gaze probability value; calculating the spatial adjacency feature vectors between the target document element and each of the M non-target document elements based on the bounding box coordinates; determining the text feature vector of the target document element based on the text content; and combining the font format feature vector, gaze probability value, spatial adjacency feature vector, and text feature vector to form the feature information of the target document element.

[0009] Optionally, updating the edges in the initial graph structure data based on feature information includes: obtaining the gaze probability value of each document element, sorting the gaze probability values ​​to obtain the target sequence; and adding temporal edges between the nodes of document elements to which adjacent gaze probability values ​​belong in the target sequence.

[0010] Optionally, updating the edges in the initial graph structure data based on feature information includes: for any first document element pair, calculating the similarity of the first document element pair based on feature information, and determining whether the similarity is greater than a similarity threshold; if the similarity is greater than the similarity threshold, determining whether there are edges between the nodes of the first document element pair in the initial graph structure data; if there are no edges between the nodes of the first document element pair in the initial graph structure data, adding semantic edges between the nodes of the first document element pair.

[0011] Optionally, after obtaining the target graph structure data of the target document, the method further includes: for any document element, obtaining a tag prompt, inputting the tag prompt and the document element into a tag recognition model to obtain the tag information of the document element, and adding the tag information to the target graph structure data; obtaining a fifth document element from M document elements whose tag information is the first information, and determining whether there is a sixth document element among the document elements with an edge to the fifth document element, wherein the tag information of the sixth document element is the second information; if there is a sixth document element, obtaining all node links containing the fifth and sixth document elements from the target graph structure data, and obtaining preset inference chain structure information; obtaining target node links that conform to the preset inference chain structure information from all node links, and determining the target node links as the target inference chain, wherein when it is detected that the user clicks on any node in the target inference chain in the target graph structure data, the target inference chain is displayed.

[0012] Optionally, after obtaining the target graph structure data of the target document, the method further includes: for any first document element pair with connecting edges, determining the semantic contradiction score of the first document element pair based on the feature information of the document elements; determining whether the semantic contradiction score is greater than a first threshold, and generating a prompt message if the semantic contradiction score is greater than the first threshold, wherein the prompt message is used to characterize that there is an error in the first document element pair; determining whether the semantic contradiction score is less than or equal to the first threshold if the semantic contradiction score is less than a second threshold, wherein the second threshold is less than the first threshold; and performing a fusion operation on the nodes of the first document element pair in the target graph structure data to obtain the updated target graph structure data if the semantic contradiction score is less than the second threshold.

[0013] Optionally, after obtaining the target graph structure data of the target document, the method further includes: determining the semantic contradiction score of each second document element pair with semantic edges, and determining the first parameter value of the target graph structure data based on the semantic contradiction score and edge weight of each second document element pair; determining the distribution entropy value of the tag information of each document element, obtaining M distribution entropy values, and determining the second parameter value of the target graph structure data based on the M distribution entropy values; determining the global energy value of the target graph structure data based on the first parameter value and the second parameter value, and determining whether the global energy value is greater than the energy threshold; if the global energy value is greater than the energy threshold, determining the maximum contributing document element pair based on the semantic contradiction score and edge weight of each second document element pair, and determining the maximum contributing document element pair as the document element pair to be processed; adjusting the edge weight of the maximum contributing document element pair, and re-executing the process of determining the global energy value of the target graph structure data using the adjusted edge weight, until the global energy value of the target graph structure data is less than or equal to the energy threshold.

[0014] To achieve the above objectives, according to another aspect of this application, a document processing apparatus is provided. The apparatus includes: a splitting unit, configured to perform content recognition on a target document to obtain document content, and split the document content according to element type to obtain M document elements, wherein each document element belongs to an element type, and M is a positive integer; a first determining unit, configured to acquire attribute information and text content of each document element, and determine the association relationship between the M document elements based on the attribute information and text content; a first generating unit, configured to generate initial graph structure data of the target document based on the association relationship, wherein the initial graph structure data includes M nodes, each node corresponding to a document element, and the association relationship corresponds to the edges between the nodes; and a second determining unit, configured to determine the feature information of each document element based on the attribute information and text content of each document element, and update the edges in the initial graph structure data based on the feature information to obtain target graph structure data of the target document.

[0015] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, the electronic device including a memory storing an executable program; and a processor for running the program, wherein the program executes the above-described document processing method during runtime.

[0016] To achieve the above objectives, according to another aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the document processing method described above.

[0017] In this embodiment, the target document is content identified to obtain document content, which is then split into M document elements according to element type. Each document element belongs to one element type, where M is a positive integer. The attribute information and text content of each document element are obtained, and the relationships between the M document elements are determined based on the attribute information and text content. An initial graph structure data of the target document is generated based on the relationships, where the initial graph structure data includes M nodes, each node corresponding to a document element, and the relationships correspond to the edges between the nodes. The attributes and text content of each document element are then used to determine the relationships between them. This method involves identifying feature information and updating the edges in the initial graph structure data based on that feature information to obtain the target graph structure data of the target document. By recognizing the content of the target document, determining the multimodal features of each element in the document, and generating graph structure data between elements based on the multimodal features and semantic associations, the method achieves the goal of improving the restoration of the document's inherent logical topology and semantic relationships through graph structure data. This achieves the technical effect of transforming unstructured documents into structured data containing document content and relationships between document elements, thereby solving the technical problem in related technologies where existing document recognition technologies only output linear text, resulting in low efficiency in document content acquisition and document usage. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 A hardware structure block diagram of a computer terminal for implementing a document processing method is shown.

[0020] Figure 2 This is a flowchart of a document processing method according to Embodiment 1 of this application;

[0021] Figure 3 This is a schematic diagram of an optional graph structure data provided according to Embodiment 1 of this application;

[0022] Figure 4 This is a schematic diagram of a document processing apparatus according to Embodiment 2 of this application;

[0023] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] It should be noted that the document processing methods, apparatuses, and electronic devices defined in this disclosure can be used in the fields of artificial intelligence and document processing, or in any field other than artificial intelligence and document processing. The application fields of the document processing methods, apparatuses, and electronic devices defined in this disclosure are not limited.

[0028] It should be noted that all information, user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) used in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant regulations and standards of the relevant regions, have taken necessary measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse use. If the user chooses to refuse, the process proceeds to the expert decision-making process. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface. After receiving consent from the aforementioned user or organization, the relevant information is obtained. Users can view the purpose of data use in real time through the authorization interface and have the right to withdraw authorization or delete data at any time. After authorization is withdrawn, the system will terminate the relevant data processing within 24 hours.

[0029] The embodiments or examples disclosed herein are not exhaustive, but merely illustrative of some embodiments or examples, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment or example can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment or example can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment or example can be arbitrarily interchanged. Furthermore, optional methods or examples in a particular embodiment or example can be arbitrarily combined; moreover, embodiments or examples can be arbitrarily combined. For example, some or all steps of different embodiments or examples can be arbitrarily combined, and a particular embodiment or example can be arbitrarily combined with optional methods or examples of other embodiments or examples.

[0030] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0031] RGB: Red Green Blue, a color model for red, green, and blue hues.

[0032] JSON: JavaScript Object Notation.

[0033] Example 1

[0034] According to an embodiment of this application, an embodiment of a document processing method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal for implementing a document processing method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, processing devices such as microprocessors or programmable logic devices), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface, a universal serial bus port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the document processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the document processing method described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] The display may be, for example, a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0040] Under the aforementioned operating environment, this application provides the following: Figure 2 The document processing method shown. Figure 2 This is a flowchart of a document processing method according to Embodiment 1 of this application, as shown below. Figure 2 As shown, the method includes:

[0041] Step S201: Perform content recognition on the target document to obtain the document content, and split the document content according to the element type to obtain M document elements, where each document element belongs to an element type and M is a positive integer.

[0042] It should be noted that the execution subject of this embodiment can be a document processing system. This system can perform content recognition and structuring processing on the received document, and construct a target graph structure that can accurately reflect the document's internal logical topology and semantic dependencies, thereby solving the problem that the existing technology cannot restore structured semantic relationships, resulting in low efficiency in content acquisition and use.

[0043] It should be noted that the target document can be an electronic document to be processed, and its format includes, but is not limited to, image format documents, scanned documents, etc. Document elements can be the most basic recognizable text elements in the document, which can include six categories: title content, paragraph content, table content, image content, formula content, and footnote content.

[0044] For example, when processing a document, multimodal visual parsing technology can first be used to semantically decode the original document to obtain the document content, that is, the text content of the document. Based on the text content, semantic segmentation of the document content is performed. Combined with a pre-trained layout analysis model, the bounding box coordinates of each text block in the document and its corresponding text content, image region or formula expression are identified. Thus, the document content is divided into text blocks, that is, document elements, which are the smallest processable units with clear semantic categories. The types of document elements can include headings, paragraphs, tables, pictures, formulas and footnotes.

[0045] For example, document element 1 can be "3.2.1 Sports Rehabilitation Topic", with the element type "Title"; document element 2 can be a table (the specific content of the table is not shown here), with the element type "Table".

[0046] Step S202: Obtain the attribute information and text content of each document element, and determine the association relationship between the M document elements based on the attribute information and text content.

[0047] It should be noted that attribute information may include, but is not limited to, layout features and visual morphological features, such as bounding box coordinates, font size, font weight, color, line spacing, page position, and relative coordinates. The relationships between document elements refer to the connections established between elements based on the spatial logic and semantic dependencies between attribute information and text content; these relationships can include hierarchy, references, etc.

[0048] For example, after obtaining each document element, the graph structure data of the target document can be generated based on the document elements. At this time, it is necessary to first determine the relationship between each document element, so as to determine which document elements in the graph structure data have edges.

[0049] For each document element, the following operations can be performed based on the document element's attribute information and text content to determine the relationships between document elements: extract its bounding box coordinates to calculate the relative horizontal distance, vertical distance, line angle, and bounding box overlap rate to form spatial adjacency features; combine font size and thickness to determine hierarchical intent and identify the inclusion relationship between headings and paragraphs; analyze whether the text content contains explicit guiding words such as "as shown in Table x" or "see paragraph x" to identify its reference relationship with image or formula elements.

[0050] All the relationships obtained from the above operations are quantitatively defined based on the attribute information and text content of document elements, forming unambiguous and computable relationships. This ensures that each relationship has a clear input basis, thereby automatically constructing a structured relationship network based on both layout and semantics. This provides a topological foundation that truly reflects the document logic for the generation of graph structure data, significantly improving the integrity and accuracy of structural relationships.

[0051] Step S203: Generate initial graph structure data of the target document based on the association relationship. The initial graph structure data includes M nodes, each node corresponds to a document element, and the association relationship corresponds to the edge between each node.

[0052] It should be noted that the initial graph structure data refers to the basic graph model composed of nodes and edges. A node refers to a unique node corresponding to each document element in the graph. The relationship between nodes refers to each relationship type being represented by a directed edge with a semantic label in the graph. The type of the edge is consistent with the relationship. Figure 3 This is a schematic diagram of an optional graph structure data provided according to Embodiment 1 of this application, such as... Figure 3 As shown, it can include 5 nodes and 4 edges, with each node corresponding to a document element.

[0053] For example, after determining the relationships, M document elements can be mapped to M nodes in a graph, with each node carrying its element type and original text content. For each pair of document elements with a relationship, a directed edge is added between the two corresponding nodes. The type of the edge corresponds to the type of relationship between the two nodes, such as hierarchy, reference, etc. The initial weight of the edge can be set according to preset rules, for example, the weight of a hierarchy edge is 1.0, and the weight of a reference edge is 1.0. Finally, an attribute graph structure containing M nodes and corresponding edges is formed, thereby establishing a formal transformation of the document from linear text to a non-linear knowledge network, realizing the graph representation of structural information, breaking through the limitations of traditional linear text extraction, and realizing the complete reconstruction of the document knowledge network.

[0054] Step S204: Determine the feature information of each document element based on the attribute information and text content of each document element, and update the edges in the initial graph structure data based on the feature information to obtain the target graph structure data of the target document.

[0055] It should be noted that the feature information can include integrated layout, visual and semantic features, generate a high-dimensional embedding vector for each node, and update the edges in the initial graph structure data by adding edges between nodes in the graph structure data based on the feature information of document elements.

[0056] For example, after obtaining the initial graph structure data, it is also necessary to determine the new associations between document elements based on the feature information of each document element, thereby adding edges between nodes in the initial graph structure. For example, the reading order is determined and a temporal relationship is established based on the gaze probability value vector. Preliminary semantic matching of the text content is performed to calculate the text semantic similarity between any two document elements and determine whether there is an implicit semantic association, thereby adding temporal edges and implicit semantic edges. The weight of the temporal edge is set according to the normalization of the reading order difference, and the weight of the implicit semantic edge is the similarity multiplied by a fixed coefficient.

[0057] After the initial graph structure data completes the operation of adding edges, which is also the update operation of the initial graph structure data, the target graph structure data of the target document is obtained, thus completing the construction process of the target graph structure data of the target document.

[0058] Through the above steps, the target document is first identified and segmented into multiple document elements with clear semantic classifications based on element type. Then, combining the attribute information of each element (such as position, font, size, and layout relationships) with the text content, the hierarchy and referencing edges are modeled to form an initial graph structure. Based on this, multimodal features of each element are further extracted, and these features are used to add edges to the graph structure, such as temporal edges and implicit semantic edges, and the node association strength is dynamically optimized. This constructs a target graph structure that accurately reflects the document's inherent logical topology and semantic dependencies. This graph structure uses nodes to represent structured elements and weighted edges to represent semantic relationships, breaking through the limitations of traditional linear text extraction and achieving a complete reconstruction of the document's knowledge network. Therefore, it effectively solves the problem that existing technologies cannot restore structured semantic relationships, leading to low efficiency in content acquisition and use.

[0059] The document processing method provided in this application embodiment involves: identifying the content of the target document to obtain document content; splitting the document content according to element type to obtain M document elements, where each document element belongs to one element type, and M is a positive integer; obtaining the attribute information and text content of each document element; determining the association relationship between the M document elements based on the attribute information and text content; generating initial graph structure data of the target document based on the association relationship, wherein the initial graph structure data includes M nodes, each node corresponding to a document element, and the association relationship corresponding to the edges between the nodes; and determining the association relationship between each document element based on the attribute information and text content of each document element. This method involves identifying the feature information of document elements and updating the edges in the initial graph structure data based on this feature information to obtain the target graph structure data of the target document. By recognizing the content of the target document, determining the multimodal features of each element in the document, and generating graph structure data between elements based on the multimodal features and semantic associations, the method achieves the goal of improving the restoration of the document's inherent logical topology and semantic relationships through graph structure data. This achieves the technical effect of transforming unstructured documents into structured data containing document content and relationships between document elements, thereby solving the technical problem in related technologies where existing document recognition technologies only output linear text, resulting in low efficiency in document content acquisition and document usage.

[0060] Optionally, in the document processing method provided in this application embodiment, splitting the document content according to element type to obtain M document elements includes: splitting the document content to obtain multiple text blocks; inputting a preset set of element types and multiple text blocks into a target recognition model to obtain the element type of each text block, wherein the target recognition model is trained through a first training set, the first training set includes multiple first sample data, each first sample data includes a sample text block and the element type of the sample text block; and determining each element type and the corresponding text block as a document element to obtain M document elements.

[0061] It should be noted that splitting document content refers to dividing the character sequences and layout information in the original document into several semantically complete and clearly defined smallest text units based on the document's visual structure and semantic continuity; obtaining multiple text blocks refers to several independent text fragments formed after splitting, each text block being a continuous, unbroken, and semantically self-consistent character sequence; the preset element type set refers to a finite set of classifications consisting of six structural units: titles, paragraphs, tables, images, formulas, and footnotes; and the target recognition model refers to a machine learning classification model used to determine the element type of each text block.

[0062] For example, when splitting a document to obtain document elements, the original format data of the target document needs to be parsed first to extract all text characters and their corresponding bounding box coordinates. Based on visual coherence, line height variation, paragraph structure and layout density, the document content is semantically segmented to generate multiple text blocks. Each text block is a semantically complete text sequence without logical breaks. Its boundaries are automatically determined by the semantic consistency of the context, rather than manually setting the length.

[0063] For example, a title can be a block of text, or a paragraph can be a block of text.

[0064] Furthermore, the preset set of element types is used as the classification label space, and all text blocks are input into the target recognition model in parallel. This model is a structured classifier based on multimodal input. Its input features include the character sequence and bounding box coordinates of the text block. The output is the probability distribution of the text block belonging to each class in the preset set of element types. The element type of each text block is determined according to the probability distribution, thus obtaining the element type of each text block. For example, the element type of text block 1 is title.

[0065] Finally, the text block is combined with its corresponding element type to determine a document element, resulting in M ​​document elements, where each document element consists of a text block and the element type of that text block.

[0066] It should be noted that the target recognition model is trained under supervision using a first training set. The first training set consists of multiple first sample data sets, each containing a sample text block and its element types annotated by experts. The sample text blocks are derived from multiple real documents such as academic papers, contracts, and textbooks, covering complex scenarios such as different fonts, layouts, language styles, cross-page tables, and formula embeddings, thereby improving the model's generalization ability. The target recognition model can be trained on any neural network model, as long as the model can perform the recognition operations described in this scheme. This embodiment does not impose any limitations.

[0067] It should be noted that if the probability of each element type output by the model is less than the preset probability threshold, it indicates that the text block does not conform to any of the above six element types. In this case, the text block can be deleted, or the text block can be merged with the text blocks adjacent to it and then re-identified, thereby completing the recognition operation of the entire document content.

[0068] This embodiment achieves automated, high-precision, and standardized element classification of each text block in the document content through a target recognition model trained on the first training set. This ensures that each text block is accurately assigned a unique category from the preset element type set, thereby identifying the text block with a defined element type as a document element and achieving the technical effect of accurately identifying document elements.

[0069] Optionally, in the document processing method provided in this application embodiment, the attribute information includes the bounding box coordinate information of the document element in the target document. Determining the association relationship between M document elements based on the attribute information and text content includes: for any first document element, determining whether there is second text content in the text content of the M document elements, wherein the second text content contains the first text content of the first document element; if there is second text content, determining that there is a reference relationship between the second document element to which the second text content belongs and the first document element, wherein the reference relationship is used to represent that the first document element references the second document element; and / or, for any third document element, determining whether there is a fourth document element containing the third document element based on the bounding box coordinate information; if there is a fourth document element, determining that there is a hierarchical relationship between the third document element and the fourth document element, wherein the hierarchical relationship is used to represent that the third document element is contained in the fourth document element.

[0070] It should be noted that attribute information refers to the structured feature data carried by each document element. Its core component is the bounding box coordinate information of the element in the target document, that is, the smallest bounding rectangle range of the element in the page space, which is jointly defined by the coordinates of the upper left corner (x_min, y_min) and the coordinates of the lower right corner (x_max, y_max). The bounding box coordinate information refers to the numerical expression of this coordinate range, which is used to accurately describe the spatial position and size of the document element in the two-dimensional page. The first document element and the third document element are both any one of the M document elements.

[0071] For example, after obtaining M document elements and their attribute information, the system first performs semantic matching on the text content of the M document elements to identify reference relationships: for any document element (e.g., the first document element), the system extracts its first text content, traverses the text content of all other document elements, and determines whether there exists a second text content whose character sequence completely contains all the characters of the first text content, and the inclusion relationship satisfies semantic continuity. If it exists, it is determined that the first document element references the second document element, and a reference relationship from the first document element to the second document element is established.

[0072] Furthermore, spatial relationship analysis can be performed based on the bounding box coordinates of each document element to identify hierarchical relationships: For any document element (e.g., the third document element), the system performs geometric inclusion calculations on its bounding box coordinates and the bounding box coordinates of each of the other fourth document elements. If all four coordinate points of the third document element are located inside the bounding box of the fourth document element, and the width and height of the third document element are both smaller than those of the fourth document element, then it is determined that the fourth document element completely contains the third document element in space, thereby establishing a hierarchical relationship between the two. This relationship is only retained when visual inclusion is established and the type conforms to semantic logic.

[0073] This embodiment achieves automated and unambiguous identification of hierarchical and referential relationships between document elements through a dual mechanism of spatial geometric analysis and precise matching of text content, thus achieving the technical effect of accurately determining the association relationships between document elements.

[0074] Optionally, in the document processing method provided in this application embodiment, the attribute information includes the bounding box coordinate information of the document element in the target document. Determining the feature information of each document element based on the attribute information and text content includes: for any target document element, determining the font format feature vector of the target document element based on the attribute information; inputting the bounding box coordinate information and text content into the gaze probability recognition model to obtain the gaze probability value of the target document element, wherein the gaze probability value is used to characterize the probability that the content gazed upon by the user for the first time in the target document is the target document element, the gaze probability recognition model is trained through a second training set, the second training set includes multiple second sample data, each second sample data includes a sample element and the gaze probability value of the sample element; calculating the spatial adjacency feature vector between the target document element and each non-target document element among M document elements based on the bounding box coordinate information; determining the text feature vector of the target document element based on the text content; and combining the font format feature vector, gaze probability value, spatial adjacency feature vector, and text feature vector into the feature information of the target document element.

[0075] It should be noted that the font format feature vector refers to the high-dimensional numerical representation generated by analyzing the text appearance attributes of the target document element, based on visual style features such as font size, weight, and color; the gaze probability value is a value output by the gaze probability recognition model, used to quantify the probability that a user's visual attention will first fall on the document element when browsing the target document for the first time; the gaze probability recognition model refers to a machine learning model trained based on the visual attention prediction mechanism, whose input is the bounding box coordinates of the document element and the text content, and whose output is the gaze probability value; the spatial adjacency feature vector refers to the quantitative expression of the relative spatial relationship between the target document element and each other non-target document element, which can include four-dimensional features: normalized horizontal distance, normalized vertical distance, line angle, and bounding box overlap rate; the text feature vector refers to the high-dimensional semantic vector generated by vectorizing the text content of the target document element through a semantic encoder, reflecting the essential attributes of its semantic content.

[0076] For example, when acquiring feature information of document elements, for any target document element, the system first extracts the font attributes from its attribute information, including font size, weight, and color. The font size is normalized and mapped to the interval [0, 1], the weight is numerically encoded (e.g., 1 for regular and 2 for bold), and the color is normalized to [0, 1] using RGB triples. The three are concatenated into a three-dimensional input vector, which is then mapped to a fixed-length font format feature vector by a pre-trained fully connected layer. .

[0077] Among them, font format feature vector It can be represented as:

[0078] ;

[0079] in, This is the weight matrix. For bias vectors, For font size, For font thickness, Set the font color.

[0080] For example, the system also needs to input the bounding box coordinates of the target document element and the text content into the gaze probability recognition model to obtain the gaze probability value, which reflects its visual priority in the overall reading path. It should be noted that the gaze probability recognition model can be a neural network model based on convolutional neural networks and attention mechanisms, which can be trained and optimized by a second training set. The second training set consists of a large amount of eye movement trajectory data of real users when reading documents. Each second sample data contains a sample element and its corresponding first gaze probability value.

[0081] Among them, the fixation probability vector is composed of fixation probability values. It can be represented as:

[0082] ;

[0083] in, This is the weight matrix. For bias vectors, This represents the fixation probability value.

[0084] For example, the system also needs to traverse all non-target document elements among the M document elements, combine each non-target document element with the target document element into a pair of document elements, and calculate the spatial adjacency features between the bounding boxes of each pair of document elements, including normalized horizontal distance, normalized vertical distance, the angle between the line connecting the centers of the two elements and the horizontal axis, and the bounding box overlap rate. These four factors constitute a four-dimensional vector. This calculation is performed sequentially for all non-target elements to form a set of spatial adjacency feature vectors. , is used to characterize the relative position semantics of the target element in the layout structure.

[0085] Among them, the set of spatial adjacency feature vectors It can be represented as:

[0086] ;

[0087] in, The normalized horizontal distance between document element i and document element j. The normalized vertical distance between document element i and document element j. Let the angle between the line connecting the centers of the bounding boxes of document elements i and j and the horizontal axis be denoted as . This represents the overlap ratio of the bounding boxes of document element i and document element j.

[0088] For example, the system also needs to input the text content of the target document element into a pre-trained semantic encoder to generate a high-dimensional text feature vector.

[0089] Finally, the system concatenates the font format feature vector, gaze probability value, spatial adjacency feature vector, and text feature vector to form the complete feature information of the target document element. .

[0090] Among them, complete feature information It can be represented as:

[0091] ;

[0092] in, This represents the text feature vector of the target document element. This represents the gaze probability value of the target document element. The layout feature vector of the target document element includes a font format feature vector and a spatial adjacency feature vector.

[0093] This embodiment achieves multi-dimensional representation of each document element by integrating four heterogeneous features: font style, visual salience, spatial location, and semantic content. This enables the feature information of text elements to simultaneously reflect their visual salience, semantic importance, and spatial context, thus achieving the technical effect of accurately determining the characteristic information of text elements.

[0094] Optionally, in the document processing method provided in this application embodiment, updating the edges in the initial graph structure data according to the feature information includes: obtaining the gaze probability value of each document element, sorting the gaze probability values ​​to obtain the target sequence; and adding temporal edges between the nodes of the document elements to which adjacent gaze probability values ​​belong in the target sequence.

[0095] It should be noted that the gaze probability value refers to the numerical value output by the gaze probability recognition model, representing the probability that a user's gaze will first focus on a document element during the initial reading process. Its value ranges from the closed interval [0, 1], with higher values ​​indicating a stronger priority for the element in the visual flow. Temporal edges refer to directed connections in the document structure graph used to represent the reading order, pointing from nodes with higher visual priority to nodes with lower priority, thus depicting the natural reading path of the document content.

[0096] For example, after the system generates feature information of all document elements, it can extract the gaze probability value corresponding to each element to form a gaze probability set containing M values, and sort these M gaze probability values ​​in descending order to generate a target sequence. Each item in the sequence corresponds to a unique document element node, and its position in the sequence directly reflects the relative order of the node in the visual reading path. The highest gaze probability value is located at the beginning of the sequence, and the lowest value is located at the end.

[0097] In the target sequence, the system establishes a one-way connection between the two document element nodes corresponding to two adjacent items. That is, a temporal edge is added between the node at position i and the node at position i+1 in the sequence. The starting point of this edge is the node with a higher gaze probability value, and the ending point is the node with a lower gaze probability value. This ensures that the direction of the temporal edge strictly follows the logic of decreasing visual priority. The establishment of the temporal edge is performed during the graph structure update phase. Its function is to transform the originally unordered set of nodes into a directed subgraph with a clear reading order. This subgraph is consistent with the natural visual flow of the document and can truly reflect the attention transfer path when humans read fixed-format documents.

[0098] This embodiment generates a target sequence based on gaze probability values ​​and automatically establishes temporal edges between adjacent nodes, achieving an objective, data-driven restoration of the document reading order. This makes the temporal relationships in the graph structure truly reflect the natural migration pattern of the user's visual attention, significantly improving the ability of the document structure graph to express the real reading logic.

[0099] Optionally, in the document processing method provided in this application embodiment, updating the edges in the initial graph structure data according to feature information includes: for any first document element pair, calculating the similarity of the first document element pair according to feature information, and determining whether the similarity is greater than a similarity threshold; if the similarity is greater than the similarity threshold, determining whether the nodes of the first document element pair have edges in the initial graph structure data; if the nodes of the first document element pair do not have edges in the initial graph structure data, adding semantic edges between the nodes of the first document element pair.

[0100] It should be noted that semantic edges refer to undirected connections in a document structure graph used to represent potential semantic associations between two nodes.

[0101] For example, after obtaining the feature information of all document elements, the system also needs to traverse all possible combinations of document elements among the M document elements to form a set of first document element pairs. For each first document element pair, the system extracts the text feature vectors of its two nodes, uses the cosine similarity formula to calculate the degree of closeness between the two in the semantic space, and compares the calculated similarity with the similarity threshold. If the similarity is less than or equal to the similarity threshold, the operation on the first document element pair is completed.

[0102] If the similarity is greater than the similarity threshold, it is necessary to query whether there is a hierarchical edge, temporal edge or reference edge between the two nodes of the first document element pair in the initial graph structure data. If any kind of edge exists, the node pair is skipped. If there is no edge between the two nodes in the initial graph structure data, a semantic edge is created between them. This edge is an undirected connection used to represent that the two nodes have an implicit semantic relationship, but it is not reflected by explicit reference or layout structure.

[0103] It should be noted that, in order to prevent the graph structure from becoming too dense, the system can limit the number of semantic edges associated with each node. Each node can retain a maximum of 20 semantic edges with the highest similarity, and any excess edges will be removed in descending order of similarity.

[0104] It should be noted that similarity can be calculated using the following formula:

[0105] ;

[0106] Where sim(vi, vj) represents the similarity between any pair of document elements. Let be the text feature vector from the feature information of the i-th document element. Let be the text feature vector in the feature information of the j-th document element.

[0107] It should be noted that the edge weight of a semantic edge can be sim(vi, vj)×0.7, that is, the edge weight is determined based on the similarity.

[0108] This embodiment achieves automated discovery of implicit semantic relationships in documents through a dual constraint mechanism of semantic similarity and initial edge state. It overcomes the problem of missing semantic relationships caused by traditional document parsing methods that rely solely on explicit references or spatial positional relationships, and significantly enhances the expressive power of graph structures for document knowledge networks.

[0109] Optionally, in the document processing method provided in this application embodiment, after obtaining the target graph structure data of the target document, the method further includes: for any document element, obtaining a tag prompt word, and inputting the tag prompt word and the document element into a tag recognition model to obtain the tag information of the document element, and adding the tag information to the target graph structure data; obtaining a fifth document element from M document elements whose tag information is the first information, and determining whether there is a sixth document element among the document elements that have an edge with the fifth document element, wherein the tag information of the sixth document element is the second information; if there is a sixth document element, obtaining all node links containing the fifth document element and the sixth document element from the target graph structure data, and obtaining preset inference chain structure information; obtaining a target node link that conforms to the preset inference chain structure information from all node links, and determining the target node link as the target inference chain, wherein when it is detected that the user clicks on any node in the target inference chain in the target graph structure data, the target inference chain is displayed.

[0110] It should be noted that the tag prompts refer to structured text instructions used to guide the tag recognition model in semantic role classification. Their content can be something like, "Please classify the following text paragraphs into semantic roles: Text content: {document element}; Optional tags: definition, example, conclusion, step, hypothesis, evidence, inference, refutation; Please output the probability distribution of each tag (summed to 1), in JSON format." The tag recognition model refers to a semantic classifier built on a large language model. Its input is the tag prompts and the text content of the document elements, and its output is an eight-dimensional probability distribution vector, corresponding to the probability of belonging to the eight semantic tag categories mentioned above. This large language model can be any large language model capable of tag recognition. The fifth document element can be any one of the M document elements. The preset inference chain structure information refers to a predefined semantic tag arrangement pattern, in the form of an ordered tag sequence such as "hypothesis, inference, evidence, conclusion."

[0111] For example, the system can call the tag recognition model for each document element in the target graph structure data, input a fixed-format tag prompt word and the text content of the document element, the model outputs the probability distribution of its eight semantic tags, and only retains the tags with a probability greater than the threshold as the tag information of the node, and writes the tag information into the node attribute.

[0112] Furthermore, the system traverses all document elements, identifies the fifth document element with the tag information "evidence" (i.e., first information), verifies all its adjacent nodes, and determines whether there is a sixth document element with the tag information "inference" (i.e., second information). If it exists, it extracts all complete paths from the target graph structure data that have the sixth and fifth document elements as the last two elements, forming a set of node links.

[0113] Furthermore, the system also needs to compare whether the node label sequence of each node link in the node link set is consistent with the preset inference chain structure information. The preset structure information can include a limited combination of "hypothesis, inference, evidence" and "hypothesis, steps, inference, evidence". Only when the consecutive node labels of a link completely match any preset structure is the link marked as the target node link and determined as the target inference chain.

[0114] Based on this, the system listens for user click events on any node in the target graph structure data in the front-end interaction module. If the node belongs to any target inference chain, the system automatically extracts all nodes and edges of the inference chain, renders them as a highlighted path and visualizes them. The nodes in the path are arranged in semantic order, and the edge type and weight retain their original values.

[0115] This embodiment achieves accurate extraction of implicit logical structures in documents through a tag-driven reasoning chain automatic identification mechanism, enabling machines to identify typical reasoning patterns and dynamically present them in user interaction, significantly enhancing the logical traceability of knowledge expression and the semantic transparency of human-computer interaction.

[0116] Optionally, in the document processing method provided in this application embodiment, after obtaining the target graph structure data of the target document, the method further includes: for any first document element pair with connecting edges, determining the semantic contradiction score of the first document element pair based on the feature information of the document elements; determining whether the semantic contradiction score is greater than a first threshold, and generating a prompt message if the semantic contradiction score is greater than the first threshold, wherein the prompt message is used to characterize that there is an error in the first document element pair; determining whether the semantic contradiction score is less than or equal to the first threshold if the semantic contradiction score is less than a second threshold, wherein the second threshold is less than the first threshold; and performing a fusion operation on the nodes of the first document element pair in the target graph structure data to obtain updated target graph structure data if the semantic contradiction score is less than the second threshold.

[0117] It should be noted that the semantic contradiction score refers to the probability value that two document elements have a logical contradiction in semantic content, calculated by the natural language reasoning model. Its value range is a closed interval [0, 1], and the higher the score, the stronger the contradiction. The prompt information refers to the structured warning information automatically generated by the system, which is used to indicate that there is a semantic contradiction between the first pair of document elements. The fusion operation refers to the graph structure update behavior of merging two nodes into a single node, retaining the node with more complete content, deleting redundant nodes, and redirecting the original connecting edges to the new node.

[0118] It should be noted that the semantic contradiction score can be calculated using the following formula:

[0119] ;

[0120] Among them, h i h represents the feature information of the i-th document element. j represents the feature information of the j-th document element, ⊕ indicates vector concatenation, NLI(·) is the probability value of the contradictory category output by the natural language inference model, Wc is the weight matrix, σ is the Sigmoid activation function with an output range of [0, 1], and Conflict(vi, vj) is the semantic contradiction score with a value range of [0, 1]. The larger the value, the deeper the contradiction.

[0121] For example, after obtaining the target graph structure data, it is also possible to traverse all first document element pairs with connecting edges in the target graph structure data, extract the text feature vectors of the two nodes, and input them into a pre-trained natural language inference model to obtain semantic contradiction scores.

[0122] When the semantic contradiction score exceeds the first threshold, the system generates a prompt message that clearly indicates that there is a logical error in the first document element pair. The prompt message includes the unique identifiers of the two nodes, a summary of their text content, the contradiction type (definition contradiction, data contradiction, or conclusion contradiction), and is marked as "logical conflict" for subsequent secondary review.

[0123] When the semantic contradiction score is less than or equal to the first threshold, the system continues to determine whether it is less than the second threshold. If it is less than the second threshold, the system determines that the first document element is highly consistent in semantics and triggers the fusion operation: the system selects the node with more characters in the text content of the two nodes as the retained node, redirects all edge relationships of the other node to the retained node, deletes the merged node, and updates the node set and edge set in the target graph structure data to improve the connectivity and consistency of the graph structure.

[0124] This embodiment achieves accurate identification of logical conflicts and automatic elimination of semantic redundancy through a dual-threshold discrimination mechanism based on semantic contradiction scores. This avoids interference from highly contradictory nodes on the reasoning chain and reduces graph structure expansion caused by repetitive expressions, significantly improving the semantic accuracy and indirectness of the graph structure.

[0125] Optionally, in the document processing method provided in this application embodiment, after obtaining the target graph structure data of the target document, the method further includes: determining the semantic contradiction score of each second document element pair with semantic edges, and determining the first parameter value of the target graph structure data based on the semantic contradiction score and edge weight of each second document element pair; determining the distribution entropy value of the tag information of each document element to obtain M distribution entropy values, and determining the second parameter value of the target graph structure data based on the M distribution entropy values; determining the global energy value of the target graph structure data based on the first parameter value and the second parameter value, and determining whether the global energy value is greater than the energy threshold; if the global energy value is greater than the energy threshold, determining the maximum contributing document element pair based on the semantic contradiction score and edge weight of each second document element pair, and determining the maximum contributing document element pair as the document element pair to be processed; adjusting the edge weight of the maximum contributing document element pair, and re-executing the process of determining the global energy value of the target graph structure data using the adjusted edge weight, until the global energy value of the target graph structure data is less than or equal to the energy threshold.

[0126] It should be noted that the second document element pair refers to any combination of document elements formed by two document element nodes connected by semantic edges in the target graph structure data; the distribution entropy value refers to the information entropy calculated based on the probability distribution of the label information of each document element; the global energy value refers to the global consistency evaluation index constructed by combining the first parameter value and the second parameter value; the maximum contributing document element pair refers to the pair of nodes with the largest product of semantic contradiction score and edge weight among all second document element pairs, and its energy contribution value is the highest.

[0127] For example, after obtaining the target graph structure data of the target document, the system can also calculate the product of the semantic contradiction score and the edge weight for each pair of second document elements with semantic edges in the target graph structure data, and sum all the product values ​​to obtain the first parameter value, which can be expressed as:

[0128] ;

[0129] Among them, E semantic Let be the set of implicit semantic edges, where vi and vj are document elements, wij is the weight of edge (vi, vj), and Conflict(vi, vj) is the semantic contradiction score of vi and vj.

[0130] Furthermore, the system calculates the distribution entropy value for the tag information of each document element, obtaining M distribution entropy values, and then sums them to obtain the second parameter value, which can be expressed as:

[0131] ;

[0132] Where H(Pi) is the entropy value of the semantic role label probability distribution of node vi, and the calculation formula is:

[0133] ;

[0134] Among them, P ik The probability distribution for each label.

[0135] Furthermore, the system multiplies the first parameter value and the second parameter value by the balance coefficient λ (default value 0.2) and then adds them together to obtain the global energy value E(G) of the target graph structure data. That is, the global energy value E(G) is calculated by the following formula:

[0136] ;

[0137] Furthermore, if the global energy value is greater than the energy threshold, the optimization process begins: the system iterates through all pairs of second document elements, calculates the product of the semantic contradiction score and the edge weight for each pair, selects the second document element pair corresponding to the maximum value as the maximum contributing document element pair, and marks it as a document element pair to be processed; the system reduces the semantic edge weight of the document element pair to be processed by 50%, and recalculates the first parameter value, the second parameter value, and the global energy value using the adjusted weight; this process is repeated until the global energy value is less than or equal to the energy threshold, or the energy value improvement is less than 1% after 20 consecutive iterations.

[0138] Finally, the document elements marked as pending processing are processed uniformly to complete the operation of reducing the global energy value.

[0139] This embodiment uses an iterative optimization mechanism based on global energy values ​​to automatically identify and weaken the semantic associations that most strongly disrupt logical consistency, while suppressing the impact of semantically ambiguous nodes on the overall structure, thereby enhancing the self-consistency of the graph structure.

[0140] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0141] Example 2

[0142] This application also provides a document processing apparatus. It should be noted that the document processing apparatus of this application can be used to execute the document processing method provided in the above embodiments. The document processing apparatus provided in this application will be described below.

[0143] According to an embodiment of this application, an apparatus for implementing the above-described document processing method is also provided. Figure 4 This is a schematic diagram of a document processing apparatus according to Embodiment 2 of this application, as shown below. Figure 4 As shown, the device includes:

[0144] The splitting unit 41 is used to perform content recognition on the target document, obtain the document content, and split the document content according to the element type to obtain M document elements, where each document element belongs to an element type and M is a positive integer.

[0145] The first determining unit 42 is used to obtain the attribute information and text content of each document element, and determine the association relationship between the M document elements based on the attribute information and text content.

[0146] The first generation unit 43 is used to generate initial graph structure data of the target document according to the association relationship. The initial graph structure data includes M nodes, each node corresponds to a document element, and the association relationship corresponds to the edge between each node.

[0147] The second determining unit 44 is used to determine the feature information of each document element based on the attribute information and text content of each document element, and update the edges in the initial graph structure data based on the feature information to obtain the target graph structure data of the target document.

[0148] The document processing apparatus provided in this application embodiment performs content recognition on the target document by a splitting unit 41 to obtain document content, and splits the document content according to element type to obtain M document elements, wherein each document element belongs to an element type and M is a positive integer; a first determining unit 42 obtains the attribute information and text content of each document element, and determines the association relationship between the M document elements according to the attribute information and text content; a first generating unit 43 generates initial graph structure data of the target document according to the association relationship, wherein the initial graph structure data includes M nodes, each node corresponds to a document element, and the association relationship corresponds to the edge between each node; a second determining unit 44 determines the feature information of each document element according to the attribute information and text content of each document element, and updates the edge in the initial graph structure data according to the feature information to obtain the target graph structure data of the target document. By recognizing the content of the target document, determining the multimodal features of each element in the document, and generating graph structure data between elements based on the multimodal features and semantic associations, the goal of improving the restoration of the document's inherent logical topology and semantic relationships through graph structure data is achieved. This realizes the technical effect of transforming unstructured documents into structured data containing document content and relationships between document elements, thereby solving the technical problem that existing document recognition technologies only output linear text, resulting in low efficiency in document content acquisition and document usage.

[0149] Optionally, in the document processing apparatus provided in this application embodiment, the splitting unit 41 includes: a splitting module, used to split the document content to obtain multiple text blocks; a first processing module, used to input a preset element type set and multiple text blocks into a target recognition model to obtain the element type of each text block, wherein the target recognition model is trained through a first training set, the first training set includes multiple first sample data, each first sample data includes a sample text block and the element type of the sample text block; and a first determining module, used to determine each element type and the corresponding text block as a document element to obtain M document elements.

[0150] Optionally, in the document processing apparatus provided in this application embodiment, the attribute information includes the bounding box coordinate information of the document element in the target document. The first determining unit 42 includes: a first determining module, used to determine whether there is second text content in the text content of M document elements for any first document element, wherein the second text content contains the first text content of the first document element; a second determining module, used to determine that there is a reference relationship between the second document element to which the second text content belongs and the first document element when the second text content exists, wherein the reference relationship is used to represent that the first document element references the second document element; and / or, the second determining module, used to determine whether there is a fourth document element containing the third document element based on the bounding box coordinate information for any third document element; and a third determining module, used to determine that there is a hierarchical relationship between the third document element and the fourth document element when the fourth document element exists, wherein the hierarchical relationship is used to represent that the third document element is contained in the fourth document element.

[0151] Optionally, in the document processing apparatus provided in this application embodiment, the attribute information includes the bounding box coordinate information of the document element in the target document. The second determining unit 44 includes: a fourth determining module, used to determine the font format feature vector of the target document element according to the attribute information for any target document element; a second processing module, used to input the bounding box coordinate information and text content into the gaze probability recognition model to obtain the gaze probability value of the target document element, wherein the gaze probability value is used to characterize the probability that the content gazed at by the user for the first time in the target document is the target document element, the gaze probability recognition model is trained through a second training set, the second training set includes multiple second sample data, each second sample data includes a sample element and the gaze probability value of the sample element; a calculation module, used to calculate the spatial adjacency feature vector between the target document element and each non-target document element among M document elements according to the bounding box coordinate information; a fifth determining module, used to determine the text feature vector of the target document element according to the text content; and a combination module, used to combine the font format feature vector, gaze probability value, spatial adjacency feature vector and text feature vector into the feature information of the target document element.

[0152] Optionally, in the document processing apparatus provided in this application embodiment, the second determining unit 44 includes: a sorting module, used to obtain the gaze probability value of each document element and sort the gaze probability values ​​to obtain a target sequence; and a first adding module, used to add temporal edges between nodes of document elements to which adjacent gaze probability values ​​belong in the target sequence.

[0153] Optionally, in the document processing apparatus provided in this application embodiment, the second determining unit 44 includes: a third determining module, configured to calculate the similarity of the first document element pair based on feature information for any first document element pair, and determine whether the similarity is greater than a similarity threshold; a fourth determining module, configured to determine whether there are edges between the nodes of the first document element pair in the initial graph structure data when the similarity is greater than the similarity threshold; and a second adding module, configured to add semantic edges between the nodes of the first document element pair when there are no edges between the nodes of the first document element pair in the initial graph structure data.

[0154] Optionally, in the document processing apparatus provided in this application embodiment, after obtaining the target graph structure data of the target document, the apparatus further includes: a first acquisition unit, configured to acquire a tag prompt for any document element, input the tag prompt and the document element into a tag recognition model to obtain the tag information of the document element, and add the tag information to the target graph structure data; a second acquisition unit, configured to acquire a fifth document element whose tag information is the first information from M document elements, and determine whether there is a sixth document element among the document elements that have an edge with the fifth document element, wherein the tag information of the sixth document element is the second information; a third acquisition unit, configured to acquire all node links containing the fifth document element and the sixth document element from the target graph structure data when the sixth document element exists, and acquire preset inference chain structure information; and a third determination unit, configured to acquire a target node link that conforms to the preset inference chain structure information from all node links, and determine the target node link as the target inference chain, wherein the target inference chain is displayed when it is detected that the user clicks on any node in the target inference chain in the target graph structure data.

[0155] Optionally, in the document processing apparatus provided in this application embodiment, after obtaining the target graph structure data of the target document, the apparatus further includes: a fourth determining unit, configured to determine the semantic contradiction score of any first document element pair having connecting edges based on the feature information of the document elements; a second generating unit, configured to determine whether the semantic contradiction score is greater than a first threshold, and generate a prompt message if the semantic contradiction score is greater than the first threshold, wherein the prompt message is used to indicate that there is an error in the first document element pair; a first judging unit, configured to determine whether the semantic contradiction score is less than a second threshold if the semantic contradiction score is less than or equal to the first threshold, wherein the second threshold is less than the first threshold; and an updating unit, configured to perform a fusion operation on the nodes of the first document element pair in the target graph structure data if the semantic contradiction score is less than the second threshold, to obtain updated target graph structure data.

[0156] Optionally, in the document processing apparatus provided in this application embodiment, after obtaining the target graph structure data of the target document, the apparatus further includes: a fifth determining unit, configured to determine the semantic contradiction score of each second document element pair with semantic edges, and determine the first parameter value of the target graph structure data based on the semantic contradiction score and edge weight of each second document element pair; a sixth determining unit, configured to determine the distribution entropy value of the tag information of each document element, obtain M distribution entropy values, and determine the second parameter value of the target graph structure data based on the M distribution entropy values; a second judging unit, configured to determine the global energy value of the target graph structure data based on the first parameter value and the second parameter value, and judge whether the global energy value is greater than the energy threshold; a seventh determining unit, configured to determine the maximum contributing document element pair based on the semantic contradiction score and edge weight of each second document element pair when the global energy value is greater than the energy threshold, and determine the maximum contributing document element pair as the document element pair to be processed; and an adjusting unit, configured to adjust the edge weight of the maximum contributing document element pair, and re-execute the process of determining the global energy value of the target graph structure data using the adjusted edge weight, until the global energy value of the target graph structure data is less than or equal to the energy threshold.

[0157] It should be noted that the aforementioned splitting unit 41, first determining unit 42, first generating unit 43, and second determining unit 44 correspond to steps S201 to S204 in Embodiment 1. The instances and application scenarios implemented by each of these units and their corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the aforementioned modules or units can be hardware or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). These modules can also run as part of a device in the computer terminal 10 provided in Embodiment 1.

[0158] Example 3

[0159] Embodiments of this application may provide an electronic device. Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (Only one is shown) processor 1002, memory 1004, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0160] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0161] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0162] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0163] Example 4

[0164] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the document processing method provided in Embodiment 1.

[0165] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0166] Embodiments of this application also provide a computer program product, which, when executed on a data processing device, is adapted to perform the steps of a document processing method.

[0167] Embodiments of this application also provide a computer-readable storage medium, which includes a stored executable program, wherein the executable program controls the device where the computer-readable storage medium is located to execute the above-described document processing method when it runs.

[0168] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0169] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0171] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0172] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0173] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0174] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A document processing method, characterized in that, include: The target document is subjected to content recognition to obtain the document content, and the document content is split according to the element type to obtain M document elements, where each document element belongs to an element type and M is a positive integer; Obtain the attribute information and text content of each document element, and determine the association relationship between the M document elements based on the attribute information and text content; An initial graph structure data for the target document is generated based on the association relationship, wherein the initial graph structure data includes M nodes, each node corresponds to a document element, and the association relationship corresponds to the edge between each node; The feature information of each document element is determined based on the attribute information and text content of each document element, and the edges in the initial graph structure data are updated based on the feature information to obtain the target graph structure data of the target document.

2. The method according to claim 1, characterized in that, The document content is split according to element type, resulting in M ​​document elements, including: The document content is split into multiple text blocks; The preset set of element types and the multiple text blocks are input into the target recognition model to obtain the element type of each text block. The target recognition model is trained through a first training set, which includes multiple first sample data. Each first sample data includes a sample text block and the element type of the sample text block. Each element type and its corresponding text block are identified as the document element, resulting in the M document elements.

3. The method according to claim 1, characterized in that, The attribute information includes the bounding box coordinates of the document element in the target document. Determining the association relationship between the M document elements based on the attribute information and the text content includes: For any first document element, determine whether there is second text content in the text content of the M document elements, wherein the second text content contains the first text content of the first document element; If the second text content exists, it is determined that there is a reference relationship between the second document element to which the second text content belongs and the first document element, wherein the reference relationship is used to indicate that the first document element references the second document element; and / or, For any third document element, determine whether there exists a fourth document element containing the third document element based on the bounding box coordinate information; In the presence of the fourth document element, a hierarchical relationship is determined between the third document element and the fourth document element, wherein the hierarchical relationship is used to indicate that the third document element is contained in the fourth document element.

4. The method according to claim 1, characterized in that, The attribute information includes the bounding box coordinates of the document element in the target document. Determining the feature information of each document element based on its attribute information and text content includes: For any target document element, determine the font format feature vector of the target document element based on the attribute information; The bounding box coordinate information and the text content are input into the gaze probability recognition model to obtain the gaze probability value of the target document element. The gaze probability value is used to characterize the probability that the user's gaze content when first gazing at the target document is the target document element. The gaze probability recognition model is trained through a second training set, which includes multiple second sample data. Each second sample data includes a sample element and the gaze probability value of the sample element. Calculate the spatial adjacency feature vector between the target document element and each non-target document element among the M document elements based on the bounding box coordinate information; The text feature vector of the target document element is determined based on the text content; The font format feature vector, the gaze probability value, the spatial adjacency feature vector, and the text feature vector are combined to form the feature information of the target document element.

5. The method according to claim 4, characterized in that, Updating the edges in the initial graph structure data based on the feature information includes: Obtain the gaze probability value of each document element, and sort the gaze probability values ​​to obtain the target sequence; Add temporal edges between the nodes of document elements whose gaze probability values ​​are adjacent in the target sequence.

6. The method according to claim 1, characterized in that, Updating the edges in the initial graph structure data based on the feature information includes: For any pair of first document elements, calculate the similarity of the first document element pair based on the feature information, and determine whether the similarity is greater than the similarity threshold. If the similarity is greater than the similarity threshold, determine whether the nodes of the first document element pair have edges in the initial graph structure data; If no edges exist between the nodes of the first document element pair in the initial graph structure data, semantic edges are added between the nodes of the first document element pair.

7. The method according to claim 1, characterized in that, After obtaining the target graph structure data of the target document, the method further includes: For any document element, obtain the tag prompt word, and input the tag prompt word and the document element into the tag recognition model to obtain the tag information of the document element, and add the tag information to the target graph structure data; Obtain the fifth document element whose tag information is the first information from the M document elements, and determine whether there is a sixth document element among the document elements that have an edge with the fifth document element, wherein the tag information of the sixth document element is the second information; In the presence of the sixth document element, all node links containing the fifth and sixth document elements are obtained from the target graph structure data, and preset inference chain structure information is obtained. Obtain target node links that conform to the preset inference chain structure information from all node links, and determine the target node links as target inference chains. When it is detected that a user clicks on any node in the target inference chain in the target graph structure data, the target inference chain is displayed.

8. The method according to claim 1, characterized in that, After obtaining the target graph structure data of the target document, the method further includes: For any first document element pair with connecting edges, determine the semantic contradiction score of the first document element pair based on the feature information of the document elements; Determine whether the semantic contradiction score is greater than a first threshold, and if the semantic contradiction score is greater than the first threshold, generate a prompt message, wherein the prompt message is used to indicate that there is an error in the first document element pair; If the semantic contradiction score is less than or equal to the first threshold, it is determined whether the semantic contradiction score is less than the second threshold, wherein the second threshold is less than the first threshold; If the semantic contradiction score is less than the second threshold, the nodes of the first document element pair in the target graph structure data are merged to obtain the updated target graph structure data.

9. The method according to claim 1, characterized in that, After obtaining the target graph structure data of the target document, the method further includes: Determine the semantic contradiction score of each pair of second document elements with semantic edges, and determine the first parameter value of the target graph structure data based on the semantic contradiction score and edge weight of each pair of second document elements; Determine the distribution entropy value of the tag information of each document element to obtain M distribution entropy values, and determine the second parameter value of the target graph structure data based on the M distribution entropy values; The global energy value of the target graph structure data is determined based on the first parameter value and the second parameter value, and it is determined whether the global energy value is greater than the energy threshold. If the global energy value is greater than the energy threshold, the document element pair with the largest contribution is determined based on the semantic contradiction score and edge weight of each second document element pair, and the document element pair with the largest contribution is determined as the document element pair to be processed. The edge weights of the pair of document elements with the largest contribution are adjusted, and the process of determining the global energy value of the target graph structure data is re-executed using the adjusted edge weights until the global energy value of the target graph structure data is less than or equal to the energy threshold.

10. A document processing apparatus, characterized in that, include: The splitting unit is used to perform content recognition on the target document, obtain the document content, and split the document content according to the element type to obtain M document elements, where each document element belongs to an element type and M is a positive integer; The first determining unit is used to obtain the attribute information and text content of each document element, and determine the association relationship between the M document elements based on the attribute information and the text content. The first generation unit is used to generate initial graph structure data of the target document according to the association relationship, wherein the initial graph structure data includes M nodes, each node corresponds to a document element, and the association relationship corresponds to the edge between each node; The second determining unit is used to determine the feature information of each document element based on the attribute information and the text content of each document element, and update the edges in the initial graph structure data based on the feature information to obtain the target graph structure data of the target document.

Images