Document information extraction method and system, and electronic device and storage medium

Abstract

The present disclosure relates to the fields of large model technology and document processing. Disclosed are a document information extraction method and system, and an electronic device and a storage medium. The method comprises: acquiring a document, wherein the document includes a plurality of semantic entities (S302); on the basis of semantic information of the semantic entities and a structural relationship between the semantic entities, constructing a target entity feature of the semantic entities (S304); and on the basis of the target entity feature, performing information extraction on the document to obtain an information extraction result of the document (S306).

Description

Document information extraction method, system, electronic device and storage medium Technical Field

[0001] The present disclosure relates to the fields of large model technology and document processing, and in particular to a document information extraction method, system, electronic device and storage medium. Background Art

[0002] Currently, most corporate data exists in the form of visual documents, such as text, images, scans, spreadsheets, online documents, and emails. However, these documents are often difficult to circulate and process. Most companies process visual documents using graph neural networks, extracting key structural information from them and then processing this information to improve processing efficiency. However, graph neural networks are mostly trained based on NLP (Neuro-Linguistic Programming) models, which have a weak ability to capture the structured knowledge in visual documents and are unable to predict dependencies between long-range semantic entities, resulting in poor information extraction from visual documents.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far.

[0004] Summary of the Invention

[0005] The embodiments of the present disclosure provide a document information extraction method, system, electronic device, and storage medium to at least solve the technical problem of poor document information extraction effect in related technologies.

[0006] According to one aspect of an embodiment of the present disclosure, a method for extracting information from a document is provided, comprising: acquiring a document, wherein the document contains multiple semantic entities; constructing target entity features of the semantic entity based on semantic information of the semantic entity and structural relationships between the semantic entities; and extracting information from the document based on the target entity features to obtain an information extraction result of the document.

[0007] According to another aspect of an embodiment of the present disclosure, a document information extraction method is also provided, including: responding to a document received in an interactive interface, wherein the document contains multiple semantic entities; constructing target entity features of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities; extracting information from the document based on the target entity features to obtain an information extraction result of the document; and displaying the information extraction result in the interactive interface.

[0008] According to another aspect of an embodiment of the present disclosure, a document information extraction method is also provided, including: obtaining a document by calling a first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is a document, and the document contains multiple semantic entities; constructing a target entity feature of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities; extracting information from the document based on the target entity feature to obtain an information extraction result of the document; outputting the information extraction result by calling a second interface, wherein the second interface includes a second parameter, the parameter value of the second parameter is the information extraction result.

[0009] According to another aspect of an embodiment of the present disclosure, a document information extraction system is also provided, including: a receiving component, configured to display an interactive interface and capture a document input in the interactive interface, wherein the document contains multiple semantic entities; a processing device, configured to construct target entity features of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities, and extract information from the document based on the target entity features to obtain an information extraction result of the document; a sending component, configured to send the information extraction result to the front-end client, wherein the information extraction result is displayed in the interactive interface by the front-end client.

[0010] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, including: a memory storing an executable program; and a processor for running the program, wherein any one of the above methods is executed when the program is run.

[0011] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute any of the above methods.

[0012] In an embodiment of the present disclosure, a method is adopted in which a document is obtained, wherein the document contains multiple semantic entities; a target entity feature of the semantic entity is constructed based on the semantic information of the semantic entity and the structural relationship between the semantic entities; information is extracted from the document based on the target entity feature to obtain an information extraction result of the document, by using the semantic information of the semantic entity as the local entity representation of the document, and constructing the global structural information of the document based on the structural relationship between the semantic entities, and then constructing a target entity feature containing the local entity representation and global structural information of the document based on the semantic information and structural information, so as to ensure that the target entity feature can be associated with the full text of the document and has high robustness, and finally information extraction is performed on the document based on the target entity feature, which can further ensure that the obtained information extraction result is consistent with the content of the document, improve the accuracy of the information extraction result, thereby achieving the technical effect of improving the accuracy of the result obtained by information extraction for the document, and further solving the technical problem of poor information extraction effect for the document in the related art.

[0013] It is easy to note that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present disclosure, and do not constitute a limitation of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0015] FIG1 is a schematic diagram of a hardware environment of a virtual reality device according to a document information extraction method in Example 1 of the present disclosure;

[0016] FIG2 is a structural block diagram of a computing environment of a document information extraction method according to Example 1 of the present disclosure;

[0017] FIG3 is a flow chart of a method for extracting information from a document according to Embodiment 1 of the present disclosure;

[0018] FIG4 is a schematic diagram of a document information extraction process according to Embodiment 1 of the present disclosure;

[0019] FIG5 is a schematic diagram of a process for determining a structural relationship of a semantic entity according to Embodiment 1 of the present disclosure;

[0020] FIG6 is a flow chart of a method for extracting information from a document according to Embodiment 2 of the present disclosure;

[0021] 7 is a flowchart of a method for extracting information from a document according to Embodiment 3 of the present disclosure;

[0022] FIG8 is a structural block diagram of a document information extraction device according to Embodiment 4 of the present disclosure;

[0023] 9 is a structural block diagram of a document information extraction device according to Example 5 of the present disclosure;

[0024] 10 is a structural block diagram of a document information extraction device according to Example 6 of the present disclosure;

[0025] 11 is a structural block diagram of a document information extraction system according to Example 7 of the present disclosure;

[0026] FIG12 is a structural block diagram of an electronic device according to Embodiment 8 of the present disclosure. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or components is not necessarily limited to those steps or components clearly listed, but may include other steps or components that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] The technical solution provided by the present disclosure is mainly implemented using large-scale model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. The large model can also be called a cornerstone model / foundation model (Foundation Model). The large model is pre-trained by large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as large-scale language models (LLMs) and multi-modal pre-training models.

[0030] It should be noted that when the large model is actually applied, the pre-trained model can be fine-tuned through a small number of samples, so that the large model can be applied to different tasks. For example, the large model can be widely used in natural language processing (NLP), computer vision, speech processing and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image description (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present disclosure, the use of a large language model to extract information from documents is used as an example for explanation.

[0031] First, some nouns or terms that appear in the description of the embodiments of the present disclosure are subject to the following explanations:

[0032] Visual documents: documents that store information in the form of text, images, scans, spreadsheets, online documents, emails, etc. Users can obtain the information they need by reading visual documents.

[0033] Document parsing: The process of taking unstructured documents as input, extracting information from the documents based on document intelligent pre-training technologies and products, and outputting processed structured data.

[0034] Example 1

[0035] According to an embodiment of the present disclosure, a method for extracting information from a document is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] Taking into account the huge amount of model parameters of the large model and the limited computing resources of the mobile terminal, the information extraction method of the above-mentioned document provided in the embodiment of the present disclosure can be applied to the application scenario shown in Figure 1, but is not limited to this. Figure 1 is a schematic diagram of the hardware environment of a document information extraction method according to embodiment 1 of the present disclosure. In the application scenario shown in Figure 1, the large model is deployed in a server 10, and the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client devices 20 here may include but are not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, car-mounted devices, etc. The client device 20 can interact with the user through a graphical user interface to implement the call to the large model, thereby implementing the method provided by the embodiment of the present disclosure.

[0037] In an embodiment of the present disclosure, a system consisting of a client device and a server can perform the following steps: obtaining a document, wherein the document contains multiple semantic entities; constructing target entity features of the semantic entities based on the semantic information of the semantic entities and the structural relationships between the semantic entities; extracting information from the document based on the target entity features to obtain information extraction results for the document, thereby resolving the technical problem of poor document information extraction results in related technologies and achieving the purpose of improving the accuracy of information extracted from the document. It should be noted that if the operating resources of the client device can meet the deployment and operation conditions of the large model, the embodiment of the present disclosure can be performed in the client device.

[0038] FIG2 is a block diagram of a computing environment for a method for extracting information from a document according to Example 1 of the present disclosure. As shown in FIG2 , a computing environment 201 includes multiple computing nodes (such as servers) running on a distributed network (shown as 210-1, 210-2, ..., in the figure). Different computing nodes contain local processing and memory resources, and an end user 202 can remotely run applications or store data in the computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3, and 220-4 in the computing environment 201, representing services "A," "D," "E," and "H," respectively.

[0039] End user 202 can provide and access services through a web browser or other software application on a client. In some embodiments, the provisioning and / or request of end user 202 can be provided to the ingress gateway 230. The ingress gateway 230 may include a corresponding agent to handle the provisioning and / or request for services (one or more services provided in the computing environment 201).

[0040] Services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, executing programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0041] In one embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). For example, as shown in Figure 2, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). The Pod may include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers in the Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0042] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201. Executing one or more functions of one service may require invoking one or more functions of another service. As shown in FIG2 , service "A" 220-1 receives a user request from end user 202 from ingress gateway 230. Service "A" 220-1 may invoke service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to execute one or more functions.

[0043] This computing environment can be a cloud computing environment, where resource allocation is managed by the cloud service provider, allowing for feature development without having to worry about implementing, adjusting, or scaling servers. This computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be partitioned to perform a set of functions that can scale independently and automatically, rather than scaling a single hardware device to handle the potential load.

[0044] In the above operating environment, the present disclosure provides a document information extraction method as shown in Figure 3. It should be noted that the document information extraction method of this embodiment can be executed by the mobile terminal of the embodiment shown in Figure 1. Figure 3 is a flow chart of a document information extraction method according to Example 1 of the present disclosure. As shown in Figure 3, the method may include the following steps:

[0045] Step S302: Acquire a document, wherein the document contains multiple semantic entities.

[0046] The above-mentioned documents may refer to documents that need to be observed by the user with the naked eye, such as visual documents, which may include but are not limited to: text, pictures, scans, spreadsheets, online documents, emails, etc. For ease of understanding, the following description is based on the text contained in the document. The above-mentioned semantic entities may refer to entities with specific meanings in the document, and may refer to data that the user can understand based on the background or terminology of the document when reading the document. For example, it may be a specific person, place, product, color, or an abstract event, date, number or symbol. For ease of understanding, if the document is an introduction text of a product, the corresponding semantic entity may refer to the product name of the product introduced in the introduction text, and the user can quickly associate the product with the product name. Generally, a document may contain multiple semantic entities.

[0047] Generally, when the amount of document data is small, for example, when the document contains only one semantic entity, the information extraction system can directly extract information from the document and extract keywords and sentences related to the semantic entity. The user can quickly obtain the content contained in the document based on the keywords and sentences without having to consider too much whether the extracted keywords and sentences can accurately reflect the overall content of the document. However, when the amount of document data is large, for example, when the document contains more semantic entities, since the document contains more content, that is, more information related to the semantic entities, when the information extraction system extracts information from the document, the extracted multiple keywords, sentences and other information may not match the semantic entities, resulting in multiple keywords, sentences and other information not matching the actual content of the document, which in turn causes the user to understand the wrong document content based on the keywords, sentences and other information. Therefore, in order to improve the accuracy of the information extracted from the document, that is, the extracted information can accurately reflect the content contained in the document, the information extraction system can first obtain the document that the user needs to understand, which contains multiple semantic entities, so that the semantic understanding model can be used to assist the user in understanding the document.

[0048] Step S304: constructing target entity features of the semantic entities based on the semantic information of the semantic entities and the structural relationships between the semantic entities.

[0049] The above-mentioned semantic information may refer to the semantics of semantic entities in a document, which may be represented in the form of a feature vector. The above-mentioned structural relationship may refer to the spatial geometric information between different semantic entities, such as the coordinate position relationship, size relationship, etc. between different semantic texts, which may be used to construct the global structural information of the document. The above-mentioned target entity features may refer to features that can reflect the local entity representation and global structural information of the document, wherein the local entity representation may refer to the information contained in the semantic entity itself in the document, such as the semantic information of the above-mentioned semantic entity, and the global structural information may refer to the association relationship information between different semantic entities in the document, such as the structural relationship of the semantic entity.

[0050] In an optional solution of this embodiment, considering that the information contained in semantic entities themselves is relatively simple, there may not be a clear pointing relationship between different semantic entities. When the information extraction system extracts information from a document based on the semantic information of different semantic entities in the document, it may erroneously match different semantic entities. For example, if a document includes the semantic entities "shoes," "display stand," "white," and "black," where "shoes" corresponds to "white" and "display stand" corresponds to "black," if information extraction is performed on the document based solely on the semantic information of the semantic entities, "shoes" may be matched with "black" and "display stand" may be matched with "white." Moreover, the longer the distance between the matching semantic entities, the lower the probability of matching them based solely on the semantic information of the semantic entities. Incorrect matching can cause the information extraction results obtained from the document to be inconsistent with the actual content contained in the document, which can mislead the user's understanding of the document. Therefore, to ensure the accuracy of the information extraction results from the document, the information extraction system needs to accurately establish the matching relationship between different semantic entities. Considering that when users or machines write documents, they usually write according to their own writing habits or document format requirements, and writing habits and format requirements usually do not change, the spatial geometric information between different semantic entities presented in the corresponding document, that is, the structural relationship mentioned above, usually does not change. For example, if user A likes to put the subject before the object, then when writing the aforementioned shoes and display stand, user A will usually write "the color of the display stand is black, and the color of the shoes is white." The corresponding semantic entity matching the semantic entity "shoes" is the latter "white", not the front "black". This means that the semantic information of the semantic entities in the document and the structural relationship between the semantic entities can be used to determine the matching of the semantic entities and the semantic information, thereby improving the accuracy of the matching relationship established between different semantic entities. The above structural relationship can be determined based on information such as the writing habits and format requirements of the user or machine that wrote the document.

[0051] Therefore, in order to ensure the accuracy of the results obtained from document extraction, that is, to ensure that the results are consistent with the actual content contained in the document, the information extraction system can introduce the structural relationship between semantic entities in the document during the information extraction process to improve the accuracy of information extraction from world documents. Considering that the global structural information of a document can better reflect the dependency relationship between different semantic entities in the document than the structural relationship between semantic entities, thereby improving the effect of document information extraction, the information extraction system can further construct global structural information that can reflect the dependency relationship between long-range semantic entities in the document based on the structural relationship between different semantic entities. When extracting information from the document, the local entity representation in the document and the global structural information are combined. That is, based on the semantic information of each semantic entity and the structural relationship between different semantic entities, the above-mentioned target entity features are constructed, so that the constructed target entity features can be associated with the document as a whole and have high robustness.

[0052] Step S306: extract information from the document based on the target entity features to obtain an information extraction result of the document.

[0053] After constructing the target entity features corresponding to the semantic entities in the document, the information extraction system can extract information from the document based on the target entity features, thereby ensuring the accuracy of the information extraction results.

[0054] In the embodiment of the present disclosure, a method is adopted in which a document is obtained, wherein the document contains multiple semantic entities; a target entity feature of the semantic entity is constructed based on the semantic information of the semantic entity and the structural relationship between the semantic entities; information is extracted from the document based on the target entity feature to obtain an information extraction result of the document, by using the semantic information of the semantic entity as the local entity representation of the document, and using the structural relationship between the semantic entities as the global structural information for constructing the document, and then constructing a target entity feature of the semantic entity containing the local entity representation and global structural information of the document based on the semantic information and structural information, so as to ensure that the target entity feature can be associated with the full text of the document and has high robustness. At the same time, information extraction is finally performed on the document based on the target entity feature, which can further ensure that the obtained information extraction result is consistent with the content of the document, improve the accuracy of the information extraction result, thereby achieving the technical effect of improving the accuracy of the result obtained by information extraction for the document, and thus solving the technical problem of poor information extraction effect for the document in the related art.

[0055] In an embodiment of the present disclosure, target entity features of semantic entities are constructed based on semantic information of semantic entities and structural relationships between semantic entities, including: constructing target entity features based on semantic information and structural relationships using a document processing model.

[0056] The above-mentioned document processing model may refer to a model that integrates and analyzes the local entity representation and global structural information of a document.

[0057] In an optional solution of this embodiment, the above-mentioned document processing model can be used to fuse the semantic information of the semantic entity in the document and the structural relationship corresponding to the semantic entity to obtain the above-mentioned target entity feature.

[0058] It should be noted that the models used in the process of extracting information from documents can all be large models. For example, the above-mentioned document processing model can be a large language model. The large language model is used to construct the above-mentioned target entity features based on the above-mentioned semantic information and structural relationships, which can ensure the accuracy of the constructed target entity features.

[0059] In an optional scheme of this embodiment, the above-mentioned document processing model may include at least: a component for obtaining semantic information of semantic entities, a component for obtaining relationships between semantic entities, such as structural relationships and semantic relationships, and a component for constructing global structural information of the document based on structural relationships, so that the information extraction system can accurately obtain local entity representations and global structural information in the document, thereby improving the accuracy of target entity features constructed based on semantic information and structural information.

[0060] In an embodiment of the present disclosure, a document parsing model includes: a feature extraction component, a relationship generation component, and an information mining component, and utilizes a document processing model to construct target entity features based on semantic information and structural relationships, including: utilizing a feature extraction component to extract features from a semantic entity to obtain initial entity features of the semantic entity, wherein the initial entity features are used to characterize the semantic information of the semantic entity; utilizing a relationship generation component to map the initial entity features to obtain relationship features of multiple semantic entities, wherein the relationship features are used to characterize the semantic relationships between multiple semantic entities; utilizing an information mining component to perform attention processing on relationship features and structural relationships to obtain global structural information; and fusing the initial entity features and the global structural information to obtain target entity features.

[0061] The above-mentioned feature extraction component may refer to a component that extracts features from semantic entities and obtains semantic information of semantic entities. The above-mentioned initial entity features may refer to feature vectors of semantic entities, which can be used to represent the semantic information of semantic entities. The above-mentioned relationship generation component may refer to a component used to preliminarily determine the association relationship between different semantic entities. The above-mentioned relationship features may refer to features that can reflect the association relationship between different semantic entities, such as semantic relationships. The above-mentioned information mining component may refer to a component used to determine the global structural information of a document based on the structural relationship of semantic entities, which can be used to mine the dependency relationship between different semantic entities. In order to improve the accuracy of the constructed target entity features, the above-mentioned document parsing model may include at least: a feature special zone component, a relationship generation component and an information mining component.

[0062] In an optional scheme of this embodiment, when the information extraction system constructs the target entity features based on the local entity representation of the semantic entity and the global structural information of the document, it can first use the above-mentioned feature extraction component to determine the local entity representation, that is, the feature extraction component can be used to extract features of the semantic entities in the document, and the feature extraction of the semantic entities can be performed to obtain the initial entity features corresponding to the semantic entities.

[0063] In an optional scheme of this embodiment, considering that by utilizing the initial entity features corresponding to the semantic entities, the semantic relationship between different semantic entities can be preliminarily determined, and the semantic relationship can preliminarily reflect the degree of association between different semantic entities, therefore, after obtaining the initial entity features corresponding to the semantic entities, the information extraction system can further utilize the above-mentioned relationship generation component to map the initial entity features corresponding to different semantic entities to obtain the relationship features between different semantic entities.

[0064] In an optional scheme of this embodiment, it is considered that the degree of association between different semantic entities is different. For example, the degree of association between "shoes" and "white" mentioned above is higher, the degree of association between "shoes" and "display stand" is second, and the degree of association between "shoes" and "black" is lower. The degree of association between different semantic entities may not be accurately expressed based on the structural relationship of the semantic entities alone, and the relationship features of the semantic entities determined above can reflect the degree of association. Therefore, when the information extraction system determines the global structural information of the document based on the structural information of the semantic entities, it can determine the above-mentioned global structural information based on the structural relationships of different semantic entities and the relationship features between different semantic entities. That is, the above-mentioned information mining component can be used to process the relationship features and structural relationships of the semantic entities to obtain the global structural information of the document, thereby ensuring the accuracy of the global structural information. Furthermore, considering that the computational complexity of the dependency relationships between different semantic entities is relatively large, for example, if a document contains N semantic entities, the number of relationship features and structural relationships between different semantic entities will be N. 2 The corresponding computational complexity is O(N 4 ), therefore, the self-attention mechanism can be used to process the above-mentioned relationship features and structural relationships, reducing the computational complexity to O(N 2 ), thereby improving the efficiency of obtaining global structural information.

[0065] After obtaining the global structural information of the document, the information extraction system can fuse the above-mentioned initial entity features with the global structural information to obtain the above-mentioned target entity features.

[0066] In an embodiment of the present disclosure, a feature extraction component includes: a pre-trained document model and two different linear layers. The feature extraction component is used to extract features of a semantic entity to obtain initial entity features of the semantic entity, including: encoding the semantic entity using the pre-trained document model to obtain an encoding vector of the semantic entity; using two different linear layers to process the encoding vector respectively to generate an initial key feature and an initial value feature of the semantic entity, wherein the initial key feature is used to characterize the entity at the head of the semantic entity, and the initial value feature is used to characterize the entity at the tail of the semantic entity; based on the initial key feature and the initial value feature, the initial entity feature is obtained.

[0067] The pre-trained document model may be a model used to encode semantic entities and obtain vectors corresponding to the semantic entities, such as LayoutLM, XYLayoutLM, or other models. The initial key feature may refer to the entity feature of the entity at the head of a semantic entity, and the initial value feature may refer to the entity feature of the entity at the tail of a semantic entity. By determining the degree of match between the initial key features and initial value features of different semantic entities, it is possible to preliminarily determine whether there is an association relationship between the different semantic entities.

[0068] In an optional solution of the present embodiment, in order to ensure the accuracy of the obtained initial entity features, the information extraction system can first use the above-mentioned pre-trained document model to encode the semantic entity to obtain the above-mentioned encoding vector, and then use two different linear layers to process the encoding vector, that is, input the above-mentioned encoding vector into two different linear layers to generate the initial key features and initial value features of the semantic entity, and then combine the initial key features and initial value features to obtain the above-mentioned initial entity features. In an optional solution of the present embodiment, the above-mentioned pre-trained document model is plug-and-play in the feature extraction component, and the user can select any pre-trained document model at any time according to needs to complete the purpose of encoding the semantic entity, and will not generate additional computational burden for the process of determining the target entity features.

[0069] In an embodiment of the present disclosure, a relationship generation component includes: a relationship determination component and a relationship generation component. The relationship generation component is used to map initial entity features to obtain relationship features of multiple semantic entities, including: using the relationship determination component to determine the semantic relationship between semantic entities based on the initial entity features; using the relationship generation component to generate relationship features based on the semantic relationship and the initial entity features.

[0070] In an optional scheme of this embodiment, the relationship generation component may include at least: a relationship determination component and a relationship generation component, wherein the relationship determination component can be configured to determine the semantic relationship between different semantic entities, which may refer to a classifier, and the relationship generation component can be configured to generate relationship features between different semantic entities, which may refer to a forward network.

[0071] When using the relationship generation component to map the initial entity features, you can first use the above-mentioned relationship determination component to map the initial entity features of different semantic entities to obtain the semantic relationship between different semantic entities, and then use the above-mentioned relationship generation component to generate the above-mentioned relationship features based on the obtained semantic relationship and the above-mentioned initial entity features.

[0072] In an embodiment of the present disclosure, a relationship determination component is used to determine a semantic relationship between semantic entities based on initial entity features, including: reading an initial key feature of the first semantic entity from the initial entity features of the first semantic entity, and reading an initial value feature of the second semantic entity from the initial entity features of the second semantic entity, wherein the first semantic entity and the second semantic entity are used to represent any two semantic entities among a plurality of semantic entities; and determining a semantic relationship based on the similarity between the initial key feature and the initial value feature.

[0073] In an optional scheme of this embodiment, as shown above, considering the degree of matching between the initial key features and initial value features of different semantic entities, it can be determined whether there is an association relationship between different semantic entities, such as the above-mentioned semantic relationship. Therefore, when using the relationship determination component to process the initial semantic entity, you can first screen out any two semantic entities from multiple semantic entities, namely the above-mentioned first semantic entity and the second semantic entity, and then read the initial key feature of the first semantic entity from the initial entity feature corresponding to the first semantic entity, and read the initial value feature of the second semantic entity from the initial entity feature corresponding to the second semantic entity, and then match the initial key feature and the initial value feature to determine the similarity between the two, and determine the semantic relationship between the first semantic entity and the second semantic entity based on the similarity.

[0074] In an embodiment of the present disclosure, an information mining component includes: a local attention layer, a global interaction layer, and a pooling layer connected to the local attention layer and the global interaction layer. The information mining component is used to perform attention processing on relationship features and structural relationships to obtain global structural information, including: using the local attention layer to perform self-attention processing on the relationship features to obtain a first attention feature; using the global interaction layer to perform cross-attention processing on the relationship features and structural relationships to obtain a second attention feature; superimposing the first attention feature and the second attention feature to obtain a superimposed feature; and pooling the superimposed feature to obtain global structural information.

[0075] In an optional solution of this embodiment, the information mining component may include at least: a local attention layer, a global interaction layer, and a pooling layer, wherein the pooling layer is connected to the local attention layer and the global interaction layer. When using the information mining component to perform attention processing on the relationship features and the structural relationship, the local attention layer can be used to perform self-attention processing on the relationship features to obtain the above-mentioned first attention feature, and at the same time, the global interaction layer can be used to perform poor attention processing on the relationship features and the structural relationship to obtain the above-mentioned second attention feature. Then, the first attention feature and the second attention feature are superimposed to obtain a superimposed feature. Finally, the superimposed feature is input into the pooling layer for processing to obtain the above-mentioned global structural information.

[0076] In an embodiment of the present disclosure, the initial entity features and the global structural information are fused to obtain the target entity features, including: fusing the initial entity features and the global structural information based on a gating mechanism to obtain the target entity features.

[0077] In an optional solution of this embodiment, after obtaining the initial entity features and global structure information, the initial entity features and global structure information can be fused to obtain the target entity features. To ensure the accuracy of the obtained target entity features, a gating mechanism can be used to perform the fusion process.

[0078] In an optional scheme of this embodiment, as shown above, considering that when a user writes a document, the writing results of different semantic entities may not conform to the user's writing habits, for example, there is a specified writing format for some data, and the writing format may not conform to the user's writing habits, then the structural relationship of the obtained semantic entity may be inaccurate, resulting in inaccurate global structural information, and then resulting in errors in the constructed target entity features. Therefore, after obtaining the target entity features, the information extraction system can further re-input the target entity features into the aforementioned relationship generation component to re-determine the above-mentioned relationship features, and then generate new target entity features based on the new relationship features and structural relationships. After repeating this process for multiple times, by improving the accuracy of the generated relationship features to reduce the errors caused by inaccurate structural relationships, a target entity feature with a smaller error can be obtained, thereby improving the accuracy of the obtained target entity features.

[0079] In an embodiment of the present disclosure, when the document parsing model includes: multiple relationship generation components and multiple information mining components, the number of multiple relationship generation components is the same as the number of multiple information mining components, and the multiple relationship generation components and the multiple information mining components are iteratively run in sequence.

[0080] In an optional scheme of this embodiment, in order to ensure the accuracy of the target entity features obtained using the document parsing model, in addition to re-inputting the generated target entity features into the relationship generation component, multiple relationship generation components and multiple information mining components can also be set in the document parsing model, and the number of relationship generation components and information mining components must be the same, and the iteration order of the two is also the same, that is, first use the relationship generation component, and then use the information mining component to continuously mine the global structural information of the semantic entity, and gradually enhance the local entity representation of the semantic entity, thereby improving the accuracy of the constructed target entity features.

[0081] In an embodiment of the present disclosure, the method further includes: performing optical recognition on the document to determine the spatial information of the bounding box to which the semantic entity belongs in the document; determining the distance and angle between the semantic entities based on the spatial information; and generating a structural relationship based on the distance and angle.

[0082] In an optional solution of this embodiment, when determining the structural relationship of semantic entities, the information extraction system may first perform optical recognition on the document, using OCR (Optical Character Recognition) technology to determine the spatial information of the bounding box to which the semantic entities in the document belong, such as the position of the vertices of the bounding box in the document, the area of ​​the bounding box, and other information. Based on this spatial information, the distance and angle between different semantic entities are then determined. Based on this distance and angle, the structural relationship between different semantic entities can be determined. The distance may refer to the Euclidean distance.

[0083] In an embodiment of the present disclosure, spatial information includes: subspace information of multiple endpoints, and based on the spatial information, the distance and angle between semantic entities are determined, including: based on the subspace information of the same endpoint among the multiple endpoints, multiple sub-distances and multiple sub-angles between semantic entities are determined; multiple sub-distances are spliced ​​to obtain the distance between semantic entities; multiple sub-angles are spliced ​​to obtain the angle between semantic entities.

[0084] In an optional solution of this embodiment, the spatial information of the bounding box to which the semantic entity belongs in the document may include at least: subspace information of multiple endpoints on the bounding box, such as endpoint coordinates, the number of endpoints, and the position of the endpoints within the bounding box. When determining the distance and angle between semantic entities, the same endpoints among the multiple endpoints of the bounding boxes to which different semantic entities belong (e.g., endpoints with the same label and position within the bounding box) may be first determined to obtain multiple sub-distances and multiple sub-angles between the semantic entities. The multiple sub-distances may then be concatenated to obtain the distances between the different semantic entities, and the multiple sub-angles may be concatenated to obtain the angles between the different semantic entities.

[0085] In an optional solution of this embodiment, the above-mentioned multiple endpoints may refer to the upper endpoint, the lower right endpoint and the center point of the bounding box.

[0086] In an embodiment of the present disclosure, the method also includes: outputting documents and information extraction results in an interactive interface; responding to modification instructions acting on the interactive interface, determining target extraction results corresponding to the modification instructions; and updating model parameters of the document processing model based on the documents and target extraction results.

[0087] In an optional solution of this embodiment, in order to facilitate users to view the process and results of information extraction from documents, the information extraction system can also output the document and the corresponding information extraction results in a preset interactive interface. The user can determine whether the current document processing model meets the information extraction requirements based on the information extraction results. If not, the user can adjust the document processing model based on the information extraction results, that is, the user can enter a modification instruction to modify the information extraction results in the interactive interface. The information extraction system can adjust the information extraction results according to the modification instruction to obtain a target extraction result that is more consistent with the content contained in the document, and then use the document and the target extraction result to update the model parameters of the document processing model to provide the accuracy of the document processing model.

[0088] In an embodiment of the present disclosure, information extraction is performed on a document based on target entity features to obtain information extraction results of the document, including: using a document recognition component to recognize the target entity features to obtain a recognition result set of semantic entities; using a normalization layer to determine the probability of the recognition results contained in the recognition result set; and based on the probability of the recognition results, determining the information extraction results from the recognition result set.

[0089] The document recognition component mentioned above may be referred to as a classifier.

[0090] In an optional scheme of this embodiment, when extracting information from a document based on target entity features, the target entity features can be first identified using the above-mentioned document recognition component, and a recognition result set corresponding to the semantic entity can be determined from the document. The recognition result set can then be normalized using a normalization layer to determine the probability that the recognition result set contains a recognition result. Finally, based on the probability of the recognition result, the above-mentioned information extraction result can be determined from the recognition result set.

[0091] In order to facilitate understanding of the above process, Figure 4 is a schematic diagram of the information extraction process of a document according to Example 1 of the present disclosure. As shown in Figure 4, the document to be extracted can be input into the document processing model. After being processed by the document processing model, a target entity feature can be obtained. The information extraction system can extract information from the document based on the target entity feature. The document processing model can be divided into three components, wherein component one represents a feature extraction component, component two represents a relationship generation component, and component three represents an information mining component. Component one includes at least one pre-trained document model A and two different linear layers, linear layer A and linear layer B. In component one, the pre-trained document model A can be used to encode the semantic entities in the document to obtain an encoding vector a, and then the encoding vector a is input into the above two linear layers for processing to obtain the initial key feature b and initial value feature c of the semantic entity. The initial entity feature H of the semantic entity can be generated based on the initial key feature b and initial value feature c. Component 2 includes at least one relationship determination component B and a relationship generation component C. Component 2 can use the relationship determination component to process the initial key features and initial value features corresponding to different semantic entities to determine the semantic relationship S between the different semantic entities. For example, the initial key feature b1 of semantic entity 1 is matched with the initial value feature c2 of semantic entity 2 to determine the similarity between the two, and then determine the semantic relationship S based on the similarity. The relationship generation component is then used to process the semantic relationship S and the initial entity features H of the semantic entities to generate the relationship features R(t) corresponding to the semantic entities.

[0092] Among them, the formula of the relationship feature R(t) can be expressed as:

[0093] Component three includes at least a local attention layer D, a global interaction layer E, and a pooling layer F. In component three, the local attention layer D can first be used to perform self-attention processing on the relationship feature R(t) to obtain a first attention feature d. At the same time, the global interaction layer E can be used to perform cross-attention processing on the relationship feature R(t) and the structural relationship e of the semantic entity to obtain a second attention feature f. Then, the first attention feature d and the second attention feature f can be superimposed to obtain a superimposed feature m, and the pooling layer F can be used to pool the superimposed feature m to obtain the global structural information of the semantic entity.

[0094] Finally, the aforementioned initial entity features and the global structure information are fused according to the gating mechanism to obtain the target entity features H'.

[0095] Among them, the formula of the target entity feature H' can be expressed as:

[0096] Among them, g is the control parameter corresponding to the gating mechanism, Represents the fused target entity feature H', represents the initial entity feature H, It represents the global structural information output by the above pooling layer F.

[0097] As shown in FIG4 , in order to improve the accuracy of the determined target entity features, the initially determined target entity features can be used as input, and component 2 can be reused to determine new relationship features, and then the above process can be repeated to perform multiple iterative processing on the target entity features.

[0098] After obtaining the target entity features, the classifier in component 2 can be further used to process the target entity features, determine a set of recognition results from the document, and use the softmax function corresponding to the normalization layer to determine the information extraction results from the recognition result set.

[0099] Figure 5 is a schematic diagram of a process for determining the structural relationship of a semantic entity according to Example 1 of the present disclosure. As shown in Figure 5, when determining the structural relationship between semantic entities, semantic entity 1 and semantic entity 2 can be first subjected to OCR recognition to determine the spatial information of the bounding box to which the semantic entity belongs, and then the subspace information of multiple endpoints on the bounding box can be determined from the spatial information. Based on the subspace information, multiple distances and multiple angles between the multiple endpoints can be determined. By splicing these multiple distances, the target distance between the semantic entities can be obtained. By splicing multiple angles, the target angle between the semantic entities can be obtained. Through the target distance and target angle, the structural relationship between the semantic entities can be determined.

[0100] The coordinates of the upper endpoint of the semantic entity 1 bounding box are (x i ,y i ), the coordinates of the upper endpoint of the semantic entity 2 bounding box are (x j ,y j ) as an example, the formula for the target distance between semantic entity 1 and semantic entity 2 can be:

[0101] The formula for the target angle between semantic entity 1 and semantic entity 2 can be:

[0102] The corresponding structural relationship formula can be expressed as:

[0103] g i,j =[W θ θ(i,j);W d d(i,j)].

[0104] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and components involved are not necessarily required by the present disclosure.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present disclosure.

[0106] Example 2

[0107] According to an embodiment of the present disclosure, a method for extracting information from a document is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0108] FIG6 is a flow chart of a method for extracting information from a document according to Embodiment 2 of the present disclosure. As shown in FIG6 , the method may include the following steps:

[0109] Step S602: responding to a document received in the interactive interface, wherein the document includes a plurality of semantic entities.

[0110] In an optional solution of this embodiment, the information extraction system may first obtain a document that needs to be understood by the user, where the document contains multiple semantic entities and may be a visual document.

[0111] Step S604: constructing target entity features of the semantic entities based on the semantic information of the semantic entities and the structural relationships between the semantic entities.

[0112] In an optional scheme of this embodiment, the information extraction system can introduce the global structural information in the document during the process of extracting information from the document, and combine the local entity representation and the global structural information in the document, that is, according to the semantic information of different semantic entities and the structural relationship between different semantic entities, construct the above-mentioned target entity features, and extract information from the document based on the target entity features.

[0113] Step S606: extract information from the document based on the target entity features to obtain an information extraction result of the document.

[0114] After constructing the target entity features corresponding to the semantic entities in the document, the information extraction system can extract information from the document based on the target entity features, thereby ensuring the accuracy of the information extraction results.

[0115] Step S608: Display the information extraction result in the interactive interface.

[0116] After extracting key information from the document, that is, obtaining the information extraction results, the information extraction system can display the information extraction results in a preset interactive interface to facilitate user reference.

[0117] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0118] Example 3

[0119] According to an embodiment of the present disclosure, a method for extracting information from a document is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0120] FIG7 is a flow chart of a method for extracting information from a document according to Embodiment 3 of the present disclosure. As shown in FIG7 , the method may include the following steps:

[0121] Step S702: Acquire the document by calling the first interface.

[0122] The first interface includes a first parameter, a parameter value of the first parameter is a document, and the document includes multiple semantic entities.

[0123] In an optional solution of this embodiment, when extracting information from a document, the information extraction system may use the above-mentioned first interface to call the first parameter, that is, to call the acquired document, which contains multiple semantic entities and may be a visual document.

[0124] Step S704: constructing target entity features of the semantic entities based on the semantic information of the semantic entities and the structural relationships between the semantic entities.

[0125] In an optional scheme of this embodiment, the information extraction system can introduce the global structural information in the document during the process of extracting information from the document, and combine the local entity representation and the global structural information in the document, that is, according to the semantic information of different semantic entities and the structural relationship between different semantic entities, construct the above-mentioned target entity features, and extract information from the document based on the target entity features.

[0126] Step S706: extract information from the document based on the target entity features to obtain an information extraction result of the document.

[0127] After constructing the target entity features corresponding to the semantic entities in the document, the information extraction system can extract information from the document based on the target entity features, thereby ensuring the accuracy of the information extraction results.

[0128] Step S708: Output the information extraction result by calling the second interface.

[0129] The second interface includes a second parameter, and the parameter value of the second parameter is the information extraction result.

[0130] In an optional solution of this embodiment, after obtaining the information extraction result, the information extraction system can call the second parameter of the second interface, that is, the above-mentioned information extraction result, and the user can view the called information extraction result according to needs.

[0131] In an optional solution of this embodiment, the determined risk index can be output by calling the second interface, so that the staff can more intuitively monitor the risk index corresponding to the area.

[0132] Example 4

[0133] According to an embodiment of the present disclosure, a device for implementing the document information extraction method described above is also provided. The device can be deployed in a target client. Figure 8 is a block diagram of a document information extraction device according to embodiment 4 of the present disclosure. As shown in Figure 8, the device 800 includes: a document acquisition component 802, a feature construction component 804, and an information extraction component 806.

[0134] Among them, the document acquisition component 802 is configured to acquire a document, wherein the document contains multiple semantic entities; the feature construction component 804 is configured to construct the target entity feature of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities; the information extraction component 806 is configured to extract information from the document based on the target entity feature to obtain the information extraction result of the document.

[0135] In the above embodiment of the present disclosure, the feature construction component 804 includes: a feature construction component configured to construct target entity features based on semantic information and structural relationships using a document processing model.

[0136] In the above embodiment of the present disclosure, the document parsing model includes: a feature extraction component, a relationship generation component and an information mining component, and the feature construction component is further configured to: use the feature extraction component to extract features of the semantic entity to obtain the initial entity features of the semantic entity, wherein the initial entity features are used to characterize the semantic information of the semantic entity; use the relationship generation component to map the initial entity features to obtain the relationship features of multiple semantic entities, wherein the relationship features are used to characterize the semantic relationships between multiple semantic entities; use the information mining component to perform attention processing on the relationship features and structural relationships to obtain global structural information; and fuse the initial entity features and the global structural information to obtain the target entity features.

[0137] In the above embodiment of the present disclosure, the feature extraction component includes: a pre-trained document model and two different linear layers, and the feature construction component is further configured to: encode the semantic entity using the pre-trained document model to obtain the encoding vector of the semantic entity; use two different linear layers to process the encoding vector respectively to generate the initial key feature and initial value feature of the semantic entity, wherein the initial key feature is used to represent the entity at the head of the semantic entity, and the initial value feature is used to represent the entity at the tail of the semantic entity; based on the initial key feature and the initial value feature, the initial entity feature is obtained.

[0138] In the above-mentioned embodiment of the present disclosure, the relationship generation component includes: a relationship determination component and a relationship generation component, and the feature construction component is further configured to: use the relationship determination component to determine the semantic relationship between semantic entities based on the initial entity features; use the relationship generation component to generate relationship features based on the semantic relationship and the initial entity features.

[0139] In the above embodiment of the present disclosure, the information mining component includes: a local attention layer, a global interaction layer, and a pooling layer connected to the local attention layer and the global interaction layer. The feature construction component is also configured to: use the local attention layer to perform self-attention processing on the relationship features to obtain a first attention feature; use the global interaction layer to perform cross-attention processing on the relationship features and the structural relationship to obtain a second attention feature; superimpose the first attention feature and the second attention feature to obtain a superimposed feature; and pool the superimposed feature to obtain global structural information.

[0140] In the above embodiment of the present disclosure, the feature construction component is further configured to: fuse the initial entity features and the global structure information based on a gating mechanism to obtain the target entity features.

[0141] In the above embodiment of the present disclosure, when the document parsing model includes: multiple relationship generation components and multiple information mining components, the number of multiple relationship generation components is the same as the number of multiple information mining components, and the multiple relationship generation components and the multiple information mining components are iteratively run in sequence.

[0142] In the above-mentioned embodiment of the present disclosure, the device also includes: an information recognition component, configured to perform optical recognition on the document and determine the spatial information of the bounding box to which the semantic entity belongs in the document; a parameter determination component, configured to determine the distance and angle between the semantic entities based on the spatial information; and a relationship generation component, configured to generate a structural relationship based on the distance and angle.

[0143] In the above-mentioned embodiment of the present disclosure, the spatial information includes: subspace information of multiple endpoints, and the parameter determination component includes: a sub-parameter determination component, which is configured to determine multiple sub-distances and multiple sub-angles between semantic entities based on the subspace information of the same endpoint among the multiple endpoints; a distance determination component, which is configured to splice multiple sub-distances to obtain the distance between the semantic entities; and an angle determination component, which is configured to splice multiple sub-angles to obtain the angle between the semantic entities.

[0144] In the above-mentioned embodiment of the present disclosure, the device also includes: an information output component, configured to output documents and information extraction results in an interactive interface; a structure modification component, configured to respond to modification instructions acting on the interactive interface and determine the target extraction results corresponding to the modification instructions; a parameter update component, configured to update the model parameters of the document processing model based on the documents and target extraction results.

[0145] In the above embodiment of the present disclosure, the information extraction component 806 includes: a feature recognition component, which is configured to use the document recognition component to recognize the target entity features and obtain a recognition result set of the semantic entity; a probability determination component, which is configured to use the normalization layer to determine the probability of the recognition results contained in the recognition result set; and a result determination component, which is configured to determine the information extraction result from the recognition result set based on the probability of the recognition result.

[0146] It should be noted that the document acquisition component 802, feature construction component 804, and information extraction component 806 described above correspond to steps S302 to S306 in Example 1. The three components and corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Example 1. It should be noted that the above components or assemblies can be hardware components or software components stored in a memory and processed by one or more processors. The above components can also be run as part of the device in the AR / VR device provided in Example 1.

[0147] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0148] Example 5

[0149] According to an embodiment of the present disclosure, a device configured to implement the above-mentioned document information extraction is also provided, which can be deployed in a target client. Figure 9 is a structural block diagram of a document information extraction device according to embodiment 5 of the present disclosure. As shown in Figure 9, the device 900 includes: a document receiving component 902, a feature construction component 904, an information extraction component 906, and a result display component 908.

[0150] Among them, the document receiving component 902 is configured to respond to the document received in the interactive interface, wherein the document contains multiple semantic entities; the feature construction component 904 is configured to construct the target entity feature of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities; the information extraction component 906 is configured to extract information from the document based on the target entity feature to obtain the information extraction result of the document; the result display component 908 is configured to display the information extraction result in the interactive interface.

[0151] It should be noted that the document receiving component 902, feature building component 904, information extraction component 906, and result display component 908 described above correspond to steps S602 to S608 in Example 2. The four components and corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Example 1. It should be noted that the above-mentioned components or assemblies can be hardware components or software components stored in a memory and processed by one or more processors. The above-mentioned components can also be run as part of the device in the AR / VR device provided in Example 1.

[0152] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0153] Example 6

[0154] According to an embodiment of the present disclosure, a device configured to implement the document information extraction method described above is also provided. This device can be deployed in a target client. Figure 10 is a block diagram of a document information extraction device according to Embodiment 6 of the present disclosure. As shown in Figure 10, the device 1000 includes: a first calling component 1002, a feature construction component 1004, an information extraction component 1006, and a second calling component 1008.

[0155] Among them, the first calling component 1002 is configured to obtain a document by calling a first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter is a document, and the document contains multiple semantic entities; the feature construction component 1004 is configured to construct the target entity feature of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities; the information extraction component 1006 is configured to extract information from the document based on the target entity feature to obtain the information extraction result of the document; the second calling component 1008 is configured to output the information extraction result by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter is the information extraction result.

[0156] It should be noted that the first calling component 1002, feature construction component 1004, information extraction component 1006, and second calling component 1008 described above correspond to steps S702 to S708 in Example 3. The four components and corresponding steps implement the same instances and application scenarios, but are not limited to those disclosed in Example 1. It should be noted that the above-mentioned components or assemblies may be hardware components or software components stored in a memory and processed by one or more processors. The above-mentioned components may also be run as part of a device in the AR / VR device provided in Example 1.

[0157] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0158] Example 7

[0159] According to an embodiment of the present disclosure, a system configured to implement the document information extraction method described above is also provided. This system can be deployed on a target client. Figure 11 is a block diagram of a document information extraction system according to Embodiment 7 of the present disclosure. As shown in Figure 11, the system 1100 includes a receiving component 1102, a processing device 1104, and a sending component 1106.

[0160] Among them, the receiving component 1102 is configured to display an interactive interface and capture a document input in the interactive interface, wherein the document contains multiple semantic entities; the processing device 1104 is configured to construct target entity features of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities, and extract information from the document based on the target entity features to obtain an information extraction result of the document; the sending component 1106 is configured to send the information extraction result to the front-end client, wherein the information extraction result is displayed in the interactive interface by the front-end client.

[0161] It should be noted that the preferred implementation scheme involved in the above embodiments of the present disclosure is the same as the solution provided in Example 1, as well as the application scenario and implementation process, but is not limited to the solution provided in Example 1.

[0162] Example 8

[0163] The embodiment of the present disclosure may provide an electronic device, which may be any electronic device in a group of electronic devices. Optionally, in this embodiment, the electronic device may also be replaced by a terminal device such as a mobile terminal.

[0164] Optionally, in this embodiment, the electronic device may be located in at least one grid device among a plurality of grid devices of a computer grid.

[0165] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the document information extraction method: obtaining a document, wherein the document contains multiple semantic entities; constructing the target entity features of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities; extracting information from the document based on the target entity features to obtain the information extraction results of the document.

[0166] Optionally, Figure 12 is a block diagram of an electronic device according to Embodiment 8 of the present disclosure. As shown in the figure, the electronic device A may include: a processor 1202 and a memory 1204, wherein an executable program is stored; the processor is configured to run the program, wherein when the program is run, the document information extraction method described in Embodiment 1 is executed.

[0167] Optionally, as shown in FIG12 , the electronic device A may further include: a storage controller and a peripheral interface, wherein the peripheral interface is connected to the radio frequency component, the audio component and the display.

[0168] The memory can be used to store software programs and components, such as program instructions / components corresponding to the image processing method and device in the embodiments of the present disclosure. The processor executes the software programs and components stored in the memory to perform various functional applications and data processing, that is, to implement the above-mentioned image segmentation method. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the terminal A via a grid. Examples of the above-mentioned grid include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0169] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: construct the target entity features of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities, including: using the document processing model to construct the target entity features based on the semantic information and structural relationship.

[0170] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: the document parsing model includes: a feature extraction component, a relationship generation component and an information mining component, and uses the document processing model to construct target entity features based on semantic information and structural relationships, including: using the feature extraction component to extract features of the semantic entity to obtain the initial entity features of the semantic entity, wherein the initial entity features are used to represent the semantic information of the semantic entity; using the relationship generation component to map the initial entity features to obtain relationship features of multiple semantic entities, wherein the relationship features are used to represent the semantic relationships between multiple semantic entities; using the information mining component to pay attention to the relationship features and structural relationships to obtain global structural information; fusing the initial entity features and the global structural information to obtain the target entity features.

[0171] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: the feature extraction component includes: a pre-trained document model and two different linear layers, and the feature extraction component is used to extract features of the semantic entity to obtain the initial entity features of the semantic entity, including: using the pre-trained document model to encode the semantic entity to obtain the encoding vector of the semantic entity; using two different linear layers to process the encoding vector respectively to generate the initial key feature and initial value feature of the semantic entity, wherein the initial key feature is used to represent the entity at the head of the semantic entity, and the initial value feature is used to represent the entity at the tail of the semantic entity; based on the initial key feature and the initial value feature, the initial entity feature is obtained.

[0172] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: the relationship generation component includes: a relationship determination component and a relationship generation component, and the relationship generation component is used to map the initial entity features to obtain relationship features of multiple semantic entities, including: using the relationship determination component to determine the semantic relationship between semantic entities based on the initial entity features; using the relationship generation component to generate relationship features based on the semantic relationship and the initial entity features.

[0173] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: using the relationship determination component to determine the semantic relationship between semantic entities based on the initial entity features, including: reading the initial key features of the first semantic entity from the initial entity features of the first semantic entity, and reading the initial value features of the second semantic entity from the initial entity features of the second semantic entity, wherein the first semantic entity and the second semantic entity are used to represent any two semantic entities among multiple semantic entities; determining the semantic relationship based on the similarity between the initial key features and the initial value features.

[0174] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: the information mining component includes: a local attention layer, a global interaction layer, and a pooling layer connected to the local attention layer and the global interaction layer. The information mining component is used to perform attention processing on the relationship features and structural relationships to obtain global structural information, including: using the local attention layer to perform self-attention processing on the relationship features to obtain a first attention feature; using the global interaction layer to perform cross-attention processing on the relationship features and structural relationships to obtain a second attention feature; superimposing the first attention feature and the second attention feature to obtain a superimposed feature; and pooling the superimposed feature to obtain global structural information.

[0175] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: fusing the initial entity features and the global structure information to obtain the target entity features, including: fusing the initial entity features and the global structure information based on the gating mechanism to obtain the target entity features.

[0176] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: when the document parsing model includes: multiple relationship generation components and multiple information mining components, the number of multiple relationship generation components is the same as the number of multiple information mining components, and the multiple relationship generation components and the multiple information mining components are iteratively run in sequence.

[0177] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: The method also includes: optically identifying the document to determine the spatial information of the bounding box of the semantic entity in the document; based on the spatial information, determining the distance and angle between the semantic entities; and generating a structural relationship based on the distance and angle.

[0178] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: the spatial information includes: subspace information of multiple endpoints, and based on the spatial information, the distance and angle between semantic entities are determined, including: based on the subspace information of the same endpoint among the multiple endpoints, multiple sub-distances and multiple sub-angles between semantic entities are determined; multiple sub-distances are spliced ​​to obtain the distance between semantic entities; multiple sub-angles are spliced ​​to obtain the angle between semantic entities.

[0179] The processor can call the information and application stored in the memory through the transmission device to execute the following steps: the method also includes: outputting documents and information extraction results in the interactive interface; responding to modification instructions acting on the interactive interface, determining the target extraction results corresponding to the modification instructions; based on the documents and target extraction results, updating the model parameters of the document processing model.

[0180] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: extract information from the document based on the target entity features to obtain the information extraction results of the document, including: using the document recognition component to recognize the target entity features to obtain a recognition result set of semantic entities; using the normalization layer to determine the probability of the recognition results contained in the recognition result set; based on the probability of the recognition results, determine the information extraction results from the recognition result set.

[0181] In an embodiment of the present disclosure, a method is adopted in which a document is obtained, wherein the document contains multiple semantic entities; a target entity feature of the semantic entity is constructed based on the semantic information of the semantic entity and the structural relationship between the semantic entities; information is extracted from the document based on the target entity feature to obtain an information extraction result of the document, by using the semantic information of the semantic entity as the local entity representation of the document, and constructing the global structural information of the document based on the structural relationship between the semantic entities, and then constructing a target entity feature containing the local entity representation and global structural information of the document based on the semantic information and structural information, so as to ensure that the target entity feature can be associated with the full text of the document and has high robustness, and finally information extraction is performed on the document based on the target entity feature, which can further ensure that the obtained information extraction result is consistent with the content of the document, improve the accuracy of the information extraction result, thereby achieving the technical effect of improving the accuracy of the result obtained by information extraction for the document, and further solving the technical problem of poor information extraction effect for the document in the related art.

[0182] Those skilled in the art will appreciate that the structure shown in FIG12 is merely illustrative, and that the computer terminal may also be a smartphone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. FIG12 does not limit the structure of the aforementioned electronic devices. For example, the computer terminal A may include more or fewer components (e.g., a grid interface, a display device, etc.) than those shown in FIG12 , or may have a configuration different from that shown in FIG12 .

[0183] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0184] Example 10

[0185] The embodiment of the present disclosure further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the document information extraction method provided in the above embodiment 1.

[0186] In this embodiment, the above-mentioned computer-readable storage medium can be located in any computer terminal in the AR / VR device terminal group in the AR / VR device network, or in any mobile terminal in the mobile terminal group.

[0187] In an embodiment of the present disclosure, a storage medium is configured to store program code for executing the following steps: obtaining a document, wherein the document contains multiple semantic entities; constructing target entity features of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities; and extracting information from the document based on the target entity features to obtain information extraction results of the document.

[0188] In an embodiment of the present disclosure, the storage medium is configured to store program code for performing the following steps: constructing target entity features of a semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities, including: constructing target entity features based on the semantic information and the structural relationship using a document processing model.

[0189] In an embodiment of the present disclosure, a storage medium is configured to store program code for executing the following steps: a document parsing model includes: a feature extraction component, a relationship generation component, and an information mining component, and a document processing model is used to construct target entity features based on semantic information and structural relationships, including: using the feature extraction component to extract features of the semantic entity to obtain initial entity features of the semantic entity, wherein the initial entity features are configured to represent the semantic information of the semantic entity; using the relationship generation component to map the initial entity features to obtain relationship features of multiple semantic entities, wherein the relationship features are used to represent the semantic relationships between multiple semantic entities; using the information mining component to perform attention processing on the relationship features and structural relationships to obtain global structural information; and fusing the initial entity features and the global structural information to obtain target entity features.

[0190] In an embodiment of the present disclosure, a storage medium is configured to store program code for performing the following steps: a feature extraction component includes: a pre-trained document model and two different linear layers, and the feature extraction component is used to extract features of a semantic entity to obtain initial entity features of the semantic entity, including: encoding the semantic entity using the pre-trained document model to obtain an encoding vector of the semantic entity; using two different linear layers to process the encoding vector respectively to generate an initial key feature and an initial value feature of the semantic entity, wherein the initial key feature is configured to represent an entity at the head of the semantic entity, and the initial value feature is used to represent an entity at the tail of the semantic entity; based on the initial key feature and the initial value feature, an initial entity feature is obtained.

[0191] In an embodiment of the present disclosure, a storage medium is configured to store program code for executing the following steps: a relationship generation component includes: a relationship determination component and a relationship generation component, and the relationship generation component is used to map initial entity features to obtain relationship features of multiple semantic entities, including: using the relationship determination component to determine the semantic relationship between semantic entities based on the initial entity features; using the relationship generation component to generate relationship features based on the semantic relationship and the initial entity features.

[0192] In an embodiment of the present disclosure, a storage medium is configured to store program code for performing the following steps: determining a semantic relationship between semantic entities based on initial entity features using a relationship determination component, including: reading an initial key feature of the first semantic entity from the initial entity features of the first semantic entity, and reading an initial value feature of the second semantic entity from the initial entity features of the second semantic entity, wherein the first semantic entity and the second semantic entity are used to represent any two semantic entities among a plurality of semantic entities; and determining a semantic relationship based on the similarity between the initial key feature and the initial value feature.

[0193] In an embodiment of the present disclosure, a storage medium is configured to store program code for executing the following steps: an information mining component includes: a local attention layer, a global interaction layer, and a pooling layer connected to the local attention layer and the global interaction layer, and the information mining component is used to perform attention processing on relationship features and structural relationships to obtain global structural information, including: using the local attention layer to perform self-attention processing on the relationship features to obtain a first attention feature; using the global interaction layer to perform cross-attention processing on the relationship features and structural relationships to obtain a second attention feature; superimposing the first attention feature and the second attention feature to obtain a superimposed feature; and pooling the superimposed feature to obtain global structural information.

[0194] In an embodiment of the present disclosure, the storage medium is configured to store program code for executing the following steps: fusing the initial entity features and the global structural information to obtain the target entity features, including: fusing the initial entity features and the global structural information based on a gating mechanism to obtain the target entity features.

[0195] In an embodiment of the present disclosure, the storage medium is configured to store program code for executing the following steps: when the document parsing model includes: multiple relationship generation components and multiple information mining components, the number of the multiple relationship generation components is the same as the number of the multiple information mining components, and the multiple relationship generation components and the multiple information mining components are iteratively run in sequence.

[0196] In an embodiment of the present disclosure, the storage medium is configured to store program code for executing the following steps: the method also includes: optically identifying the document to determine the spatial information of the bounding box to which the semantic entity belongs in the document; based on the spatial information, determining the distance and angle between the semantic entities; and generating a structural relationship based on the distance and angle.

[0197] In an embodiment of the present disclosure, a storage medium is configured to store program code for performing the following steps: spatial information includes: subspace information of multiple endpoints, and based on the spatial information, determining the distance and angle between semantic entities, including: determining multiple sub-distances and multiple sub-angles between semantic entities based on the subspace information of the same endpoint among the multiple endpoints; splicing the multiple sub-distances to obtain the distance between the semantic entities; splicing the multiple sub-angles to obtain the angle between the semantic entities.

[0198] In an embodiment of the present disclosure, the storage medium is configured to store program code for executing the following steps: the method also includes: outputting documents and information extraction results in an interactive interface; responding to modification instructions acting on the interactive interface, determining target extraction results corresponding to the modification instructions; and updating model parameters of the document processing model based on the documents and target extraction results.

[0199] In an embodiment of the present disclosure, a storage medium is configured to store program code for executing the following steps: extracting information from a document based on target entity features to obtain information extraction results of the document, including: identifying target entity features using a document recognition component to obtain a set of recognition results of semantic entities; determining the probability of recognition results contained in the recognition result set using a normalization layer; and determining an information extraction result from the recognition result set based on the probability of the recognition results.

[0200] In an embodiment of the present disclosure, a method is adopted in which a document is obtained, wherein the document contains multiple semantic entities; a target entity feature of the semantic entity is constructed based on the semantic information of the semantic entity and the structural relationship between the semantic entities; information is extracted from the document based on the target entity feature to obtain an information extraction result of the document, by using the semantic information of the semantic entity as the local entity representation of the document, and constructing the global structural information of the document based on the structural relationship between the semantic entities, and then constructing a target entity feature containing the local entity representation and global structural information of the document based on the semantic information and structural information, so as to ensure that the target entity feature can be associated with the full text of the document and has high robustness, and finally information extraction is performed on the document based on the target entity feature, which can further ensure that the obtained information extraction result is consistent with the content of the document, improve the accuracy of the information extraction result, thereby achieving the technical effect of improving the accuracy of the result obtained by information extraction for the document, and further solving the technical problem of poor information extraction effect for the document in the related art.

[0201] Example 11

[0202] An embodiment of the present disclosure further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned document information extraction method.

[0203] Example 12

[0204] An embodiment of the present disclosure further provides a computer program product, which includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned document information extraction method.

[0205] Example 13

[0206] An embodiment of the present disclosure further provides a computer program, wherein when the computer program is executed by a processor, the above-mentioned document information extraction method is implemented.

[0207] The serial numbers of the above-mentioned embodiments of the present disclosure are for description only and do not represent the advantages or disadvantages of the embodiments.

[0208] In the above embodiments of the present disclosure, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0209] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of components is only a logical function division. In actual implementation, there may be other division methods, such as multiple components or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of components or components can be electrical or other forms.

[0210] Components described as separate parts may or may not be physically separate, and components shown as components may or may not be physical components, that is, they may be located in one place or distributed across multiple network components. Some or all of these components may be selected based on actual needs to achieve the objectives of this embodiment.

[0211] In addition, the functional components in the various embodiments of the present disclosure may be integrated into a single processing component, each component may exist physically separately, or two or more components may be integrated into a single component. The aforementioned integrated components may be implemented in the form of hardware or software functional components.

[0212] If the integrated components are implemented in the form of software functional components and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0213] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as within the scope of protection of the present disclosure. Industrial Applicability

[0214] The solution provided by the embodiment of the present disclosure can be applied to the information extraction process of documents in a document information extraction platform. The solution can be obtained by obtaining a document, wherein the document contains multiple semantic entities; constructing a target entity feature of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities; extracting information from the document based on the target entity feature to obtain the information extraction result of the document, using the semantic information of the semantic entity as the local entity representation of the document, and constructing the global structural information of the document based on the structural relationship between the semantic entities, and then constructing the target entity feature containing the local entity representation and global structural information of the document based on the semantic information and structural information to ensure that the target entity feature can be associated with the full text of the document and has high robustness. Finally, information extraction is performed on the document based on the target entity feature, which can further ensure that the obtained information extraction result is consistent with the content of the document, improve the accuracy of the information extraction result, thereby achieving the technical effect of improving the accuracy of the result obtained from information extraction of the document, and thus solving the technical problem of poor information extraction effect of the document in the related art.

Claims

1. A method for extracting information from a document, comprising: Obtaining a document, wherein the document contains a plurality of semantic entities; constructing target entity features of the semantic entities based on the semantic information of the semantic entities and the structural relationships between the semantic entities; Information extraction is performed on the document based on the target entity features to obtain an information extraction result of the document.

2. The method according to claim 1, constructing a target entity feature of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities, comprising: The target entity features are constructed based on the semantic information and the structural relationship using a document processing model.

3. The method according to claim 2, wherein the document parsing model comprises: The feature extraction component, the relationship generation component, and the information mining component utilize a document processing model to construct the target entity features based on the semantic information and the structural relationship, including: Performing feature extraction on the semantic entity using the feature extraction component to obtain initial entity features of the semantic entity, wherein the initial entity features are used to represent the semantic information of the semantic entity; Mapping the initial entity features using the relationship generation component to obtain relationship features of the plurality of semantic entities, wherein the relationship features are used to represent the semantic relationships between the plurality of semantic entities; Using the information mining component to perform attention processing on the relationship features and the structural relationship to obtain global structural information; The initial entity features and the global structure information are fused to obtain the target entity features.

4. The method according to claim 3, wherein the feature extraction component comprises: The pre-trained document model and two different linear layers perform feature extraction on the semantic entity using the feature extraction component to obtain initial entity features of the semantic entity, including: Encoding the semantic entity using the pre-trained document model to obtain an encoding vector of the semantic entity; Using the two different linear layers to process the encoding vector respectively, generating an initial key feature and an initial value feature of the semantic entity, wherein the initial key feature is used to represent the entity at the head of the semantic entity, and the initial value feature is used to represent the entity at the tail of the semantic entity; The initial entity feature is obtained based on the initial key feature and the initial value feature.

5. The method according to claim 3, wherein The relationship generation component includes: a relationship determination component and a relationship generation component. The relationship generation component is used to map the initial entity features to obtain relationship features of multiple semantic entities, including: Determining the semantic relationship between the semantic entities based on the initial entity features using the relationship determination component; The relationship feature is generated by using the relationship generation component based on the semantic relationship and the initial entity feature.

6. The method according to claim 5, wherein determining the semantic relationship between the semantic entities based on the initial entity features by using the relationship determination component comprises: Reading an initial key feature of the first semantic entity from the initial entity feature of the first semantic entity, and reading an initial value feature of the second semantic entity from the initial entity feature of the second semantic entity, wherein the first semantic entity and the second semantic entity are used to represent any two semantic entities among the plurality of semantic entities; The semantic relationship is determined based on a similarity between the initial key feature and the initial value feature.

7. The method according to claim 3, wherein the information mining component comprises: The local attention layer, the global interaction layer, and the pooling layer connected to the local attention layer and the global interaction layer use the information mining component to perform attention processing on the relationship features and the structural relationship to obtain global structural information, including: Performing self-attention processing on the relationship feature using the local attention layer to obtain a first attention feature; Using the global interaction layer to perform cross attention processing on the relationship feature and the structural relationship to obtain a second attention feature; superimposing the first attention feature and the second attention feature to obtain a superimposed feature; Pooling is performed on the superimposed features to obtain the global structure information.

8. The method according to claim 3, fusing the initial entity features with the global structure information to obtain the target entity features, comprising: The initial entity features and the global structure information are fused based on a gating mechanism to obtain the target entity features.

9. The method according to claim 3, wherein the document parsing model comprises: In the case of multiple relationship generation components and multiple information mining components, the number of the multiple relationship generation components is the same as the number of the multiple information mining components, and the multiple relationship generation components and the multiple information mining components are iteratively run in sequence.

10. The method according to claim 1, further comprising: Performing optical recognition on the document to determine spatial information of a bounding box to which the semantic entity belongs in the document; Determining distances and angles between the semantic entities based on the spatial information; The structural relationship is generated based on the distance and the angle.

11. The method according to claim 10, wherein the spatial information comprises: Subspace information of a plurality of endpoints, and determining distances and angles between the semantic entities based on the subspace information, including: determining a plurality of sub-distances and a plurality of sub-angles between the semantic entities based on subspace information of a same endpoint among the plurality of endpoints; concatenating the multiple sub-distances to obtain the distance between the semantic entities; The multiple sub-angles are spliced ​​together to obtain the angle between the semantic entities.

12. The method according to claim 2, further comprising: Outputting the document and the information extraction result in an interactive interface; In response to a modification instruction applied to the interactive interface, determining a target extraction result corresponding to the modification instruction; Based on the document and the target extraction result, the model parameters of the document processing model are updated.

13. The method according to claim 1, extracting information from the document based on the target entity features to obtain an information extraction result for the document, comprising: Using a document recognition component to identify the target entity features, and obtain a recognition result set of the semantic entity; Determining the probability of the recognition results included in the recognition result set using a normalization layer; The information extraction result is determined from the recognition result set based on the probability of the recognition result.

14. A method for extracting information from a document, comprising: Responding to a document received in the interactive interface, wherein the document includes a plurality of semantic entities; constructing target entity features of the semantic entities based on the semantic information of the semantic entities and the structural relationships between the semantic entities; Extracting information from the document based on the target entity features to obtain an information extraction result for the document; The information extraction result is displayed in the interactive interface.

15. A document information extraction system comprising: a receiving component configured to display an interactive interface and capture a document input in the interactive interface, wherein the document contains a plurality of semantic entities; a processing device configured to construct a target entity feature of the semantic entity based on the semantic information of the semantic entity and the structural relationship between the semantic entities, and extract information from the document based on the target entity feature to obtain an information extraction result of the document; The sending component is configured to send the information extraction result to the front-end client, wherein the information extraction result is displayed in the interactive interface by the front-end client.

16. An electronic device comprising: a memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 14 when running.

17. A computer-readable storage medium comprising a stored executable program, wherein: When the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 14.

18. A computer program product, wherein The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 14.

19. A computer program product, wherein The invention comprises a non-volatile computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 14 when executed by a processor.

20. A computer program, wherein When the computer program is executed by a processor, the method according to any one of claims 1 to 14 is implemented.

Images