Training Document Information Extraction Model, Method and Device for Document Information Extraction

Abstract

The present disclosure provides a method and apparatus for training a document information extraction model and document information extraction, relating to the field of artificial intelligence, and particularly to the field of natural language processing. The specific implementation solution is as follows: obtaining training data and a document information extraction model annotated with answers corresponding to preset questions, wherein the training data includes layout document training data and streaming document training data; extracting at least one feature from the training data; obtaining a fused feature after fusing the at least one feature; inputting the preset question, the fused feature, and the training data into the document information extraction model to obtain a prediction result; and adjusting network parameters of the document information extraction model based on the prediction result and the answer. This implementation manner can integrate the streaming document information extraction ability and the layout document information extraction ability into a model with a unified architecture, improve the generality of the model while maintaining the information extraction effect of the model, and reduce the privatization cost.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, particularly to the field of natural language processing, and specifically to a method, apparatus, and device for training a document information extraction model and extracting document information, as well as a storage medium and a computer program product. Background Art

[0002] In real user business scenarios, the cost of annotating text is often very high. Therefore, the zero-shot or few-shot learning ability of the model is very important, which determines whether the information extraction model can be widely used and deployed in multiple application scenarios of different vertical categories.

[0003] At the same time, the small amount of labeled data given by the user may include both streaming documents (*.doc, *.docx, *.Wps, *.Txt, *.excel, etc.) and layout documents (*.pdf, *.jpg, *.Jpeg, *.Png, *.Bmp, *.tif, etc.). In order to make the best use of the labeled data given by the user and fully train the model according to the user's needs, it is very necessary to integrate the information extraction capabilities of streaming documents and layout documents into a model with a unified architecture.

[0004] In the process of document information extraction in related technologies, the information extraction is disassembled into two stages, resulting in the accumulation of errors, high time consumption, and the inability to fully combine visual and semantic information. It cannot make full use of all training data, requires more annotations, and has a high labor cost. Summary of the Invention

[0005] The present disclosure provides a method, apparatus, device, equipment, storage medium, and computer program product for training a document information extraction model and extracting document information.

[0006] According to a first aspect of the present disclosure, a method for training a document information extraction model is provided, including: obtaining training data and a document information extraction model annotated with answers corresponding to preset questions, where the training data includes layout document training data and streaming document training data; extracting at least one feature from the training data; obtaining a fused feature after fusing the at least one feature; inputting the preset question, the fused feature, and the training data into the document information extraction model to obtain a prediction result; and adjusting the network parameters of the document information extraction model based on the prediction result and the answer.

[0007] According to a second aspect of the present disclosure, there is provided a method for extracting document information, including: obtaining the document information to be extracted; extracting at least one feature from the document information; fusing the at least one feature to obtain a fused feature; inputting a preset question, the fused feature, and the document information into a document information extraction model trained by the method according to any one of the first aspects to obtain an answer.

[0008] According to a third aspect of the present disclosure, there is provided an apparatus for training a document information extraction model, including: an obtaining unit configured to obtain training data labeled with answers corresponding to preset questions and a document information extraction model, where the training data includes layout document training data and streaming document training data; an extraction unit configured to extract at least one feature from the training data; a fusion unit configured to fuse the at least one feature to obtain a fused feature; a prediction unit configured to input a preset question, the fused feature, and the training data into the document information extraction model to obtain a prediction result; and an adjustment unit configured to adjust network parameters of the document information extraction model based on the prediction result and the answer.

[0009] According to a fourth aspect of the present disclosure, there is provided a document information extraction apparatus, including: an obtaining unit configured to obtain the document information to be extracted; an extraction unit configured to extract at least one feature from the document information; a fusion unit configured to fuse the at least one feature to obtain a fused feature; a prediction unit configured to input a preset question, the fused feature, and the document information into a document information extraction model trained by the apparatus according to any one of the second aspects to obtain an answer.

[0010] According to a fifth aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; where the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of the first aspects.

[0011] According to a sixth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to cause the computer to execute the method according to any one of the first aspects.

[0012] According to a seventh aspect of the present disclosure, there is provided a computer program product including a computer program, where the computer program, when executed by a processor, implements the method according to any one of the first aspects.

[0013] The training document information extraction model, the method and device for document information extraction provided by the embodiments of the present disclosure empower the needs of enterprise customers for document information extraction by using natural language processing technology, and integrate the information extraction capabilities of stream documents and layout documents. By introducing new features to mark and distinguish stream documents and layout documents, the generality of the model is improved while maintaining the information extraction effect of the model, reducing the privatization cost. At the same time, the two-dimensional spatial layout information of the document is introduced to improve the information extraction effect of the layout document.

[0014] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0016] Figure 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;

[0017] Figure 2 is a flowchart of an embodiment of the method for training a document information extraction model according to the present disclosure;

[0018] Figures 3a - 3b is a schematic diagram of an application scenario of the method for training a document information extraction model according to the present disclosure;

[0019] Figure 4 is a flowchart of an embodiment of the document information extraction method according to the present disclosure;

[0020] Figure 5 is a schematic structural diagram of an embodiment of the device for training a document information extraction model according to the present disclosure;

[0021] Figure 6 is a schematic structural diagram of an embodiment of the document information extraction device according to the present disclosure;

[0022] Figure 7 is a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The exemplary embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0024] Figure 1 An exemplary system architecture 100 for a method of training a document information extraction model, an apparatus for training a document information extraction model, a document information extraction method, or a document information extraction apparatus to which the embodiments of the present disclosure can be applied is shown.

[0025] As Figure 1 shown, the system architecture 100 may include terminals 101, 102, a network 103, a database server 104, and a server 105. The network 103 is used to provide a medium for communication links between the terminals 101, 102, the database server 104, and the server 105. The network 103 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0026] The user 110 may use the terminals 101, 102 to interact with the server 105 through the network 103 to receive or send messages, etc. Various client applications may be installed on the terminals 101, 102, such as model training applications, document information extraction applications, shopping applications, payment applications, web browsers, and instant messaging tools, etc.

[0027] The terminals 101, 102 here may be hardware or software. When the terminals 101, 102 are hardware, they may be various electronic devices with a display screen, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), laptop computers, and desktop computers, etc. When the terminals 101, 102 are software, they may be installed in the above-listed electronic devices. It may be implemented as multiple software or software modules (such as for providing distributed services), or it may be implemented as a single software or software module. No specific limitation is made here.

[0028] The database server 104 can be a database server that provides various services. For example, a sample set can be stored in the database server. The sample set contains a large number of samples, that is, training data. Among them, the samples can include layout document training data and streaming document training data. In this way, the user 110 can also select samples from the sample set stored in the database server 104 through the terminals 101 and 102.

[0029] The server 105 can also be a server that provides various services. For example, it is a background server that supports various applications displayed on the terminals 101 and 102. The background server can use the samples in the sample set sent by the terminals 101 and 102 to train the initial model, and can send the training results (such as the generated document information extraction model) to the terminals 101 and 102. In this way, the user can apply the generated document information extraction model to extract document information.

[0030] Here, the database server 104 and the server 105 can be either hardware or software. When they are hardware, they can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When they are software, they can be implemented as multiple software or software modules (for example, used to provide distributed services), or as a single software or software module. No specific limitation is made here. The database server 104 and the server 105 can also be servers of a distributed system, or servers combined with a blockchain. The database server 104 and the server 105 can also be cloud servers, or intelligent cloud computing servers or intelligent cloud hosts with artificial intelligence technology.

[0031] It should be noted that the method for training a document information extraction model or the document information extraction method provided by the embodiments of the present disclosure is generally executed by the server 105. Correspondingly, the device for training a document information extraction model or the document information extraction device is generally also set in the server 105.

[0032] It should be pointed out that in the case where the server 105 can implement the related functions of the database server 104, the database server 104 may not be provided in the system architecture 100.

[0033] It should be understood that Figure 1 the numbers of terminals, networks, database servers, and servers in

[0034] Continue to refer to Figure 2 , which shows a flow 200 of an embodiment of the method for training a document information extraction model according to the present disclosure. The method for training a document information extraction model may include the following steps:

[0035] Step 201: Obtain training data with answers corresponding to preset questions and a document information extraction model.

[0036] In this embodiment, the execution entity (such as Figure 1 the server 105 shown) of the method for training a document information extraction model can obtain the training data and the document information extraction model in various ways. For example, the execution entity can obtain the existing document information extraction model and training data stored therein from a database server (such as Figure 1 the database server 104 shown) through a wired connection method or a wireless connection method. For another example, a user can collect training data through terminals (such as Figure 1 the terminals 101 and 102 shown), including: layout document training data and stream document training data. In this way, the execution entity can receive the training data collected by the terminals and store these training data locally to generate a sample set. The training data is labeled with answers corresponding to preset questions. For example, for the question "Name", the answer "Zhang San" is labeled. The training data can be labeled manually or by an automatic labeling method. Stream documents support free editing, and page layout calculation and drawing are performed in a stream filling and arranging manner during browsing. Stream documents generally include metadata, styles, bookmarks, hyperlinks, objects, sections (the largest typesetting unit, and the document content with different page styles forms different sections), paragraphs, sentences, and other elements and attributes. These contents are described in a certain hierarchical structure to form the format of stream documents, such as word, txt, etc. Layout documents refer to non-editable, that is, fixed-layout documents, such as pdf, jpg, etc. Layout documents will not "run out of layout", and the display and printing effects on any device are highly precisely consistent. The content, position, style, etc. of the text elements in the file are fixed when the file is generated, and it is not easy for others to modify and edit them. Only some information such as annotations and signatures can be added, and it can maintain a high degree of consistency in different software, operating system, and other environments.

[0037] The document information extraction model is a reading comprehension model, including but not limited to ERNIE, BERT, etc.

[0038] Step 202: Extract at least one feature from the training data.

[0039] In this embodiment, for each layout text or stream document, at least one feature can be extracted using existing tools. For example, semantic features, stream reading order information, spatial position information of text characters, text segmentation information, document type, etc.

[0040] The sequential reading order information in streaming refers to reading text characters in the order from left to right and from top to bottom. For a layout document, it is first divided into columns in the order from left to right and from top to bottom, and then read in each column in the order from left to right and from top to bottom.

[0041] The spatial position information of text characters refers to understanding the overall layout information of the document based on the positions of text characters in a two-dimensional space. For example, based on the distribution positions and font sizes of all characters on the entire page, it can be determined where the title is, where the columns are divided, where the table is, etc. There are six representations of the two-dimensional position of characters (position embedding 2D): x0, y0 (the x and y coordinates of the upper left corner point of the character bounding box); x1, y1 (the x and y coordinates of the lower right corner point of the character bounding box); w, h (the width and height of the character bounding box). We establish mapping tables for x, y, w, and h respectively, so that through continuous learning, the model can learn the corresponding representation vectors for the four features of x, y, w, and h of the character.

[0042] Text segmentation information refers to information such as each paragraph of the document text, each cell of the table, etc. Existing tools, such as Textmind, can be used to parse the document structure to obtain information such as each paragraph of the document text, each cell of the table, etc., and assign different segment ids to different paragraphs and different cells.

[0043] The document type refers to a streaming document or a layout document. Since the model architecture proposed in this application is an open-domain unified information extraction model, which needs to solve the information extraction tasks of both streaming documents and layout documents at the same time, a flag bit task id is added to help the model understand whether the current document is a streaming document or a layout document. The document type can be determined by the document extension or some attribute information in the document (for example, columns, titles, etc.).

[0044] In summary, the model structure proposed in this application can cleverly combine the input information of these four parts, enabling the model to jointly understand the text semantic information and spatial position information, better learn the global features, and improve the model's overall understanding of the document content.

[0045] Step 203: Obtain a fused feature by fusing at least one feature.

[0046] In this embodiment, the fused feature can be directly obtained by adding the vectors of each feature. Optionally, weights for different features can be set, and the weighted sum of different features can be used as the fused feature. Different features can be pre-converted into vectors of the same length.

[0047] Step 204: Input the preset question, the fused feature, and the training data into the document information extraction model to obtain a prediction result.

[0048] In this embodiment, the answers corresponding to the questions have been marked in the training data. The document information extraction model can understand the semantic information of the text contained in the document. For example, if you want to extract someone's date of birth (i.e., the question), the model must understand that the format of "xxxx xx xx" represents date information, and then combine the name of the person input, and finally can correctly extract the content you want (i.e., the answer). This mainly includes the representation of text content (Embedding) and the representation of one-dimensional position information (position embedding 1D), that is, the streaming reading order.

[0049] The document information extraction model is a reading comprehension model. By inputting the question and document information, the answer, that is, the prediction result, can be found from the document information.

[0050] Step 205: Adjust the network parameters of the document information extraction model based on the prediction result and the answer.

[0051] In this embodiment, the loss value is calculated based on the distance (such as cosine similarity or Euclidean distance) between the prediction result and the answer, and the mean squared error loss function can be used. If the loss value is greater than or equal to the predetermined loss threshold, the network parameters of the document information extraction model need to be adjusted. Then reselect the training data, or use the original training data to repeat steps 201-205 to obtain the updated loss value. Repeat steps 201-205 until the loss value is less than the predetermined loss value.

[0052] The method for training the document information extraction model in this embodiment proposes an open-domain general document unified information extraction model, which improves the generalization of the solution; and can simultaneously ensure that the information extraction effects of both streaming documents and layout documents are very powerful.

[0053] In some optional implementation manners of this embodiment, obtaining the training data marked with the answers corresponding to the preset questions includes: obtaining the text content of the web page and the corresponding key-value pair information by crawling and parsing the web page; constructing the streaming document training data marked with the answers corresponding to the preset questions according to the text content and the corresponding key-value pair information. For example, by crawling and parsing HTML web pages such as Baidu Encyclopedia or Wikipedia, the text content of the web page and the corresponding key-value pair information can be obtained. Then, using the remote supervision scheme, a large amount of labeled pure document information extraction model training data in different fields and different vertical categories can be constructed.

[0054]

【For example:

[0055] Web text: Grilled Pepper Cake. The grilled pepper cake is a delicious dish. The main ingredients are dough and lean and fat minced meat. The ingredients are coriander and fat, and the seasonings are oyster sauce, sugar, sesame oil, etc. This delicious dish is mainly made by the method of charcoal grilling.

[0056] Key-value pair: Chinese name - Grilled Pepper Cake. Taste - salty and fragrant. Type - A delicious dish.

[0057] In the key-value pair, the "key" is the question and the "value" is the answer.

[0058] This implementation method greatly enhances the zero-shot and few-shot capabilities of the model. By using a large amount of document data for pre-training, it is possible to analyze and judge texts in different fields without additional training data, which is convenient for reuse in multiple projects and saves manpower and material resources.

[0059] In some alternative implementation methods of this embodiment, obtaining training data annotated with answers corresponding to preset questions includes: obtaining streaming document training data and a collection of layout documents; clearing the text content in the collection of layout documents and retaining the document structure; filling the streaming document training data into the document structure to generate layout document training data. The above method can be used to obtain streaming document training data, or other automatic annotation methods or manual annotation methods can be used to obtain streaming training data. By mining the layout styles, chart structures, etc. of billions of real documents, and filling the obtained pure text with annotated information extraction data, a large amount of annotated document information extraction training data with rich styles, that is, layout document training data, is obtained.

[0060] This implementation method greatly enhances the zero-shot and few-shot capabilities of the model. By using a large amount of document data for pre-training, it is possible to analyze and judge texts in different fields without additional training data, which is convenient for reuse in multiple projects and saves manpower and material resources.

[0061] In some alternative implementation methods of this embodiment, extracting at least one feature from the training data includes: extracting at least one of the following from the training data: streaming reading order information, spatial position information of text characters, text segmentation information, document type. This implementation method deeply combines text semantic information and two-dimensional spatial position information, enabling the model to obtain more comprehensive and multi-dimensional features and improving the performance of the model.

[0062] See further Figures 3a - 3b , Figures 3a - 3b is a schematic diagram of an application scenario of the method for training a document information extraction model according to this embodiment. In Figures 3a - 3b the application scenario, the input information of this task includes multiple features:

[0063] 1. Text content and streaming reading order information. The document pre-training language model ERNIE-layout is used to understand the semantic information of the text contained in the document. For example, if we want to extract someone's date of birth, the model must understand that the format of xxxx-year-xx-month-xx-day represents date information, and then combined with the input name of someone, we can finally correctly extract the content we want. This part mainly includes the representation of text content (Embedding) and the representation of one-dimensional position information (position embedding 1D).

[0064] 2. Spatial position information of text characters. The model can understand the overall layout information of the document based on the position of text characters in two-dimensional space. For example, based on the distribution position and font size of all characters on the entire page, it can determine where the title is, where the column is, where the table is, etc. There are six representations of the two-dimensional spatial position of characters (position embedding 2D): x0, y0 (the x and y coordinates of the point in the upper left corner of the character frame); x1, y1 (the x and y coordinates of the point in the lower right corner of the character frame); w, h (the width and height of the character frame). We establish mapping tables for x, y, w, and h respectively, so that the model can learn the corresponding representation vectors for the four features of x, y, w, and h of the characters through continuous learning.

[0065] 3. Text segmentation information. To help the model understand the content and layout of the text, tools such as Textmind can be used to parse the document structure, obtain information about each paragraph of the document text, each cell of the table, and assign different segment IDs to different paragraphs and cells.

[0066] 4. Distinguish the information of streaming documents and layout documents. Since the proposed model architecture is an open domain unified information extraction model, it is necessary to solve the information extraction tasks of streaming documents and layout documents at the same time. Therefore, a flag taskid is added to help the model understand whether the current document is a streaming document or a layout document.

[0067] In summary, the model structure proposed in this application can cleverly combine these four parts of input information, allowing the model to understand the text semantic information and spatial location information together, better learn the global features, and improve the model's overall understanding of the document content.

[0068] In order to improve the generalization of the model and the accuracy of information extraction, this application can adopt the most advanced large-scale document pre-training model ERNIE-layout (structure) as the base and infrastructure of the model. This architecture introduces two-dimensional spatial position information, allowing the model to learn rich multimodal features.

[0069] Concatenate all the input characters in sequence, and use special symbols such as

CLS

SEP

[0070] Please refer to Figure 4 , which shows the flowchart 400 of an embodiment of the document information extraction method provided by the present disclosure. The document information extraction method may include the following steps:

[0071] Step 401, obtain the document information to be extracted.

[0072] In this embodiment, the execution subject of the document information extraction method (such as Figure 1 the server 105 shown) can obtain the document information to be extracted in various ways. For example, the execution subject can obtain the document information to be extracted stored therein from the database server (such as Figure 1 the database server 104 shown) through a wired connection method or a wireless connection method. For another example, the execution subject can also receive the document information to be extracted collected by the terminal (such as Figure 1 the terminals 101, 102 shown) or other devices. The document information to be extracted can be a streaming document or a layout document.

[0073] Step 402, extract at least one feature from the document information.

[0074] In this embodiment, the document information is equivalent to the training data in step 202, and at least one feature can be extracted from the document information by using the method described in step 202, which will not be elaborated here.

[0075] Step 403, obtain a fused feature after fusing at least one feature.

[0076] In this embodiment, the method described in step 303 can be used to obtain a fused feature after fusing at least one feature, which will not be elaborated here.

[0077] Step 404, input the preset question, the fused feature, and the document information into the document information extraction model to obtain an answer.

[0078] In this embodiment, the execution subject may input the document information obtained in step 401, the fusion features obtained in step 403, and the preset questions into the document information extraction model, so as to generate a prediction result. The prediction result is the answer extracted from the document information.

[0079] In this embodiment, the document information extraction model may be generated by using the method described in the above Figure 2 embodiment. For the specific generation process, please refer to Figure 2 the relevant description of the embodiment, which will not be elaborated here.

[0080] It should be noted that the document information extraction method in this embodiment can be used to test the document information extraction models generated in the above embodiments. Furthermore, the document information extraction model can be continuously optimized according to the test results. This method can also be the actual application method of the document information extraction models generated in the above embodiments. Using the document information extraction models generated in the above embodiments for document information extraction helps to improve the performance of the document information extraction models, improve the efficiency and accuracy of document information extraction, etc., and reduce labor costs. At the same time, it can shorten the document information extraction time and prevent users from perceiving the document information extraction, which may affect the user experience.

[0081] Continuing to refer to Figure 5 , as an implementation of the methods shown in the above figures, an embodiment of an apparatus for training a document information extraction model is provided in the present disclosure. This apparatus embodiment corresponds to Figure 2 the method embodiment shown, and this apparatus can be specifically applied to various electronic devices.

[0082] As Figure 5 shown, the apparatus 500 for training a document information extraction model in this embodiment may include: an acquisition unit 501, an extraction unit 502, a fusion unit 503, a prediction unit 504, and an adjustment unit 505. Among them, the acquisition unit 501 is configured to acquire training data and a document information extraction model annotated with answers corresponding to preset questions, where the training data includes layout document training data and streaming document training data; the extraction unit 502 is configured to extract at least one feature from the training data; the fusion unit 503 is configured to fuse the at least one feature to obtain a fusion feature; the prediction unit 504 is configured to input the preset questions, the fusion feature, and the training data into the document information extraction model to obtain a prediction result; the adjustment unit 505 is configured to adjust the network parameters of the document information extraction model based on the prediction result and the answer.

[0083] In some alternative implementation manners of this embodiment, the obtaining unit 501 is further configured to: obtain the text content of the web page and the corresponding key-value pair information by crawling and parsing the encyclopedia web page; construct streaming document training data annotated with the answers corresponding to the preset questions according to the text content and the corresponding key-value pair information.

[0084] In some alternative implementation manners of this embodiment, the obtaining unit 501 is further configured to: obtain streaming document training data and a collection of layout documents; empty the text content in the collection of layout documents and retain the document structure; fill the streaming document training data into the document structure to generate layout document training data.

[0085] In some alternative implementation manners of this embodiment, the extraction unit 502 is further configured to extract at least one of the following from the training data: streaming reading order information, spatial position information of text characters, text segmentation information, document type.

[0086] Continue to refer to Figure 6 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a document information extraction device. This device embodiment corresponds to Figure 4 the method embodiment shown, and this device can be specifically applied to various electronic devices.

[0087] As Figure 6 shown, the document information extraction device 600 of this embodiment may include: an obtaining unit 601, an extraction unit 602, a fusion unit 603, and a prediction unit 604. Among them, the obtaining unit 601 is configured to obtain the document information to be extracted; the extraction unit 602 is configured to extract at least one feature from the document information; the fusion unit 603 is configured to obtain a fusion feature after fusing the at least one feature; the prediction unit 604 is configured to input the preset question, the fusion feature, and the document information into the document information extraction model trained according to the device 500 to obtain an answer.

[0088] In the technical solution of the present disclosure, the processing of collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0089] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0090] An electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the device according to any one of processes 200 or 400.

[0091] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the device according to any one of processes 200 or 400.

[0092] A computer program product, comprising a computer program which, when executed by a processor, implements the device according to any one of processes 200 or 400.

[0093] Figure 7 A schematic block diagram of an exemplary electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0094] As Figure 7 shown, the device 700 includes a computing unit 701 which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0095] A plurality of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0096] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various apparatuses and processes described above, such as the document information extraction method. For example, in some embodiments, the document information extraction method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the document information extraction method described above can be executed. Alternatively, in other embodiments, the computing unit 701 can be configured to execute the document information extraction method by any other suitable means (e.g., by means of firmware).

[0097] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor, receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] The program code for implementing the devices of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0099] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0100] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0101] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0102] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, can also be a server of a distributed system, or a server incorporating a blockchain.

[0103] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present disclosure can be achieved, and no limitations are imposed herein.

[0104] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A method for training a document information extraction model, comprising: Obtaining training data with answers corresponding to preset questions and a document information extraction model, wherein the training data includes layout document training data and streaming document training data; Extracting streaming reading order information and at least one of the following features from the training data: semantic features, spatial position information of text characters, text segmentation information, document type, wherein the spatial position information of text characters includes the coordinates, width, and height of the character bounding box; Taking the weighted sum of different features after converting them into vectors of the same length as the fused feature; Inputting the preset question, the fused feature, and the training data into the document information extraction model to obtain a prediction result, wherein the prediction result includes the representation of the text content and the streaming reading order; Adjusting the network parameters of the document information extraction model based on the prediction result and the answer.

2. The method according to claim 1, wherein, The obtaining of training data with answers corresponding to preset questions includes: Obtaining the text content of the web page and the corresponding key-value pair information through web page crawling and parsing; Constructing streaming document training data with answers corresponding to preset questions according to the text content and the corresponding key-value pair information.

3. The method according to claim 1, wherein, The obtaining of training data with answers corresponding to preset questions includes: Obtaining streaming document training data and a set of layout documents; Clearing the text content in the set of layout documents and retaining the document structure; Filling the streaming document training data into the document structure to generate layout document training data.

4. A document information extraction method, comprising: Obtaining the document information to be extracted; Extracting at least one feature from the document information; Fusing the at least one feature to obtain a fused feature; Inputting the preset question, the fused feature, and the document information into the document information extraction model trained according to any one of claims 1-3 to obtain an answer.

5. An apparatus for training a document information extraction model, comprising: An obtaining unit configured to obtain training data with answers corresponding to preset questions and a document information extraction model, wherein the training data includes layout document training data and streaming document training data; An extracting unit configured to extract streaming reading order information and at least one of the following features from the training data: semantic features, spatial position information of text characters, text segmentation information, document type, wherein the spatial position information of text characters includes the coordinates, width, and height of the character bounding box; A fusing unit configured to take the weighted sum of different features after converting them into vectors of the same length as the fused feature; A predicting unit configured to input the preset question, the fused feature, and the training data into the document information extraction model to obtain a prediction result, wherein the prediction result includes the representation of the text content and the streaming reading order; An adjusting unit configured to adjust the network parameters of the document information extraction model based on the prediction result and the answer.

6. The device according to claim 5, wherein, The obtaining unit is further configured to: Obtain the text content of the web page and the corresponding key-value pair information through web page crawling and parsing; Construct streaming document training data annotated with answers corresponding to preset questions according to the text content and the corresponding key-value pair information.

7. The device according to claim 5, wherein The obtaining unit is further configured to: Obtain streaming document training data and a collection of layout documents; Empty the text content in the collection of layout documents while retaining the document structure; Fill the streaming document training data into the document structure to generate layout document training data.

8. A document information extraction device, comprising: An obtaining unit configured to obtain document information to be extracted; An extraction unit configured to extract at least one feature from the document information; A fusion unit configured to obtain a fusion feature after fusing the at least one feature; A prediction unit configured to input a preset question, the fusion feature, and the document information into a document information extraction model trained by the device according to any one of claims 5-7 to obtain an answer.

9. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

11. A computer program product, comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-4.

Images