Information extraction method, model training method and device for information extraction

Abstract

The present disclosure provides an information extraction method, a model training method and device for information extraction, relating to the field of artificial intelligence technology, and particularly to the field of natural language processing technology. The specific implementation solution is as follows: obtaining a search term, and obtaining multi-page image maps of a document to be extracted, classifying the multi-page image maps to determine the category to which any image map belongs, selecting a target image map with the target category from the multi-page image maps according to the target category associated with the search term, and performing keyword extraction on the text recognized in the target image map based on the search term to obtain a target word related to the search term. By determining the target image map from the multi-page image maps and extracting the target word related to the search term from the target image map, the number of characters to be processed in one extraction is reduced, and the accuracy of information extraction from multi-image documents is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more particularly to the field of natural language processing technology, and specifically relates to an information extraction method, a model training method and device for information extraction. Background Art

[0002] Image documents are ubiquitous in people's lives. For example, bills, contracts, etc. all contain image documents, and important information is recorded in these image documents.

[0003] In the scenario of image document target information extraction, a document usually contains multiple pages of image documents. When processing a document composed of multiple pages of image documents, how to accurately extract the target information contained therein is a technical problem to be solved. Summary of the Invention

[0004] The present disclosure provides an information extraction method, a model training method and device for information extraction, which improve the accuracy of information extraction.

[0005] According to one aspect of the present disclosure, there is provided an information extraction method, including:

[0006] Obtaining a retrieval term, and obtaining multiple pages of image maps in a document to be extracted;

[0007] Classifying the multiple pages of the image maps to determine the category to which any one of the image maps belongs;

[0008] Selecting target image maps having the target category from the multiple pages of image maps according to the target category associated with the retrieval term;

[0009] Based on the retrieval term, performing keyword extraction on the text recognized in the target image maps to obtain target words related to the retrieval term.

[0010] According to another aspect of the present disclosure, there is provided a model training method for information extraction, including:

[0011] Obtaining a sample retrieval term, and obtaining training text obtained by performing text recognition on multiple pages of image maps in a sample document, wherein the training text is labeled with target words related to the sample retrieval term;

[0012] Based on the categories to which the multiple pages of the image maps belong, selecting target image maps having the target category from the multiple pages of image maps, wherein the target category is the category associated with the training retrieval term;

[0013] Inputting the retrieval term and the training text recognized in the target image maps into an extraction model for keyword extraction to obtain predicted words related to the retrieval term;

[0014] Train the extraction model based on the difference between the predicted word and the target word.

[0015] According to another aspect of the present disclosure, an information extraction device is provided, including:

[0016] An acquisition module, configured to acquire a retrieval term and acquire multi-page image maps in a document to be extracted;

[0017] A classification module, configured to classify the multi-page image maps to determine the category to which any one of the image maps belongs;

[0018] A selection module, configured to select a target image map with the target category from the multi-page image maps according to the target category associated with the retrieval term;

[0019] An extraction module, configured to perform keyword extraction on the text recognized in the target image map based on the retrieval term to obtain a target word related to the retrieval term.

[0020] According to another aspect of the present disclosure, a model training device for information extraction is provided, including:

[0021] An acquisition module, configured to acquire a sample retrieval term and acquire training text obtained by performing text recognition on multi-page image maps in a sample document, wherein the target word related to the sample retrieval term is marked in the training text;

[0022] A selection module, configured to select a target image map with a target category from the multi-page image maps based on the category to which the multi-page image maps belong, wherein the target category is the category associated with the training retrieval term;

[0023] An extraction module, configured to input the retrieval term and the training text recognized in the target image map into an extraction model for keyword extraction to obtain a predicted word related to the retrieval term;

[0024] A first training module, configured to train the extraction model based on the difference between the predicted word and the target word.

[0025] According to another aspect of the present disclosure, an electronic device is provided, including:

[0026] At least one processor; and

[0027] A memory communicatively connected to the at least one processor; wherein,

[0028] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the information extraction method described in the foregoing aspect, or the model training method for information extraction described in the foregoing other aspect.

[0029] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the information extraction method described in the foregoing aspect, or the model training method for information extraction described in the foregoing other aspect.

[0030] According to another aspect of the present disclosure, there is provided a computer program product including a computer program, and the computer program implements the information extraction method described in the foregoing aspect, or the model training method for information extraction described in the foregoing other aspect when executed by a processor.

[0031] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0032] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0033] Figure 1 is a schematic flowchart of an information extraction method provided by an embodiment of the present disclosure;

[0034] Figure 2 is a schematic flowchart of another information extraction method provided by an embodiment of the present disclosure;

[0035] Figure 3 is a schematic flowchart of another information extraction method provided by an embodiment of the present disclosure;

[0036] Figure 4 is a schematic extraction diagram of a model provided by an embodiment of the present disclosure;

[0037] Figure 5 is a schematic flowchart of another information extraction method provided by an embodiment of the present disclosure;

[0038] Figure 6 is a schematic flowchart of a model training method for information extraction provided by an embodiment of the present disclosure;

[0039] Figure 7 is a schematic flowchart of another model training method for information extraction provided by an embodiment of the present disclosure;

[0040] Figure 8 It is a schematic structural diagram of a model provided by an embodiment of the present disclosure;

[0041] Figure 9 It is a schematic structural diagram of an information extraction device provided by an embodiment of the present disclosure;

[0042] Figure 10 It is a schematic structural diagram of a model training device for information extraction provided by an embodiment of the present disclosure;

[0043] Figure 11 It is a schematic block diagram of an exemplary electronic device 1100 provided by an embodiment of the present disclosure. Detailed implementation manners

[0044] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to assist understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted below.

[0045] It should be noted that all actions of obtaining signals, information, or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where it is located and obtaining authorization from the owner of the corresponding device.

[0046] Figure 1 It is a flowchart of an information extraction method provided by an embodiment of the present disclosure. The execution subject of the information extraction method is an information extraction device, and the information extraction device can be set in an electronic device. The electronic device can be a terminal device, such as a smart phone, a smart wearable device, a personal digital assistant, etc., which is not limited in this embodiment.

[0047] As Figure 1 shown, the method includes the following steps.

[0048] Step 101, obtain a search term and obtain multi-page image maps in the document to be extracted.

[0049] Among them, the document to be extracted contains multi-page image maps. The document to be extracted can be, for example, a contract document, an agreement document, a draft document, a bill document, or a statement document, etc., which are not listed one by one here. Among them, the search term is used to extract target words related to the search term from the multi-page image maps in the document to be extracted, and the target words are the key information to be extracted. The search term can be a single word or a word list.

[0050] As an example, in a financial scenario, the document to be extracted is a loan contract document, which contains image files of the main body page and the signature and seal page of the loan contract. Among them, the main body page can be multiple pages. Among the multiple image files included in the loan contract document, there is information related to the retrieval term that the user needs to obtain, that is, the target term. For example, the retrieval term is the borrower.

[0051] Step 102: Classify the multiple-page image to determine the category to which any image belongs.

[0052] In one implementation manner of the embodiments of the present application, the multiple-page image can be input into a trained model for classification to determine the category to which any image belongs, improving the efficiency and accuracy of determining the category to which any image belongs.

[0053] In another implementation manner of the embodiments of the present application, text recognition can be performed on the multiple-page image to determine the text recognized in any image. As an implementation manner, for each page of the image, text recognition can be performed on the page of the image based on the OCR technology to determine the text recognized in the page of the image. Furthermore, according to the text recognized in the multiple-page image, the multiple-page image is classified to determine the category to which any image belongs. For example, the text recognized from an image of a certain page includes key information such as "Party A" and "official seal", and then based on the text recognition, it can be determined that the categories of the page of the image include: "main body page" and "signature and seal page". By performing text recognition on the image and classifying based on the recognized text, the semantic information and context semantic information carried in the text can be obtained, and classification is performed according to the semantic information and context semantic information carried in the text to classify each page of the image, with high classification accuracy.

[0054] As an example, in a financial scenario, when classifying the multiple-page images included in the loan contract document, the classification of each page of the image can be obtained. For example, if the loan contract contains two pages, namely the main body page and the signature and seal page, then classification is performed according to the text recognized from the images of these 2 pages. For example, the recognized images are "signing date page", "payment terms page", "signed amount page" or "signature and seal page", etc.

[0055] Step 103: Select the target image with the target category from the multiple-page images according to the target category associated with the retrieval term.

[0056] In the embodiments of the present application, the categories associated with the search terms are different in different scenarios, and may also be different in the same scenario. Therefore, it is necessary to determine the target category associated with the search terms. Furthermore, according to the target category, the target image maps with the target category are selected from multiple-page image maps, realizing the positioning of the target image maps from multiple-page image maps, determining the truly required and concerned target image maps, reducing the number of image maps to be processed, and lowering the complexity and data volume of processing.

[0057] As an example, in the financial scenario, taking a loan contract as an example, the electronic document of the loan contract consists of multiple-page image maps, and each page of the image map corresponds to a page of the paper loan contract. The information to be extracted may be distributed on any page of the image map. For example, the information of "Party A" may be on the [first page of the main text] or on the [signature and seal page] at the end of the contract. However, the information that the customer needs to obtain is the value corresponding to "Party A" on the [signature and seal page], and the values of "Party A" on other pages do not need to be obtained. If all the image maps are processed, the number of image maps to be processed is increased, which increases the difficulty of subsequent processing and reduces the accuracy of information acquisition. Therefore, determining the target image maps from multiple-page image maps can reduce the complexity of subsequent processing and improve the accuracy of information acquisition.

[0058] Step 104: Based on the search terms, extract keywords from the text recognized in the target image maps to obtain target words related to the search terms.

[0059] Among them, the target words can be entity words related to the search terms or relationship words related to the search terms. An entity corresponds to a word for an object or a person. A relationship word is a word associated with the search term. For example, if the search term is "mastoid process", the relationship word is "tenderness".

[0060] In the embodiments of the present application, using the search terms as prompt words, keywords are extracted from the text recognized in the target image maps, and the target words related to the search terms are determined according to the extracted keywords, so as to extract the target words related to the search terms from the target image maps, reducing the number of characters to be processed in one extraction, and at the same time, improving the accuracy of extraction.

[0061] In the information extraction method according to the embodiments of the present disclosure, a retrieval term is obtained, and multi-page image maps in a document to be extracted are obtained. The multi-page image maps are classified to determine the category to which any image map belongs. Based on the target category associated with the retrieval term, target image maps with the target category are selected from the multi-page image maps. Based on the retrieval term, keywords are extracted from the text recognized in the target image maps to obtain target words related to the retrieval term. By determining the target image maps from the multi-page image maps and extracting the target words related to the retrieval term from the target image maps, the number of characters to be processed in one extraction is reduced, and the accuracy of information extraction from multi-image documents is improved.

[0062] Based on the above embodiments, Figure 2 is a schematic flowchart of another information extraction method provided by the embodiments of the present disclosure. As Figure 2 shown, the method includes the following steps:

[0063] Step 201, obtain a retrieval term and obtain multi-page image maps in a document to be extracted.

[0064] Among them, Step 201 can refer to the examples in the foregoing embodiments, with the same principle, and will not be elaborated here.

[0065] Step 202, perform feature extraction according to the text distribution positions of the multi-page image maps to obtain the image features of the multi-page image maps.

[0066] In one implementation manner of the embodiments of the present application, for each page of the image map, the text distribution position in the page of the image map can be determined. The text distribution position indicates the position where the text is located in the page of the image map. As an implementation manner, the pixels of the part with text in the page of the image map can be set to 1, and the pixels of the part without text can be set to 0, and then convolution processing is performed for feature extraction to obtain the image features of the page of the image map.

[0067] Step 203, classify the multi-page image maps according to the image features extracted from the multi-page image maps to determine the category to which any image map belongs.

[0068] As an implementation, the classification of multi-page image maps can be achieved based on the model of the trained deep neural network. That is, according to the model of the deep neural network, feature extraction is performed based on the text distribution positions of the multi-page image maps to obtain the image features of the multi-page image maps. According to the image features extracted from the multi-page image maps, the multi-page image maps are classified to determine the category to which any image map belongs. Among them, the text distribution positions of the multi-page image maps indicate the distribution of the text parts and non-text parts in each page of the image maps. Furthermore, according to the distribution positions of the text in each page of the image maps and the image information contained in each position, the image features are extracted. The image features include the distribution positions of the text in each page of the image maps, features such as the color and texture of the images, increasing the information contained in the image features. Thus, based on the obtained image features for classification, the category to which each page of the image maps belongs can be effectively determined, improving the accuracy of category recognition.

[0069] Among them, the model of the deep neural network can be the BERT model. Among them, the training method of the BERT model will be described in the subsequent embodiments.

[0070] Step 204, according to the target category associated with the search term, select the target image maps with the target category from the multi-page image maps.

[0071] Step 205, based on the search term, perform keyword extraction on the text recognized in the target image maps to obtain the target words related to the search term.

[0072] Among them, steps 204 and 205 can refer to the explanations in the foregoing embodiments. The principles are the same and will not be elaborated here.

[0073] In the information extraction method of the embodiments of the present disclosure, a search term is obtained, and multi-page image maps in the document to be extracted are obtained. The multi-page image maps are classified to determine the category to which any image map belongs. According to the target category associated with the search term, the target image maps with the target category are selected from the multi-page image maps. Based on the search term, keyword extraction is performed on the text recognized in the target image maps to obtain the target words related to the search term. By determining the target image maps from the multi-page image maps and extracting the target words related to the search term from the target image maps, the number of characters to be processed in one extraction is reduced, and the accuracy of information extraction from multi-image documents is improved.

[0074] Based on the above embodiments, Figure 3 is a schematic flowchart of another information extraction method provided by the embodiments of the present disclosure, as Figure 3 shown. This method includes the following steps:

[0075] Step 301, obtain a search term and obtain multi-page image maps in the document to be extracted.

[0076] Step 302: Perform text recognition on the multi-page image to determine the text recognized in any of the images.

[0077] Step 303: Classify the multi-page image according to the text recognized in the multi-page image to determine the category to which any of the images belongs.

[0078] Step 304: Select the target image with the target category from the multi-page image according to the target category associated with the search term.

[0079] Among them, Steps 301 to 304 can refer to the examples in the foregoing embodiments. The principles are the same and will not be elaborated here.

[0080] Step 305: Encode the search term to obtain a first encoded sequence.

[0081] In an implementation manner of the embodiment of the present application, the search term is segmented and vectorized to obtain a first encoded sequence.

[0082] Step 306: Encode the text recognized in the target image to obtain a second encoded sequence.

[0083] In the embodiment of the present application, the text recognized in the target image is segmented into multiple characters, and each character is vectorized to obtain a second encoded sequence.

[0084] Step 307: Add set characters to the head and tail of the first encoded sequence, and splice it with the second encoded sequence to obtain an input encoded sequence.

[0085] Among them, the set characters are, for example, CLS and SEP. As Figure 4 shown, for the first encoded sequence, that is Figure 4 the CLS is added before the Prompt in the first encoded sequence corresponding to the search term, and the SEP is added after the first encoded sequence, and then spliced with the second encoded sequence corresponding to the text of the target image to obtain the input encoded sequence. Among them, the SEP is added after the input encoded sequence. The set character CLS is placed at the beginning of the first sentence, and the representation vector obtained through BERT can be used for subsequent classification tasks. The set character SEP is used to separate the encoded sequences corresponding to the two input texts, that is, to separate the first encoded vector and the second encoded vector. For example, for input texts A and B, the set character SEP needs to be added between texts A and B.

[0086] It should be noted that in this embodiment, there is no limitation on the execution timing of adding the set characters to the head and tail of the first encoded sequence and splicing it with the second encoded sequence.

[0087] Step 308: Input the input encoded sequence into the extraction model to obtain the extraction positions output by the extraction model.

[0088] Among them, the extraction model is a pre-trained model ERNIE, such as ERNIE 3.0, which uses the Unified Information Extraction Framework UIE.

[0089] Step 309: Extract the text located at the extraction positions from the text recognized in the target image to obtain the target words related to the retrieval term.

[0090] In the embodiments of this application, when the input encoded sequence is input into the extraction model, the extraction model can identify the extraction positions for information extraction of the text recognized in the target image. Among them, the extraction positions include the start position and the end position. Furthermore, the text located at the start position and the end position included in the extraction positions is extracted from the text recognized in the target image to obtain the target words related to the retrieval term.

[0091] As an example, as Figure 4 shown, in the scenario of entity extraction, the extraction model extracts the target word "Chest CT plain scan" according to the text corresponding to the input target image and the retrieval term "examination". In the scenario of relation extraction, according to the input target image and the retrieval term "part symptom", the extracted target word is "No tenderness in the mastoid process".

[0092] In the information extraction method of the embodiments of this application, when performing content extraction, an encoded sequence is generated according to the text recognized from the target image and the retrieval term, and the encoded sequence is input into the extraction model to implement using the retrieval term as a prompt word and executing the entity extraction task using the extraction framework UIE to determine the extraction positions of the extracted entities, and to extract the target words related to the retrieval term based on the extraction positions, thereby improving the accuracy.

[0093] Based on the above embodiments, Figure 5 is a schematic flowchart of another information extraction method provided by the embodiments of the present disclosure. As Figure 5 shown, this method includes the following steps:

[0094] Step 501: Obtain the retrieval term and obtain multiple-page images in the document to be extracted.

[0095] Step 502: Perform text recognition on the multiple-page images to determine the text recognized in any one of the images.

[0096] Step 503: Classify the multiple-page images according to the text recognized in the multiple-page images to determine the category to which any one of the images belongs.

[0097] Step 504: Select target image maps with the target category from multiple-page image maps according to the target category associated with the search term.

[0098] Step 505: Encode the search term to obtain a first encoded sequence.

[0099] Step 506: Encode the text recognized in the target image map to obtain a second encoded sequence.

[0100] Step 507: Add set characters to the head and tail of the first encoded sequence and splice it with the second encoded sequence to obtain an input encoded sequence.

[0101] Step 508: Input the input encoded sequence into the extraction model to obtain the extraction positions output by the extraction model.

[0102] Step 509: Extract the text located at the extraction positions from the text recognized in the target image map to obtain target words related to the search term.

[0103] Among them, the relevant explanations in the foregoing embodiments are also applicable to Steps 501 to 509. The principles are the same and will not be elaborated here.

[0104] Step 510: Query the verification rules associated with the search term and verify the target words related to the search term based on the verification rules.

[0105] Among them, the verification rules can be determined according to the specific business logic.

[0106] As an example, if the target word is "date", the verification rule is to verify whether it contains English letters and punctuation marks other than ['-', '.', ' / ', '\']; if the target word is "lowercase amount", the corresponding verification rule is to verify whether it contains other characters other than ['yuan', 'ten thousand yuan', '￥', ',', ',', '.']. Through verification, the target words with errors are identified. These target words are usually the words that are misrecognized during the text recognition process, and the target words with errors are deleted, improving the accuracy of target word acquisition.

[0107] For example, the search term is "institution", and the determined target word corresponding to the search term is "North X University". Among them, "North X University" is actually "North Y University", which may be caused by misrecognition during the text recognition process. Therefore, the target word "North X University" will be deleted.

[0108] Step 511: In the case where there are multiple target words, sort the multiple target words according to the confidence level of the extraction positions.

[0109] In the embodiments of the present application, when multiple target words are extracted according to the extraction positions, the extraction model also outputs the confidence of the extraction positions. The confidence of the extraction positions indicates the reliability of the extracted target words. Then, the target words are sorted according to the confidence of the extraction positions. For example, they are sorted from high to low confidence. In one implementation of the embodiments of the present application, the extraction positions include the start position and the end position. When there are multiple target words, for any target word, it is sorted based on the product between the confidence of the start position and the confidence of the end position. That is to say, for any target word, the product of the confidence of the start position and the confidence of the end position of this target word is used as the position confidence of this target word, and it is sorted based on the obtained position confidence. By using the confidence of the start position of the target word extraction output by the extraction model and the confidence of the target position, the confidence of the extraction position corresponding to the target word is determined, improving the accuracy of the determination of the confidence of the extraction position. Then, based on the confidence, the target words are sorted, which can improve the accuracy of the determination of the target words.

[0110] As Figure 4 shown, the extraction model determines 2 extraction positions for extracting the text recognized from the target image according to the text corresponding to the input target image and the retrieval word. Each extraction position includes the start position Start and the end position End. According to each start position Start and end position End, the target words corresponding to the retrieval word are extracted, such as Figure 4 Result1 output by the model in

[0111] Step 512, based on the sorting, select at least one target word from multiple target words as the attribute value, and use the retrieval word as the corresponding attribute item.

[0112] In an implementation of the embodiment of the present application, when the search term and the target term are in a one-to-one relationship, based on sorting, the target term with the highest confidence is selected from multiple target terms as the attribute value, and the search term is used as the corresponding attribute item, thus obtaining the attribute item corresponding to the attribute value. For example, in a loan contract document, there is only one target term corresponding to search terms such as "signing date", "Party A", and "total contract price", so the key-value relationship formed by the search term and the target term is one-to-one.

[0113] In an implementation of the embodiment of the present application, when the search term and the target term are in a one-to-many relationship, based on sorting, multiple target terms with a confidence level higher than a set threshold are selected from multiple target terms as the attribute value, and the search term is used as the corresponding attribute item, thus obtaining multiple attribute items corresponding to the attribute value, increasing the amount of information extracted from the target image and improving the accuracy of information extraction.

[0114] For example, in the financial field, taking a loan contract as an example, the search term is the collateral. Depending on the amount of the loan, the collateral can be one or multiple. That is to say, as an attribute item, the corresponding attribute value of the collateral can be one or multiple. In practical applications, the attribute item with the largest amount can be determined, or multiple attribute items corresponding to a relatively large amount can be determined from all the attribute items. In the information extraction method of the embodiment of the present application, when multiple target terms are obtained according to the extraction position, the extraction model also outputs the confidence level of the extraction position. Among them, the confidence level of the extraction position indicates the reliability of the extracted target term. Then, based on the sorting, at least one target term is selected from multiple target terms as the attribute value, and the search term is used as the corresponding attribute item, realizing the use of the search term as a prompt word to extract the attribute value information corresponding to the search term from the located target image, reducing the number of characters processed by the model during one extraction, and solving the technical problem in the related art that when processing a document composed of multiple-page images, due to the long number of document characters, extraction cannot be performed or the extraction effect is poor. The present application greatly improves the extraction accuracy of key information in image documents in the case of low resources, multiple pages, and long text.

[0115] Based on the above embodiments, as an implementation, after extracting the text located at the extraction position from the text recognized from the target image to obtain the target term related to the search term, it further includes:

[0116] When there are multiple target terms, the multiple target terms are respectively used as the attribute value, and the search term is used as the corresponding attribute item.

[0117] In the embodiments of the present application, if the retrieval term and the target term are one-to-many, in the case where there are multiple target terms, the multiple target terms are respectively used as attribute values, and the retrieval term is used as the corresponding attribute item, realizing the determination of all attribute values related to the retrieval term, increasing the number of information extracted from the target image, and improving the diversity and accuracy of extraction.

[0118] Based on the above embodiments, Figure 6 is a schematic flowchart of a model training method for information extraction provided by an embodiment of the present disclosure, as Figure 6 shown, the method includes the following steps:

[0119] Step 601, obtain a sample retrieval term, and obtain training text obtained by performing text recognition on multi-page image maps in a sample document.

[0120] Among them, the training text is marked with target terms related to the sample retrieval term. As an implementation manner, the doccano tool can be used to mark the target terms related to the retrieval term in a high-precision marking manner.

[0121] In the embodiments of the present application, the training text contains multiple words, and target terms related to the sample retrieval term are determined from the training text according to the sample retrieval term as annotation information.

[0122] Among them, the sample document contains multi-page image maps. The sample document is, for example, a contract document, an agreement document, a draft document, a bill document, or a statement document, etc., which are not listed one by one here. Among them, the sample retrieval term is used to determine target terms related to the retrieval term from multi-page image maps in the sample document, and the target term is the key information to be extracted.

[0123] As an example, in a financial scenario, the sample document is a loan contract document. The loan contract document contains image copies of the main body page and the signature and seal page of the loan contract. Among them, the main body page can be multiple pages. The multi-page image copies included in the loan contract document contain information related to the retrieval term that the user needs to obtain, that is, contain multiple words. Corresponding target terms are determined according to the retrieval term as annotation information for the retrieval term. For example, if the retrieval term is "borrower", the target term corresponding to the retrieval term "borrower" marked in the loan contract is "Name: Zhang San".

[0124] Step 602, based on the category to which the multi-page image maps belong, select target image maps with the target category from the multi-page image maps.

[0125] Among them, the target category is the category associated with the sample retrieval term.

[0126] Specifically, reference can be made to the explanation in the foregoing embodiments. The principle is the same and will not be elaborated here.

[0127] Step 603: Input the retrieval term and the training text recognized from the target image into the extraction model for keyword extraction to obtain predicted terms related to the retrieval term.

[0128] Step 604: Train the extraction model based on the difference between the predicted terms and the target terms.

[0129] In the embodiments of the present application, the training process adopts mini - batch gradient descent (mini_batch = 8, depending on the video memory size of the image processor GPU), that is, this process uses a batch of samples for training, achieving a certain degree of parallelism during the training process. For the finally obtained extraction model, a test set is used for evaluation. If the extraction model meets specific delivery metrics, the extraction model will be released. Otherwise, the hyperparameter tuning module will be used to re - formulate the hyperparameters to train a new extraction model until the requirements are met.

[0130] In the model training method for information extraction according to the embodiments of the present application, by obtaining a sample retrieval term and the training text obtained by text recognition of multi - page images in a sample document, where the training text is marked with target terms related to the sample retrieval term, and based on the category to which the multi - page images belong, target images with the target category are selected from the multi - page images, and then the extraction model is trained based on the training text recognized from the target images and the retrieval term, so that the extraction model is trained based on the target images obtained by positioning, reducing the length of the characters included in the input training text, enabling the extraction model to obtain the context semantic information in the training text during the training process, and using the retrieval term as a prompt word to improve the accuracy of extracting information from the training text.

[0131] Based on the above - mentioned embodiments, Figure 7 is a schematic flowchart of another model training method for information extraction provided by the embodiments of the present disclosure, specifically illustrating how to train a classification model, as Figure 7 shown, this method includes the following steps:

[0132] Step 701: Obtain a sample retrieval term and the training text obtained by text recognition of multi - page images in a sample document.

[0133] Among them, the training text is marked with target terms related to the sample retrieval term.

[0134] It should be noted that the relevant explanations in the foregoing embodiments also apply to this embodiment, with the same principle and will not be elaborated here.

[0135] Among them, the multi - page images are all marked with the target category, that is, each page of the image is marked with the target category.

[0136] Step 702: Query the verification rules associated with the search terms. Based on the verification rules, verify the target words marked in the training text to retain the target words that pass the verification.

[0137] Among them, the verification rules can be determined according to the specific business logic.

[0138] As an example, if the target word marked in the training text is "date", the verification rule is to verify whether it contains English letters and punctuation marks other than ['-', '.', ' / ', '\']; if the target word is "lowercase amount", the corresponding verification rule is to verify whether it contains other characters other than ['yuan', 'ten thousand yuan', '￥', ',', ',', '.']. Through verification, identify the target words with errors in the training text. These target words are usually the words that are misrecognized during the text recognition process, and delete the target words with errors, which improves the accuracy of the target words marked in the training samples corresponding to the search terms, avoids training the extraction model with incorrect marked information, and improves the training effect of the extraction model.

[0139] Step 703: Use the classification model to classify the multi-page image maps to determine the predicted category to which any image map belongs.

[0140] Among them, the classification model is a model of a deep neural network, such as a BERT model.

[0141] Step 704: Train the classification model based on the difference between the predicted category and the target category.

[0142] The training process of the BERT model is as follows:

[0143] Obtain the dataset for training. For each page of the image map in the dataset, it is a classification sample. There are N overall labels, and each label indicates the category of the page image map. For each sample, it can correspond to the first label or the Nth label. Multiple labels can be marked for each sample. For example: In a contract, if a certain page has key information such as "Party A" and "official seal", then its category is [Text Page], or, it is also [Signature and Seal Page]. The specific category indicated by the label should be determined by the product requirements side according to the actual business.

[0144] Furthermore, divide the samples obtained after annotating the dataset to get the training set: validation set: test set = 8:1:1.

[0145] Use the training set to train the BERT model. Specifically, as Figure 8As shown in the figure, the parameters of all layers of the pre-trained BERT model are unfrozen and connected to a fully-connected layer. One neuron in the fully-connected layer represents one category of multi-labels, and the Sigmoid function is used for activation. Finally, the probability of each category in the forward calculation is output. The loss function loss uses the binary cross-entropy loss function to fine-tune the BERT model, achieving the training of the classification model based on the categories annotated in each page of the image, enabling the trained classification model to determine the corresponding categories of each page of the image and improving the training effect. After the training is completed, the validation set and the test set are used to evaluate each model, and the model with the highest accuracy is obtained and saved for release. As an implementation method, the BERT model is deployed on a Linux server using containerization technology, and the FastAPI backend framework of Python is used to provide inference services, and the GPU is enabled to improve the inference performance.

[0146] Step 705: Based on the category to which the multi-page image belongs, select the target image with the target category from the multi-page images.

[0147] Among them, the target category is the category associated with the sample retrieval term.

[0148] Step 706: Input the retrieval term and the training text recognized from the target image into the extraction model for keyword extraction to obtain the predicted words related to the retrieval term.

[0149] Step 707: Train the extraction model based on the difference between the predicted words and the target words.

[0150] Among them, Steps 705 to 707 can refer to the explanations in the foregoing embodiments, and the principles are the same and will not be elaborated here.

[0151] In the model training method for information extraction according to the embodiments of the present application, by obtaining the sample retrieval term and the training text obtained by text recognition of the multi-page images in the sample document, where the training text is marked with the target words related to the sample retrieval term, and based on the category to which the multi-page images belong, select the target images with the target category from the multi-page images, and then train the extraction model based on the training text recognized from the target images and the retrieval term, so that the extraction model is trained based on the target images located, reducing the length of the characters included in the input training text, enabling the extraction model to obtain the context semantic information in the training text during the training process, and being able to use the retrieval term as a prompt word to improve the accuracy of extracting information from the training text.

[0152] To implement the above embodiments, Figure 9 is a schematic structural diagram of an information extraction device provided by an embodiment of the present disclosure.

[0153] As shown Figure 9 below, the device includes:

[0154] An acquisition module 910, configured to acquire retrieval terms and acquire multi-page image maps in a document to be extracted.

[0155] A classification module 920, configured to classify the multi-page image maps to determine the category to which any one of the image maps belongs.

[0156] A selection module 930, configured to select target image maps with the target category from the multi-page image maps according to the target category associated with the retrieval terms.

[0157] An extraction module 940, configured to perform keyword extraction on the text recognized in the target image maps based on the retrieval terms to obtain target words related to the retrieval terms.

[0158] Furthermore, in an implementation manner of the embodiment of the present application, the extraction module 940 is specifically configured to:

[0159] Encode the retrieval terms to obtain a first encoding sequence;

[0160] Encode the text recognized in the target image maps to obtain a second encoding sequence;

[0161] Add set characters to the head and tail of the first encoding sequence and splice it with the second encoding sequence to obtain an input encoding sequence;

[0162] Input the input encoding sequence into an extraction model to obtain extraction positions output by the extraction model;

[0163] Extract the text located at the extraction positions from the text recognized in the target image maps to obtain target words related to the retrieval terms.

[0164] In an implementation manner of the embodiment of the present application, the output of the extraction model further includes the confidence of the extraction positions;

[0165] The device further includes:

[0166] A processing module, configured to, when there are multiple target words, sort the multiple target words according to the confidence of the extraction positions; based on the sorting, select at least one target word from the multiple target words as an attribute value, and use the retrieval terms as the corresponding attribute items.

[0167] In an implementation manner of the embodiment of the present application, the processing module is further configured to:

[0168] When there are multiple target words, use the multiple target words as attribute values respectively, and use the retrieval terms as the corresponding attribute items.

[0169] In one implementation of the embodiment of the present application, the extraction position includes a start position and an end position;

[0170] The processing module is further configured to: when there are multiple target words, for any one of the target words, sort based on the product between the confidence of the start position and the confidence of the end position.

[0171] In one implementation of the embodiment of the present application, the apparatus further includes:

[0172] A verification module, configured to query the verification rules associated with the retrieval term; based on the verification rules, verify the target words related to the retrieval term.

[0173] In one implementation of the embodiment of the present application, the classification module 920 is further specifically configured to:

[0174] Perform text recognition on multiple pages of the image map to determine the text recognized in any one of the image maps;

[0175] Classify the multiple pages of image maps according to the text recognized in the multiple pages of image maps to determine the category to which any one of the image maps belongs.

[0176] In one implementation of the embodiment of the present application, the classification module 920 is further specifically configured to:

[0177] Extract features according to the text distribution positions of the multiple pages of image maps to obtain the image features of the multiple pages of image maps;

[0178] Classify the multiple pages of image maps according to the image features extracted from the multiple pages of image maps to determine the category to which any one of the image maps belongs.

[0179] It should be noted that the foregoing explanation of the method embodiment also applies to the apparatus of this embodiment, with the same principle and will not be elaborated here.

[0180] In the information extraction apparatus of the embodiment of the present disclosure, a retrieval term is obtained, and multiple pages of image maps in a document to be extracted are obtained, the multiple pages of image maps are classified to determine the category to which any one of the image maps belongs, according to the target category associated with the retrieval term, target image maps with the target category are selected from the multiple pages of image maps, and based on the retrieval term, keyword extraction is performed on the text recognized in the target image maps to obtain target words related to the retrieval term. By determining the target image maps from the multiple pages of image maps and extracting the target words related to the retrieval term from the target image maps, the number of characters to be processed in one extraction is reduced, and the accuracy of information extraction from multiple-image documents is improved.

[0181] To implement the above embodiments,Figure 10 It is a schematic structural diagram of a model training device for information extraction provided by an embodiment of the present disclosure.

[0182] As Figure 10 shown, the device includes:

[0183] An acquisition module 110, configured to acquire sample retrieval terms and training texts obtained by performing text recognition on image maps of multiple pages in a sample document, wherein target words related to the sample retrieval terms are marked in the training texts.

[0184] A selection module 120, configured to select target image maps with a target category from the multiple pages of image maps based on the category to which the multiple pages of image maps belong, wherein the target category is the category associated with the training retrieval terms.

[0185] An extraction module 130, configured to input the retrieval terms and the training texts recognized in the target image maps into an extraction model for keyword extraction to obtain prediction words related to the retrieval terms.

[0186] A first training module 140, configured to train the extraction model based on the difference between the prediction words and the target words.

[0187] Furthermore, in an implementation manner of an embodiment of the present application, the device further includes:

[0188] A verification module, configured to query the verification rules associated with the retrieval terms; based on the verification rules, verify the target words marked in the training texts to retain the target words that pass the verification.

[0189] In an implementation manner of an embodiment of the present application, the device further includes: the multiple pages of image maps are all marked with a target category, and the method further includes:

[0190] A second training module, configured to classify the multiple pages of image maps using a classification model to determine the predicted category to which any one of the image maps belongs; based on the difference between the predicted category and the target category, train the classification model.

[0191] It should be noted that the foregoing explanations of the method embodiments also apply to the device of this embodiment, with the same principle, and will not be elaborated here.

[0192] In the model training device for information extraction according to the embodiments of the present application, by obtaining a sample retrieval term and training text obtained by performing text recognition on multi-page image graphs in a sample document, where the training text is marked with a target term related to the sample retrieval term, and based on the category to which the multi-page image graphs belong, target image graphs with a target category are selected from the multi-page image graphs, and then the extraction model is trained based on the training text and the retrieval term recognized from the target image graphs, so that the extraction model is trained based on the target image graphs located, reducing the length of the characters included in the input training text, enabling the extraction model to obtain the context semantic information in the training text during the training process, and being able to use the retrieval term as a prompt word to improve the accuracy of extracting information from the training text.

[0193] To implement the above embodiments, another aspect of the present disclosure provides an electronic device, including:

[0194] At least one processor; and

[0195] A memory communicatively connected to the at least one processor; wherein,

[0196] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the information extraction method of the foregoing embodiments, or the model training method for information extraction of the foregoing.

[0197] To implement the above embodiments, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the information extraction method of the foregoing embodiments, or the model training method for information extraction of the foregoing.

[0198] To implement the above embodiments, the present disclosure provides a computer program product, including a computer program, where the computer program, when executed by a processor, implements the information extraction method of the foregoing embodiments, or the model training method for information extraction of the foregoing.

[0199] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0200] Figure 11FIG. shows a schematic block diagram of an exemplary electronic device 1100 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementations of the present disclosure described and / or claimed herein.

[0201] As Figure 11 shown, the device 1100 includes a computing unit 1101 that can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1102 or a computer program loaded from a storage unit 1108 into a RAM (Random Access Memory) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An I / O (Input / Output) interface 1105 is also connected to the bus 1104.

[0202] A plurality of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a magnetic disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0203] The computing unit 1101 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 executes the various methods and processes described above, such as the methods of the foregoing embodiments. For example, in some embodiments, the methods of the foregoing embodiments may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the methods described above may be executed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the methods of the foregoing embodiments in any other suitable manner (e.g., by means of firmware).

[0204] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chip), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0205] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0206] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory), or flash memory, optical fibers, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0207] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or an LCD (Liquid Crystal Display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0208] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0209] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server can also be a server of a distributed system, or a server combined with blockchain.

[0210] Herein, it should be noted that artificial intelligence is a discipline that studies to make a computer simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and there are both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0211] It should be understood that various forms of the processes shown above can be used, reordering, adding, or deleting steps. For example, the steps recited in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and no limitations are imposed herein.

[0212] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. An information extraction method, comprising: Obtaining a retrieval term, and obtaining multi-page image maps in a document to be extracted; Classifying the multi-page image maps to determine the category to which any one of the image maps belongs; Selecting target image maps with the target category from the multi-page image maps according to the target category associated with the retrieval term; Performing keyword extraction on the text recognized in the target image maps based on the retrieval term to obtain target words related to the retrieval term; Wherein, performing keyword extraction on the text recognized in the target image maps based on the retrieval term to obtain target words related to the retrieval term includes: Encoding the retrieval term to obtain a first encoding sequence; Encoding the text recognized in the target image maps to obtain a second encoding sequence; Adding set characters to the head and tail of the first encoding sequence and splicing it with the second encoding sequence to obtain an input encoding sequence; Inputting the input encoding sequence into an extraction model to obtain the extraction positions output by the extraction model; Extracting the text located at the extraction positions from the text recognized in the target image maps to obtain target words related to the retrieval term.

2. The method according to claim 1, wherein The output of the extraction model further includes the confidence of the extraction positions; The method further includes: When there are multiple target words, sorting the multiple target words according to the confidence of the extraction positions; Based on the sorting, selecting at least one target word from the multiple target words as an attribute value, and using the retrieval term as the corresponding attribute item.

3. The method according to claim 1, wherein The method further includes: When there are multiple target words, using the multiple target words as attribute values respectively, and using the retrieval term as the corresponding attribute item.

4. The method according to claim 2, wherein, The extraction positions include a start position and an end position; When there are multiple target words, sorting the multiple target words according to the confidence of the extraction positions includes: When there are multiple target words, for any one of the target words, sorting is performed based on the product between the confidence of the start position and the confidence of the end position.

5. The method according to claim 1, wherein The method further includes: Querying the verification rules associated with the retrieval term; Verifying the target words related to the retrieval term based on the verification rules.

6. The method according to any one of claims 1-5, wherein, The classifying the multi-page image maps to determine the category to which any one of the image maps belongs includes: Performing text recognition on the multi-page image maps to determine the text recognized in any one of the image maps; Classifying the multi-page image maps according to the text recognized in the multi-page image maps to determine the category to which any one of the image maps belongs.

7. The method according to any one of claims 1-5, wherein, The classifying the multi-page image maps to determine the category to which any one of the image maps belongs includes: Performing feature extraction according to the text distribution positions of the multi-page image maps to obtain the image features of the multi-page image maps; Classifying the multi-page image maps according to the image features extracted from the multi-page image maps to determine the category to which any one of the image maps belongs.

8. The method according to claim 1, wherein The training method of the extraction model includes: Obtain a sample retrieval term and training text obtained by performing text recognition on image maps of multiple pages in a sample document, where the training text is marked with a target term related to the sample retrieval term; Based on the category to which the multiple pages of image maps belong, select target image maps with a target category from the multiple pages of image maps, where the target category is the category associated with the sample retrieval term; Input the sample retrieval term and the training text recognized in the target image map into an extraction model for keyword extraction to obtain a predicted term related to the sample retrieval term; Train the extraction model based on the difference between the predicted term and the target term related to the sample retrieval term.

9. The method according to claim 8, the method further comprising: Query the verification rule associated with the sample retrieval term; Based on the verification rule, verify the target terms marked in the training text to retain the target terms that pass the verification.

10. The method according to claim 8 or 9, where the multiple pages of image maps are all marked with a target category, the method further comprising: Use a classification model to classify the multiple pages of image maps to determine the predicted category to which any one of the image maps belongs; Train the classification model based on the difference between the predicted category and the target category.

11. An information extraction device, comprising: An acquisition module, configured to acquire a retrieval term and multiple pages of image maps in a document to be extracted; A classification module, configured to classify the multiple pages of image maps to determine the category to which any one of the image maps belongs; A selection module, configured to select target image maps with the target category from the multiple pages of image maps according to the target category associated with the retrieval term; An extraction module, configured to perform keyword extraction on the text recognized in the target image map based on the retrieval term to obtain a target term related to the retrieval term; The extraction module is specifically configured to: encode the retrieval term to obtain a first encoded sequence; Encode the text recognized in the target image map to obtain a second encoded sequence; Add set characters to the head and tail of the first encoded sequence and splice it with the second encoded sequence to obtain an input encoded sequence; Input the input encoded sequence into an extraction model to obtain an extraction position output by the extraction model; Extract the text located at the extraction position from the text recognized in the target image map to obtain a target term related to the retrieval term.

12. The device according to claim 11, wherein, The training device of the extraction model includes: An acquisition module, configured to acquire a sample retrieval term and training text obtained by performing text recognition on multiple pages of image maps in a sample document, where the training text is marked with a target term related to the sample retrieval term; A selection module, configured to select target image maps with a target category from the multiple pages of image maps based on the category to which the multiple pages of image maps belong, where the target category is the category associated with the sample retrieval term; An extraction module, configured to input the sample retrieval term and the training text recognized from the target image into an extraction model to perform keyword extraction, so as to obtain a predicted term related to the sample retrieval term; A first training module, configured to train the extraction model based on the difference between the predicted term and the target term related to the sample retrieval term.

13. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method according to any one of claims 1-10.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

15. A computer program product, comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-10.

Images