Document information identification method and system and related product

Abstract

The embodiment of the invention discloses a receipt information identification method and system and a related product. The method comprises the following steps: searching a target cue word correspondingly matched with a to-be-identified receipt from a cue word library; acquiring a historical session record; the example document recorded by the historical session record is the same as the document type to which the to-be-identified document belongs, and the historical session record at least records the content typesetting format of the example document; and based on the target cue word and the indication of the content typesetting format, identifying receipt information specified to be output by the target cue word from the to-be-identified receipt. According to the method, a mechanism that historical session records are combined with cue words for information recognition is adopted, so that the recognition accuracy of the receipt information can be comprehensively improved in multiple aspects, and errors caused by single-factor recognition are avoided. Besides, the historical session records and the target prompt words have corresponding relations with the to-be-identified receipts, so that the method and the device can effectively adapt to diversified receipts with different receipt types or continuously updated formats, and manual intervention and cost input are reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of information identification technology, and in particular, to document information identification methods, systems and related products. Background Art

[0002] In the era of digital development, document information can summarize and feedback the required transaction information.

[0003] Traditional document information products mainly use optical character recognition (OCR) technology and template matching methods to identify document information. Specifically, a pre-set document template is used to select characters at the corresponding position of the current document, and then the selected characters are recognized by OCR to extract the required document information, such as payer, payment account, payee, receiving account, transaction amount, etc.

[0004] The above-mentioned document information products are effective in identifying standardized documents, but the recognition accuracy and adaptability are limited when faced with documents issued by different institutions with variable formats and strong personalization. The layout format and information filling location of documents issued by different institutions are different. If it is a new document, it is difficult to extract valid information in the corresponding position using the old document template, and the recognition accuracy is not high. In order to adapt to each new document and develop a separate document template for it, developers need to do a lot of data training and debugging work, which is costly. Summary of the invention

[0005] The embodiments of the present application provide a document information recognition method, system and related products for efficiently improving the accuracy of information recognition of diversified documents and reducing maintenance costs.

[0006] A first aspect of an embodiment of the present application provides a method for identifying document information, comprising:

[0007] Searching for a target prompt word that matches the document to be identified from a prompt word library;

[0008] Acquire a historical conversation record; the example document recorded in the historical conversation record is of the same document type as the document to be identified, and the historical conversation record at least records the content layout format of the example document;

[0009] Based on the target prompt word and the indication of the content layout format, the document information designated to be output by the target prompt word is identified from the document to be identified.

[0010] Optionally, the process of constructing the prompt word library includes:

[0011] Collecting document samples of different document types, and constructing initial prompt words corresponding to each type of document sample;

[0012] Inputting each type of the document sample and its initial prompt word into a pre-trained information recognition model to recognize the document information output by each type of the document sample specified by the corresponding initial prompt word;

[0013] If the document information output by the document sample does not meet the expected result, analyze the reason why the document sample is recognized incorrectly, and modify the initial prompt word of the document sample based on the reason to obtain a new prompt word;

[0014] The document sample and its new prompt word are input into the information recognition model until the output document information meets the expected result, and the latest prompt word is used as the prompt word matching the document type of the document sample.

[0015] Optionally, if the document information is identified by using an information recognition model, then based on the target prompt word and the indication of the content layout format, before identifying the document information specified by the target prompt word to be output from the document to be identified, the method further includes:

[0016] Collecting error marking information and / or correction data of the document information output by the information recognition model from the client;

[0017] The information recognition model is incrementally learned using the error labeling information and / or the correction data to obtain a new information recognition model for replacing the original information recognition model.

[0018] Optionally, if the document information is identified by using an information recognition model, then based on the target prompt word and the indication of the content layout format, before identifying the document information specified by the target prompt word to be output from the document to be identified, the method further includes:

[0019] An attention mechanism is introduced into the information recognition model to enhance the information recognition model's focus on key information areas in the document; the key information includes at least one prompt word to specify the output document information.

[0020] Optionally, if the target prompt word cannot be found in the prompt word library, the method further includes:

[0021] Obtaining a default prompt word; the default prompt word is determined at least by summarizing the common content of various sample documents and the document information required to be output in the past;

[0022] The default prompt word is used as the target prompt word.

[0023] Optionally, based on the target prompt word and the indication of the content layout format, after identifying the document information to be outputted specified by the target prompt word from the document to be identified, the method further includes:

[0024] Performing a logical check on the identified document information;

[0025] If the verification fails, an error prompt is initiated, and / or at least the step of identifying the document information specified by the target prompt word to be output from the document to be identified is returned based on the instructions of the target prompt word and the content layout format, so as to retry to obtain the document information with successful verification.

[0026] Optionally, based on the target prompt word and the indication of the content layout format, before identifying the document information designated to be output by the target prompt word from the document to be identified, the method further includes:

[0027] Converting the document to be identified into an image format file;

[0028] The clarity of the image format file is improved to identify clearer document information therefrom.

[0029] When the method described in the first aspect of the present application is implemented, the content described in the second aspect of the present application can be adopted to achieve it.

[0030] A second aspect of an embodiment of the present application provides a document information identification system, including:

[0031] A processing unit, used for searching a target prompt word that matches the document to be identified from a prompt word library;

[0032] An acquisition unit, configured to acquire a historical conversation record; the example document recorded in the historical conversation record is of the same document type as the document to be identified, and the historical conversation record at least records the content layout format of the example document;

[0033] The processing unit is further configured to identify the document information designated to be output by the target prompt word from the document to be identified based on the target prompt word and the indication of the content typesetting format.

[0034] A third aspect of an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0035] The processor is configured to communicate with the memory and execute instructions in the memory to implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0036] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, they implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0037] A fifth aspect of an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0038] It can be seen from the above technical solutions that the embodiments of the present application have at least the following advantages:

[0039] The embodiment of the present application adopts a mechanism of combining historical conversation records with prompt words for information recognition, which can comprehensively improve the recognition accuracy of document information in many aspects and avoid errors caused by single factor recognition. In addition, the historical conversation records and target prompt words are in a corresponding relationship with the documents to be recognized, so that the embodiment of the present application can effectively adapt to different document types or diversified documents with constantly updated formats, reducing manual intervention and cost investment. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0041] It should be noted that, although the steps in the process diagrams (if any) involved in the embodiments are drawn in sequence as indicated by the arrows, unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0042] Figure 1 This is a schematic diagram of the system architecture of the document information identification method according to an embodiment of the present application;

[0043] Figure 2 A schematic diagram of a flow chart of a method for identifying document information according to an embodiment of the present application;

[0044] Figure 3This is a schematic diagram of an application service of the document information identification method according to an embodiment of the present application;

[0045] Figure 4 A structural diagram of a document information identification system according to an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0048] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims and drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0049] In the following description, similar expressions such as "a specific implementation" or "a specific example" are involved, which describe a subset of all possible embodiments, but it can be understood that "a specific implementation" or "a specific example" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. In the following description, the term multiple refers to at least two. In some specific examples, the numerical value mentioned in this application reaches the threshold value (if any), which may include the case where the former is greater than the latter of the threshold value; if similar expressions such as "any" or "at least one" are mentioned, it can specifically refer to any one of the listed examples or any combination of these examples.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0051] The method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 101 through the network, and the data storage system 100 can store the data that the server 101 needs to process or needs. The data storage system 100 can be integrated on the server 101, or it can be placed on the cloud or other network servers. The terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 101 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0052] The terminal 102 can obtain the document to be identified input by the user (such as "a bank online banking electronic receipt"), and send the document to be identified to the server 101. The server 101 can search for the target prompt word corresponding to the document to be identified from the prompt word library. The target prompt word may include: if it contains words such as "collection" and "income", it may be the payee information; obtain the historical session record, the example document recorded in the historical session record is of the same document type as the document to be identified, such as a certain "a bank online banking electronic receipt" in the previous quarter, and the historical session record at least records the content layout format of the example document, such as "payee information on the left, payee information on the right", etc.; the server 101 can identify the document information specified by the target prompt word from the document to be identified based on the target prompt word and the indication of the content layout format, for example, accurately identify the payee information such as the name of the payee in the document to be identified as Zhang San and the collection date as September 2, 2024.

[0053] It should be noted that the method provided in the embodiment of the present application can be implemented jointly by the terminal device and the server as described above, or it can be implemented entirely on the server side, or it can be implemented entirely on the terminal device side. The specific method can be determined according to the actual application scenario and is not limited here.

[0054] The method of the present application will be described in further detail below.

[0055] See also Figure 2 In a first aspect, the present application provides a specific embodiment of a document information identification method, which includes the following steps:

[0056] Step S21, searching the prompt word library for a target prompt word that matches the document to be identified;

[0057] Existing document information recognition technology is difficult to adapt to the diverse formats of different documents under different banks (such as receipts of various banks), and has high requirements for image quality, and has poor recognition effect on blurred or damaged receipts; at the same time, with the continuous update of bank receipt formats, the existing technology needs to frequently adjust templates and train and deploy new models to adapt to newly emerging bank receipts, which has high maintenance costs. For this reason, the embodiment of the present application proposes that a prompt word library can be summarized and sorted, and then the target prompt word corresponding to the document to be identified can be found from the prompt word library, so that the required document information can be quickly identified from the document to be identified through the guidance of the prompt word.

[0058] If the document to be identified is a document of the type "e-bank payment receipt", the following prompt words can be found for this type of document:

[0059] Identify the file content and output it in json format bankName: bank, createDate: transaction date, expendcustomer: payer, expendAcount: payer account number: incomeCustomer: payee, incomeAcount: payee account number, abstract summary or purpose, amount lowercase amount balance: balance, remark remark: transNo: serial number, curName: currency field content. Note: If the file contains multiple receipts, please return the contents of multiple receipts in a list from top to bottom. All amount fields are in numeric format. If a field is not provided, it is represented by an empty character. Make sure that the recognition order of the payer and the payee is consistent with the layout on the receipt, that is, the payer information is on the left and the payee information is on the right.

[0060] Step S22: Obtain historical session records;

[0061] The sample document recorded in the historical session record and the document to be identified belong to the same document type, and the historical session record at least records the content and format of the sample document. The document type can be specifically composed of the institution name and the business type, such as "e-bank payment receipt", "a-bank online banking electronic receipt", etc.

[0062] The present application embodiment introduces historical conversation records, which can assist the above-mentioned target prompt words to more effectively identify information. For example, historical conversation records can serve as auxiliary prompt words, so that more effective information can be retrieved through less background code or without compiling a large amount of code content, narrowing the search scope of the documents to be identified, thereby quickly and accurately completing the document information identification task. In addition, because the historical conversation records are a summary of real historical examples, such as recording the correct content layout format such as "the payee information is recorded on the left side of the document, the text layout format is aligned at both ends, and the date format is xxxx-xx-xx", in some examples, the historical conversation records can play a greater practical reference value than the target prompt words, that is, if the bill of lading (i.e., the document to be identified) cannot be accurately identified only by the prompt words, then the constructed historical conversation records (i.e., the combination of the target prompt words + historical conversation records) can be used to assist in identification. Of course, the number and / or specific content of the historical conversation records associated with each document type can be specifically set according to experience or actual conditions, and are not limited here.

[0063] Step S23: Based on the target prompt word and the indication of the content layout format, the document information to be outputted specified by the target prompt word is identified from the documents to be identified.

[0064] As described above, combined with the target prompt word + the instructions of the historical session record, the location of the required information in the document to be identified can be targeted, so as to quickly and accurately identify the document information specified by the target prompt word. For example, according to the limitation of the above default prompt word, output bankName: bank, createDate: transaction date, expendcustomer: payer, expendAcount: payer account number: incomeCustomer: payee, incomeAcount: payee account number, abstract summary or purpose, amount lowercase amount balance: balance, remark remark: transNo: serial number, curName: currency field content, etc. Among them, the symbol ":" is followed by the specific content identified from the document to be identified, such as "bankName: a bank, createDate: September 2, 2024", etc.

[0065] In summary, the embodiment of the present application adopts a mechanism of combining historical conversation records with prompt words for information recognition, which can comprehensively improve the recognition accuracy of document information in many aspects and avoid errors caused by single factors (such as recognition only by prompt words). In addition, the historical conversation records and target prompt words are in a corresponding relationship with the document to be recognized (the document type), so that the embodiment of the present application can effectively adapt to different document types or diversified documents with constantly updated formats, reducing manual intervention and cost investment.

[0066] Based on the above example descriptions, some specific possible implementation examples will be provided below. In actual applications, the implementation contents between these examples can be combined or implemented separately as needed according to the corresponding functional principles and application logic. If implemented in combination, the execution order between the combined examples can be determined according to their respective processing logics, which can be determined by the actual scenario.

[0067] based on Figure 2 The present application provides another specific embodiment of a document information identification method, which includes the following steps:

[0068] Step S21, searching the prompt word library for a target prompt word that matches the document to be identified;

[0069] In some examples, before step S23, the method of the embodiment of the present application may also include (document preprocessing): converting the document to be identified into an image format file; improving the clarity of the image format file to identify clearer document information therefrom.

[0070] Specifically, we can first unify the file format, that is, classify the uploaded document files by suffix or extension. If it is an image file, it will be processed as a single page; if it is a PDF file, we will analyze whether the PDF has multiple pages. If it has multiple pages, each page will be saved as an image file. In this way, all input file formats are unified into one image file. Then the content of the image file can be recognized through the OCR module (some large model interfaces can have their own file parsing interface).

[0071] The above-mentioned method for improving the clarity of the image format file may be at least one of the following methods:

[0072] 1. Image denoising: Use filters to remove image noise and improve image quality.

[0073] 2. Contrast enhancement: Adjust the image contrast to make the text information clearer.

[0074] 3. Size standardization: adjust all receipt images to a uniform size to facilitate model processing.

[0075] In some specific examples, we can first build a default receipt prompt word, which has a certain degree of universality, summarizes some overall rules of the bill of lading and stipulates the format of the content to be returned. The more common prompt word expression here can be: identify the file content and output it in json format bankName: bank, createDate: transaction date, incomeCustomer: payee, incomeAccount: payee account number, expendCustomer: payee, expendAccount: payee account number, abstract: summary or purpose, amount: lowercase amount, balance: balance, remark: remark, trtransNo: serial number, curName: currency field content. Note: If the file contains multiple receipts, please put the contents of multiple receipts in a list from top to bottom and return them. All amount fields are in numerical format. If any field is not provided, it is represented by a null character. Note: If the file contains multiple receipts, please put the contents of multiple receipts in a list from top to bottom and return them. All amount fields are in numerical format. If any field is not provided, it is represented by a null character.

[0076] The above-mentioned default prompt words (or universal prompt words) are only exemplary. The specific expressions can be set according to historical situations or experience and are not limited here. The universality of this default prompt word can be compared with other specific prompt words. For example, for a specific prompt word with a document type (bill_type) of "a bank invoice" or "b bank statement", the universality of the default prompt word is greater than that of this specific prompt word, that is, it can be applied to more document types.

[0077] In some specific examples, if the default prompt word cannot completely and accurately identify a certain type of receipt, a prompt word that matches this type of receipt can be defined separately and used as the content of the interface bill_type parameter. Therefore, in addition to constructing the above-mentioned default prompt words, special prompt words that correspond to each type of receipt can also be constructed one by one for each type of receipt. Therefore, the construction process of the prompt word library may include: collecting document samples of different document types, and initial prompt words corresponding to each type of document sample; inputting each type of document sample and its initial prompt words into a pre-trained information recognition model to identify the document information output by each type of document sample specified by the corresponding initial prompt words; if the document information output by a document sample does not meet the expected results, analyze the reasons why the document sample is recognized incorrectly, and modify the initial prompt words of the document sample based on the reasons to obtain new prompt words; input the document sample and its new prompt words into the information recognition model until the output document information meets the expected results, and use the latest prompt words as prompt words that match the document type of the document sample.

[0078] The above-mentioned document types can be specifically composed of the name of the institution and the type of business, and can include: at least one of the receipts, invoices, statements, receipts, etc. submitted by different institutions (such as a bank or other institutions or individuals), wherein the receipts can be specifically divided into: online banking electronic receipts, payment business receipts, transfer business receipts, etc., that is, the document types can be: a bank online banking electronic receipt, b bank online banking electronic receipt, a bank payment business receipt, b bank payment business receipt, a bank invoice, b bank invoice... Different banks can have the same or different typesetting formats for various documents (such as fonts, font sizes, line spacing, cell formats, text alignment, etc.), which can be determined according to actual conditions and are not limited here. For the convenience of explanation and understanding, the embodiments of this application mainly use bank receipts as an example.

[0079] The information recognition model of the embodiment of the present application can be called a generative artificial intelligence (AIGC, Artificial Intelligence Generated Content) big model. This AIGC model is a generalized concept, including but not limited to: 1. Application Programming Interface (API, Application Programming Interface) of the big model: such as GPT-4o interface, moonshotAI, Zhipu AI, etc. 2. Self-deployed open source models: MiniCPM-Llama3, Qwen-VL-Max, GLM-4v-9B, etc. 3. Industry big model: financial big model, etc.

[0080] Exemplarily, a certain type of receipt samples from various banks can be collected. The big model API attempts to identify and analyze the content of the receipt samples through initial prompt words (such as default prompt words). If the content recognition is found to be inaccurate, the prompt words are adjusted until the obtained prompt words can accurately identify this type of receipt. If errors in recognition are found during the process, the corresponding big model interface can be used to let the big model give an optimization strategy for the prompt words; this optimization strategy can be: the payee and the payee in the above-mentioned recognition content are reversed, please analyze the cause of the recognition error and give suggestions for modifying the prompt words; after obtaining the modified new prompt words, the above-mentioned attempt to identify and analyze the content of the receipt samples through the prompt words can be repeated until the output document information meets the expected results, and the latest prompt words are used as prompt words that match the document type (bill_type) of the document sample. At this point, corresponding matching prompt words can be constructed for each type of document sample, and each prompt word can be found through the bill_type index, for example:

[0081] When bill_type is "e-commercial bank transfer receipt", the corresponding prompt words can be as follows:

[0082] Identify the file content and output it in json format: bankName: bank, transactionDate: transaction date, expendCustomer: payer, expendAccount: payer account number, incomeCustomer: payee, incomeAccout: payee account number, abstract: summary or purpose, amount: lowercase amount, balance: balance remark: remark, transNo: serial number, curName: currency field content. Note: If the file contains multiple receipts, please return the contents of multiple receipts in a list from top to bottom. All amount fields are in numeric format. If a field is not provided, it is represented by a null character, and the payment and collection account number is determined in combination with the receipt record.

[0083] When bill_type is "f bank online banking electronic receipt", the corresponding prompt words can be as follows:

[0084] Identify bank electronic receipts. Please note: a receipt may contain information about the payer and the payee, but sometimes the information on one side may be empty. For accurate identification, please pay special attention to the following points: 1. Page layout identification: The page layout of the receipt is usually that the payer information is on the left and the payee information is on the right. Please identify the account name, account number and bank account fields based on this layout. 2. Summary or purpose information utilization: The summary or purpose field usually contains a brief description of the transaction, which helps to determine the nature of the transaction. It is generally used as a reference when one of the payer and payee information is empty. For example, if the summary contains words such as "payment", "payment", "purchase", etc., it may be the payee information; if it contains words such as "collection", "income", "interest", etc., it may be the payee information. 3. Field identification rules: If the summary or purpose field provides enough information to assist in judgment, but the account name or account number field is empty, please leave the corresponding field blank in the JSON output. At the same time, if the summary information is "interest" and the payment direction is not clearly indicated, it should be defaulted to the payee information. 4. If the summary or purpose field information is unclear, it should be combined with the opening bank. The opening bank information is an important clue to determine whether the account belongs to the payer or the payee. 5. The opening bank information is an important clue to determine whether the account belongs to the payer or the payee. It can be judged in combination with other receipts in the file to ensure that the account name and account number are correctly matched. 6. Double verification: After identifying the information, perform double verification to ensure that the information in all fields is logically consistent and complies with the normal transaction process. 7. JSON format output requirements: Please output the recognition results in the following format, including bank name, transaction date, payer and payee name and account number, summary or purpose, amount (numeric format), remarks, serial number and currency. If the information of one party is missing, please make sure that the corresponding field is an empty string. Sample JSON output: [{"bankName":"f bank","createDate":"transaction date","expendcustomer":"payer name (leave blank if empty)","expendAccount":"payer number (leave blank if empty)","incomecustomer":"payee name (leave blank if empty)","incomeAcount":"payee account number (leave blank if empty)","abstract":"summary or purpose","amount":"amount value","balance":"balance value (leave blank if none)","remark":"remark information","transNo":"serial number","curName":"currency"}] Please identify and output receipt information according to the above requirements.

[0085] As mentioned above, for the recognition of various receipts, the correspondence between various receipts and corresponding prompt words can be debugged before the release of this service, so that the corresponding prompt words can be indexed according to the document type.

[0086] In some examples, the priority of a special prompt word (i.e., the target prompt word corresponding to each document type) can be set higher than the general prompt word, that is, if a special prompt word can be found, the special prompt word is used first to identify the document to be identified, but if the target prompt word cannot be found in the prompt word library, the method of the embodiment of the present application can also include: obtaining a default prompt word; the default prompt word is determined by at least the common content of various sample documents and the summary of the document information required to be output in history; and the default prompt word is used as the target prompt word. This is equivalent to providing a redundancy mechanism, so that even if there is no special prompt word for the current document to be identified, there is a general prompt word as a substitute for identification, thereby ensuring that each document identification task is completed smoothly. The above-mentioned common content can include the general title name (incomeCustomer) of the payee, the amount, etc. Of course, as another possible implementation method, without considering the cost of user waiting time, it is also possible to construct a special prompt word for the current document to be identified as soon as possible to make the recognition result more accurate.

[0087] Step S22: Obtain historical session records;

[0088] For example, if the document type of the current document to be identified is "electronic receipt of online banking of a certain bank in China", the expression of the corresponding matching historical session record can be as follows: [{"bankName":"A certain bank in China","createDate":"September 18, 2023","expendCustomer":"A certain industrial company in Hubei","expendAccount":"0000040101030007","incomeCustomer":"A certain construction company in Hubei","incomeAccount":"0000002519200100002","abstract":"Inter-bank","amount":"11000","balance":"","remark":"Payment transaction sequence number: 9140496 Message type: Small customer initiates ordinary credit business entrustment date: 2023-09-18 Business type (type): Ordinary remittance","tansN0":"247 83436","curName":"RMB"},{"bankName":"A Chinese bank","createDate":"2023-09-15","expendCustomer":"A Hubei Construction Co., Ltd.","expendAcount":"1813002519200102402","incomecustomer":"","incomeAcount":"","abstract":"Interbank remittance fee","amount":"9","balance":"","remark":"Product name: PBOC domestic large remittance fee Name: Corporate interbank remittance fee Receivable amount: 10.00 Actual amount collected: 9.00 Charging channel: Business date: 2023-09-15 Business account number: 000002519200100002","transNo":"000000","curName":"RMB"}]

[0089] In some specific examples, each type of document may correspond to at least one historical session record; multiple historical session records of the same type of document (i.e., the documents have the same institution and business type) can be distinguished in chronological order. As for which of the multiple historical session records to use to identify the document, it can be determined by the needs. Of course, each historical session record can also be used separately, and then the recognition results corresponding to the output of each record can be combined as the final result to enhance the accuracy of the document recognition results. There is no specific limitation.

[0090] Step S23: Based on the target prompt word and the indication of the content layout format, the document information to be outputted specified by the target prompt word is identified from the documents to be identified.

[0091] In some specific examples, if an information recognition model is used to recognize document information, the above-mentioned information recognition model can be continuously improved based on user feedback and recognition results, including model optimization without retraining the model. Specifically, before step S23, the method of the embodiment of the present application may also include (introducing additional knowledge to fine-tune the information recognition model): collecting error marking information and / or correction data of the document information output by the information recognition model by the client; using the error marking information and / or correction data, incrementally learning the information recognition model to obtain a new information recognition model for replacing the original information recognition model.

[0092] For example, 1. User feedback collection: After the recognition result is output, a user interface is provided to allow users to mark the information of recognition errors, that is, to allow users to participate in the error correction process and input additional knowledge into the model. Afterwards, the server can collect the user's correction data on the recognition result.

[0093] 2. Error analysis: The server can analyze user feedback to identify common error types, such as incorrect recognition of receipts in a specific format (such as an electronic receipt for online banking of a certain bank), inaccurate recognition of specific fields (such as amounts in capital letters), etc.

[0094] 3. Model fine-tuning: The server fine-tunes the model based on the results of error analysis instead of completely retraining it. This may include adjusting model parameters or optimizing algorithms, such as adjusting the model's learning rate, batch size and other hyperparameters to improve the model's performance on specific types of data.

[0095] 4. Incremental learning: Use correction data submitted by users for incremental learning, that is, learn new data based on the existing model instead of replacing the original knowledge.

[0096] 5. Model update: Update the model regularly to include newly learned knowledge, but ensure that the overall performance of the model is not affected.

[0097] 6. Performance monitoring: Continuously monitor model performance, such as regularly monitoring the model's recall rate, accuracy, mean square error and other parameters to ensure that optimization measures are effective and to promptly identify new performance bottlenecks.

[0098] It should be noted that the incremental learning strategy mentioned above in the embodiment of the present application is different from the optimization strategy of retraining the model. Incremental learning: gradually add new data knowledge to the model, only process the newly added data knowledge, and there is no need to reprocess the old data knowledge, so that the model can be optimized while reducing the training cost of the model; that is, use the existing model knowledge and new data to update the model parameters, so that the model can gradually adapt to the new data while maintaining the memory of the old data. In contrast, retraining: is to use all the training data to retrain the model, which usually involves large-scale data processing and calculation. Each retraining will consume a lot of computing resources and time costs. Of course, regardless of the cost, the embodiment of the present application can also adopt the optimization strategy of retraining the model, which can be selected according to the needs.

[0099] In some specific examples, if the information recognition model is used to recognize the document information, the attention mechanism can be introduced as one of the optimization strategies of the model. Therefore, specifically, before step S23, the method of the embodiment of the present application can also include (introducing the attention mechanism to optimize the information recognition model): introducing the attention mechanism to the information recognition model to enhance the information recognition model's attention to the key information area in the document; the key information includes at least one prompt word to specify the output document information.

[0100] Specifically, strengthening the model's attention to key information areas (payee information, payer information, etc.) can be achieved by adjusting the model structure or assigning attention weights. Specifically, the strategy for adjusting the model structure can be to add an attention layer (such as a convolutional layer or a pooling layer) at the appropriate position of the model for weighted processing. For example, in a convolutional neural network (CNN), by adding a convolutional layer, higher-level features can be extracted, and these features are often more representative of the key information in the image; and adding a pooling layer can help reduce the size of the feature map while retaining important information, so that the model pays more attention to the key area. In addition, the strategy for assigning attention weights can be to introduce spatial attention: assigning attention weights in the spatial dimension so that the model can focus on specific areas in the image. This is usually achieved by generating an attention map of the same size as the input image. Each pixel value in the map represents the importance of the corresponding position. The larger the pixel value, the more important it is. For example, it can be set empirically that the pixel value at the location of the bank icon logo is smaller than the pixel value at the location of the transaction amount.

[0101] Furthermore, the optimization strategy for the information identification model in the embodiment of the present application may also include at least one of the following strategies:

[0102] 1. Data enhancement: Perform data enhancement on documents that are incorrectly recognized by the model, such as rotating, scaling, and cropping the images of the original documents, to generate more training samples for the model optimization process.

[0103] 2. Ensemble learning: Integrate the prediction results of multiple models to improve the overall recognition accuracy.

[0104] 3. Transfer learning: Use the model trained on other tasks as a starting point and adapt it to the current task through transfer learning. That is, the models for different types of documents such as receipts, statements, invoices, etc. can be borrowed from each other to adapt to the recognition of a certain type of document.

[0105] In some specific examples, the accuracy of the recognition result can be ensured by logical verification. Therefore, after step S23, the method of the embodiment of the present application may also include (accuracy verification): performing a logical verification on the recognized document information; if the verification fails, initiating an error prompt, and / or at least returning an instruction based on the target prompt word and the content layout format, and identifying the document information specified by the target prompt word from the document to be recognized, so as to retry to obtain the document information that has been successfully verified.

[0106] Specifically, after the document to be identified, the target prompt word and the historical conversation record are input into the information recognition model as above, the model can output the corresponding document information as the recognition result. Next, the recognition result can be logically checked, such as checking whether the recognized amount is a reasonable value (i.e., it should be a numeric type and a positive number).

[0107] In summary, accuracy verification is generally based on defining verification rules based on the relationship between the fields of the bank receipt and the type of the output field, or the logical relationship between the accounts on multiple pages in the same file. The defined verification rules can be at least one of the following rules, which can be set according to the actual situation. This is only an example:

[0108] The amount field of the bank must be a numeric type [Rule 1]; the payment account and the receiving account are different [Rule 2]. If the recognition result of an external transfer bill is "the payment account and the receiving account are the same", it can generally be considered that the recognition result is wrong; for multiple receipts on the same page, the account name + account number should be unique (except for those not listed) [Rule 3]. Therefore, if the account number of the payee / payee account A1 identified in the first receipt is B1, then if the recognition result of another receipt is: the payee / payee account corresponding to account B1 is C1, then it can generally be considered that the recognition result is wrong. Because the account names corresponding to the same bank account should not be different, the countermeasures can be to optimize the model, re-recognize the document information, or prompt to check the document content.

[0109] In summary, the embodiments of the present application are mainly based on the generalization ability and common sense learning ability of the AIGC large model, and dynamically adjust the prompt words to adapt to the recognition of bank receipts of different formats. Specifically, it can be summarized as follows: preprocessing of documents (such as file format unification, image denoising), loading of pre-trained AIGC models or directly selecting a model API to ensure that the computing environment required for model operation is configured normally; building a prompt word library and building historical session records to assist prompt words in document recognition; (key content matching) regular expressions can be used to try to match the content of the document to be identified to the document type already defined in the system, because the identification content often includes the bank name and business type of the receipt, and a previously defined document type (that is, the document type of the document to be identified) can be automatically matched through matching; the accuracy of the recognition result is ensured through logical verification.

[0110] Special note: for the functional scenario of receipt identification, the encapsulated service interface name can be recorded as huidan_identify. In this scenario, it is generally necessary to upload the document file and its document type (ie, bill_type parameter) when calling. The file type can be limited to supporting PDF format and image file format.

[0111] A prompt word automatic matching module can be designed: this module is responsible for automatically matching the corresponding prompt words in the prompt word library according to the files uploaded by the user, so that even if the specific document type of the document is not clear (such as the user does not explicitly tell the document type), the module can preliminarily identify the title (content) and other contents of the document by itself to determine the document type, and thus can also match a more appropriate prompt word for input into the AIGC model for recognition.

[0112] Inputting different prompt words into the large model can solve the application of different scenarios (such as identifying document information such as invoices or remittance slips). Therefore, a flexible interface definition method can be used to customize different application scenarios. For example, in addition to the above-mentioned recognition scenario of bank receipts, you can also define your own interface to identify information on documents such as invoices or remittance slips. The service interface name corresponding to the invoice recognition scenario can be recorded as invoice_identify. This recognition process is similar to the above-mentioned recognition of bank receipts, and the details are not repeated here.

[0113] In addition, the embodiments of the present application can develop paradigm changes under the support of the above-mentioned large model: Figure 3As shown, based on the development of prompt words, developers or even product managers can fine-tune the recognition of specific receipts by defining prompt words in the service interface in the future, and return the results through sample data. After verification, the service can be released. In simple terms, when faced with a new version of the receipt format submitted, the product manager can use this new receipt to try to define the prompt words on the system interface, such as adding, deleting or modifying certain words on the basis of the existing prompt words to build new prompt words that can adapt to this new receipt. There is no need for developers to develop supporting running codes in the background for new receipts every time, which reduces the workload of developers and is equivalent to dynamically opening the adjustment authority of the prompt words to users, thereby improving the user experience.

[0114] Considering that document identification requests may be highly concurrent, in some instances, in order to improve service concurrency performance, specific functions can be implemented asynchronously. Therefore, an interface for result query can be published on demand. For example, when faced with multiple receipts that may be dozens of pages in size, the user can be prompted to wait for a certain period of time and then query the results of the receipt through a certain identification number in a specific result query interface, so as to reduce the high concurrency processing pressure on the server and the user's timeout waiting time.

[0115] It can be seen from the above description that the embodiments of the present application have the following advantages:

[0116] High recognition accuracy: Adapt to bank receipts of different formats and improve recognition accuracy.

[0117] Quick adaptability: By adjusting the prompt words, you can quickly adapt to the new receipt format without making a lot of changes to the background code. Dynamic management of prompt words can successfully complete the document recognition process.

[0118] Intelligent processing: Model self-learning and optimization, reducing manual intervention and maintenance costs.

[0119] User-friendly: Simplify the operation process and improve the user experience.

[0120] Compared to Figure 2 The examples shown illustrate that the above-mentioned additional or detailed example contents or possible implementation methods (such as document preprocessing and accuracy verification) do not necessarily have to be executed during the specific implementation. If more than two examples or possible implementation methods are added, these examples or possible implementation methods can be implemented in combination or separately. If implemented in combination, the execution order of the combined examples can be determined according to their respective processing logic, which can be determined according to the actual scenario.

[0121] See also Figure 4 The second aspect of the present application provides a specific embodiment of a document information identification system, the system comprising:

[0122] The processing unit 401 is used to search the prompt word library for a target prompt word that matches the document to be identified;

[0123] The acquisition unit 402 is used to acquire the historical conversation record; the example document recorded in the historical conversation record is of the same document type as the document to be identified, and the historical conversation record at least records the content layout format of the example document;

[0124] The processing unit 401 is further configured to identify the document information to be outputted specified by the target prompt word from the document to be identified based on the target prompt word and the indication of the content typesetting format.

[0125] In some examples, the processing unit 401 is specifically configured to:

[0126] Collect document samples of different document types, and construct corresponding initial prompt words for each type of document sample;

[0127] Input various document samples and their initial prompt words into the pre-trained information recognition model to identify the document information output by various document samples specified by the corresponding initial prompt words;

[0128] If the document information output by a document sample does not meet the expected result, analyze the reason why the document sample is recognized incorrectly, and modify the initial prompt word of the document sample based on the reason to obtain a new prompt word;

[0129] The document sample and its new prompt word are input into the information recognition model until the output document information meets the expected result, and the latest prompt word is used as the prompt word matching the document type of the document sample.

[0130] In some examples, if the information recognition model is used to recognize the document information, the processing unit 401 is further used to:

[0131] Collecting error marking information and / or correction data of the document information output by the information recognition model from the client;

[0132] Using the error-labeled information and / or the correction data, the information recognition model is incrementally learned to obtain a new information recognition model for replacing the original information recognition model.

[0133] In some examples, if the information recognition model is used to recognize the document information, the processing unit 401 is further used to:

[0134] An attention mechanism is introduced into the information recognition model to enhance the information recognition model's attention to key information areas in the document; the key information includes at least one prompt word to specify the output document information.

[0135] In some examples, if the target prompt word cannot be found in the prompt word library, the processing unit 401 is further configured to:

[0136] Obtaining a default prompt word; the default prompt word is determined at least by the common content of various sample documents and the summary of the document information required to be output in the past;

[0137] Use the default cue word as the target cue word.

[0138] In some examples, the processing unit 401 is further configured to:

[0139] Perform logical verification on the identified document information;

[0140] If the verification fails, an error prompt is initiated, and / or at least an instruction based on the target prompt word and the content layout format is returned to identify the document information specified by the target prompt word from the document to be identified, so as to retry to obtain the document information with successful verification.

[0141] In some examples, the processing unit 401 is further configured to:

[0142] Convert the document to be identified into an image format file;

[0143] Improve the clarity of image format files to identify clearer document information from them.

[0144] In the embodiment of the present application, the operations performed by each unit of the document information identification system are similar to the operations described in the first aspect or any specific method embodiment of the first aspect, and will not be described in detail here. Of course, the specific implementation process of each operation in the first aspect of the present application can also refer to the relevant description of the second aspect.

[0145] See also Figure 5 The electronic device 500 of the embodiment of the present application may include one or more processors (such as central processing units (CPU)) 501 and a memory 505, in which one or more applications or data are stored.

[0146] The memory 505 may be a volatile storage or a persistent storage. The program stored in the memory 505 may include one or more modules, each of which may include a series of instruction operations in the electronic device. Furthermore, the processor 501 may be configured to communicate with the memory 505 and execute a series of instruction operations in the memory 505 on the electronic device 500.

[0147] The electronic device 500 may also include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input and output interfaces 504, and / or one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0148] The processor 501 can execute the operations performed by the aforementioned first aspect or any specific method embodiment of the first aspect, and the details will not be repeated here.

[0149] The present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect or any specific implementation of the first aspect.

[0150] The present application provides a computer program product comprising instructions or a computer program. When the computer program product is run on a computer, the computer is enabled to execute the method described in the first aspect or any specific implementation of the first aspect.

[0151] It is understood that in various embodiments of the present application, the sequence number of each step does not mean the order of execution, and the execution order of each step should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The operation content added or refined in each example scheme of the above method, system or device (if any) does not necessarily have to be executed in the specific implementation. If more than two operations are added, these operations can be implemented in combination or separately, depending on the actual scenario.

[0152] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system (if any) and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0153] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system or device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0154] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product (or computer program product) is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a business server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A document information identification method, characterized in that: include: Searching for a target prompt word that matches the document to be identified from a prompt word library; Acquire a historical conversation record; the example document recorded in the historical conversation record is of the same document type as the document to be identified, and the historical conversation record at least records the content layout format of the example document; Based on the target prompt word and the indication of the content layout format, the document information designated to be output by the target prompt word is identified from the document to be identified.

2. The document information recognition method according to claim 1, characterized in that: The construction process of the prompt word library includes: Collecting document samples of different document types, and constructing initial prompt words corresponding to each type of document sample; Inputting each type of the document sample and its initial prompt word into a pre-trained information recognition model to recognize the document information output by each type of the document sample specified by the corresponding initial prompt word; If the document information output by the document sample does not meet the expected result, analyzing the reason why the document sample is recognized incorrectly, and modifying the initial prompt word of the document sample based on the reason to obtain a new prompt word; The document sample and its new prompt word are input into the information recognition model until the output document information meets the expected result, and the latest prompt word is used as the prompt word matching the document type of the document sample.

3. The document information recognition method according to claim 1 or 2, characterized in that: If the document information is identified by using the information recognition model, then based on the target prompt word and the indication of the content layout format, before identifying the document information specified by the target prompt word to be output from the document to be identified, the method further includes: Collecting error marking information and / or correction data of the document information output by the information recognition model from the client; The information recognition model is incrementally learned using the error labeling information and / or the correction data to obtain a new information recognition model for replacing the original information recognition model.

4. The document information recognition method according to claim 1 or 2, characterized in that: If the document information is identified by using the information recognition model, then based on the target prompt word and the indication of the content layout format, before identifying the document information specified by the target prompt word to be output from the document to be identified, the method further includes: An attention mechanism is introduced into the information recognition model to enhance the information recognition model's focus on key information areas in the document; the key information includes at least one prompt word to specify the output document information.

5. The document information recognition method according to claim 1 or 2, characterized in that: If the target prompt word cannot be found in the prompt word library, the method further includes: Obtaining a default prompt word; the default prompt word is determined at least by summarizing the common content of various sample documents and the document information required to be output in the past; The default prompt word is used as the target prompt word.

6. The document information recognition method according to claim 1 or 2, characterized in that: After identifying the document information designated to be output by the target prompt word from the document to be identified based on the target prompt word and the indication of the content layout format, the method further includes: Performing a logical check on the identified document information; If the verification fails, an error prompt is initiated, and / or at least the step of identifying the document information specified by the target prompt word to be output from the document to be identified is returned based on the instructions of the target prompt word and the content layout format, so as to retry to obtain the document information with successful verification.

7. The document information recognition method according to claim 1 or 2, characterized in that: Based on the target prompt word and the indication of the content layout format, before identifying the document information designated to be output by the target prompt word from the document to be identified, the method further includes: Converting the document to be identified into an image format file; The clarity of the image format file is improved to identify clearer document information therefrom.

8. A document information recognition system, characterized in that: include: A processing unit, used for searching a target prompt word that matches the document to be identified from a prompt word library; An acquisition unit, configured to acquire a historical conversation record; the example document recorded in the historical conversation record is of the same document type as the document to be identified, and the historical conversation record at least records the content layout format of the example document; The processing unit is further configured to identify the document information designated to be output by the target prompt word from the document to be identified based on the target prompt word and the indication of the content typesetting format.

9. An electronic device, characterized in that: include: Processor and memory; The processor is configured to communicate with the memory and execute instructions in the memory to implement the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The computer program product comprises computer instructions, which implement the method according to any one of claims 1 to 7 when executed by a processor.

Images