Contract information extraction and automatic warehousing method and system based on large model

Abstract

The invention discloses a contract information extraction and automatic storage method and system based on a large model, and the method comprises the steps: receiving a contract file, and converting the contract file into a character string in a preset coding format; extracting text content from the contract file according to the file type of the contract file; according to different types of contract files, a prompt word template of a corresponding type is constructed, and the prompt word template comprises a contract text and a field needing to be extracted; according to the file type of the contract file, based on the prompt word template of the corresponding type, initiating a preset round of large model questions and answers to the text content; and analyzing and merging results returned by the large model, and storing the results in a database. According to the technical scheme, automatic processing of the whole process of contract information extraction and storage is achieved, data management is optimized, manual intervention is reduced, and the accuracy and efficiency of overall data processing are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of feature extraction, and in particular to a method for extracting and automatically storing contract information based on a large model and a system for extracting and automatically storing contract information based on a large model. Background Art

[0002] In the process of enterprise operation, there are many types of contract documents, including project contract documents, sales bidding documents, acceptance report documents, etc. These contract documents are not only in various forms, including Word, PDF (ordinary version and scanned version), pictures (JPG, PNG), etc., but also complex in content, and the key information that needs to be extracted is different.

[0003] Traditional contract information processing methods mainly rely on manual operations, where staff need to manually read and enter the contents of contract documents. This method is not only labor-intensive and inefficient, but also prone to errors, resulting in low efficiency and high costs. In addition, when processing a large number of contract documents, manual operations cannot meet timeliness requirements.

[0004] Although some companies use optical character recognition (OCR) technology and template matching methods to improve the efficiency of contract information extraction, these technologies still face many challenges when processing contract documents with complex formats and diverse contents.

[0005] Traditional OCR technology has limited recognition accuracy for scanned PDF and image files, especially for handwriting and low-quality scans. Even for ordinary PDF and Word files, template matching methods are difficult to flexibly handle contract texts of different formats and layouts.

[0006] Moreover, the existing technical solutions still require a lot of manual sorting and proofreading after extracting information, which makes it difficult to achieve true automation. This not only increases labor costs, but also easily leads to data omissions and errors, affecting the accuracy and completeness of information extraction.

[0007] In the process of data migration and information processing, data consistency comparison is an important link. The traditional line-by-line comparison method is inefficient, especially when there are many data tables and large amounts of data. It requires a lot of manpower to configure quality audit rules, which is not only costly but also prone to errors. In addition, for data without primary keys, it is difficult to match source and target data line by line, and traditional audit methods cannot meet batch audit and timeliness requirements.

[0008] These defects indicate that the existing contract information processing technology still has a lot of room for improvement in dealing with complex and diverse contract documents and efficient and automated information extraction and management. As enterprises continue to increase their demand for data processing efficiency and accuracy, an innovative solution is urgently needed to overcome the above defects and improve the overall level of contract information processing. Summary of the invention

[0009] In response to the above problems, the present invention provides a method and system for contract information extraction and automatic warehousing based on a big model. By combining optical character recognition (OCR) technology and big model technology, automatic extraction and warehousing of contract information is realized, and contract texts in various formats can be efficiently identified and extracted, and the extracted information can be structured and stored in a database, thereby realizing automatic processing of the entire information extraction process, optimizing data management, reducing manual intervention, and improving the accuracy and efficiency of overall data processing.

[0010] To achieve the above objectives, the present invention provides a method for extracting and automatically storing contract information based on a large model, comprising:

[0011] Receiving a contract document and converting the contract document into a character string in a preset encoding format;

[0012] Extracting text content from the contract document according to the file type of the contract document;

[0013] Constructing corresponding types of prompt word templates for different types of contract documents, wherein the prompt word templates include the contract text and the fields to be extracted;

[0014] According to the file type of the contract document, based on the corresponding type of prompt word template, a preset round of large model question and answer is initiated on the text content;

[0015] The results returned by the large model are parsed and merged and then stored in the database.

[0016] In the above technical solution, preferably, the receiving of the contract document and converting the contract document into a character string in a preset coding format specifically includes:

[0017] The contract file uploaded by the user is received through the HTTP interface, and the contract file is converted into a Base64-encoded string using a file processing program.

[0018] In the above technical solution, preferably, the text content is extracted from the contract document according to the file type of the contract document, and the specific process includes:

[0019] For contract files of Word type and ordinary PDF type, directly read the contents of the contract files;

[0020] For scanned PDF contract documents, convert them into images, identify the image orientation and rotate them to the positive orientation, and obtain the text in the image based on PaddleOCR recognition;

[0021] For image files, directly use PaddleOCR to recognize the text in the image;

[0022] The text obtained by reading or recognizing is segmented in a preset round to obtain sliced ​​text as the extracted text content.

[0023] In the above technical solution, preferably, the process of constructing corresponding types of prompt word templates for different types of contract documents includes:

[0024] Generate prompt words according to input parameters;

[0025] If it is detected that the type parameter is not passed in, the interface call fails and an error message is returned;

[0026] If an incoming prompt parameter is detected, the incoming prompt value is used directly and the placeholder is replaced;

[0027] If it is detected that the prompt parameter is not passed in, the built-in prompt word template is used and the placeholder is replaced;

[0028] Different types of prompt word templates are constructed for different types of contract documents, so that the same type of prompt word templates can be used to extract the contract documents during the large-model question-answering process.

[0029] In the above technical solution, preferably, according to the file type of the contract document and based on the corresponding type of prompt word template, a preset round of large model question and answer is initiated on the text content, and the specific process includes:

[0030] According to the file type of the uploaded contract file, the same type of prompt word template is used to call the large model interface to perform preset rounds of questions and answers;

[0031] In the large model question-answering process, the prompt words in the prompt word template are sequentially input into the large model to obtain a return result in JSON format;

[0032] If the Q&A fails or the result is not in JSON format, repeat the Q&A;

[0033] If the correct JSON result still cannot be obtained, abandon the current round of extraction and continue to extract the next text and field.

[0034] In the above technical solution, preferably, the results returned by the large model are parsed and merged and then stored in a database, and the specific process includes:

[0035] The JSON result returned by the large model for the text content of the entire contract document is parsed, and the extracted field contents are merged and stored in the MySQL database.

[0036] The present invention also proposes a large model-based contract information extraction and automatic warehousing system, which uses the large model-based contract information extraction and automatic warehousing method disclosed in any one of the above technical solutions, including:

[0037] A contract format conversion module, used for receiving a contract document and converting the contract document into a character string in a preset coding format;

[0038] A contract content extraction module, used to extract text content from the contract file according to the file type of the contract file;

[0039] A prompt template construction module, used to construct prompt word templates of corresponding types for different types of contract documents, wherein the prompt word templates include the contract text and the fields to be extracted;

[0040] A large model extraction module is used to initiate a preset round of large model question and answer on the text content based on the file type of the contract document and the prompt word template of the corresponding type;

[0041] The result parsing and storage module is used to parse and merge the results returned by the large model and store them in the database.

[0042] In the above technical solution, preferably, the contract content extraction module is specifically used to:

[0043] For contract files of Word type and ordinary PDF type, directly read the contents of the contract files;

[0044] For scanned PDF contract documents, convert them into images, identify the image orientation and rotate them to the positive orientation, and obtain the text in the image based on PaddleOCR recognition;

[0045] For image files, directly use PaddleOCR to recognize the text in the image;

[0046] The text obtained by reading or recognizing is segmented in a preset round to obtain sliced ​​text as the extracted text content.

[0047] In the above technical solution, preferably, the prompt template construction module is specifically used to:

[0048] Generate prompt words according to input parameters;

[0049] If it is detected that the type parameter is not passed in, the interface call fails and an error message is returned;

[0050] If an incoming prompt parameter is detected, the incoming prompt value is used directly and the placeholder is replaced;

[0051] If it is detected that the prompt parameter is not passed in, the built-in prompt word template is used and the placeholder is replaced;

[0052] Different types of prompt word templates are constructed for different types of contract documents, so that the same type of prompt word templates can be used to extract the contract documents during the large-model question-answering process.

[0053] In the above technical solution, preferably, the large model extraction module is specifically used for:

[0054] According to the file type of the uploaded contract file, the same type of prompt word template is used to call the large model interface to perform preset rounds of questions and answers;

[0055] In the large model question-answering process, the prompt words in the prompt word template are sequentially input into the large model to obtain a return result in JSON format;

[0056] If the Q&A fails or the result is not in JSON format, repeat the Q&A;

[0057] If the correct JSON result still cannot be obtained, abandon the current round of extraction and continue to extract the next text and field.

[0058] Compared with the prior art, the beneficial effects of the present invention are as follows: by combining optical character recognition (OCR) technology and large model technology, the automatic extraction and storage of contract information is realized, and contract texts in various formats can be efficiently identified and extracted, and the extracted information is structured and stored in the database, thereby realizing the automatic processing of the entire information extraction process, optimizing data management, reducing manual intervention, and improving the accuracy and efficiency of overall data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a flow chart of a method for extracting and automatically storing contract information based on a large model disclosed in an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of the modules of a large-model-based contract information extraction and automated warehousing system disclosed in an embodiment of the present invention.

[0061] In the figure, the corresponding relationship between each component and the reference numeral is as follows:

[0062] 1. Contract format conversion module, 2. Contract content extraction module, 3. Prompt template construction module, 4. Large model extraction module, 5. Result parsing and storage module. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0065] like Figure 1 As shown, a method for extracting and automatically storing contract information based on a large model provided by the present invention includes:

[0066] Receive the contract document and convert the contract document into a character string in a preset encoding format;

[0067] Extracting text content from the contract document according to the file type of the contract document;

[0068] Construct corresponding prompt word templates for different types of contract documents, wherein the prompt word templates include the contract text ({inputText}) and the fields to be extracted ({fields});

[0069] According to the file type of the contract document, based on the corresponding type of prompt word template, a preset round of large-scale model question and answer is initiated for the text content;

[0070] The results returned by the large model are parsed and merged and stored in the database.

[0071] In this implementation, by combining optical character recognition (OCR) technology and large model technology, automatic extraction and storage of contract information is achieved, which can efficiently identify and extract contract texts in various formats, and store the extracted information in a structured manner in a database, thereby realizing automated processing of the entire information extraction process, optimizing data management, reducing manual intervention, and improving the accuracy and efficiency of overall data processing.

[0072] In the above implementation, preferably, receiving the contract document and converting the contract document into a character string in a preset coding format includes:

[0073] The contract file uploaded by the user is received through the HTTP interface, and the contract file is converted into a Base64-encoded string using the file processing program for subsequent processing.

[0074] In the above implementation, preferably, according to the file type of the contract file, the text content is extracted from the contract file, and the specific process includes:

[0075] For contract files of Word type and ordinary PDF type, directly read the contents of the contract file;

[0076] For scanned PDF contract files, convert them into images, identify the image orientation and rotate them to the positive orientation, and use PaddleOCR to identify the text in the image.

[0077] For image files (such as JPG and PNG formats), directly use PaddleOCR to recognize the text in the image;

[0078] The text obtained by reading or recognizing is segmented in a preset round to obtain sliced ​​text as the extracted text content.

[0079] In the above implementation, preferably, corresponding types of prompt word templates are constructed for different types of contract documents, and the specific process includes:

[0080] Generate prompt words according to input parameters;

[0081] If it is detected that the type parameter is not passed in, the interface call fails and an error message is returned;

[0082] If an incoming prompt parameter is detected, the incoming prompt value is used directly and the placeholder is replaced;

[0083] If it is detected that the prompt parameter is not passed in, the built-in prompt word template is used and the placeholder is replaced;

[0084] For different types of contract documents (such as sales contracts and purchase contracts), different types of prompt word templates are constructed so that the same type of prompt word templates can be used for extraction during the large-model question-answering process for the contract documents.

[0085] In the above implementation, preferably, according to the file type of the contract document, based on the corresponding type of prompt word template, a preset round of large model question and answer is initiated for the text content, and the specific process includes:

[0086] According to the file type of the uploaded contract file, the same type of prompt word template is used to call the large model interface to perform preset rounds of questions and answers;

[0087] In the big model question-answering process, the prompt words in the prompt word template are input into the big model in sequence to obtain the return results in JSON format;

[0088] If the Q&A fails or the result is not in JSON format, repeat the Q&A;

[0089] If the correct JSON result still cannot be obtained, abandon the current round of extraction and continue to extract the next text and field.

[0090] In the above implementation, preferably, the results returned by the large model are parsed and merged and then stored in the database. The specific process includes:

[0091] The JSON result returned by the large model for the text content of the entire contract document is parsed, and the extracted field contents are merged and stored in the MySQL database, including a retry mechanism to ensure the accuracy and integrity of the data.

[0092] like Figure 2 As shown, the present invention further proposes a contract information extraction and automatic warehousing system based on a large model, which applies the contract information extraction and automatic warehousing method based on a large model disclosed in any one of the above embodiments, including:

[0093] The contract format conversion module 1 is used to receive the contract document and convert the contract document into a character string in a preset coding format;

[0094] The contract content extraction module 2 is used to extract text content from the contract file according to the file type of the contract file;

[0095] A prompt template construction module 3 is used to construct a prompt word template of a corresponding type for different types of contract documents, wherein the prompt word template includes the contract text and the fields to be extracted;

[0096] The large model extraction module 4 is used to initiate a preset round of large model question and answer on the text content based on the file type of the contract document and the prompt word template of the corresponding type;

[0097] The result analysis and storage module 5 is used to analyze and merge the results returned by the large model and store them in the database.

[0098] In the above implementation, preferably, the contract content extraction module 2 is specifically used for:

[0099] For contract files of Word type and ordinary PDF type, directly read the contents of the contract file;

[0100] For scanned PDF contract files, convert them into images, identify the image orientation and rotate them to the positive orientation, and use PaddleOCR to identify the text in the image.

[0101] For image files, directly use PaddleOCR to recognize the text in the image;

[0102] The text obtained by reading or recognizing is segmented in a preset round to obtain sliced ​​text as the extracted text content.

[0103] In the above implementation, preferably, the prompt template construction module 3 is specifically used for:

[0104] Generate prompt words according to input parameters;

[0105] If it is detected that the type parameter is not passed in, the interface call fails and an error message is returned;

[0106] If an incoming prompt parameter is detected, the incoming prompt value is used directly and the placeholder is replaced;

[0107] If it is detected that the prompt parameter is not passed in, the built-in prompt word template is used and the placeholder is replaced;

[0108] Different types of prompt word templates are constructed for different types of contract documents, so that the same type of prompt word templates can be used to extract the contract documents during the large-model question-answering process.

[0109] In the above implementation, preferably, the large model extraction module 4 is specifically used for:

[0110] According to the file type of the uploaded contract file, the same type of prompt word template is used to call the large model interface to perform preset rounds of questions and answers;

[0111] In the big model question-answering process, the prompt words in the prompt word template are input into the big model in sequence to obtain the return results in JSON format;

[0112] If the Q&A fails or the result is not in JSON format, repeat the Q&A;

[0113] If the correct JSON result still cannot be obtained, abandon the current round of extraction and continue to extract the next text and field.

[0114] According to the big model-based contract information extraction and automatic warehousing system disclosed in the above-mentioned implementation mode, the functions to be implemented by each module correspond to the steps in the big model-based contract information extraction and automatic warehousing method disclosed in the above-mentioned implementation mode. During the implementation process, operations are performed in reference to the above-mentioned implementation mode, which will not be repeated here.

[0115] According to the large model-based contract information extraction and automatic warehousing method and system disclosed in the above implementation mode, after the above implementation process, the following effects are achieved.

[0116] (1) Multiple document format processing and unified conversion: The present invention can efficiently process multiple document formats, including Word, PDF (normal version and scanned version), pictures (JPG, PNG), etc., by converting them into Base64 strings, and then performing subsequent text extraction and processing in a unified manner. This unified conversion method simplifies the processing flow of documents in different formats and improves the compatibility and processing efficiency of the system.

[0117] (2) Automated structured data extraction: By calling the structured extraction interface, the present invention can automatically extract the content of specified fields from different types of contract documents and output the results in JSON format. This automated process reduces manual intervention, improves the accuracy and efficiency of data extraction, and enables enterprises to more quickly obtain and utilize key information in documents.

[0118] (3) Highly accurate text extraction and recognition: The present invention uses advanced text extraction and recognition technologies, including direct text reading and PaddleOCR technology, which can accurately extract text content from regular PDF, scanned PDF and image files. In particular, for scanned PDF and low-quality images, the system can improve recognition accuracy through image preprocessing and OCR technology, achieving an accuracy rate of 99% for printed text recognition and 90% for handwritten text recognition.

[0119] (4) Flexible prompt word generation and large model call: The present invention designs a flexible prompt word generation strategy, which automatically generates prompt words according to input parameters and calls the large model interface for question answering. Whether using built-in templates or custom prompt words, the system can efficiently obtain structured data and ensure the accuracy and usability of question answering results.

[0120] (5) Reliable result processing and data storage: The present invention provides a complete result processing and storage mechanism that can parse the JSON results returned by the large model and store the extracted field content in the MySQL database. The system's built-in retry mechanism ensures the accuracy and integrity of the data. Even if the large model interface call fails, the reliability of the data extraction process can be guaranteed through the retry mechanism.

[0121] (6) Modular design and high scalability: The present invention adopts a modular system architecture design, and each functional module works independently and collaboratively, including a file processing module, a text extraction module, a prompt word generation module, a large model calling module, and a result processing module. This modular design improves the scalability and maintainability of the system, and facilitates the subsequent expansion of functions and upgrades of the system.

[0122] (7) Complete risk identification and countermeasures: This paper proposes corresponding countermeasures for potential risks such as text extraction accuracy, large model interface call stability, and data storage security. For example, through multiple OCR technology comparisons and interface call retry mechanisms, the accuracy of text extraction and the stability of the system are ensured, while the security of data storage is guaranteed through a temporary file processing mechanism.

[0123] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for extracting and automatically storing contract information based on a large model, characterized in that: include: Receiving a contract document and converting the contract document into a character string in a preset encoding format; Extracting text content from the contract document according to the file type of the contract document; Constructing corresponding types of prompt word templates for different types of contract documents, wherein the prompt word templates include the contract text and the fields to be extracted; According to the file type of the contract document, based on the corresponding type of prompt word template, a preset round of large model question and answer is initiated on the text content; The results returned by the large model are parsed and merged and then stored in the database.

2. The method for extracting and automatically storing contract information based on a large model according to claim 1 is characterized in that: The receiving of the contract document and converting the contract document into a character string in a preset coding format specifically includes: The contract file uploaded by the user is received through the HTTP interface, and the contract file is converted into a Base64-encoded string using a file processing program.

3. The method for extracting and automatically storing contract information based on a large model according to claim 2 is characterized in that: The process of extracting text content from the contract file according to the file type of the contract file includes: For contract files of Word type and ordinary PDF type, directly read the contents of the contract files; For scanned PDF contract documents, convert them into images, identify the image orientation and rotate them to the positive orientation, and obtain the text in the image based on PaddleOCR recognition; For image files, directly use PaddleOCR to recognize the text in the image; The text obtained by reading or recognizing is segmented in a preset round to obtain sliced ​​text as the extracted text content.

4. The method for extracting and automatically storing contract information based on a large model according to claim 3 is characterized in that: The specific process of constructing corresponding prompt word templates for different types of contract documents includes: Generate prompt words according to input parameters; If it is detected that the type parameter is not passed in, the interface call fails and an error message is returned; If an incoming prompt parameter is detected, the incoming prompt value is used directly and the placeholder is replaced; If it is detected that the prompt parameter is not passed in, the built-in prompt word template is used and the placeholder is replaced; Different types of prompt word templates are constructed for different types of contract documents, so that the same type of prompt word templates can be used to extract the contract documents during the large-model question-answering process.

5. The method for extracting and automatically storing contract information based on a large model according to claim 4 is characterized in that: According to the file type of the contract document and based on the corresponding type of prompt word template, a preset round of large model question and answer is initiated for the text content. The specific process includes: According to the file type of the uploaded contract file, the same type of prompt word template is used to call the large model interface to perform preset rounds of questions and answers; In the large model question-answering process, the prompt words in the prompt word template are sequentially input into the large model to obtain a return result in JSON format; If the Q&A fails or the result is not in JSON format, repeat the Q&A; If the correct JSON result still cannot be obtained, abandon the current round of extraction and continue to extract the next text and field.

6. The method for extracting and automatically storing contract information based on a large model according to claim 5 is characterized in that: After parsing and merging the results returned by the large model, the results are stored in the database. The specific process includes: The JSON result returned by the large model for the text content of the entire contract document is parsed, and the extracted field contents are merged and stored in the MySQL database.

7. A contract information extraction and automatic warehousing system based on a large model, characterized in that: The method for extracting and automatically storing contract information based on a large model as claimed in any one of claims 1 to 6 comprises: A contract format conversion module, used for receiving a contract document and converting the contract document into a character string in a preset coding format; A contract content extraction module, used to extract text content from the contract file according to the file type of the contract file; A prompt template construction module, used to construct prompt word templates of corresponding types for different types of contract documents, wherein the prompt word templates include the contract text and the fields to be extracted; A large model extraction module is used to initiate a preset round of large model question and answer on the text content based on the file type of the contract document and the prompt word template of the corresponding type; The result parsing and storage module is used to parse and merge the results returned by the large model and store them in the database.

8. The large model-based contract information extraction and automatic warehousing system according to claim 7 is characterized in that: The contract content extraction module is specifically used for: For contract files of Word type and ordinary PDF type, directly read the contents of the contract files; For scanned PDF contract documents, convert them into images, identify the image orientation and rotate them to the positive orientation, and obtain the text in the image based on PaddleOCR recognition; For image files, directly use PaddleOCR to recognize the text in the image; The text obtained by reading or recognizing is segmented in a preset round to obtain sliced ​​text as the extracted text content.

9. The large model-based contract information extraction and automatic warehousing system according to claim 8 is characterized in that: The prompt template construction module is specifically used for: Generate prompt words according to input parameters; If it is detected that the type parameter is not passed in, the interface call fails and an error message is returned; If an incoming prompt parameter is detected, the incoming prompt value is used directly and the placeholder is replaced; If it is detected that the prompt parameter is not passed in, the built-in prompt word template is used and the placeholder is replaced; Different types of prompt word templates are constructed for different types of contract documents, so that the same type of prompt word templates can be used to extract the contract documents during the large-model question-answering process.

10. The large model-based contract information extraction and automatic warehousing system according to claim 9 is characterized in that: The large model extraction module is specifically used for: According to the file type of the uploaded contract file, the same type of prompt word template is used to call the large model interface to perform preset rounds of questions and answers; In the large model question-answering process, the prompt words in the prompt word template are sequentially input into the large model to obtain a return result in JSON format; If the Q&A fails or the result is not in JSON format, repeat the Q&A; If the correct JSON result still cannot be obtained, abandon the current round of extraction and continue to extract the next text and field.

Images