Novel efficient application tool integrating spreadsheet, OCR and large model capability
By developing an efficient application tool that integrates spreadsheets, OCR and large model capabilities, it solves the problems of inefficiency, poor accuracy, lack of automation and integration tools, and data source separation in the data entry and verification process in the financial and audit fields, and achieves efficient and accurate data entry and verification.
Patent Information
- Application Number
- CN202411886203.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-04-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing technology has inefficient, poor accuracy, lack of automation and integration tools in the data entry and verification process in the fields of finance and audit, and the data source fragmentation has led to a large amount of manual verification work.
Develop an efficient application tool that integrates spreadsheets, OCR and large model capabilities, including file recognition modules, large model modules, user interface modules, permission management modules and core functional modules. Text recognition is performed through OCR technology, and information extraction and matching functions are realized in combination with large language models, supporting the direct interaction between spreadsheets and original files.
It improves the efficiency and accuracy of data entry and verification, reduces the workload of manual verification, improves the accuracy and efficiency of verification, and solves the problems of inefficiency, poor accuracy, lack of automation and integration tools, and data source fragmentation in the existing technology.
Smart Images
Figure CN119830880A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of financial audit office technology, and in particular to a new and efficient application tool integrating electronic spreadsheet, OCR and large model capabilities. Background Art
[0002] In the current financial and auditing fields, data entry and verification work still highly relies on traditional spreadsheets (such as Excel) for operation. Specifically, staff manually enter information from scanned documents, pictures, PDF files and other files into spreadsheets, and then compare and verify them with the original vouchers. Although this method meets daily needs to a certain extent, with the increase in data volume and the increase in work complexity, the model that relies on manual entry and verification obviously has the following deficiencies and shortcomings: 1. Low efficiency. Many financial and auditing personnel still rely on spreadsheets such as Excel to manually enter voucher data, and the processing process is cumbersome and inefficient; 2. Poor accuracy. Manual entry and verification work is prone to errors. Due to the increase in financial The amount of data in the integrated audit scenario is huge. When manually checking the details of the vouchers, omissions, misoperations and data comparison errors are common; 3. Lack of automation and integrated tools. The disconnection between the existing OCR technology and Excel means that users still need to download the data files exported by the OCR software and then manually copy them to their own tables. This process still consumes a lot of time and has the risk of manual errors, and fails to achieve true automation and efficiency; 4. The manual verification workload caused by the fragmentation of the data source is large. In the data verification link, the list of vouchers to be verified usually exists in Excel tables, while scanned copies and image files are stored separately as additional materials. This fragmentation of the data source leads to the repetitiveness and complexity of the verification work.
[0003] At present, there are no technical products on the market that can completely solve the above problems. Although some existing products have introduced OCR technology or automated data processing tools, they often fail to be deeply integrated with spreadsheet tools such as Excel, or there are obvious gaps in user experience and data processing capabilities. Therefore, the market still mainly relies on manual operations. Therefore, a new efficient application tool that integrates spreadsheets, OCR and large model capabilities is proposed to solve the above problems. Summary of the invention
[0004] 1. Technical issues to be solved
[0005] In view of the deficiencies of the prior art, the present invention provides a new and efficient application tool that integrates electronic spreadsheets, OCR and large model capabilities, which has the advantages of effectively improving the efficiency and accuracy of data entry and verification, and solves the problems of low efficiency, poor accuracy, lack of automation and integrated tools, and large workload of manual verification caused by the fragmentation of data sources in the prior art.
[0006] (II) Technical solution
[0007] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities, including a file recognition module, a large model module, a user interface module, a rights management module and a core function module;
[0008] The core functional modules include a text interception submodule, a verification mark submodule, an abnormal mark submodule, an interception summation submodule, a table interception submodule, a file matching submodule, an information extraction submodule, an extraction matching submodule and a file removal submodule.
[0009] Preferably, the file recognition module performs text recognition on the uploaded file through OCR technology, and stores the recognition result, coordinate information and original image data.
[0010] Preferably, the OCR technology uses an open source OCR engine or an integrated commercial OCR engine.
[0011] Preferably, the data storage uses a SQL or NoSQL database to store the above information and generates a unique identifier for each file.
[0012] Preferably, the coordinate information of the text is output through the "bounding box" of the OCR engine, and the position information of each text is associated with a coordinate value to facilitate subsequent precise positioning.
[0013] Preferably, the large model module is used to interface with the large language model to achieve information extraction and extraction matching functions.
[0014] Preferably, the large language model is Alibaba's Qwen2.5 7B.
[0015] Preferably, the docking interface between the large model module and the large language model is through HTTP or gRPC protocol. The system sends a request to the large model, passes in the keywords entered by the user, and the model returns the corresponding text value. The request and response data are transmitted in JSON format.
[0016] Preferably, the user interface module is responsible for quickly rendering and displaying the data returned by the core function module.
[0017] Preferably, the authority management module is used to manage the authority of different users, and intercept and control the user's access to system resources.
[0018] (III) Beneficial effects
[0019] Compared with the prior art, the present invention provides a new and efficient application tool integrating electronic spreadsheet, OCR and large model capabilities, which has the following beneficial effects:
[0020] 1. This new efficient application tool integrates spreadsheet, OCR and large model capabilities. By realizing direct interaction between spreadsheet and original file, it can write character positioning information in original file directly into spreadsheet and support highlighting. Users do not need to enter text manually or copy text twice, but directly select relevant text in the original image, and automatically write the selected content into the spreadsheet. This process not only improves data entry efficiency, but also supports original text positioning and viewing, which greatly facilitates data verification and accuracy verification.
[0021] 2. This new efficient application tool integrates spreadsheet, OCR and large model capabilities. Through the interception and summation technology, users can directly select the numbers to be added in the original image. The system automatically completes the addition and writes the results into the spreadsheet, avoiding the tedious process of manual addition. It also supports original text positioning and viewing to ensure the accuracy of the verification results.
[0022] 3. This new efficient application tool integrates spreadsheet, OCR and large model capabilities. Through information extraction technology, users only need to input the key value to be extracted, and the system can use the OCR recognition results and large language model to automatically extract information from the file. The large language model can identify the key value and corresponding value of each file from the identified file content, thereby realizing batch and efficient extraction of file content.
[0023] 4. This new efficient application tool integrates spreadsheets, OCR and large model capabilities. Through the extraction and matching function, users can put the detailed data that needs to be verified into the spreadsheet. The system automatically extracts the corresponding Key value and the corresponding Value value from the original file, and compares it with the data in the spreadsheet to automatically complete the verification process. This method effectively reduces the workload of manual verification and improves the accuracy and efficiency of verification. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 It is a schematic diagram of the module architecture of the present invention;
[0025] Figure 2 It is a schematic diagram of the file processing flow in the present invention. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0027] See also Figure 1-2 , a new and efficient application tool integrating spreadsheet, OCR and large model capabilities, including file recognition module, large model module, user interface module, permission management module and core function module.
[0028] The core functional modules include text interception submodule, verification mark submodule, exception mark submodule, interception and summation submodule, table interception submodule, file matching submodule, information extraction submodule, extraction matching submodule and file removal submodule.
[0029] Specifically, the file recognition module performs text recognition on the uploaded file through OCR technology, and stores the recognition result, coordinate information and original image data.
[0030] Specifically, the OCR technology uses an open source OCR engine or an integrated commercial OCR engine.
[0031] Specifically, the data storage uses a SQL or NoSQL database to store the above information and generates a unique identifier for each file.
[0032] Specifically, the coordinate information of the text is output through the "bounding box" of the OCR engine. The location information of each text is associated with a coordinate value to facilitate subsequent precise positioning.
[0033] Specifically, the large model module is used to connect with the large language model to realize information extraction and extraction matching functions. The large language model is Alibaba's Qwen2.5 7B.
[0034] Specifically, the docking interface between the large model module and the large language model uses the HTTP or gRPC protocol. The system sends a request to the large model, passes in the keywords entered by the user, and the model returns the corresponding text value. The request and response data are transmitted in JSON format.
[0035] Furthermore, the big model module calls the local or cloud-based big model data interface and combines the built-in prompt words to connect to the information extraction submodule and extraction matching submodule in the core function module. The big model module receives and processes the data returned by the core function module, and feeds back the results in JSON format to other related modules or user interface modules for subsequent operations and display.
[0036] Specifically, the user interface module is responsible for quickly rendering and displaying the data returned by the core functional modules. Through a simple and intuitive operation interface, users can interact through clear buttons and menu options. In addition, the user interface module also integrates the spreadsheet API, providing powerful custom editing functions, allowing users to comprehensively edit and manage table data.
[0037] Furthermore, the interface layout of the user interface module is developed using a modern front-end framework (Vue.js) with a modular design, and the page consists of multiple components, such as file upload, data display, buttons, and menus.
[0038] Specifically, the permission management module is used to manage the permissions of different users, and intercept and control users' access to system resources, ensuring that only authorized users can access and operate specific resources or functions, thereby ensuring the security of the system.
[0039] Furthermore, the permission management module controls the user's access rights to system resources. The system implements permission control in the following ways:
[0040] 1) User role management: The system sets different user roles, such as administrator, ordinary user, etc. Each role has different access rights;
[0041] 2) Resource access control: Through the JWT (JSON Web Token) authentication mechanism, users are authenticated to ensure that only authorized users can perform specific operations (such as file upload, data modification, etc.).
[0042] Specifically, the text capture submodule: the user selects or selects the text area to be captured through the front-end interface. The system obtains the corresponding text data from the file recognition module through the coordinate information returned by the front-end, and calls the spreadsheet API through the user interface module to render the data into the cells of the spreadsheet. Each cell contains not only the captured text content, but also the coordinate information of the text. When the user clicks on a cell, the corresponding area in the original file image will be highlighted.
[0043] Specifically, the verification mark submodule: the user also selects the required text area through the front-end interface. The system obtains the corresponding content from the file recognition module based on the coordinate information, and displays a "√" symbol in the cell of the spreadsheet to indicate that the text has passed the verification. The user can also right-click the image area in the original file to mark the area as "text interception", "abnormal mark", etc. for further operations.
[0044] Specifically, the abnormal marking submodule is used to mark text that fails verification. The user selects the text area through the front-end interface. The system recognizes the coordinate information and displays an "×" symbol in the cell of the spreadsheet, indicating that there is a problem with the content. The user can also right-click the image area to change the label, such as changing it to "text interception" or "verification mark."
[0045] Specifically, the summation submodule is intercepted: the user selects the text area that needs to be summed through the front-end interface. The system identifies the corresponding text based on the coordinate information and performs the summation through the built-in formula. The summation result will be displayed in the cell through the spreadsheet API. In addition to the summed value, the cell also contains the coordinate information of the text. When the cell is clicked, the corresponding area in the original file image will be highlighted.
[0046] Specifically, the table capture submodule automatically identifies and extracts table content in the document through the OCR table recognition technology in the file recognition module. Users can view and select the table to be captured through the front-end interface. The system renders the table content into the cells of the spreadsheet through the spreadsheet API. The cells contain not only the table data but also the corresponding coordinate information. When you click a cell, the table area in the original file will be highlighted.
[0047] Specifically, the file matching submodule: the user enters the detailed data that needs to be matched in advance in the spreadsheet, usually including the Key value and its corresponding Value value. The system will compare the input Key and Value with the content in the original file. The user can select the Key value and Value value to be matched in the spreadsheet. The system searches and matches the corresponding content in the file through the file identification module, and displays the matching results in the cell through the spreadsheet API. The user can compare whether they are consistent. When clicking the cell, the matching area in the original file will be highlighted for easy comparison.
[0048] Specifically, the information extraction submodule: the user selects the file to be extracted and enters the Key value to be extracted. The system will start the large model module and automatically extract the corresponding Value value based on the Key value entered by the user. The extraction result will be rendered to the cells of the spreadsheet through the spreadsheet API, thereby automatically extracting basic information from non-fixed format files. In addition, the user can also save the defined Key value template so that the same template can be used for information extraction next time to improve operational efficiency.
[0049] Specifically, extraction and matching sub-module: the user enters the Key value and its Value value to be matched in the spreadsheet, and the system passes these values to the large model module for extraction. The large model module returns the Value value that matches the Key value based on the file content, and displays the result in the cell through the spreadsheet API. The user can compare whether the extracted value is consistent with the input value, and further verify it through the highlight display in the original file image.
[0050] Specifically, remove the file submodule: users can remove original files that are no longer needed from the system as needed, including files that are not marked as screenshots or files marked as abnormal. This function helps users manage files and retain important files for subsequent operations.
[0051] The workflow of the present invention is as follows: first, the original file is uploaded, and the tool is used to perform OCR recognition on all files to extract text and coordinate information, and then the recognition result data is stored. Finally, text interception, verification marking, exception marking, interception summation, table interception, and file matching are applied to data that does not require the participation of a large model, and information extraction, extraction matching, and other application operations are performed on data that requires the participation of a large language model.
[0052] In summary, this new efficient application tool that integrates spreadsheets, OCR, and large model capabilities can write character positioning information in the original file directly into the spreadsheet by realizing direct interaction between the spreadsheet and the original file, and supports highlighting. Users do not need to manually enter or copy the text twice, but directly select the relevant text in the original image, and automatically write the selected content into the spreadsheet. This process not only improves the efficiency of data entry, but also supports original text positioning and viewing, which greatly facilitates data verification and accuracy verification. Through the interception and summation technology, users can directly select the numbers that need to be totaled in the original image, and the system automatically completes the totalization and writes the results into the spreadsheet, avoiding the tedious process of manual totalization, and supports original text positioning and viewing to ensure the accuracy of the verification results.
[0053] Moreover, through information extraction technology, users only need to input the Key value to be extracted, and the system can use the OCR recognition results and the large language model to automatically extract information from the file. The large language model can identify the Key value and the corresponding Value value of each file from the identified file content, thereby realizing batch and efficient extraction of file content. Through the extraction and matching function, users can put the detailed data that needs to be verified into a spreadsheet. The system automatically extracts the corresponding Key value and the corresponding Value value from the original file, and compares it with the data in the spreadsheet to automatically complete the verification process. This method effectively reduces the workload of manual verification and improves the accuracy and efficiency of verification, solving the problems of low efficiency, poor accuracy, lack of automation and integration tools, and large workload of manual verification caused by the fragmentation of data sources in the existing technology.
[0054] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0055] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A new and efficient application tool integrating electronic spreadsheet, OCR and large model capabilities, characterized by: It includes file identification module, large model module, user interface module, authority management module and core function module; The core functional modules include a text interception submodule, a verification mark submodule, an abnormal mark submodule, an interception summation submodule, a table interception submodule, a file matching submodule, an information extraction submodule, an extraction matching submodule and a file removal submodule.
2. A new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 1, characterized in that: The file recognition module performs text recognition on the uploaded file through OCR technology, and stores the recognition result, coordinate information and original image data.
3. A new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 2, characterized in that: The OCR technology uses an open source OCR engine or an integrated commercial OCR engine.
4. A new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 2, characterized in that: The data storage uses a SQL or NoSQL database to store the above information and generates a unique identifier for each file.
5. The new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 3, characterized in that: The coordinate information of the text is output through the "bounding box" of the OCR engine. The position information of each text is associated with a coordinate value to facilitate subsequent precise positioning.
6. A new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 1, characterized in that: The large model module is used to connect with the large language model to realize information extraction and extraction matching functions.
7. A new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 6, characterized in that: The large language model is Alibaba's Qwen2.5 7B.
8. The new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 1, characterized in that: The docking interface between the large model module and the large language model uses the HTTP or gRPC protocol. The system sends a request to the large model, passes in the keywords entered by the user, and the model returns the corresponding text value. The request and response data are transmitted in JSON format.
9. The new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 1, characterized in that: The user interface module is responsible for quickly rendering and displaying the data returned by the core function module.
10. A new efficient application tool integrating electronic spreadsheet, OCR and large model capabilities according to claim 1, characterized in that: The authority management module is used to manage the authority of different users and intercept and control the user's access to system resources.