Invoice information positioning and interception system and method based on image recognition

Abstract

The present invention relates to the field of image processing technology, and specifically to an invoice information positioning and interception system and method based on image recognition, including a data processing unit, a recognition, matching and extraction unit, an information integration and verification unit, a data storage and management unit and a security assurance unit. In the present invention, the processed image text and pattern are clarified, which can improve the accuracy of text positioning by the text detection and positioning module, thereby reducing the dependence on image quality. After the characters are recognized by the recognition, matching and extraction unit, the format is adapted by the format adaptation module, and then the information is checked and quickly extracted by the verification and extraction module, so that invoices of different types and formats can be quickly recognized, matched and extracted, reducing the interference of font and symbol recognition limitations and the diversity of invoice formats, ensuring the accuracy of invoice information recognition and positioning, and ensuring the integrity and correctness of information extraction.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a system and method for locating and intercepting invoice information based on image recognition. Background Art

[0002] An invoice is a receipt or payment voucher issued and collected when buying and selling goods, providing or receiving services, or engaging in other business activities. By recognizing the invoice image, the reimbursement information related to the invoice can be obtained.

[0003] For example, the "invoice reimbursement information positioning method based on image recognition" provided by Chinese patent publication number CN108460381A includes separating and extracting a binary image of all machine-printed characters from an invoice image; converting the binary image of all machine-printed characters into black and white to obtain a corresponding inverted image; connecting the inverted image to a connected domain to obtain an eroded block image; and performing external rectangular boundary detection on the eroded block image to detect the vertex and bottom point position information of each reimbursement information block.

[0004] In the above patent, although the problems of low efficiency, long process and high error rate of manual invoice reimbursement are solved through image processing and computer vision technology, when collecting invoice images, it is easy to appear blur, tilt, distortion or stain due to factors such as shooting equipment, light, angle and paper quality, making it difficult for the image recognition algorithm to accurately identify and locate the information on the invoice, resulting in incomplete or erroneous information extraction.

[0005] In view of this, the present application proposes a system and method for locating and intercepting invoice information based on image recognition. Summary of the invention

[0006] The purpose of the present invention is to provide an invoice information positioning and interception system and method based on image recognition to solve the above problems.

[0007] To achieve the above object, the present invention provides the following technical solutions: An invoice information positioning and interception system based on image recognition, comprising a data processing unit, an identification, matching and extraction unit, an information integration and verification unit, a data storage and management unit and a security guarantee unit; The data processing unit is used to collect and process the image information of the invoice; The data processing unit includes a data acquisition module, an image preprocessing module and a text detection and positioning module. The data acquisition module is used to obtain invoice images through a scanning device and a shooting device; The image preprocessing module is used to perform denoising, binarization, tilt correction and image enhancement processing on the collected images; The text detection and positioning module is used to detect and locate the invoice text area using a deep learning algorithm; The recognition, matching and extraction unit is used to perform character recognition on the located text area, match the format of the invoice and extract information; The information integration and verification unit is used to integrate the information extracted by the identification, matching and extraction units, and compare and verify it with the external data source; The data storage and management unit is used to store the invoice information processed by the information integration and verification unit in a local database, and to perform classification management and retrieval; The security assurance unit is used to encrypt the stored data and set user permissions.

[0008] Furthermore, the image preprocessing module includes a denoising processing module, a binarization processing module, a tilt correction module and an image enhancement module. The denoising processing module is used to remove salt and pepper noise and Gaussian noise in the image by using a filtering method, and retains the detailed information of the image while removing the noise; The binarization processing module is used to convert the invoice image into a black and white binary image, and automatically determine the threshold using the Otsu algorithm to maximize the contrast between the text and the background, so as to recognize and locate the text.

[0009] Furthermore, the tilt correction module is used to detect straight line features in the invoice image using algorithms such as Hough transform, determine the tilt angle of the invoice, and perform rotation correction to make the invoice image in a horizontal state; The image enhancement module is used to enhance the contrast of the image by using the histogram equalization method to make the text and patterns in the image clearer. For images with uneven lighting, the homomorphic filtering method is used to process and adjust the lighting effect of the image.

[0010] Furthermore, when calibrating the invoice image, the tilt correction module detects the direction of the invoice text through a deep learning model and automatically performs rotation correction on the image.

[0011] Furthermore, in the text detection and positioning module, the deep learning algorithm is combined with multi-scale detection and context information to accurately locate the invoice information.

[0012] Further, the recognition, matching and extraction unit includes a character recognition module, a format adaptation module and a verification and extraction module, wherein the character recognition module is used to select OCR technology, process characters through character segmentation and recognition, and then perform post-processing correction on the characters; The format adaptation module is used to match and identify the current invoice format with the template library, and automatically adjust the subsequent information positioning and interception strategy according to the identification result; The checking and extracting module is used to extract information from the located text, perform logic checking on the extracted information, compare the checked data with the external data source, and output the comparison result.

[0013] Furthermore, when matching the invoice format in the format adaptation module, the following contents are also included: A1. Feature extraction and matching: When receiving the invoice image to be processed, the format adaptation module automatically extracts the image features and matches the extracted features with the templates in the template library; A2. Format recognition: By calculating similarity and feature matching, the format type of the invoice can be quickly and accurately identified; A3. Strategy adjustment: Dynamically adjust the parameters and rules of the text detection and positioning module and the character recognition module based on the position differences of the amount, tax rate, and invoicing date in invoices of different formats.

[0014] Furthermore, the security assurance unit includes an access management module, a data desensitization module and a security audit module. The access management module is used to manage user rights, and authorized users can access and process invoice data; The data desensitization module is used to desensitize sensitive information and hide some taxpayer identification numbers and amounts; The security audit module is used to establish a security audit mechanism, record system work logs, and track and audit system operations through work logs.

[0015] Furthermore, in the security audit module, when the audit mechanism is established, the system operation behavior is recorded through logs, a user behavior analysis model is established, the user operation behavior is monitored and analyzed in real time, the audit data is stored, and audit reports are generated regularly. At the same time, permission management and security incident response processes are set.

[0016] A method for locating and intercepting an invoice information system based on image recognition, comprising the following steps: S1. Use the acquisition device of the data processing unit to collect invoice images, pre-process the collected images, implement image denoising, binarization, tilt correction and image enhancement, and use a deep learning algorithm to detect and locate the text area of ​​the invoice; S2. Perform character recognition and post-processing correction on the located text area through the recognition, matching and extraction unit. After character recognition, match the invoice format with the template library. After the match is successful, extract key information. Then, integrate and verify the information through the information integration and verification unit. S3, encrypting and storing the information verified by the information integration and verification unit in the database through the data storage and management unit; S4. User permissions and sensitive information are processed through the security assurance unit, and the operation of the system is supervised through the security audit mechanism.

[0017] Beneficial effects of the present invention: 1. In the present invention, the collected image is preprocessed by an image preprocessing module, and is processed in sequence by a denoising processing module, a binarization processing module, a tilt correction module and an image enhancement module, so that the processed image text and pattern are clear, and the accuracy of text positioning by a text detection and positioning module can be improved, thereby reducing the dependence on image quality. After the characters are recognized by the recognition, matching and extraction unit, the format is adapted by the format adaptation module, and then the information is checked and quickly extracted by the checking and extraction module, so that invoices of different types and formats can be quickly recognized, matched and extracted, reducing the interference of font and symbol recognition limitations and the diversity of invoice formats, ensuring the accuracy of invoice information recognition and positioning, and ensuring the completeness and correctness of information extraction.

[0018] 2. In the present invention, the invoice image and the extracted information are encrypted, stored and transmitted through the security assurance unit to prevent data leakage during the transmission and storage process. The user authority and sensitive information are processed respectively through the access management module and the data desensitization module, and strict user authority management is set. Only authorized users can access and process invoice data, hide part of the taxpayer identification number and amount, protect user privacy, establish a security audit mechanism through the security audit module, record the system operation log, supervise the operation of the system, facilitate tracking and auditing, and ensure the security of the entire system when in use. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a system block diagram of an invoice information positioning and interception system based on image recognition according to the present invention; Figure 2 This is a system diagram of an image preprocessing module of an invoice information positioning and interception system based on image recognition according to the present invention; Figure 3 This is a system diagram of the recognition, matching and extraction units of an invoice information positioning and interception system based on image recognition of the present invention; Figure 4 This is a system diagram of a security assurance unit of an invoice information positioning and interception system based on image recognition according to the present invention.

[0020] In the figure: 1. Data processing unit; 11. Data acquisition module; 12. Image preprocessing module; 121. De-noising processing module; 122. Binarization processing module; 123. Tilt correction module; 124. Image enhancement module; 13. Text detection and positioning module; 2. Recognition, matching and extraction unit; 21. Character recognition module; 22. Format adaptation module; 23. Verification and extraction module; 3. Information integration and verification unit; 4. Data storage and management unit; 5. Security assurance unit; 51. Access management module; 52. Data desensitization module; 53. Security audit module. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example 1: Please refer to Figure 1-Figure 3 ,This design proposes an implementation method, an invoice information positioning and interception system based on image recognition, including a data processing unit 1, an identification, matching and extraction unit 2, an information integration and verification unit 3, a data storage and management unit 4 and a security assurance unit 5; The data processing unit 1 is used to collect and process the image information of the invoice; The data processing unit 1 includes a data acquisition module 11, an image preprocessing module 12 and a text detection and positioning module 13. The data acquisition module 11 is used to obtain the invoice image through a scanning device and a shooting device; the image preprocessing module 12 is used to perform denoising, binarization, tilt correction and image enhancement processing on the collected image; The image preprocessing module 12 includes a denoising processing module 121, a binarization processing module 122, a tilt correction module 123 and an image enhancement module 124. The denoising processing module 121 is used to remove salt and pepper noise and Gaussian noise in the image by using a filtering method, and retain the detailed information of the image while removing the noise; The binarization processing module 122 is used to convert the invoice image into a black and white binary image, and automatically determine the threshold value through a suitable threshold selection method, such as the Otsu algorithm, so as to maximize the contrast between the text and the background, so as to facilitate subsequent text recognition and positioning; the tilt correction module 123 is used to detect the straight line features in the invoice image using algorithms such as the Hough transform, determine the tilt angle of the invoice, and then perform rotation correction to make the invoice image horizontal. It can also detect the direction of the invoice text through a deep learning model and automatically perform rotation correction on the image; the image enhancement module 124 is used to enhance the contrast of the image using methods such as histogram equalization to make the text and patterns in the image clearer. For images with uneven lighting, methods such as homomorphic filtering can be used to adjust the lighting effect of the image.

[0023] The text detection and positioning module 13 is used to detect and locate the invoice text area using a deep learning algorithm; A text detection algorithm based on a convolutional neural network is used to automatically learn text features in invoice images and accurately locate text areas. A multi-scale detection method is then used to perform text detection on images of different scales and text of different sizes. By detecting the title and column name of the invoice, other relevant information can be located. A language model or rule engine is used to perform semantic analysis on the detected text to determine whether it conforms to the logical structure of the invoice information. The invoice information can be accurately located based on the judgment results.

[0024] The recognition, matching and extraction unit 2 is used to perform character recognition on the located text area, match the format of the invoice and extract information; the recognition, matching and extraction unit 2 includes a character recognition module 21, a format adaptation module 22 and a verification and extraction module 23. The character recognition module 21 is used to select OCR technology to recognize characters, and for contiguous or overlapping characters, a character segmentation algorithm is used to separate them and then recognize them. For special characters or handwritten characters that are difficult to recognize, a special character library or training samples is established, and the recognized characters are post-processed, and the recognition results are corrected through dictionary matching and grammar checking. The format template and business rules of the invoice are used to verify and correct the recognized information, and the post-processing correction is completed; The format adaptation module 22 is used to collect various invoice samples, covering different regions, industries, invoice issuers and invoice types, extract the format features, text layout, and field positions of the invoice, and build an invoice format template library. When receiving an invoice image to be processed, the module will automatically match the image features with the templates in the template library, and quickly and accurately identify the format type of the invoice by calculating similarity, feature matching and other indicators, and match and identify the current invoice format with the template library. According to the recognition results, the parameters and rules of the text detection and positioning module 13 and the character recognition module 21 are dynamically adjusted according to the position differences of key information such as the amount, tax rate, and invoice date in invoices of different formats; When matching the invoice format in the format adaptation module 22, the following contents are also included: 1. Feature extraction and matching: When receiving the invoice image to be processed, the format adaptation module 22 automatically extracts the image features and matches the extracted features with the templates in the template library; 2. Format recognition: By calculating similarity and feature matching, the format type of the invoice can be quickly and accurately identified; 3. Strategy adjustment: dynamically adjust the parameters and rules of the text detection and positioning module 13 and the character recognition module 21 according to the position differences of the amount, tax rate and invoicing date in invoices of different formats; The verification and extraction module 23 is used to extract information from the located text, extract key information from the invoice image according to the identified invoice format, perform logical verification on the extracted information, check the relevance and rationality between the data, compare the extracted invoice information with the external data source, compare the authenticity of the invoice code and number in the tax system, verify whether the purchaser and seller information is consistent with the internal records of the enterprise, and output the compared results.

[0025] The information integration and verification unit 3 is used to integrate the information extracted by the identification, matching and extraction unit 2, and compare and verify it with the external data source; the data storage and management unit 4 is used to store the invoice information processed by the information integration and verification unit 3 in the local database, and classify and manage it; the security assurance unit 5 is used to encrypt the stored data and set user permissions.

[0026] In this example, the character segmentation algorithm is used to process the contiguous or overlapping characters, and the processed characters are then recognized by OCR technology. The recognized characters are post-processed to ensure the accuracy of character recognition, thereby facilitating subsequent information correction. The establishment and update of the template library facilitates the rapid matching of different types of invoice samples, and facilitates the adjustment of information positioning and interception strategies according to the position differences of key invoice information to ensure the accuracy of interception. The accuracy and completeness of invoice information can be further verified through information extraction, logical verification and external data source comparison. When the information integration and verification unit 3 integrates information, it can be combined with other information of the invoice, such as QR code, barcode, etc., through multimodal recognition technology, to complement and verify each other, so as to improve the completeness and accuracy of information extraction. When verifying, the extracted invoice information can be compared and verified with external data sources such as tax systems and internal corporate databases to promptly discover and correct erroneous information.

[0027] Example 2: Please refer to Figure 1-Figure 4 , This design proposes an implementation method, an invoice information positioning and interception system based on image recognition, the security assurance unit 5 includes an access management module 51, a data desensitization module 52 and a security audit module 53, the access management module 51 is used to manage user permissions, and authorized users can access and process invoice data; the data desensitization module 52 is used to desensitize sensitive information and hide part of the taxpayer identification number and amount; the security audit module 53 is used to establish a security audit mechanism, record the system work log, and track and audit the system operation through the work log; When the audit mechanism is established, the following contents shall also be included: B1. Log Records: The system comprehensively records various operation behaviors, including user login and logout time, login IP address, viewing, modification, and deletion of invoice data, and data import and export records, and also records the system's operating status.

[0028] B2. User behavior analysis: Use big data analysis technology to establish a user behavior analysis model to monitor and analyze user operation behaviors in real time. By setting normal behavior patterns, once it is found that user operations deviate from the normal pattern, an alarm will be triggered in time.

[0029] B3. Audit data storage: Audit data is stored in an independent security database, isolated from invoice business data. Encryption technology is used to encrypt and store audit data to prevent data tampering or theft. Audit data is backed up and stored regularly.

[0030] B4. Audit report generation: Generate audit reports regularly. The report content includes system operation statistics, security event summary, and user behavior analysis results. The audit report includes intuitive charts and detailed text descriptions.

[0031] B5. Authority management: strictly control the access rights to audit data. Only authorized security personnel and auditors can access and view audit data. The auditors' operations are recorded and audited, and operation logs are generated.

[0032] B6. Security incident response: Establish a security incident response process. When a security incident is discovered during the audit process, take different measures based on the level of the security incident, such as freezing relevant user accounts, blocking abnormal IP addresses, and notifying relevant personnel to conduct investigations and handling.

[0033] In this embodiment, by real-time monitoring of user behavior and system operations, abnormal situations can be discovered in a timely manner. For example, if unauthorized access attempts or frequent data modifications are encountered, the system can issue an alarm in advance to provide security personnel with processing time and reduce potential risks when the system is used. When a system failure or error occurs, the audit log can help technicians quickly discover the problem and locate the root cause of the problem, understand the sequence of system operations and related parameters, accelerate the fault repair process, reduce system downtime, and improve system stability and reliability.

[0034] Example 3: Please refer to Figure 4 This design proposes an implementation method, a method for locating and intercepting an invoice information system based on image recognition, comprising the following steps: Step 1: Use the acquisition device of the data processing unit 1 to collect the invoice image, pre-process the collected image, implement image denoising, binarization, tilt correction and image enhancement, and use the deep learning algorithm to detect and locate the text area of ​​the invoice; Step 2: Perform character recognition and post-processing correction on the located text area through the recognition, matching and extraction unit 2. After completing character recognition, match the invoice format with the template library. After the match is successful, extract key information, and then integrate and verify the information through the information integration and verification unit 3; Step 3: The information verified by the information integration and verification unit 3 is encrypted and stored in the database through the data storage and management unit 4; Step 4: Process user permissions and sensitive information through the security assurance unit 5, and monitor the operation of the system through the security audit mechanism.

[0035] In the present invention, firstly, the data acquisition module 11 is used to obtain the relevant image of the invoice by scanning equipment, mobile phone photography, etc., and then the image preprocessing module 12 is used to preprocess the collected image. After denoising, binarization, tilt correction and image enhancement, the accuracy of text positioning by the text detection and positioning module 13 can be improved, thereby reducing the dependence on image quality and ensuring the integrity and accuracy of information extraction. After the characters are recognized by the recognition, matching and extraction unit 2, the format is adapted by the format adaptation module 22, and then the information is checked and quickly extracted by the verification and extraction module 23, so that invoices of different types and formats can be quickly recognized, matched and extracted, reducing the interference of font and symbol recognition limitations and diversified invoice formats. The invoice image and the extracted information are encrypted, stored and transmitted by the security guarantee unit 5 to prevent data leakage, control access, set strict user authority management, and only authorized users can access and process invoice data. Sensitive information is desensitized to protect user privacy, establish a security audit mechanism, record system operation logs, facilitate tracking and auditing, and ensure the security of the entire system when in use.

[0036] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An invoice information positioning and interception system based on image recognition, characterized in that: It includes a data processing unit (1), an identification, matching and extraction unit (2), an information integration and verification unit (3), a data storage and management unit (4) and a security assurance unit (5); The data processing unit (1) is used to collect and process image information of the invoice; The data processing unit (1) comprises a data acquisition module (11), an image preprocessing module (12) and a text detection and positioning module (13); the data acquisition module (11) is used to obtain an invoice image through a scanning device and a photographing device; The image preprocessing module (12) is used to perform denoising, binarization, tilt correction and image enhancement processing on the collected image; The text detection and positioning module (13) is used to detect and locate the invoice text area using a deep learning algorithm; The recognition, matching and extraction unit (2) is used to perform character recognition on the located text area, match the format of the invoice and extract information; The information integration and verification unit (3) is used to integrate the information extracted by the identification, matching and extraction unit (2), and compare and verify it with the external data source; The data storage and management unit (4) is used to store the invoice information processed by the information integration and verification unit (3) in a local database, and to perform classification management and retrieval; The security assurance unit (5) is used to encrypt stored data and set user permissions.

2. The invoice information positioning and interception system based on image recognition according to claim 1 is characterized by: The image preprocessing module (12) comprises a denoising processing module (121), a binarization processing module (122), a tilt correction module (123) and an image enhancement module (124); the denoising processing module (121) is used to remove salt and pepper noise and Gaussian noise in the image by using a filtering method, thereby retaining detailed information of the image while removing the noise; The binarization processing module (122) is used to convert the invoice image into a black and white binary image, and automatically determine the threshold using the Otsu algorithm to maximize the contrast between the text and the background, thereby identifying and locating the text.

3. The invoice information positioning and interception system based on image recognition according to claim 2 is characterized by: The tilt correction module (123) is used to detect straight line features in the invoice image using algorithms such as Hough transform, determine the tilt angle of the invoice, and perform rotation correction to make the invoice image in a horizontal state; The image enhancement module (124) is used to enhance the contrast of the image using a histogram equalization method to make the text and patterns in the image clearer. For images with uneven lighting, a homomorphic filtering method is used to process the images and adjust the lighting effect of the images.

4. The invoice information positioning and interception system based on image recognition according to claim 2 is characterized in that: When calibrating the invoice image, the tilt correction module (123) detects the direction of the invoice text through a deep learning model and automatically performs rotation correction on the image.

5. The invoice information positioning and interception system based on image recognition according to claim 1 is characterized in that: In the text detection and positioning module (13), a deep learning algorithm is combined with multi-scale detection and context information to accurately locate the invoice information.

6. The invoice information positioning and interception system based on image recognition according to claim 1 is characterized in that: The recognition, matching and extraction unit (2) comprises a character recognition module (21), a format adaptation module (22) and a verification and extraction module (23), wherein the character recognition module (21) is used to select an OCR technology, process characters through character segmentation and recognition, and then perform post-processing correction on the characters; The format adaptation module (22) is used to match and identify the current invoice format with the template library, and automatically adjust the subsequent information positioning and interception strategy according to the identification result; The checking and extraction module (23) is used to extract information from the located text, perform logic verification on the extracted information, compare the verified data with an external data source, and output the comparison result.

7. The invoice information positioning and interception system based on image recognition according to claim 6 is characterized by: When matching the invoice format in the format adaptation module (22), the following contents are also included: A1. Feature extraction and matching: When receiving the invoice image to be processed, the format adaptation module (22) automatically extracts the image features and matches the extracted features with the templates in the template library; A2. Format recognition: By calculating similarity and feature matching, the format type of the invoice can be quickly and accurately identified; A3. Strategy adjustment: Based on the position differences of the amount, tax rate, and invoice date in invoices of different formats, the parameters and rules of the text detection and positioning module (13) and the character recognition module (21) are dynamically adjusted.

8. The invoice information positioning and interception system based on image recognition according to claim 1 is characterized by: The security assurance unit (5) comprises an access management module (51), a data desensitization module (52) and a security audit module (53), wherein the access management module (51) is used to manage user rights, and authorized users can access and process invoice data; The data desensitization module (52) is used to desensitize sensitive information and hide part of the taxpayer identification number and amount; The security audit module (53) is used to establish a security audit mechanism, record the system work log, and track and audit the system operation through the work log.

9. The invoice information positioning and interception system based on image recognition according to claim 8 is characterized in that: In the security audit module (53), when the audit mechanism is established, the system operation behavior is recorded through logs, a user behavior analysis model is established, the user operation behavior is monitored and analyzed in real time, the audit data is stored, and audit reports are generated regularly. At the same time, permission management and security incident response processes are set.

10. A method for locating and intercepting an invoice information system based on image recognition, characterized in that: An invoice information positioning and interception system based on image recognition according to any one of claims 1 to 9 is used, comprising the following steps: S1. Using the acquisition device of the data processing unit (1) to acquire an invoice image, pre-processing the acquired image to achieve image denoising, binarization, tilt correction and image enhancement, and using a deep learning algorithm to detect and locate the text area of ​​the invoice; S2, performing character recognition and post-processing correction on the located text area through the recognition, matching and extraction unit (2), matching the invoice format with the template library after character recognition, extracting key information after successful matching, and then integrating and verifying the information through the information integration and verification unit (3); S3, encrypting and storing the information verified by the information integration and verification unit (3) in a database through the data storage and management unit (4); S4. User permissions and sensitive information are processed through the security assurance unit (5), and the operation of the system is supervised through the security audit mechanism.

Images