Image feature information extraction processing method and device
Through deep learning and multimodal fusion algorithm, the problems of incomplete image quality and inaccurate information extraction in invoice and commercial image processing are solved, efficient and accurate information extraction and processing are achieved, and the frequency and error rate of manual intervention are reduced. , and ensures data security.
Patent Information
- Application Number
- CN202411096790.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-22
AI Technical Summary
When processing invoices and commercial ticket images, the prior art faces problems such as uneven image quality, inaccurate positioning of key information areas, and inaccurate information extraction, resulting in low information processing efficiency and high error rate, making it difficult to achieve high-precision automated processing. .
The deep learning convolutional neural network is used for image denoising and instance segmentation, combined with the fuzzy C-mean clustering algorithm and local thresholding method for adaptive binary processing, integrated deep OCR model for text and graphic logo recognition, and through multi-mode The state fusion algorithm performs semantic understanding and structured processing, and introduces an intelligent recommendation system and data encryption transmission protocol.
It significantly improves the automation level of invoice and commercial information processing, improves processing speed and accuracy, reduces error rates and operation costs, and ensures the correctness and security of information.
Smart Images

Figure CN119007219A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and device for extracting and processing image feature information. Background Art
[0002] In recent years, with the rapid development of information technology, digital transformation has become the goal pursued by all walks of life. Especially in the field of financial management, the traditional manual processing of invoices and commercial bills can no longer meet the needs of modern enterprises for efficient operation. Such paper document processing is labor-intensive, inefficient, error-prone, and difficult to achieve real-time data analysis and sharing. Therefore, automated processing of invoices and commercial bills has become a key link in improving financial management efficiency. The widespread application of optical character recognition (OCR) technology provides technical support for this transformation, but traditional OCR technology often exhibits problems of low recognition rate and inaccurate information extraction when faced with complex backgrounds, low-quality images, and diverse document formats.
[0003] When processing invoice and commercial bill images, existing technologies face problems such as uneven image quality, inaccurate positioning of key information areas, and imperfect verification and correction mechanisms after information extraction. Especially in information-intensive and format-changing financial documents, it is difficult to achieve high-precision information extraction and processing effects by relying solely on OCR technology, which leads to low information processing efficiency, high error rate, and increased burden of manual review and correction. Summary of the invention
[0004] The purpose of the present invention is to provide a method and device for extracting and processing image feature information, which significantly improves the level of automation in invoice and commercial bill information processing, reduces the frequency of manual intervention, greatly improves processing speed and accuracy, reduces error rate and operating costs, so as to solve the problems in the prior art raised in the above background technology.
[0005] To achieve the above object, the present invention provides a method for extracting and processing image feature information, comprising the following steps:
[0006] Step 1: Receive invoice or commercial bill images from different sources, perform preliminary classification based on the features of the images, and distinguish different types of bills;
[0007] Step 2: Perform deep denoising on the classified image, followed by adaptive binarization, dynamically adjust the threshold according to the local contrast of the image, optimize the distinction between text and background, and then segment the key information area of the image;
[0008] Step 3: In the segmented key information area, perform operations of extracting text information and identifying graphic signs to obtain text information;
[0009] Step 4: Perform semantic understanding and structural processing on the extracted text information, parse out the key fields of the invoice or commercial bill, combine the text, graphics and structural information in the image, and cross-validate the extracted information through a multimodal fusion algorithm;
[0010] Step 5: Check the rationality of the amount, the validity of the date, and the legitimacy of the invoice code and number. For errors or uncertain information found during the verification process, provide a user interface for interactive correction, allow manual intervention to verify and correct, and determine the final data;
[0011] Step 6: The final processed data is output in a standardized format, supporting multiple interfaces to connect to the enterprise financial system. At the same time, the original image, processing process records and final recognition results are uniformly stored in a secure database.
[0012] In the deep denoising process, a convolutional neural network based on deep learning is used to denoise the image, and the network is pre-trained to identify and remove spots, stripes and other unstructured noise in the image.
[0013] In the adaptive binarization process, the fuzzy C-means clustering algorithm and the local threshold method are combined to calculate the optimal binarization threshold for different areas of the image, including: dividing the denoised image into multiple small blocks or pixel areas, the size of each area is pre-set according to the image resolution and content complexity; in each divided image block, the fuzzy C-means clustering algorithm is applied to calculate the similarity between pixels, and the pixels are soft-classified into different categories, each category representing a possible hue range; for each divided block, based on the result of the fuzzy C-means clustering, the pixel intensity distribution of each category is calculated, an optimal binarization threshold is determined for each block, and the binarization process is performed separately on each divided block according to the calculated local threshold; all the image blocks that have undergone local binarization processing are re-merged into a complete image.
[0014] In the step of segmenting the key information area of the image, a deep learning-based instance segmentation technology is used to identify and mark the key areas in the invoice or commercial bill.
[0015] In the operations of extracting text information and identifying graphic symbols, a deep OCR model is integrated to recognize text and graphic symbols.
[0016] The semantic understanding and structured processing steps include: information extraction and preliminary cleaning: collecting text information obtained from OCR recognition and graphic logo decoding; entity recognition and annotation: identifying key entities in the text; knowledge graph matching and linking: using the pre-built financial field knowledge graph to match and link the identified entities, and through the entity linking technology, connecting the extracted text information with the concepts in the graph; semantic parsing and relationship construction: conducting in-depth semantic parsing, understanding the implicit semantic relationship in the text, parsing sentence components, clarifying the subject-predicate-object relationship, and constructing a structured data framework; structured data generation and verification: based on the parsing results, converting the information of the invoice or commercial bill into a structured data format, each key information has a corresponding label and value, and performing data verification to ensure the logical consistency of the structured data.
[0017] In the multimodal fusion algorithm, a joint learning model integrating visual and text features is used to comprehensively analyze text, graphics and structural information, and the attention mechanism is used to focus on key information.
[0018] In the process of information verification and correction, an intelligent recommendation system is established to provide users with possible correct options based on historical data and contextual information when incorrect information is found.
[0019] In the result output and storage step, a data encryption transmission protocol is used during the process of outputting to the enterprise financial system or storing in the database.
[0020] On the other hand, the present invention provides an image feature information extraction and processing device, comprising:
[0021] The image preprocessing module is used to receive invoice or commercial bill images from different sources, and perform preliminary classification based on the features of the images to distinguish different types of bills;
[0022] The image classification module is used to perform deep denoising on the classified images, followed by adaptive binarization, dynamically adjusting the threshold according to the local contrast of the image, optimizing the distinction between text and background, and then segmenting the key information area of the image;
[0023] An image feature extraction module is used to extract text information and recognize graphic signs in the segmented key information area to obtain text information;
[0024] The information recognition and parsing module is used to perform semantic understanding and structural processing on the extracted text information, parse out the key fields of the invoice or commercial bill, combine the text, graphics and structural information in the image, and cross-validate the extracted information through a multimodal fusion algorithm;
[0025] The information verification and correction module is used to check the rationality of the amount, the validity of the date, and the legitimacy of the invoice code and number. For errors or uncertain information found during the verification process, a user interface is provided for interactive correction, allowing manual intervention to verify and correct, and determine the final data;
[0026] The result output and storage module is used to output the final processed data in a standardized format and supports multiple interfaces to connect to the enterprise financial system. At the same time, the original image, processing process records and final recognition results are uniformly stored in a secure database.
[0027] Technical effects and advantages of the present invention: Compared with the prior art, the image feature information extraction and processing method and device proposed by the present invention have the following advantages:
[0028] The present invention can realize efficient and accurate processing of invoice and commercial bill images. It can not only automatically classify and pre-process images to improve the distinction between text and background, but also accurately extract and identify key information, including text information and graphic symbols. Through semantic understanding and structured processing, combined with multimodal fusion algorithms, the accuracy and completeness of information extraction are improved. At the same time, intelligent verification and user interaction correction mechanisms are introduced to ensure the correctness of the information. Ultimately, the standardized output data can be directly connected to the enterprise financial system, and all processing records and original images are securely stored, providing strong support for the company's financial management and auditing. It significantly improves the automation level of invoice and commercial bill information processing, reduces the frequency of manual intervention, greatly improves processing speed and accuracy, and reduces error rates and operating costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a flow chart of the image feature information extraction and processing method of the present invention;
[0030] Figure 2 It is a block diagram of the image feature information extraction and processing device of the present invention. DETAILED DESCRIPTION
[0031] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0032] The present invention provides Figure 1The image feature information extraction and processing method shown is suitable for automatic information extraction of financial documents such as invoices and commercial bills, and aims to improve the efficiency, accuracy and automation level of information extraction.
[0033] In this embodiment, a method for extracting and processing image feature information includes the following steps:
[0034] Receive invoice or commercial bill images from different sources, perform preliminary classification based on the features of the images, and distinguish different types of bills;
[0035] The classified images are subjected to deep denoising, followed by adaptive binarization, which dynamically adjusts the threshold according to the local contrast of the image, optimizes the distinction between text and background, and then segments the key information areas of the image;
[0036] In the segmented key information area, operations of extracting text information and recognizing graphic signs are performed to obtain text information;
[0037] Perform semantic understanding and structural processing on the extracted text information, parse out the key fields of the invoice or commercial bill, combine the text, graphics and structural information in the image, and cross-validate the extracted information through a multimodal fusion algorithm;
[0038] Check the rationality of the amount, the validity of the date, and the legitimacy of the invoice code and number. Provide a user interface for interactive correction of errors or uncertain information found during the verification process, allowing manual intervention to verify and correct and determine the final data;
[0039] The final processed data is output in a standardized format, supporting multiple interfaces to connect to the enterprise financial system. At the same time, the original image, processing process records and final recognition results are uniformly stored in a secure database.
[0040] Furthermore, the image feature information extraction and processing method is characterized in that in the deep denoising processing step, a convolutional neural network based on deep learning is used to denoise the image, and the network is pre-trained to identify and remove spots, stripes and other unstructured noise in the image to maintain the clarity of text and lines in the image.
[0041] In the image feature information extraction and processing method, the specific method steps for the deep denoising process using a convolutional neural network based on deep learning are as follows:
[0042] Data preparation and preprocessing: Invoice and commercial bill images containing various types of noise are collected as training data sets, including unstructured noise such as spots, stripes, blur, etc. The training data is preprocessed, including size standardization, grayscale processing, and brightness and contrast adjustment for images under extreme lighting conditions to ensure the consistency and quality of the data set.
[0043] Deep learning network architecture design: Design a convolutional neural network (CNN) model optimized for image denoising, which usually contains multiple convolutional layers, pooling layers, and denoising-specific feature extraction layers. Introduce skip connections or residual structures so that the model can better preserve image detail information and maintain the clarity of text and lines while removing noise. The last layer of the network is usually a denoising output layer, which is designed to be the same size as the input image to directly output the denoised image.
[0044] Network pre-training and fine-tuning: Use existing public image denoising datasets to pre-train the network so that the network can learn to identify and separate noise and useful information in images. Fine-tune the pre-trained model for specific noise types in invoices and commercial bills, and use a dedicated labeled dataset to optimize network parameters and improve its performance on the target task.
[0045] Denoising training and verification: The preprocessed invoice and commercial bill images are input into the trained CNN model, and the model automatically learns and removes unstructured noise such as spots and stripes in the image.
[0046] Implement a cross-validation strategy, divide the dataset into training set, validation set, and test set, adjust the model parameters through the validation set to ensure the generalization ability of the model. Regularly evaluate the denoising effect of the model on the test set, including indicators such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index) to quantify the denoising quality and image fidelity.
[0047] Denoising post-processing and integration: Post-process the denoised image output by the model, such as slight smoothing filtering, to remove any remaining denoising traces while keeping the edges of text and lines sharp. Integrate the denoised image into the entire image processing pipeline as input for subsequent binarization, feature extraction and other steps to ensure the efficiency and accuracy of the entire image processing process.
[0048] Through the above steps, the deep learning denoising method can effectively remove various types of unstructured noise while maintaining the clarity of text and lines in the image, providing a purer and easier-to-analyze image resource for subsequent image processing and information extraction.
[0049] Furthermore, the image feature information extraction and processing method is characterized in that in the adaptive binarization processing step, the fuzzy C-means clustering algorithm and the local threshold method are combined to calculate the optimal binarization threshold for different areas of the image respectively to adapt to the uneven lighting and complex background in the image and improve the recognition of text information.
[0050] The specific steps of the adaptive binarization processing step combined with the fuzzy C-means clustering algorithm and the local threshold method are as follows:
[0051] Image segmentation into uniform blocks: First, the denoised image is segmented into multiple small blocks or pixel regions, where the size of each region is pre-set based on the image resolution and content complexity, in order to more finely process different lighting and texture areas in the image.
[0052] Fuzzy C-means clustering analysis: Within each segmented image block, a fuzzy C-means clustering algorithm is applied. The algorithm soft-classifies pixels into different categories by calculating the similarity between pixels, each category representing a possible tonal range (background or foreground).
[0053] Calculate local threshold: For each segmented block, based on the results of fuzzy C-means clustering, calculate the pixel intensity distribution of each category, and use Otsu method or other adaptive threshold algorithms to determine an optimal binarization threshold for each block. This threshold reflects the optimal segmentation point of the foreground and background in the current block, ensuring that text information can still be effectively distinguished in a complex background.
[0054] Local binarization application: Binarization is performed on each segmented block separately according to the calculated local threshold.
[0055] Image block merging and post-processing: All the local binarization image blocks are merged back into a complete image. Since the binarization threshold of each area is optimal, the merged image can better display the overall text information.
[0056] Through the above steps, the adaptive binarization processing method combining the fuzzy C-means clustering algorithm and the local threshold method can effectively adapt to the complex lighting and background conditions in the image, significantly improve the recognition of text information, and lay a solid foundation for subsequent information extraction and recognition.
[0057] Furthermore, the image feature information extraction and processing method is characterized in that in the step of segmenting the key information area of the image, a deep learning-based instance segmentation technology is used to automatically identify and mark the key areas in the invoice or commercial bill, such as the invoice title, amount column, date column, etc., to achieve precise area cutting.
[0058] Regarding the step of segmenting the key information area of the image, the specific method steps using the instance segmentation technology based on deep learning are as follows:
[0059] Dataset construction and preprocessing: Collect a large number of labeled invoice and commercial bill image samples, in which key areas such as invoice title, amount column, date column, etc. have been accurately marked. These samples will be used to train the deep learning model.
[0060] Model selection and architecture design: Choose a deep learning model suitable for instance segmentation tasks, such as MaskR-CNN, U-Net, or DeepLabV3+, which can simultaneously predict the object category in the image and the accurate pixel-level segmentation mask.
[0061] Training model: Use the constructed data set to train the selected model. During the training process, the back propagation algorithm is used to optimize the model parameters and minimize the loss function, so that the model can learn how to distinguish and mark the key information areas in the image.
[0062] Model validation and tuning: Evaluate model performance on an independent validation set, and measure the accuracy of the model in segmenting key areas based on evaluation indicators such as mean average precision (mAP) and IoU (intersection over union).
[0063] Automatic segmentation application and post-processing: Apply the trained model to perform real-time segmentation on new invoice or commercial bill images. The model outputs a segmentation mask for each key area, based on which key information areas such as the invoice title, amount column, date column, etc. are automatically identified and marked.
[0064] Through the above steps, the deep learning-based instance segmentation technology can automatically and accurately segment the key areas in the invoice or commercial bill images, providing an accurate basis for subsequent information extraction and processing, greatly improving the accuracy and efficiency of information extraction.
[0065] Furthermore, the image feature information extraction and processing method is characterized in that a deep OCR model is integrated in the operation steps of extracting text information and identifying graphic signs. The model not only recognizes text, but also can recognize specific graphic signs such as QR codes, barcodes and special seals through convolutional neural networks, thereby enhancing the comprehensiveness of information extraction.
[0066] Regarding the steps for extracting text information and recognizing graphic symbols, the specific steps for integrating the deep OCR model are as follows, for invoice or commercial bill scenarios:
[0067] Image preprocessing and feature extraction: After the image has been preprocessed (such as denoising, binarization, and tilt correction), feature extraction is performed on the image. First, a convolutional neural network (CNN) is used to extract global features, which are not only helpful for text recognition, but also crucial for the recognition of graphic signs (such as QR codes and barcodes).
[0068] Text area positioning: Through the object detection technology in the deep learning model, such as YOLO (You Only Look Once) or Faster R-CNN, the text area in the image is automatically located. At the same time, possible graphic logo areas are identified, such as the rectangular frames of QR codes and barcodes.
[0069] Application of deep OCR model: In the located text area, the deep OCR model is applied for text recognition. This model usually includes stages such as character segmentation and character recognition, and uses recurrent neural network (RNN) or Transformer architecture to serialize and recognize characters, so as to achieve accurate recognition of complex text on invoices and commercial bills.
[0070] Graphic logo recognition: For the identified graphic logo area, a specially designed convolutional neural network or pre-trained model is used for decoding. These models are optimized for specific types of graphic logos and can directly decode the encoded information from the image, such as product codes, URLs, or digital information.
[0071] Information integration and verification: The recognized text information and the decoded graphic logo information are integrated to form a complete invoice or commercial bill data set. Subsequently, information verification steps are performed, such as amount rationality verification and date format verification, to ensure that the extracted information is accurate. For special seal recognition, deep learning models can be used for pattern matching. By comparing the seal image with the preset legal seal database, the authenticity of the seal can be confirmed, further enhancing the comprehensiveness and security of information extraction.
[0072] Through the above steps, the solution not only efficiently extracts the text information on invoices or commercial bills, but also realizes accurate recognition of graphic logos by integrating deep learning models, improves the comprehensiveness and automation level of information extraction, and provides a reliable data foundation for subsequent financial processing processes.
[0073] Furthermore, the image feature information extraction and processing method is characterized in that in the semantic understanding and structured processing steps, natural language processing technology is combined with knowledge graphs to perform semantic analysis on the extracted text, identify and link to predefined entities and concepts, such as associating the "invoicing date" with the actual date value to form structured data.
[0074] Regarding the steps of semantic understanding and structured processing, the specific method steps of using natural language processing technology combined with knowledge graph are as follows, taking invoices or commercial bills as the application scenario:
[0075] Information extraction and preliminary cleaning: First, collect text information obtained from OCR recognition and graphic logo decoding, including key fields such as invoice number, date, amount, recipient, payee, etc. Entity recognition and annotation: Identify key entities in the text, such as date, amount, company name, etc.
[0076] Knowledge graph matching and linking: Use the pre-built financial domain knowledge graph to match and link the identified entities. The knowledge graph contains a rich set of entities and concepts and their relationships, such as the mapping of "invoicing date" to the actual date, and the correspondence between "company name" and corporate registration information. Through entity linking technology, the extracted text information is connected to the concepts in the graph, giving the information more contextual meaning.
[0077] Semantic analysis and relationship construction: Conduct in-depth semantic analysis to understand the implicit semantic relationship in the text, such as the payment behavior and amount relationship implied in "XX Company paid XX yuan to YY Company". Use syntactic analysis and semantic role labeling (SRL) technology to parse sentence components, clarify the subject-verb-object relationship, and build a structured data framework.
[0078] Structured data generation and verification: Based on the parsing results, the invoice or commercial bill information is converted into a structured data format, and each key information has a corresponding label and value. For example, {"invoice date": "2023-04-01", "amount": "1000 yuan", "payee": "XX company", "payer": "YY company"}.
[0079] Finally, perform data validation to ensure the logical consistency of structured data, such as checking whether the date format is correct, whether the amount value is reasonable, etc. If necessary, return to the previous step to make corrections.
[0080] Through the above steps, the solution converts the unstructured text information in invoices or commercial bills into structured data, which not only facilitates subsequent automated processing and analysis, but also improves the availability and accuracy of the data, providing a solid foundation for financial automation, data analysis, compliance auditing and other tasks.
[0081] Furthermore, the image feature information extraction and processing method is characterized in that in the multimodal fusion algorithm step, a joint learning model integrating visual and text features is used to comprehensively analyze text, graphics and structural information, and the attention mechanism is used to focus on key information to improve the accuracy of cross-validation. The method steps specifically applied to invoice or commercial bill processing are as follows:
[0082] Image preprocessing and feature extraction: Perform basic processing on invoice or commercial bill images, including denoising, binarization, tilt correction, etc., to optimize visual feature extraction conditions. Use deep learning models (such as convolutional neural networks (CNNs)) to extract visual features of images, with special attention paid to areas with significant information, such as text areas such as amounts and dates, and specific graphic signs (such as QR codes and barcodes).
[0083] OCR recognition and text feature extraction: Use OCR technology to recognize text information in images and convert it into text data. Use natural language processing technology (NLP) to extract text features, such as keywords and part-of-speech tags, to prepare for subsequent text analysis.
[0084] Multimodal feature fusion: Design a multimodal fusion layer to integrate visual features and text features into a unified feature space. Deep learning models such as multi-layer perceptron (MLP), recurrent neural network (RNN) or transformer can be used to achieve joint learning of features.
[0085] Joint learning and key information identification: Based on the fusion features, the model is jointly trained to learn the relationship between images and texts, and how they jointly indicate the location and content of key information.
[0086] Cross-validation and result optimization: Implement a cross-validation strategy to divide the data set into multiple subsets, which are used as training sets and test sets in turn to ensure the generalization ability of the model. Use the focus guided by the attention mechanism to cross-validate the key information identified, such as comparing the visual features of the amount area in the image with the text information extracted by OCR to ensure the consistency of the two. For incorrectly identified information, a feedback mechanism is used to correct it, such as user interactive confirmation or rule-based automatic correction, to further improve the accuracy and robustness of the overall system.
[0087] Through the above steps, the solution realizes the effective integration of visual and text information, improves the accuracy and efficiency of key information extraction in invoice and commercial bill processing, and lays a solid foundation for automated processing and data analysis.
[0088] Furthermore, the image feature information extraction and processing method is characterized in that in the steps of information verification and correction, an intelligent recommendation system is established. When the information is found to be incorrect, possible correct options are provided to the user based on historical data and contextual information, thereby accelerating the manual correction process.
[0089] The process of establishing an intelligent recommendation system in the information verification and correction step, specifically applied to invoice or commercial bill processing, is as follows:
[0090] Data collection and historical information accumulation: First, collect a large number of invoice or commercial bill samples and their correct information, including manually corrected data, to establish a rich historical database. The database includes not only correct information, but also erroneous information and its correction records.
[0091] Error pattern recognition and analysis: Use machine learning algorithms (such as decision trees, random forests, or neural networks) to analyze historical data and identify common error patterns, such as incorrect date formats, confusion over amounts, etc.
[0092] Intelligent recommendation model construction: Based on the identified error patterns, an intelligent recommendation model is constructed. The model can be a rule-based system or a more complex machine learning model, such as a deep learning model, which can be based on the currently identified error information.
[0093] Real-time error detection and recommendation: During the information extraction and verification process, the system monitors the identified information in real time. Once a suspected error is found, the intelligent recommendation system is immediately activated. Based on the error type, the system matches similar cases from the historical database, analyzes the context information, and provides users with several most likely correct options.
[0094] User interaction and modification confirmation: Provide a user interface to display the correct options for intelligent recommendations and allow users to select or modify recommendations with a simple click or input. The user interface should be intuitive and easy to use, supporting quick browsing and selection. The user's choice will be fed back to the system.
[0095] Feedback loop and model iteration: Record users’ adoption of recommended options and any additional manual corrections. This feedback information flows back into model training to continuously optimize the recommendation algorithm, making it more accurate and accelerating the automatic correction process of similar errors in the future.
[0096] Through the above steps, the intelligent recommendation system can significantly speed up the manual correction process and reduce processing delays caused by information errors. At the same time, through continuous learning and optimization, it can improve the accuracy and efficiency of overall information extraction.
[0097] Furthermore, the image feature information extraction and processing method is characterized in that in the result output and storage steps, a data encryption transmission protocol is implemented to ensure the security and privacy of information during the process of outputting to the enterprise financial system or storing in the database, while supporting data backup and recovery functions to cope with potential data loss risks.
[0098] The specific method steps for implementing the data encryption transmission protocol and ensuring information security in the result output and storage steps are as follows, for the invoice or commercial bill processing scenario:
[0099] Data encryption processing: Before the structured data is output, the Advanced Encryption Standard (AES), RSA or other industry-recognized encryption algorithms are used to encrypt the identified and processed invoice or commercial bill information. This includes sensitive information such as invoice number, amount, date, etc.
[0100] Implement SSL / TLS protocol: Use Secure Sockets Layer (SSL) or Transport Layer Security (TLS) protocol to provide a secure channel for data transmission. This involves establishing an encrypted connection between the server and the corporate financial system to ensure the confidentiality and integrity of data during transmission and prevent man-in-the-middle attacks and data tampering.
[0101] Data backup strategy: Set up an automatic backup mechanism to regularly back up processed structured data and original images to off-site or cloud storage services.
[0102] Data recovery drills: Perform data recovery drills regularly to verify the integrity and availability of backup data.
[0103] Access control and permission management: During the data storage stage, strict access control and permission management policies are implemented. Only specific financial personnel or systems are authorized to access the stored data, and a role-based access control (RBAC) model is adopted to ensure that each user can only access information that matches their responsibilities, further enhancing data privacy protection.
[0104] Through the above steps, this solution ensures the security and privacy of invoice or commercial bill information during transmission, storage, backup and recovery, effectively prevents data leakage risks, and ensures the compliance and reliability of corporate financial data processing.
[0105] On the other hand, the present invention provides an image feature information extraction and processing device, such as Figure 2 As shown, including:
[0106] The image preprocessing module is used to receive invoice or commercial bill images from different sources, and perform preliminary classification based on the features of the images to distinguish different types of bills;
[0107] The image classification module is used to perform deep denoising on the classified images, followed by adaptive binarization, dynamically adjusting the threshold according to the local contrast of the image, optimizing the distinction between text and background, and then segmenting the key information area of the image;
[0108] An image feature extraction module is used to extract text information and recognize graphic signs in the segmented key information area to obtain text information;
[0109] The information recognition and parsing module is used to perform semantic understanding and structural processing on the extracted text information, parse out the key fields of the invoice or commercial bill, combine the text, graphics and structural information in the image, and cross-validate the extracted information through a multimodal fusion algorithm;
[0110] The information verification and correction module is used to check the rationality of the amount, the validity of the date, and the legitimacy of the invoice code and number. For errors or uncertain information found during the verification process, a user interface is provided for interactive correction, allowing manual intervention to verify and correct, and determine the final data;
[0111] The result output and storage module is used to output the final processed data in a standardized format and supports multiple interfaces to connect to the enterprise financial system. At the same time, the original image, processing process records and final recognition results are uniformly stored in a secure database.
[0112] In summary, the present invention can realize efficient and accurate processing of invoice and commercial bill images. It can not only automatically classify and pre-process images, improve the distinction between text and background, but also accurately extract and identify key information, including text information and graphic symbols. Through semantic understanding and structured processing, combined with multimodal fusion algorithms, the accuracy and completeness of information extraction are improved. At the same time, intelligent verification and user interaction correction mechanisms are introduced to ensure the correctness of information. Ultimately, the standardized output data can be directly connected to the enterprise financial system, and all processing records and original images are securely stored, providing strong support for the enterprise's financial management and auditing. It significantly improves the automation level of invoice and commercial bill information processing, reduces the frequency of manual intervention, greatly improves processing speed and accuracy, and reduces error rates and operating costs.
[0113] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for extracting and processing image feature information, It is characterized in that The following steps are involved: Step 1: Receive invoice or commercial bill images from different sources, perform preliminary classification based on the features of the images, and distinguish different types of bills; Step 2: Perform deep denoising on the classified image, followed by adaptive binarization, dynamically adjust the threshold according to the local contrast of the image, optimize the distinction between text and background, and then segment the key information area of the image; Step 3: In the segmented key information area, perform operations of extracting text information and identifying graphic signs to obtain text information; Step 4: Perform semantic understanding and structural processing on the extracted text information, parse out the key fields of the invoice or commercial bill, combine the text, graphics and structural information in the image, and cross-validate the extracted information through a multimodal fusion algorithm; Step 5: Check the rationality of the amount, the validity of the date, and the legitimacy of the invoice code and number. For errors or uncertain information found during the verification process, provide a user interface for interactive correction, allow manual intervention to verify and correct, and determine the final data; Step 6: The final processed data is output in a standardized format, supporting multiple interfaces to connect to the enterprise financial system. At the same time, the original image, processing process records and final recognition results are uniformly stored in a secure database.
2. The method for extracting and processing image feature information according to claim 1, Features: In the deep denoising process, a convolutional neural network based on deep learning is used to denoise the image, and the network is pre-trained to identify and remove spots, stripes and other unstructured noise in the image.
3. The method for extracting and processing image feature information according to claim 2, Features: In the adaptive binarization process, the fuzzy C-means clustering algorithm and the local threshold method are combined to calculate the optimal binarization threshold for different areas of the image, including: Segment the denoised image into multiple small blocks or pixel regions, with the size of each region pre-set based on the image resolution and content complexity; In each segmented image block, the fuzzy C-means clustering algorithm is applied to calculate the similarity between pixels and softly classify the pixels into different categories, each of which represents a possible tonal range; For each segmented block, based on the result of fuzzy C-means clustering, the pixel intensity distribution of each category is calculated, an optimal binarization threshold is determined for each block, and binarization processing is performed separately for each segmented block according to the calculated local threshold; Merge all locally binarized image patches back into a complete image.
4. The method for extracting and processing image feature information according to claim 3, Features: In the step of segmenting the key information area of the image, a deep learning-based instance segmentation technology is used to identify and mark the key areas in the invoice or commercial bill.
5. The method for extracting and processing image feature information according to claim 4, Features: In the operations of extracting text information and identifying graphic symbols, a deep OCR model is integrated to recognize text and graphic symbols.
6. The method for extracting and processing image feature information according to claim 5, Features: The semantic understanding and structured processing steps include: Information extraction and preliminary cleaning: Collect text information obtained from OCR recognition and graphic logo decoding; Entity recognition and annotation: identifying key entities in text; Knowledge graph matching and linking: Use the pre-built financial domain knowledge graph to match and link the identified entities. Use entity linking technology to connect the extracted text information with the concepts in the graph. Semantic analysis and relationship construction: Conduct in-depth semantic analysis to understand the implicit semantic relationships in the text, analyze sentence components, clarify the subject-verb-object relationship, and construct a structured data framework; Structured data generation and verification: Based on the parsing results, the invoice or commercial bill information is converted into a structured data format. Each key information has a corresponding label and value, and data verification is performed to ensure the logical consistency of the structured data.
7. The method for extracting image feature information according to claim 6, Features: In the multimodal fusion algorithm, a joint learning model integrating visual and text features is used to comprehensively analyze text, graphics and structural information, and the attention mechanism is used to focus on key information.
8. The method for extracting and processing image feature information according to claim 1, Features: In the process of information verification and correction, an intelligent recommendation system is established to provide users with possible correct options based on historical data and contextual information when incorrect information is found.
9. The method for extracting and processing image feature information according to claim 1, Features: In the result output and storage step, a data encryption transmission protocol is used during the process of outputting to the enterprise financial system or storing in the database.
10. An image feature information extraction and processing device, It is characterized in that include: The image preprocessing module is used to receive invoice or commercial bill images from different sources, and perform preliminary classification based on the features of the images to distinguish different types of bills; The image classification module is used to perform deep denoising on the classified images, followed by adaptive binarization, dynamically adjusting the threshold according to the local contrast of the image, optimizing the distinction between text and background, and then segmenting the key information area of the image; An image feature extraction module is used to extract text information and recognize graphic signs in the segmented key information area to obtain text information; The information recognition and parsing module is used to perform semantic understanding and structural processing on the extracted text information, parse out the key fields of the invoice or commercial bill, combine the text, graphics and structural information in the image, and cross-validate the extracted information through a multimodal fusion algorithm; The information verification and correction module is used to check the rationality of the amount, the validity of the date, and the legitimacy of the invoice code and number. For errors or uncertain information found during the verification process, a user interface is provided for interactive correction, allowing manual intervention to verify and correct, and determine the final data; The result output and storage module is used to output the final processed data in a standardized format and supports multiple interfaces to connect to the enterprise financial system. At the same time, the original image, processing process records and final recognition results are uniformly stored in a secure database.
Citation Information
Patent Citations
Image segmentation threshold determination method and device, and electronic equipment
CN115810024A
Multi-mode invoice automatic classification identification method, verification method and system
CN116052186A
Bill identification method based on deep learning and knowledge graph
CN116343237A
Image processing method and device, computer equipment and storage medium
CN116955686A
Unsupervised color geological image segmentation method and system based on attention mechanism
CN117173193A
Cited By
Data processing method and system based on financial information bill
CN119249105A
Invoice information positioning and interception system and method based on image recognition
CN119785369A
Intelligent control method of multi-frequency modified adsorption system
CN119847050A
Intelligent control method of multi-frequency modified adsorption system
CN119847050B
Electromechanical product solid waste attribute intelligent identification method and system
CN120064279A
