Intelligent customs declaration management system based on multi-model cooperation

Abstract

The invention discloses an intelligent customs declaration management system based on multi-model collaboration. The intelligent customs declaration management system comprises the steps of file format judgment, picture preprocessing, file uploading, information data extraction, storage and translation and the like. By means of the mode, the intelligent customs declaration management system based on multi-model cooperation can achieve automatic file sorting and direction correction, reduce manual intervention, improve the OCR recognition accuracy under complex format and multi-language scenes through layout analysis and AI large model understanding semantics, and improve the user experience. Therefore, the robustness and efficiency of key information extraction are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent systems technology, and in particular to an intelligent customs declaration management system based on multi-model collaboration. Background Technology

[0002] Customs declaration documents (such as invoices, packing lists, bills of lading, etc.) contain multilingual and multi-format structured / unstructured data. Traditional processing requires manual sorting of document types, text recognition, and location of key information.

[0003] Disadvantages of traditional processing methods:

[0004] 1. Reliance on manual labor leads to low efficiency and high costs:

[0005] Existing technology requires manual sorting and orientation adjustment of documents. If 1,000 documents are processed per day, it takes about 8 hours per person, and sorting errors are easily caused by fatigue (error rate of about 5%).

[0006] 2. OCR recognition accuracy is affected by layout and orientation:

[0007] Customs declaration documents often have issues such as skewed scanning and mixed language layout (e.g., Chinese and English). Current technology lacks orientation judgment and layout analysis, resulting in an OCR recognition error rate as high as 15% (especially for handwritten text or complex tables).

[0008] 3. Poor robustness in extracting key information:

[0009] Data extraction based solely on keyword matching cannot handle non-fixed formats (such as differences in invoice formats between different countries), leading to omissions or errors in extracting key information (omission rate of approximately 20%).

[0010] In summary, this solution addresses the problems inherent in traditional customs document information extraction processes, including reliance on manual sorting, low OCR accuracy (affected by document orientation and layout complexity), low efficiency in extracting key data, insufficient cross-language processing capabilities, high error rates, and high labor costs. With the growth of international trade, manual processing can no longer meet the demands for efficient customs clearance, necessitating an intelligent solution. Summary of the Invention

[0011] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0012] A smart customs declaration management system based on multi-model collaboration is provided, the steps of which include:

[0013] S1. Obtain the uploaded file and process it according to its format type;

[0014] If the uploaded file is in image format, proceed to step S2; if the uploaded file is not in image format, convert the file to image format.

[0015] S2. Preprocess the image;

[0016] S2.1. Determine and correct the image orientation: The image rotation orientation determination model uses deep learning to identify the image's tilt angle: If the current tilt angle of the image is not greater than the preset tilt angle threshold, then proceed to step S3; if the current tilt angle of the image is greater than the preset tilt angle threshold, then automatically correct the image's orientation.

[0017] S2.2. Perform layout analysis and segmentation on images: The layout analysis model detects each file image, divides it into structured regions according to a preset region segmentation template, and marks each region for subsequent extraction and positioning;

[0018] S3. Intelligent document sorting;

[0019] S3.1. Information Recognition and Extraction: Based on the layout division results, the text recognition model performs text recognition on the text regions and generates recognized text with the coordinates of the text positions in the image;

[0020] S3.2. File Classification: Based on the preset set of classification keywords for each type of file, traverse and query the recognized text of each current file to determine the matching degree between the current file and the set of classification keywords corresponding to each type of file;

[0021] S3.2.1. If the matching degree between the current file and a certain category keyword set is greater than or equal to the preset matching degree threshold, then the type of the current file is determined to be the file type corresponding to the category keyword set;

[0022] S3.2.2. If the matching degree of the current file with all category keyword sets is less than the preset matching degree threshold, then determine that the type of the current file is other or notify manual classification.

[0023] S4. Screening and uploading of customs declaration materials;

[0024] S4.1. Obtain the company name of the pre-declaration company, and find the matching customs declaration template in the template library based on the company name;

[0025] S4.2. Based on the customs declaration template, select the corresponding current file and upload it;

[0026] S4.3. Based on the user's selection, optimize the image or proceed to step S5;

[0027] S5. Extraction and verification of customs declaration information and data;

[0028] S5.1. Based on the preset key fields of customs declaration information and data, the AI ​​big model filters the image areas and recognized text in the uploaded file to extract information and data that match the key fields;

[0029] S5.2. Determine whether the extracted information and data need to be validated using JSON, and save the information data that passes JSON validation and the information data that is not JSON data to the database, and then execute step S6;

[0030] S6. The AI ​​big model determines whether the filtered information or data conforms to the target language; if the filtered information or data is not in the target language, the AI ​​big model translates the information or data and saves it.

[0031] In a preferred embodiment of the present invention, in step S2.2, the structured area includes a header area and a product details area.

[0032] In a preferred embodiment of the present invention, in step S3.1, the text recognition model includes an OCR recognition model or the Google Cloud Vision API; the recognized text is generated according to a preset text template.

[0033] In a preferred embodiment of the present invention, in step S3.2, the document types include customs declarations, contracts, and invoices. Each type of document has at least one category keyword group in its category keyword set, and the matching degree is detected one by one according to the document type.

[0034] In a preferred embodiment of the present invention, in step S4, the company name can be obtained by retrieving system emails, retrieving user information uploaded to the system, or by having the pre-registered company open an API to connect with the system for direct access.

[0035] In a preferred embodiment of the present invention, in step S4.1, the customs declaration template includes the type, quantity, and format of customs declaration documents.

[0036] In a preferred embodiment of the present invention, image optimization includes: stamp removal, image orientation determination, image color removal, layout analysis, and merging of cross-page images in a table.

[0037] In a preferred embodiment of the present invention, in step S5.2, if there is information data that fails JSON validation, the information data is marked as failed and a prompt to transfer to human assistance is issued.

[0038] In a preferred embodiment of the present invention, in step S6, the target language is switched according to a preset or user selection.

[0039] In a preferred embodiment of the present invention, after the translation is completed, the translated information data replaces the originally saved information data; alternatively, both the information and data before and after translation can be saved simultaneously, and the specific method can be adjusted and controlled according to actual usage requirements.

[0040] The beneficial effects of this invention are: it can realize automated document sorting and orientation correction, reduce manual intervention, and improve the accuracy of OCR recognition in complex layout and multilingual scenarios through layout analysis and AI large model semantic understanding, thereby improving the robustness and efficiency of key information extraction. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0042] Figure 1 This is a flowchart illustrating a preferred embodiment of an intelligent customs declaration management system based on multi-model collaboration according to the present invention. Detailed Implementation

[0043] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Please see Figure 1 The embodiments of the present invention include:

[0045] A multi-model collaborative intelligent customs declaration management system enables automated and intelligent information extraction from customs declaration documents. Its steps include:

[0046] S1. Obtain the uploaded file and determine its format type;

[0047] If the uploaded file is in image format, no processing will be performed;

[0048] If the uploaded file is not in an image format (PDF, Word, TXT, Excel, etc.), the file will be converted into an image format.

[0049] S2. Preprocess the image;

[0050] S2.1. Determine and correct the image orientation.

[0051] Image rotation direction determination models use deep learning (such as CNN) to identify the tilt angle (0° / 90° / 180° / 270°) of an image:

[0052] If the current tilt angle of the image is not greater than the preset tilt angle threshold, then there is no need to correct the angle of the image.

[0053] If the current tilt angle of the image is greater than the preset tilt angle threshold, the orientation of the image will be automatically corrected.

[0054] S2.2. Image Analysis and Segmentation: The layout analysis model detects tables, images, text, and other areas in each file image, and divides them into structured areas (such as "header area" and "product details area") according to the preset area segmentation template, and marks each area for subsequent extraction and positioning.

[0055] The layout analysis model can be replaced by the LayoutLM series model.

[0056] S3. Intelligent document sorting

[0057] S3.1. Information Recognition and Extraction: Based on the layout division results, the text recognition model performs multilingual OCR recognition on the text region (supporting Chinese, English, Japanese, Korean, etc.) and generates recognized text with the coordinates of the text position in the image;

[0058] The text recognition model includes an OCR recognition model or the Google Cloud Vision API; the recognized text is generated according to a preset text template.

[0059] S3.2. File Classification:

[0060] Based on the preset set of category keywords for each type of file, the system iterates through and queries the recognized text of each current file to determine the degree of matching between the current file and the set of category keywords corresponding to each type of file.

[0061] (1) If the matching degree between the current file and a certain category keyword set is greater than or equal to the preset matching degree threshold, then the type of the current file is determined to be the file type corresponding to the category keyword set;

[0062] (2) If the matching degree of the current file with all the category keyword sets is less than the preset matching degree threshold, then the current file is determined to be of the other type or a reminder / notification is made to manually classify it.

[0063] Furthermore, the document types include customs declarations, contracts, invoices, etc., and each type of document has at least one category keyword group in its category keyword set.

[0064] Furthermore, matching can be performed on a document-by-document basis; for example, the current document can be matched with the category keyword set of customs declarations. If the current document is not a customs declaration, the current document can be matched with the category keyword set of contracts, and so on.

[0065] S4. Screening and Uploading of Customs Declaration Documents

[0066] S4.1. Obtain the company name of the pre-declaration company, and find the matching customs declaration template in the template library based on the company name; the customs declaration template includes the type, quantity, and format of customs declaration documents.

[0067] Furthermore, the company name can be obtained by retrieving system emails or user information uploaded to the system, or the pre-declaration company can open an API to connect with this system for direct access.

[0068] S4.2. Based on the customs declaration template, select the corresponding current file and upload it;

[0069] S4.3. Optimize the image or perform step S5 based on the user's selection.

[0070] Furthermore, image optimization utilizes a plugin for batch image processing. Its functions include: stamp removal, image orientation determination, image color removal (directly converting RGB images to grayscale), layout analysis, and merging images across multiple pages of a table. Specific functions can be selected or added / removed based on actual practical needs. The optimization process follows a certain order; for example, stamp removal typically precedes image orientation determination, and image color removal precedes layout analysis.

[0071] The image optimization functions mentioned above are all existing technologies. For example, image restoration algorithms (such as Photoshop's clone stamp, patch tool, and content-aware fill) or deep learning models (GAN generative adversarial networks) can be used to remove stamps. Deep learning models can be trained to identify image orientation, and color removal can be achieved through color space conversion (such as RGB to grayscale) or channel separation techniques. Image analysis can be achieved by combining image segmentation (such as U-Net semantic segmentation) and layout detection algorithms (such as OCR + geometric analysis).

[0072] S5. Extraction and verification of customs declaration information and data

[0073] S5.1. Based on the preset key fields of customs declaration information and data, the AI ​​big data model (big data extraction model) filters the image areas and recognized text in the uploaded file to extract information and data that match the key fields;

[0074] S5.2. Verify the extracted information and data;

[0075] S5.2. Determine whether there is JSON data in the extracted information data. If there is JSON data, validate the JSON data and save the information data that passes the JSON validation and the information data that is not JSON data to the database, and execute step S6. If there is information data that fails the JSON validation, mark the information data as failed and issue a prompt to transfer to human assistance.

[0076] Furthermore, the verification includes determining whether the format of the information or data is correct and whether the filtered information matches. The verification method used is a conventional technical means in this field.

[0077] S6. The AI ​​big data model determines whether the filtered information or data matches the user's selected target language;

[0078] If the selected information or data is not in the target language, the AI ​​model will translate and save the information or data.

[0079] Furthermore, after translation is completed, the translated information or data can replace the original information or data, or both the information and data before and after translation can be saved simultaneously. The specific settings can be adjusted and controlled according to actual usage needs.

[0080] The beneficial effects of the intelligent customs declaration management system based on multi-model collaboration of the present invention are:

[0081] 1. High efficiency: Automated sorting and pre-processing reduce manual intervention, increasing processing efficiency by 80% (processing 1,000 documents per day takes only 1 hour).

[0082] 2. High accuracy: By combining multiple steps such as direction judgment, layout analysis and AI big model understanding of semantics, the recognition accuracy in complex layout and multilingual scenarios is improved, reducing the recognition error rate to 3%. Moreover, the robustness and efficiency of key information extraction are improved, and the AI ​​big model achieves a key information extraction accuracy of 95% (75% higher than keyword matching).

[0083] 3. Strong robustness: Supports multiple formats and languages ​​(covering 10 languages ​​including English, Japanese, and Korean), adapts to customs declaration document formats of different countries, and meets different usage needs;

[0084] 4. Low labor costs: Reduces manual operations by 90%, lowering labor costs.

[0085] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An intelligent customs declaration management system based on multi-model collaboration, characterized by the following steps: include: S1. Obtain the uploaded file and process it according to its format type; If the uploaded file is in image format, proceed to step S2; if the uploaded file is not in image format, convert the file to image format. S2. Preprocess the image; S2.

1. Determine and correct the image orientation: The image rotation orientation determination model uses deep learning to identify the image's tilt angle: If the current tilt angle of the image is not greater than the preset tilt angle threshold, then proceed to step S3; if the current tilt angle of the image is greater than the preset tilt angle threshold, then automatically correct the image's orientation. S2.

2. Perform layout analysis and segmentation on images: The layout analysis model detects each file image, divides it into structured regions according to a preset region segmentation template, and marks each region for subsequent extraction and positioning; S3. Intelligent document sorting; S3.

1. Information Recognition and Extraction: Based on the layout division results, the text recognition model performs text recognition on the text regions and generates recognized text with the coordinates of the text positions in the image; S3.

2. File Classification: Based on the preset set of classification keywords for each type of file, traverse and query the recognized text of each current file to determine the matching degree between the current file and the set of classification keywords corresponding to each type of file; S3.2.

1. If the matching degree between the current file and a certain category keyword set is greater than or equal to the preset matching degree threshold, then the type of the current file is determined to be the file type corresponding to the category keyword set; S3.2.

2. If the matching degree of the current file with all category keyword sets is less than the preset matching degree threshold, then determine that the type of the current file is other or notify manual classification. S4. Screening and uploading of customs declaration materials; S4.

1. Obtain the company name of the pre-declaration company, and find the matching customs declaration template in the template library based on the company name; S4.

2. Based on the customs declaration template, select the corresponding current file and upload it; S4.

3. Based on the user's selection, optimize the image or proceed to step S5; S5. Extraction and verification of customs declaration information and data; S5.

1. Based on the preset key fields of customs declaration information and data, the AI ​​big model filters the image areas and recognized text in the uploaded file to extract information and data that match the key fields; S5.

2. Determine whether the extracted information and data need to be validated using JSON, and save the information data that passes JSON validation and the information data that is not JSON data to the database, and then execute step S6; S6. The AI ​​big model determines whether the filtered information or data conforms to the target language; if the filtered information or data is not in the target language, the AI ​​big model translates the information or data and saves it.

2. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, In step S2.2, the structured area includes a header area and a product details area.

3. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, In step S3.1, the text recognition model includes an OCR recognition model or the Google Cloud Vision API; the recognized text is generated according to a preset text template.

4. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, In step S3.2, the document types include customs declarations, contracts, and invoices. Each document type has at least one category keyword set, and the matching degree is checked one by one according to the document type.

5. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, In step S4, the company name can be obtained by retrieving system emails, retrieving user information uploaded to the system, or by having the pre-declaration company open an API to connect with this system for direct access.

6. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, In step S4.1, the customs declaration template includes the type, quantity, and format of customs declaration documents.

7. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, Image optimization includes: stamp removal, image orientation determination, image color removal, layout analysis, and merging images across tables.

8. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, In step S5.2, if there is information data that fails JSON validation, the information data is marked as failed and a prompt to transfer to human assistance is issued.

9. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, In step S6, the target language is switched according to a preset or user selection.

10. The intelligent customs declaration management system based on multi-model collaboration according to claim 1, characterized in that, After translation is complete, the translated information replaces the original saved information; alternatively, both the pre-translation and post-translation information can be saved simultaneously, depending on the specific usage requirements.

Images