Customs declaration document identification method and device

Abstract

This invention provides a method and apparatus for recognizing customs declaration documents, relating to the field of artificial intelligence technology. The method includes: acquiring a target image of a customs declaration document to be recognized; inputting the target image and prompt words into a visual language model to obtain a first recognition result output by the visual language model; the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of a CogVLM40B model; performing optical character recognition on the target image to obtain a second recognition result; and correcting the first recognition result based on the second recognition result to obtain a target recognition result corresponding to the customs declaration document. The customs declaration document recognition method provided by this invention improves the efficiency and accuracy of customs declaration document recognition by integrating the semantic understanding advantages of visual language models and the character accuracy advantages of optical character recognition through automated collaborative verification.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for recognizing customs declaration documents. Background Technology

[0002] With the rapid rise of emerging business models such as cross-border e-commerce, international logistics, and multimodal transport, enterprises are dealing with increasingly complex types of customs documents in practice, including bills of lading, invoices, packing lists, declaration elements, contracts, and other documents. These documents have diverse formats, complex content, and often involve different languages, currencies, and declaration requirements.

[0003] Existing technologies mainly rely on manual data entry and verification to extract key information from various customs declaration documents and convert it into standard field data that conforms to customs regulations.

[0004] However, manual data entry and verification not only takes a lot of time, but also suffers from human error in understanding information or inputting errors, resulting in low efficiency and accuracy in customs document recognition. Summary of the Invention

[0005] This invention provides a method and apparatus for identifying customs declaration documents, which solves the technical problem of low efficiency and accuracy in the identification of customs declaration documents in the prior art.

[0006] This invention provides a method for identifying customs declaration documents, comprising the following steps: Obtain the target image of the customs declaration document to be identified; The target image and prompt words are input into the visual language model to obtain the first recognition result output by the visual language model; the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model. Optical character recognition is performed on the target image to obtain a second recognition result; Based on the second identification result, the first identification result is corrected to obtain the target identification result corresponding to the customs declaration document.

[0007] According to a method for identifying customs declaration documents provided by the present invention, the step of correcting the first identification result based on the second identification result to obtain the target identification result corresponding to the customs declaration document includes: Obtain all key-value pairs in the first recognition result; wherein, the key-value pairs are composed of field names as keys and corresponding field values ​​as values; Obtain all text values ​​and their corresponding frequencies from the second recognition result; Based on the frequency of occurrence, determine the target similarity between each field value and the target text value; Based on the target similarity, the value of each field is corrected to obtain the target recognition result corresponding to the customs declaration document.

[0008] According to a customs declaration document recognition method provided by the present invention, determining the target similarity between each field value and the target text value based on the frequency of occurrence includes: The target text value is defined as all text values ​​whose frequency of occurrence is greater than 0. Calculate the similarity between each field value and each target text value; The highest similarity score among all similarities is taken as the target similarity score.

[0009] According to a method for identifying customs declaration documents provided by the present invention, the step of correcting the value of each field based on the target similarity includes: If the target similarity is greater than a preset threshold, the field value is replaced with the text value, and the frequency of occurrence of the text value is reduced by one. If the target similarity is less than or equal to the preset threshold, the field value remains unchanged.

[0010] According to a method for recognizing customs declaration documents provided by the present invention, the step of acquiring a target image of the customs declaration document to be recognized includes: Acquire the initial images of the customs declaration documents; The initial image is cropped and rotated according to preset requirements to obtain the target image.

[0011] According to a method for identifying customs declaration documents provided by the present invention, after correcting the first identification result based on the second identification result to obtain the target identification result corresponding to the customs declaration document, the method further includes: Obtain the field names from the target recognition results; The field code corresponding to the field name is retrieved from the standard field database; the standard field database is constructed based on preset business standards. Replace the field name with the field code.

[0012] The present invention also provides a customs declaration document recognition device, comprising the following modules: The acquisition module is used to acquire the target image of the customs declaration document to be identified; The first recognition module is used to input the target image and prompt words into a visual language model to obtain a first recognition result output by the visual language model; the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model with a large language model. The second recognition module is used to perform optical character recognition on the target image to obtain a second recognition result; The correction module is used to correct the first identification result based on the second identification result to obtain the target identification result corresponding to the customs declaration document.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the customs declaration document identification method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the customs declaration document recognition method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the customs declaration document recognition method as described above.

[0016] The customs declaration document recognition method and apparatus provided by this invention acquires a target image of the customs declaration document to be recognized; inputs the target image and prompt words into a visual language model to obtain a first recognition result output by the visual language model; wherein, the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model, thereby utilizing the visual semantic understanding capability of the multimodal large model after domain fine-tuning to achieve intelligent and context-related recognition of key fields of the customs declaration document; performs optical character recognition on the target image to obtain a second recognition result, thereby achieving character-level recognition of all text content in the target image, providing a high-precision text benchmark for subsequent verification and correction of the visual language model output; based on the second recognition result, the first recognition result is corrected to obtain the target recognition result corresponding to the customs declaration document, thereby achieving an automated collaborative verification by integrating the semantic understanding advantage of the visual language model and the character precision advantage of optical character recognition, achieving a recognition effect of 1+1>2, and eliminating the need for manual input and verification, thus improving the efficiency and accuracy of customs declaration document recognition. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the customs declaration document identification method provided by the present invention.

[0019] Figure 2 This is a schematic diagram illustrating an example of mapping a set of document fields to a set of business fields provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the business semantic mapping process provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the customs declaration document recognition device provided by the present invention.

[0022] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] With the rapid rise of emerging business models such as cross-border e-commerce, international logistics, and multimodal transport, enterprises are dealing with increasingly complex customs documents, including bills of lading, invoices, packing lists, declaration elements, contracts, and other documents. These documents are diverse in format and complex in content, often involving different languages, currencies, and declaration requirements. Traditional manual data entry and verification methods are not only inefficient but also prone to errors due to misunderstandings or input, leading to declaration failures, customs clearance delays, and even cargo detentions, resulting in additional time costs and compliance risks for enterprises.

[0025] Faced with the increasing demand for document processing and increasingly stringent regulatory standards, how to efficiently and accurately extract key information from various customs declaration-related documents and transform it into standard field data that conforms to customs regulations has become one of the core issues that foreign trade enterprises and customs brokerage agencies urgently need to solve.

[0026] To address this, this invention proposes a customs document recognition method based on artificial intelligence (AI) for automated identification and intelligent extraction technology, aimed at improving customs clearance efficiency and reducing error rates. Specifically, by integrating cutting-edge technologies such as optical character recognition (OCR), natural language processing (NLP), machine learning, and generative AI, the method intelligently parses original documents in various formats (such as scanned images, PDF files, or spreadsheets), automatically identifying and extracting key fields such as product name, Harmonized System (HS) code, quantity, unit price, total price, mode of transport, and country of origin. This information is then standardized and structured for output into the data format required for customs declaration. This automated processing method, combining intelligent identification and field mapping, reduces manual intervention, improves data accuracy and consistency, and effectively avoids declaration failures caused by field mismatches or format errors. This, in turn, improves customs clearance efficiency, reduces labor costs and compliance risks, and helps enterprises achieve efficient, compliant, and intelligent operations in the face of fierce international competition. Furthermore, the customs document recognition method proposed in this invention can flexibly adapt to different customs declaration templates, supporting multilingual and multi-currency information conversion and output, meeting the needs of enterprises expanding their global business.

[0027] The following is combined Figures 1 to 5 The present invention describes a method and apparatus for identifying customs declaration documents.

[0028] Figure 1 This is a flowchart illustrating the customs document identification method provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps: Step 101: Obtain the target image of the customs declaration document to be identified; Step 102: Input the target image and prompt words into the visual language model to obtain the first recognition result output by the visual language model; the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model with a large language model; Step 103: Perform optical character recognition on the target image to obtain a second recognition result; Step 104: Based on the second identification result, correct the first identification result to obtain the target identification result corresponding to the customs declaration document.

[0029] Specifically, customs declaration documents include, but are not limited to, key customs declaration documents such as bills of lading, invoices, packing lists, declaration elements, and contracts used for customs cargo declaration. First, high-resolution images are captured from numerous documents using a scanner or high-definition photographic device (such as a mobile phone or camera) as target images. Then, for the extracted target images, corresponding large-scale model prompts are written according to preset business requirements to guide the visual language model in extracting key information from each document in the target images, obtaining the first recognition result, and outputting it in JSON format.

[0030] For example, for any customs declaration document, the first recognition result output by the visual language model can be represented in JSON format as: { "Domestic consignor / consignee": "", "Contract Agreement Number": "", "Transaction method / trade term": "", "Number of pieces": "", Gross weight (kg): "", Net weight (kg): "", "Chinese name of overseas consignor / consignee": "", "Overseas shipper's name (foreign language)": "", "Overseas consignee name (foreign language)": "", "Name of transport vehicle and voyage number": "", "Bill of Lading Number": "", "Regulatory methods / trade methods": "", "Port of Departure": "", "Qiyun Kingdom": "", "Trading Country": "", "Transit Port": "", "Types of shipping packaging": "", "Product List": [ { Item Number: 1, Product ID: "", Product Name: "", "Specifications and Models (Application Elements)":"", "Transaction (First) Quantity": "", "Transaction Quantity (First / Second) Unit":"", "unit price": "", Total Price: "", "Currency": "", "Final Destination Country Name": "", "Domestic source of goods": "", "Country of Origin": "" } ] } In addition, in order to improve the accuracy of model recognition and avoid field omissions or confusion, this embodiment of the invention uses the CogVLM40B model as the base model and performs low-rank adaptation (LORA) fine-tuning of the large language model to obtain a visual language model.

[0031] After obtaining the first recognition result through the visual language model, the optical character recognition (OCR) technology is further combined to recognize the character sequence in the target image and obtain the second recognition result.

[0032] The text value in the second recognition result is compared with the field name and field value in the first recognition result to check for character errors in the first recognition result. If any character errors are found, they are corrected to obtain the target recognition result corresponding to the customs declaration document.

[0033] For example, for any ocean bill of lading, first take a picture of the ocean bill of lading with a scanner or mobile phone to obtain the target image, which is numbered as target image_1; Then, the target image_1 and the preset prompts (such as: "Extract the following fields: bill of lading number, consignee, shipper, vessel name, voyage number...") are input into the finely tuned CogVLM40B model. After analyzing the image, the model outputs the first recognition result JSON_1, including: {"Bill of Lading Number": "COSU6348902176", "Vessel Name": "EVER GOLDEN",...}; Then, a high-precision OCR engine (such as Google Tesseract, PaddleOCR, etc.) is used to perform full-image recognition on the target image_1 to obtain the second recognition result, which is a set of all recognized text and their coordinates, OCR_List_1. Finally, the first recognition result is corrected based on the second recognition result. For example, if the model outputs "ship name" as "EVER GOIDEN", ​​but the text "EVER GOLDEN" appears frequently in OCR_List_1, and calculations show that the similarity (e.g., edit distance) between "EVER GOIDEN" and "EVER GOLDEN" is extremely high, exceeding a preset threshold (e.g., 0.9), then the system determines that the letter 'L' was incorrectly recognized as 'I' by the model. The system then replaces the model's output "EVER GOIDEN" with the correctly recognized text "EVERGOLDEN" from the OCR, generating the final target recognition result.

[0034] Visual language models possess powerful visual-linguistic semantic understanding capabilities, enabling them to comprehend document layout and contextual semantics as a whole, thereby accurately extracting and associating key fields. OCR excels at accurately recognizing character sequences in images, but is weaker in field classification and semantic understanding. This invention combines a domain-adjusted visual language model with OCR technology to form a "master-slave collaborative" verification and correction mechanism. Specifically, by fine-tuning the visual language model, a deep understanding of document semantics is ensured, fundamentally reducing the problems of incorrect or missing field extractions. By introducing OCR for precise character-level verification, subtle character errors that may occur in the visual language model (such as confusion between the number '0' and the letter 'O', or missing character readings) are effectively corrected. This achieves accurate and rapid automated recognition of structured information in customs declaration documents without relying on manual input and verification, even under unrestricted document formats.

[0035] The customs declaration document recognition method provided by this invention acquires the target image of the customs declaration document to be recognized; inputs the target image and prompt words into a visual language model to obtain a first recognition result output by the visual language model; wherein, the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model, thereby utilizing the visual semantic understanding capability of the multimodal large model after domain fine-tuning to achieve intelligent and context-related recognition of key fields of the customs declaration document; optical character recognition is performed on the target image to obtain a second recognition result, thereby achieving character-level recognition of all text content in the target image, providing a high-precision text benchmark for subsequent verification and correction of the visual language model output; based on the second recognition result, the first recognition result is corrected to obtain the target recognition result corresponding to the customs declaration document, thereby achieving an automated collaborative verification by integrating the semantic understanding advantage of the visual language model and the character precision advantage of optical character recognition, achieving a recognition effect of 1+1>2, and eliminating the need for manual input and verification, thus improving the efficiency and accuracy of customs declaration document recognition.

[0036] Further, acquiring the target image of the customs declaration document to be identified includes: Acquire the initial images of the customs declaration documents; The initial image is cropped and rotated according to preset requirements to obtain the target image.

[0037] Specifically, since the performance of visual language models is highly dependent on the quality of the input data, and the raw images often suffer from perspective distortion, tilt, shadows, and background interference, directly inputting them into the visual language model will severely reduce recognition accuracy. Proper preprocessing can improve image quality, providing optimal input for subsequent visual language models and OCR engines, thereby improving the recognition accuracy of customs declaration documents.

[0038] For example, if the scanned invoice image has a lot of black background and noise at the edges, you can first remove the useless background by cropping, and then use image enhancement algorithms (such as adaptive thresholding) to enhance the contrast between the text and the background to get a clear and clean target image.

[0039] This invention improves the accuracy and stability of subsequent visual language modeling and OCR recognition by preprocessing the initial images of customs declaration documents.

[0040] Further, the step of correcting the first identification result based on the second identification result to obtain the target identification result corresponding to the customs declaration document includes: Obtain all key-value pairs in the first recognition result; wherein, the key-value pairs are composed of field names as keys and corresponding field values ​​as values; Obtain all text values ​​and their corresponding frequencies from the second recognition result; Based on the frequency of occurrence, determine the target similarity between each field value and the target text value; Based on the target similarity, the value of each field is corrected to obtain the target recognition result corresponding to the customs declaration document.

[0041] Further, determining the target similarity between each field value and the target text value based on the frequency of occurrence includes: The target text value is defined as all text values ​​whose frequency of occurrence is greater than 0. Calculate the similarity between each field value and each target text value; The highest similarity score among all similarities is taken as the target similarity score.

[0042] Furthermore, the step of correcting each field value based on the target similarity includes: If the target similarity is greater than a preset threshold, the field value is replaced with the text value, and the frequency of occurrence of the text value is reduced by one. If the target similarity is less than or equal to the preset threshold, the field value remains unchanged.

[0043] Specifically, key-value pairs are formed by using the field names in the first recognition result as keys and the corresponding field values ​​as values, and a KV set is established with all the key-value pairs. The second recognition result output by OCR is all the text values ​​recognized from the target image. Through statistical calculation, all text values ​​and their corresponding frequencies can be obtained, thus obtaining the value-frequency set of the field.

[0044] Iterate through all field values ​​in the KV set and calculate the target similarity between each field value and the target text value in turn.

[0045] If the target similarity is greater than a preset threshold, the text value in the second recognition result is determined to be highly reliable and correct text. That is, the field value recognized by the visual language model is replaced with the text value recognized by the OCR, and the frequency of occurrence of the corresponding text value is reduced by one. If the target similarity is less than or equal to the preset threshold, the field value is kept unchanged. That is, the result obtained by the visual language model based on semantic understanding is retained first to avoid the correct field extracted by the visual language model being replaced incorrectly.

[0046] Specifically, if the frequency of any text value decreases to 0, that text value is deleted. Therefore, in calculating the similarity between each field value and the target text value, the target text value is only all text values ​​that occur more frequently than 0.

[0047] Meanwhile, since each field value will be used to calculate a similarity value with each target text value, before performing the above threshold judgment, it is necessary to determine the highest similarity value as the target similarity value for threshold judgment, and replace the field value recognized by the visual language model with a target text value corresponding to the target similarity value if the target similarity value is greater than the preset threshold.

[0048] For example, parse all key-value pairs from the first identification result JSON_1, such as K_V_Set = [("Bill of Lading Number","COSU6348902176"), ("Vessel Name", "EVER GOIDEN"), ...]; Count all text values ​​and their frequencies from the second recognition result OCR_List_1, such as Value_Freq_Set= [("EVER", 3), ("GOLDEN", ​​1), ("EVER GOLDEN", ​​5), ...]; Calculate similarity: Traverse K_V_Set. For the value V_0 = "EVER GOIDEN" corresponding to the key "ship name", traverse Value_Freq_Set and calculate its similarity with each Value (such as "EVER GOLDEN"). Calculate the similarity between "EVER GOIDEN" and all OCR texts including "EVER", "GOLDEN", ​​"EVER GOLDEN", ​​and "VOYAGE". Assuming the calculated similarity values ​​are 0.4, 0.5, 0.95, and 0.1, the maximum value of 0.95 is selected as the target similarity s.

[0049] If the preset threshold s_threshold = 0.9, then: For the "ship name" field, since the target similarity s=0.95>0.9, "EVER GOLDEN" is used to replace "EVER GOLDEN" in the visual language model output. At the same time, the frequency of "EVER GOLDEN" in Value_Freq_Set is reduced from 5 to 4. Suppose another field is "Consignee". The visual language model outputs "ABC Trading Co., Ltd.", but the OCR may only recognize fragmented text "Trading" due to the seal obscuring it. The calculated target similarity s=0.4<0.9. The system then judges it as incomplete or inaccurate OCR recognition and retains the original value output by the visual language model.

[0050] This invention improves the accuracy of the target recognition result corresponding to the customs declaration document by correcting the first recognition result based on the second recognition result. Specifically, by using frequency statistics, text that appears repeatedly (with higher credibility) in the OCR result is given priority, thus improving the accuracy of the corrected result. By setting a reasonable preset threshold for verification, the OCR recognition result can be used to correct low-level character errors that may occur in the visual language model recognition result. It also avoids incorrect corrections when the visual language model recognizes correctly but the OCR recognition is incorrect or incomplete (such as complex tables, stamp areas, or blurred text), thus preventing the expansion of errors and enhancing the robustness of the customs declaration document recognition process.

[0051] Furthermore, after correcting the first identification result based on the second identification result to obtain the target identification result corresponding to the customs declaration document, the method further includes: Obtain the field names from the target recognition results; The field code corresponding to the field name is retrieved from the standard field database; the standard field database is constructed based on preset business standards. Replace the field name with the field code.

[0052] Specifically, since the final data is customs declaration business data, while the document fields (i.e. target recognition results) are information described in natural language, the two also need to be aligned and converted into codes to meet customs business standards.

[0053] Figure 2 This is a schematic diagram illustrating an example of mapping a set of document fields to a set of business fields provided by the present invention, as shown below. Figure 2 As shown, by constructing a set of document fields and a set of business fields, and combining business semantics to perform field mapping, it is ensured that the fields identified in the customs declaration documents are correctly mapped to fields that meet customs standards.

[0054] In some embodiments, the document field set may include: customs number, domestic consignee, port of entry / exit, mode of transport, bill of lading number, production and sales unit, supervision method, and port of departure, etc.; the business field set may be the corresponding field codes, including: preentryid, tradename, ieport, trafmode, billno, ownername, trademode, and despportcode, etc.

[0055] In the business semantic mapping process, the field names in the target recognition results are first obtained, and then the field codes corresponding to the field names are searched from the standard field database. Finally, the field names are replaced with the field codes to obtain the fields that meet customs standards. The standard field database is built based on preset business standards.

[0056] Figure 3 This is a schematic diagram of the business semantic mapping process provided by the present invention, such as... Figure 3 As shown, the business field set is first sorted out and partitioned according to the business field type; then, the embedding model is used to convert the business fields of different partitions into vectors and insert them into Milvus to configure the Milvus vector database; finally, a function query is constructed to query by business type, which is used to perform semantic queries based on the input natural language to obtain the query results with the highest semantic similarity, realizing a one-to-one mapping from the document fields described in natural language (i.e., target recognition results) to fields that meet customs standards.

[0057] For example, any port name is described in natural language in customs documents, such as "Point A," but in actual business, it needs to be represented in a more professional port code form, i.e., "Point A" corresponds to "CHN752." Through the above business semantic mapping, "Point A" in the target recognition result can be automatically replaced with "CHN752," generating data that can be directly used for customs declaration filling.

[0058] This invention, through the construction of a business field set and the reasonable selection of an embedding model, transforms the target recognition results generated by intelligent recognition into standardized data that can be directly used by the business system. This avoids the manual code lookup and conversion process, realizes automated processing from customs declaration documents to customs declaration data, shortens business flow time, eliminates errors that may be introduced by manual conversion, realizes customs declaration document extraction and standardized information output, and improves data processing efficiency.

[0059] The customs document recognition device provided by the present invention is described below. The customs document recognition device described below and the customs document recognition method described above can be referred to in correspondence.

[0060] Figure 4 This is a schematic diagram of the structure of the customs declaration document recognition device provided by the present invention, as shown below. Figure 4 As shown. An embodiment of the present invention provides a customs declaration document recognition device, comprising an acquisition module 401, a first recognition module 402, a second recognition module 403, and a correction module 404, wherein: The acquisition module 401 is used to acquire the target image of the customs declaration document to be identified; the first recognition module 402 is used to input the target image and prompt words into a visual language model to obtain a first recognition result output by the visual language model; the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model; the second recognition module 403 is used to perform optical character recognition on the target image to obtain a second recognition result; the correction module 404 is used to correct the first recognition result based on the second recognition result to obtain the target recognition result corresponding to the customs declaration document.

[0061] The customs declaration document recognition device provided by this invention acquires a target image of the customs declaration document to be recognized; inputs the target image and prompt words into a visual language model to obtain a first recognition result output by the visual language model; wherein, the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model, thereby utilizing the visual semantic understanding capability of the multimodal large model after domain fine-tuning to achieve intelligent and context-related recognition of key fields of the customs declaration document; performs optical character recognition on the target image to obtain a second recognition result, thereby achieving character-level recognition of all text content in the target image, providing a high-precision text benchmark for subsequent verification and correction of the visual language model output; based on the second recognition result, the first recognition result is corrected to obtain the target recognition result corresponding to the customs declaration document, thereby achieving an automated collaborative verification by integrating the semantic understanding advantage of the visual language model and the character precision advantage of optical character recognition, achieving a recognition effect of 1+1>2, and eliminating the need for manual input and verification, thus improving the efficiency and accuracy of customs declaration document recognition.

[0062] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a customs declaration document identification method, which includes: Obtain the target image of the customs declaration document to be identified; The target image and prompt words are input into the visual language model to obtain the first recognition result output by the visual language model; the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model. Optical character recognition is performed on the target image to obtain a second recognition result; Based on the second identification result, the first identification result is corrected to obtain the target identification result corresponding to the customs declaration document.

[0063] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0064] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the customs declaration document identification method provided by the above methods, the method comprising: Obtain the target image of the customs declaration document to be identified; The target image and prompt words are input into the visual language model to obtain the first recognition result output by the visual language model; the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model. Optical character recognition is performed on the target image to obtain a second recognition result; Based on the second identification result, the first identification result is corrected to obtain the target identification result corresponding to the customs declaration document.

[0065] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the customs declaration document identification method provided by the above methods, the method comprising: Obtain the target image of the customs declaration document to be identified; The target image and prompt words are input into the visual language model to obtain the first recognition result output by the visual language model; the prompt words are generated based on preset business requirements; the visual language model is obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model. Optical character recognition is performed on the target image to obtain a second recognition result; Based on the second identification result, the first identification result is corrected to obtain the target identification result corresponding to the customs declaration document.

[0066] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0067] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0068] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0069] It should also be noted that the terms "target," "first," and "second" in this application are used to distinguish similar objects, and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, and the number of objects is not limited; for example, the first object can be one or more.

[0070] In this application's embodiments, "determine B based on A" means that factor A must be considered when determining B. It is not limited to "B can be determined based solely on A," but should also include: "determine B based on A and C," "determine B based on A, C, and E," "determine C based on A, and further determine B based on C," etc. Additionally, it can include using A as a condition for determining B, for example, "when A meets the first condition, determine B using the first method"; another example, "when A meets the second condition, determine B," etc.; another example, "when A meets the third condition, determine B based on the first parameter," etc. Of course, it can also be a condition where A is a factor in determining B, for example, "when A meets the first condition, determine C using the first method, and further determine B based on C," etc.

[0071] In this invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying customs declaration documents, characterized in that, include: Obtain the target image of the customs declaration document to be identified; The target image and the prompt words are input into a visual language model to obtain a first recognition result output by the visual language model; the prompt words are generated based on preset business requirements. The visual language model was obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model. Optical character recognition is performed on the target image to obtain a second recognition result; Based on the second identification result, the first identification result is corrected to obtain the target identification result corresponding to the customs declaration document.

2. The customs declaration document identification method according to claim 1, characterized in that, The step of correcting the first identification result based on the second identification result to obtain the target identification result corresponding to the customs declaration document includes: Obtain all key-value pairs in the first recognition result; wherein, the key-value pairs are composed of field names as keys and corresponding field values ​​as values; Obtain all text values ​​and their corresponding frequencies from the second recognition result; Based on the frequency of occurrence, determine the target similarity between each field value and the target text value; Based on the target similarity, the value of each field is corrected to obtain the target recognition result corresponding to the customs declaration document.

3. The customs declaration document identification method according to claim 2, characterized in that, The step of determining the target similarity between each field value and the target text value based on the frequency of occurrence includes: The target text value is defined as all text values ​​whose frequency of occurrence is greater than 0. Calculate the similarity between each field value and each target text value; The highest similarity score among all similarities is taken as the target similarity score.

4. The customs declaration document identification method according to claim 3, characterized in that, The step of correcting each field value based on the target similarity includes: If the target similarity is greater than a preset threshold, the field value is replaced with the text value, and the frequency of occurrence of the text value is reduced by one. If the target similarity is less than or equal to the preset threshold, the field value remains unchanged.

5. The customs declaration document identification method according to claim 1, characterized in that, The acquisition of the target image of the customs declaration document to be identified includes: Acquire the initial images of the customs declaration documents; The initial image is cropped and rotated according to preset requirements to obtain the target image.

6. The customs declaration document identification method according to claim 1, characterized in that, After correcting the first identification result based on the second identification result to obtain the target identification result corresponding to the customs declaration document, the method further includes: Obtain the field names from the target recognition results; The field code corresponding to the field name is retrieved from the standard field database; the standard field database is constructed based on preset business standards. Replace the field name with the field code.

7. A customs declaration document identification device, characterized in that, include: The acquisition module is used to acquire the target image of the customs declaration document to be identified; The first recognition module is used to input the target image and the prompt words into a visual language model to obtain a first recognition result output by the visual language model; the prompt words are generated based on preset business requirements. The visual language model was obtained by performing low-rank adaptive fine-tuning of the CogVLM40B model as a large language model. The second recognition module is used to perform optical character recognition on the target image to obtain a second recognition result; The correction module is used to correct the first identification result based on the second identification result to obtain the target identification result corresponding to the customs declaration document.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the customs declaration document recognition method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the customs declaration document identification method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the customs declaration document identification method as described in any one of claims 1 to 6.

Images