Document intelligent identification method and device, equipment and storage medium

Abstract

This invention discloses a document intelligent recognition method, apparatus, device, and storage medium. The method includes: acquiring a document to be recognized and determining its document type; when the document type is a single-page certificate, performing visual enhancement processing on the document to be recognized to determine the region of interest; standardizing the region of interest to obtain a standardized image; and inputting the standardized image into a multimodal large language model for structured information extraction to obtain the recognition result of the document to be recognized. Because this invention achieves differentiated processing through document type discrimination, and performs visual enhancement and standardization processing on single-page certificate documents first, it effectively suppresses the interference of shooting distortion and background noise on the multimodal large language model. Compared with existing technologies, this invention improves the recognition accuracy of certificate fields in complex shooting scenarios.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a document intelligent recognition method, apparatus, device, and storage medium. Background Technology

[0002] In business scenarios such as license and permit verification and cooperation agreement archiving on business management platforms, automated information recognition of unstructured documents is a key step in improving business workflow efficiency. Currently, intelligent recognition technology for this type of document mainly relies on two solutions: traditional optical character recognition (OCR) engines and multimodal large language models (VLM).

[0003] Traditional OCR solutions extract text through character segmentation and template matching, but they are highly sensitive to the quality of the input image. In real-world applications, user-uploaded certificates and documents often suffer from issues such as shifted shooting perspectives, complex background interference, watermark occlusion, and varying text scales. These factors make it difficult for OCR engines to accurately segment and recognize characters, resulting in a significant drop in field-level accuracy. Furthermore, traditional OCR only outputs discrete text sequences and cannot effectively model the structured relationships between semantic fields in certificates and documents.

[0004] While multimodal large language model (VLM) solutions possess strong visual semantic understanding capabilities, they also face bottlenecks in complex shooting scenarios. Because VLMs are sensitive to geometric distortions and background noise in the input image, directly performing visual inference on the original captured image can easily lead to the loss of small target text and recognition errors. Furthermore, VLM's inference mechanism is designed for overall semantic understanding, lacking explicit attention to key information regions of documents and certificates. When the background is cluttered or the target area is too small, the model's attention is easily distracted by irrelevant areas, further reducing the reliability of key field recognition.

[0005] Therefore, there is an urgent need for an intelligent document recognition method that can improve the recognition accuracy of certificate fields in complex shooting scenarios. Summary of the Invention

[0006] The main objective of this invention is to provide a document intelligent recognition method, apparatus, device, and storage medium, aiming to solve the technical problem of insufficient recognition accuracy of certificate fields under complex shooting conditions in the prior art.

[0007] To achieve the above objectives, the present invention provides a document intelligent recognition method, the method comprising the following steps: Obtain the document to be identified and determine the document type of the document to be identified; When the document type is a single-page certificate, the document to be identified is subjected to visual enhancement processing to determine the region of interest; The region of interest is standardized to obtain a standardized image; The standardized image is input into a multimodal large language model for structured information extraction to obtain the recognition result of the document to be recognized.

[0008] Optionally, the step of performing visual enhancement processing on the document to be identified to determine the region of interest includes: The document to be identified is converted to grayscale to obtain a grayscale image, and the gradient energy field of the document to be identified is constructed based on the grayscale image. The gradient energy field is smoothed to obtain a smooth energy distribution map; Iterative boundary optimization is performed based on the smooth energy distribution map to determine the initial region of interest of the document to be identified; The initial region of interest is expanded by a preset ratio to obtain the region of interest.

[0009] Optionally, the step of standardizing the region of interest to obtain a standardized image includes: Obtain the length of the long side of the region of interest, and construct a square canvas with the length of the long side as the side length; The square canvas is filled with a preset background color, and the region of interest is placed in the center within the square canvas. The square canvas is scaled proportionally to a preset target size to obtain a standardized image.

[0010] Optionally, after the steps of obtaining the document to be identified and determining the document type of the document to be identified, the method further includes: When the document type is a multi-page contract, semantic value filtering is performed on each page of the document to be identified to obtain a candidate page set; Each candidate page in the candidate page set is input into a multimodal large language model for structured information extraction to obtain the single-page structured recognition result of each candidate page; The single-page structured recognition results are aggregated across pages to obtain the recognition results of the document to be recognized.

[0011] Optionally, the step of performing semantic value filtering on each page of the document to be identified to obtain a candidate page set includes: The document to be identified is subjected to non-uniform sampling to obtain a set of sampled pages; Multidimensional semantic value scoring is performed on each page in the sampled page set to obtain the semantic value score of each page; Based on the semantic value score of each page, a candidate page set is selected from the sampled page set.

[0012] Optionally, the step of performing cross-page aggregation processing on each of the single-page structured recognition results to obtain the recognition result of the document to be recognized includes: Obtain the single-page structured recognition result for each candidate page, wherein the single-page structured recognition result includes the extracted values ​​and confidence scores corresponding to the fields; For the same target field, cluster the same extracted values ​​to obtain at least one extracted value cluster; The comprehensive score of each extracted value cluster is determined based on the sum of the confidence scores corresponding to each extracted value in each extracted value cluster and the frequency of occurrence of the extracted value. The extraction value corresponding to the cluster with the highest comprehensive score is selected as the final recognition result of the target field in the document to be recognized.

[0013] Optionally, the step of performing multi-dimensional semantic value scoring on each page in the sampled page set to obtain the semantic value score of each page includes: Based on a preset set of semantic anchor points, keyword matching is performed on each page in the sampled page set to obtain the keyword semantic score of each page; The positional prior weight of each page is determined based on its page number position in the document to be identified. Optical character recognition is performed on each of the pages, and the recognition quality coefficient and text density features of each page are determined based on the optical character recognition results; The semantic value score of each page is obtained by weighted fusion of the keyword semantic score, the location prior weight, the recognition quality coefficient and the text density feature.

[0014] Furthermore, to achieve the above objectives, the present invention also proposes a document intelligent recognition device, the device comprising: A type determination module is used to acquire the document to be identified and determine the document type of the document to be identified; The visual enhancement module is used to perform visual enhancement processing on the document to be identified and determine the region of interest when the document type is a single-page certificate. The image processing module is used to perform standardization processing on the region of interest to obtain a standardized image; The result output module is used to input the standardized image into a multimodal large language model for structured information extraction to obtain the recognition result of the document to be recognized.

[0015] Furthermore, to achieve the above objectives, the present invention also proposes a document intelligent recognition device, the device comprising: a memory, a processor, and a document intelligent recognition program stored in the memory and executable on the processor, the document intelligent recognition program being configured to implement the steps of the document intelligent recognition method as described above.

[0016] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a document intelligent recognition program, which, when executed by a processor, implements the steps of the document intelligent recognition method as described above.

[0017] This invention discloses a method for acquiring a document to be identified and determining its document type. When the document type is a single-page certificate, visual enhancement processing is performed on the document to be identified to determine the region of interest. The region of interest is then standardized to obtain a standardized image. The standardized image is input into a multimodal large language model for structured information extraction to obtain the recognition result of the document. Because this invention achieves differentiated processing through document type discrimination, and performs visual enhancement and standardization processing on single-page certificate documents first, it effectively suppresses the interference of shooting distortion and background noise on the multimodal large language model. Compared with existing technologies, this invention improves the recognition accuracy of certificate fields in complex shooting scenarios. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the document intelligent recognition method of the present invention; Figure 2 This is a flowchart illustrating the second embodiment of the document intelligent recognition method of the present invention; Figure 3 This is a flowchart illustrating the third embodiment of the document intelligent recognition method of the present invention; Figure 4 This is a structural block diagram of the first embodiment of the document intelligent recognition device of the present invention; Figure 5 This is a schematic diagram of the structure of a document intelligent recognition device in the hardware operating environment involved in the embodiments of the present invention.

[0019] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0021] This invention provides a document intelligent recognition method, referring to... Figure 1 , Figure 1This is a flowchart illustrating the first embodiment of the document intelligent recognition method of the present invention.

[0022] In this embodiment, the document intelligent recognition method includes steps S10 to S40: Step S10: Obtain the document to be identified and determine the document type of the document to be identified.

[0023] It should be noted that the executing entity in this embodiment can be a computer server device with data processing, network communication, and program execution functions applied in image recognition scenarios, such as a server, tablet computer, or personal computer, or an electronic device capable of performing the above functions (such as a document intelligent recognition device). The following uses a system including a document intelligent recognition device (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.

[0024] It should be understood that the aforementioned document to be identified may refer to an electronic document that requires information identification and extraction, such as a certificate image file or a multi-page contract PDF file.

[0025] It should be noted that the document type mentioned above refers to the category to which the document to be identified belongs, typically including single-page certificates and licenses and multi-page contracts. Single-page certificates and licenses can include images of single-page certificates and licenses such as business licenses, ID cards, and bank account opening permits, while multi-page contracts and licenses can include PDF files containing multiple pages of content, such as cooperation agreements and lease contracts.

[0026] Understandably, document types can be determined based on file extensions, header information, or page count. For example, by detecting the page count of a PDF file, if the page count is 1 and it is in image format, it is determined to be a single-page certificate; if the page count is greater than 1, it is determined to be a multi-page contract.

[0027] In its implementation, the system first acquires the document to be identified during intelligent document recognition. This acquisition can be achieved through various methods, such as receiving user-uploaded files, retrieving documents from a database, or receiving transmitted document data via a network interface. After successfully acquiring the document, the system further determines its document type. Specifically, the system performs type discrimination on the acquired document. This discrimination process can involve a preliminary judgment based on the file extension, combined with a comprehensive assessment of the document's actual content characteristics. For example, for files with the .pdf extension, the system further parses the page number information; for files with the .jpg or .png extensions, the system classifies them as single-page image documents. After this discrimination process, the system outputs the document type determination result, enabling subsequent triggering of corresponding differentiated processing paths based on the document type.

[0028] To facilitate understanding, the following examples illustrate the concepts, but do not impose specific limitations on this embodiment. For instance, suppose in an enterprise management platform application scenario, a user uploads a scanned copy of a business license through the front-end interface, with the filename "business license.jpg". The system obtains this file as the document to be identified, and then determines the document type as a single-page certificate based on the file extension ".jpg" and the image format information in the file header. As another example, if a user uploads a 20-page cooperation agreement PDF file, the system obtains the PDF file and, by parsing the page number information, determines that it contains multiple pages, thus identifying the document type as a multi-page contract.

[0029] Step S20: When the document type is a single-page certificate, perform visual enhancement processing on the document to be identified to determine the region of interest.

[0030] It should be noted that the aforementioned visual enhancement processing refers to a series of image preprocessing operations performed on single-page certificate documents. Its purpose is to suppress background noise in the original image, compensate for distortion caused by shooting angle shift, and highlight key structural information such as text, table lines, and seals in the certificate, thereby improving the accuracy of the subsequent recognition process.

[0031] It should be explained that the aforementioned region of interest can refer to a local area containing the main document or certificate content extracted from the image of the document to be recognized after visual enhancement processing. For example, the main document area in a business license image after removing the surrounding background, or the card area in an ID card image after removing distracting objects such as desktop surfaces and fingers. Regions of interest are typically represented by a rectangular bounding box, which has a definite coordinate position and size in the original image coordinate system.

[0032] In its implementation, after determining that the document to be identified is a single-page certificate, the system triggers a processing path specifically for single-page certificates. The system performs visual enhancement processing on the document. Specifically, it analyzes the acquired single-page certificate image, identifies densely textual regions and structural edges, and based on the recognition results, crops a region of interest (ROI) containing the core content of the certificate from the original image. This visual enhancement processing eliminates irrelevant background interference in the original image, improves image contrast, and preserves the key text structure of the certificate. After completing the visual enhancement processing, the system outputs the identified ROI, which serves as the input image data for subsequent standardization processing steps. By separating the ROI from the original image, the system ensures that subsequent processing steps operate only on this local area, thereby reducing the interference of irrelevant background on the recognition results.

[0033] It should be noted that the step of performing visual enhancement processing on the document to be identified and determining the region of interest may include: performing grayscale processing on the document to be identified to obtain a grayscale image, and constructing a gradient energy field of the document to be identified based on the grayscale image; performing smoothing processing on the gradient energy field to obtain a smooth energy distribution map; performing iterative boundary optimization based on the smooth energy distribution map to determine the initial region of interest of the document to be identified; and expanding the initial region of interest by a preset ratio to obtain the region of interest.

[0034] Understandably, the aforementioned grayscale processing can refer to the operation of converting a color image into a grayscale image. Specifically, a weighted average method or a luminance method can be used to merge the pixel values ​​of the red, green, and blue channels into a single luminance value, thereby reducing the amount of data required for subsequent calculations.

[0035] It should be explained that the gradient energy field mentioned above can refer to a two-dimensional scalar field used to characterize the edge intensity or structural saliency at each pixel in an image. The gradient energy value of each pixel reflects the degree of grayscale change at that point. High-frequency structures such as text strokes, table lines, and stamp edges correspond to higher gradient energy values, while smooth background areas correspond to lower gradient energy values.

[0036] It should be noted that the above smoothing process can refer to the operation of local neighborhood averaging of the gradient energy field, which is used to suppress energy spikes caused by image noise and fill energy gaps at the breaks in the strokes of text, making the energy distribution more continuous and stable.

[0037] Accordingly, the smoothed energy distribution map mentioned above can refer to the gradient energy distribution map obtained after smoothing.

[0038] It should be explained that the above iterative boundary optimization can refer to a loop optimization process that approximates the true boundary of the region of interest by gradually shrinking the candidate boundary. Each iteration determines whether to continue shrinking based on the energy distribution characteristics of the region inside the current boundary.

[0039] Accordingly, the initial region of interest mentioned above can refer to a temporary local region obtained after iterative boundary optimization.

[0040] It is understandable that the aforementioned expansion of the preset ratio may refer to the operation of expanding the four sides of the initial region of interest outward by a certain ratio, in order to prevent the edge text from being truncated.

[0041] It should be understood that by constructing grayscale and gradient energy fields, the structural features of the text in the certificate are characterized from the perspective of information density; noise is suppressed and stroke breaks are repaired through smoothing; iterative boundary optimization achieves adaptive and accurate cropping of key semantic regions; and safe expansion avoids the truncation of edge information, effectively improving the recognition accuracy of certificate documents under complex shooting conditions.

[0042] In its implementation, the system first performs grayscale processing on the document to be recognized when performing visual enhancement processing. The system reads the color image data of the document, and then weights and sums the brightness values ​​of the red, green, and blue channels of each pixel according to preset weights to obtain the grayscale value of the corresponding pixel, thereby generating a grayscale image.

[0043] Next, the system constructs a gradient energy field based on the grayscale image. Specifically, the system uses the Sobel operator to calculate the gradient value in the horizontal and vertical directions for each pixel in the grayscale image, then adds the absolute values ​​of the horizontal and vertical gradients, and uses the sum as the gradient energy value of that pixel. The system iterates through all pixels of the grayscale image to obtain the complete gradient energy field.

[0044] Subsequently, the system smooths the gradient energy field. Specifically, the system uses a 3×3 mean convolution kernel to perform convolution operations on the gradient energy field, replacing the gradient energy value of each pixel with the arithmetic mean of its own gradient energy value and the gradient energy values ​​of its eight neighboring pixels, thereby obtaining a smooth energy distribution map.

[0045] Subsequently, the system performs iterative boundary optimization based on a smoothed energy distribution map. Using the entire image boundary as the initial candidate box, the system evaluates the energy level of the regions inside each of the four edges of the candidate box in each iteration. If the energy level inside a certain edge is consistently significantly lower than the energy level of the regions inside the candidate box, that edge is shrunk inward by a preset step size. The system repeats this boundary shrinkage process until all four edges meet the cessation shrinkage condition, at which point the current candidate box is determined as the initial region of interest.

[0046] Finally, the system expands the initial region of interest by a preset ratio. The system extends each edge of the initial region of interest outwards by a preset ratio, and outputs the expanded region as the final region of interest.

[0047] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, suppose the system acquires a color photograph of a business license taken against a complex background. The system first converts the color photograph into a grayscale image. Then, the system uses the Sobel operator to calculate the gradient energy value of each pixel in the grayscale image, constructing a gradient energy field. In this energy field, the text and border areas of the business license exhibit high energy values, while the desktop texture and shadow areas exhibit low energy values. Next, the system uses a 3×3 mean convolution kernel to smooth the gradient energy field, filling in the tiny energy breaks in the strokes of the text caused by uneven lighting. Afterward, starting from the overall boundary of the photograph, the system iteratively evaluates the inner energy levels of the four edges. It finds that the inner energy of the left and top edges is consistently lower than that of the inner region. Therefore, the system gradually shrinks the left edge to the right and the top edge downward until the inner energy of the edges reaches more than 60% of the energy of the inner region, at which point the shrinkage stops, obtaining the initial region of interest. Finally, the system extends each of the four edges of the initial region of interest outward by 3% to obtain the final region of interest, which exactly covers all the edge text of the business license.

[0048] Step S30: Standardize the region of interest to obtain a standardized image.

[0049] Understandably, the above standardization process can refer to the operation of size normalization and geometric correction of the region of interest. Its purpose is to convert regions of interest with different original sizes and aspect ratios into standardized images with the same specifications, so as to meet the requirements of multimodal large language models for the consistency of input image size.

[0050] Accordingly, the aforementioned standardized image can refer to image data with uniform size specifications that has been output after standardization processing.

[0051] To improve the reliability of structured information extraction, the step of standardizing the region of interest to obtain a standardized image may include: obtaining the long side dimension of the region of interest and constructing a square canvas with the long side dimension as the side length; filling the square canvas with a preset background color and placing the region of interest in the center within the square canvas; scaling the square canvas proportionally according to a preset target size to obtain a standardized image.

[0052] Understandably, the aforementioned longer side dimension can refer to the larger of the width and height values ​​of the region of interest. For example, if the width of the region of interest is 800 pixels and the height is 600 pixels, then the longer side dimension is 800 pixels.

[0053] The aforementioned square canvas can refer to a blank image area with equal width and height constructed with the longer side as the side length.

[0054] The aforementioned preset background color can refer to the pixel color value set in advance to fill the square canvas. This color value usually adopts a light color that is significantly different from the background color of the certificate, such as light gray (RGB value 240, 240, 240) or white (RGB value 255, 255, 255), so as to visually distinguish the filled area from the main content of the certificate.

[0055] It should be noted that the above-mentioned centering can refer to aligning the geometric center of the region of interest with the geometric center of the square canvas, so that the region of interest is centered in both the horizontal and vertical directions of the square canvas.

[0056] Understandably, the aforementioned preset target size can refer to a pre-defined target side length pixel value for a standardized image, such as 600 pixels, 800 pixels, or 1024 pixels. This size is used to unify the input specifications of all standardized images. The aforementioned proportional scaling can refer to the operation of simultaneously scaling the width and height of a square canvas at the same ratio. This operation maintains the aspect ratio of all content within the square canvas unchanged, without causing stretching or compression distortion.

[0057] In its implementation, after obtaining the region of interest (ROI), the system first acquires the width and height values ​​of the ROI and compares them, selecting the larger value as the longer side dimension. Next, the system uses this longer side dimension as the side length of a square, constructing a blank square canvas. Then, the system fills all pixels of the square canvas with a preset background color, ensuring the entire canvas is filled with the preset background color. After filling, the system places the ROI inside the square canvas, aligning its geometric center point with the geometric center point of the square canvas, ensuring equal horizontal and vertical margins relative to the left and right sides of the square canvas. Following this centering operation, the system scales the square canvas proportionally to a preset target size. The system calculates the scaling ratio between the preset target size and the current square canvas side length, and then adjusts both the width and height of the square canvas according to this scaling ratio, ensuring the side length of the square canvas equals the preset target size. Because the square canvas itself is square and proportional scaling is used, the scaled normalized image retains its square dimensions, and the content of the region of interest within it does not undergo geometric distortion. The system outputs the normalized image obtained after proportional scaling.

[0058] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, assume that the system obtains a region of interest with a width of 1200 pixels and a height of 900 pixels. The system obtains the width and height values ​​of the region of interest, compares them, and determines that the longer side is 1200 pixels. The system constructs a 1200×1200 pixel square canvas with a side length of 1200 pixels, and fills all pixels of the square canvas with a light gray RGB value of (240, 240, 240). Subsequently, the system aligns the geometric center of the region of interest with the geometric center of the square canvas, placing the region of interest centered within the square canvas. At this point, the region of interest has a gray-filled area of ​​150 pixels wide on each side and a gray-filled area of ​​300 pixels high on each side. Next, the system scales the image proportionally to a preset target size of 600 pixels, calculating a scaling factor of 600 / 1200 = 0.5. The width and height of the square canvas are then multiplied by 0.5 to obtain a standardized image of 600×600 pixels. In this standardized image, the content of the original region of interest maintains a 3:4 aspect ratio, without any stretching or distortion.

[0059] It should be understood that by constructing a square canvas based on the long side and placing the region of interest in the center, and scaling the model input to a uniform scale, geometric distortion caused by non-uniform stretching is avoided, effectively maintaining the stability of the physical form of the text, ensuring that the multimodal large language model obtains a standardized and distortion-free input image, and improving the reliability of structured information extraction.

[0060] Step S40: Input the standardized image into a multimodal large language model for structured information extraction to obtain the recognition result of the document to be recognized.

[0061] It should be noted that the aforementioned multimodal large language model can refer to a large neural network model that simultaneously possesses visual understanding and natural language processing capabilities. Such models can take images as input, extract visual features from the images through a visual encoder, and then convert the visual features into structured text output through a language decoder. For example, the Qwen3-VL model, the GPT-4V model, or similar multimodal large language models with visual understanding capabilities can be used.

[0062] It should be understood that the aforementioned structured information extraction refers to the process of extracting specific content corresponding to preset business fields from document images and outputting the extraction results according to a preset data organization format. The extracted fields may include key business information such as the company name, unified social credit code, legal representative's name, registered capital, and establishment date from certificates and licenses.

[0063] It should be noted that the above recognition results may refer to structured data that contains the correspondence between field names and field values ​​after structured information extraction, such as a collection of key-value pairs organized in JSON format.

[0064] In its implementation, standardized images are input into a multimodal large language model. The system calls the pre-deployed inference interface of the multimodal large language model, passing the standardized image data and preset structured extraction instructions to the model. Upon receiving the standardized image, the multimodal large language model performs visual encoding processing to extract visual semantic features. Then, based on the extracted visual features, it performs cross-modal inference to identify the text content corresponding to each business field contained in the image. According to preset field extraction rules, the multimodal large language model organizes the identified text content into structured data based on the correspondence between field names and field values. For example, it extracts and combines the values ​​of the "Name," "Unified Social Credit Code," and "Legal Representative" fields from a business license. Subsequently, the multimodal large language model outputs the recognition result in structured data form. The system receives the recognition result output by the multimodal large language model, which contains the extracted values ​​of each preset business field in the document to be recognized. The system uses this recognition result as the final structured business data for a single-page certificate document.

[0065] For ease of understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, assume that the above system obtains a standardized image of a business license, with a size of 600×600 pixels. The system inputs this standardized image into the Qwen3-VL multimodal large language model, along with instructions to extract fields from the business license. After analyzing the image, the multimodal large language model outputs the recognition result in the following JSON format: {"Company Name": "XX Technology Co., Ltd.", "Unified Social Credit Code": "91440101MA5XXXXXXX", "Legal Representative": "Zhang San", "Registered Capital": "RMB 10 million", "Date of Establishment": "January 1, 2020"}. The system receives this recognition result and uses it as the final recognition result for the business license document.

[0066] This embodiment discloses a process for acquiring a document to be identified and determining its document type. When the document type is a single-page certificate, visual enhancement processing is performed on the document to be identified to determine the region of interest. The region of interest is then standardized to obtain a standardized image. The standardized image is input into a multimodal large language model for structured information extraction to obtain the recognition result of the document. Because this embodiment achieves differentiated processing through document type discrimination, and performs visual enhancement and standardization processing on single-page certificate documents first, it effectively suppresses the interference of shooting distortion and background noise on the multimodal large language model. Compared with existing technologies, this embodiment improves the recognition accuracy of certificate fields in complex shooting scenarios.

[0067] refer to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the document intelligent recognition method of the present invention.

[0068] Based on the first embodiment described above, in this embodiment, after step S10, steps S101 to S103 are further included: Step S101: When the document type is a multi-page contract, perform semantic value filtering on each page of the document to be identified to obtain a candidate page set.

[0069] Step S102: Input each candidate page in the candidate page set into the multimodal large language model for structured information extraction to obtain the single-page structured recognition result of each candidate page.

[0070] Step S103: Perform cross-page aggregation processing on each of the single-page structured recognition results to obtain the recognition result of the document to be recognized.

[0071] It should be explained that the aforementioned semantic value screening refers to the process of assessing and filtering the value of pages from multiple pages of a multi-page contract document based on the richness and criticality of the business information contained on each page. For example, pages containing information about the contracting parties, bank account numbers, monetary terms, or signatures have higher semantic value, while pages containing only a table of contents, copyright notices, or repetitive standard clauses have relatively lower semantic value.

[0072] Furthermore, the aforementioned candidate page set can refer to the set of pages selected from all pages of the document to be identified after semantic value filtering for subsequent multimodal large language model in-depth reading.

[0073] It should be noted that the above-mentioned single-page structured recognition result can refer to the structured data output by the multimodal large language model after a single candidate page from the candidate page set is input into the single page. This data contains the correspondence between field names and field values.

[0074] It should be explained that the above-mentioned cross-page aggregation processing can refer to the operation process of merging and integrating the single-page structured recognition results obtained from multiple pages. This process is used to solve the problem of inconsistent or conflicting extraction results of the same field between different pages, and to obtain a unified final recognition result by integrating information from multiple pages.

[0075] In its implementation, after determining that the document to be identified is a multi-page contract, the system triggers a processing path for multi-page contract documents. First, the system performs semantic value filtering on each page of the document. Specifically, it analyzes all pages of the document, evaluates the business information value of each page relative to a preset set of business fields, and filters out pages carrying core key information based on the evaluation results, forming a candidate page set. Then, the system inputs each candidate page into a multimodal large language model for structured information extraction. For each candidate page, the system separately calls the inference interface of the multimodal large language model, inputting the image data of each candidate page into the model to obtain the corresponding single-page structured recognition result. Finally, the system obtains the single-page structured recognition results for all candidate pages and performs cross-page aggregation processing on all obtained single-page structured recognition results. The system comprehensively analyzes multiple extraction results of the same target field from different candidate pages, eliminates the illusions or biases that may be introduced by single-page reasoning by fusing information from multiple pages, and resolves conflicting field results, ultimately forming a consistent and complete structured recognition result for the entire document to be recognized.

[0076] To improve the efficiency of intelligent document recognition, the step of performing semantic value filtering on each page of the document to be recognized to obtain a candidate page set may include: performing non-uniform sampling on the document to be recognized to obtain a sampled page set; performing multi-dimensional semantic value scoring on each page in the sampled page set to obtain a semantic value score for each page; and filtering out a candidate page set from the sampled page set based on the semantic value scores of each page.

[0077] Understandably, the aforementioned non-uniform sampling can refer to a sampling strategy that selects a subset of pages from all pages of a multi-page contract document at non-equidistant intervals for preliminary analysis. Unlike uniform sampling, non-uniform sampling prioritizes specific pages with high information value within the document, such as the first page, the last page, and equidistant pages in the middle. Its purpose is to cover the main areas of the document that may contain critical business information with fewer sampled pages.

[0078] Furthermore, the aforementioned sampled page set can refer to a set of pages selected from all pages of the document to be identified for subsequent semantic value scoring after non-uniform sampling.

[0079] It should be explained that the aforementioned multidimensional semantic value scoring refers to the process of quantitatively evaluating the business information value of a page from multiple dimensions. These dimensions may include the keywords contained on the page, the page's position within the document, and the quality of text recognition. The semantic value score mentioned above refers to the numerical score calculated for each page after multidimensional semantic value scoring, representing the richness and importance of the page's business information.

[0080] In its implementation, after determining that the document to be identified is a multi-page contract, the system first performs non-uniform sampling on the document. Specifically, the system obtains the total number of pages in the document and then selects pages from all pages according to a preset non-uniform sampling rule. The system prioritizes selecting the first and last pages of the document, for example, the first three pages and the last two pages. For the middle section, the system calculates a sampling step size based on the total number of pages and selects several pages evenly spaced from the middle section according to the calculated step size. The system then forms a sampled page set from all pages selected according to the above rules. Subsequently, the system performs multi-dimensional semantic value scoring on each page in the sampled page set. Specifically, for each page in the sampled page set, the system calculates a sub-score corresponding to that page from multiple different evaluation dimensions, and then weights and merges the sub-scores of each dimension according to preset weights to calculate the semantic value score of each page. The system iterates through all pages in the sampled page set to obtain the semantic value score corresponding to each page. Finally, based on the semantic value scores of each page, the system selects a candidate page set from the sampled page set. Specifically, the system sorts the pages in the sampled page set according to their semantic value scores from high to low, and selects a preset number of the top-ranked pages to form a candidate page set.

[0081] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, suppose the system obtains a PDF file containing a cooperation agreement with 23 pages. The system performs non-uniform sampling, first selecting pages 1, 2, and 3 as the initial sampling pages, and pages 22 and 23 as the final sampling pages. Then, it calculates the sampling step size for the middle section as (23-5) / 3=6, and selects pages 10 and 16 from pages 4 to 21 as intermediate sampling pages according to the step size of 6. Thus, the system obtains a set of sampling pages consisting of pages 1, 2, 3, 10, 16, 22, and 23. Subsequently, the system performs multi-dimensional semantic value scoring on these 7 sampling pages, calculating the semantic value score for each page. Assume page 1 scores 92 points, page 3 scores 88 points, page 16 scores 75 points, and the scores of other pages are all below 60 points. The system selects the highest-scoring pages (page 1, page 3, and page 16) to form a candidate page set, based on a preset number of 3 candidate pages.

[0082] It should be understood that by prioritizing the coverage of key pages at the beginning and end and sampling middle pages at equal intervals through non-uniform sampling, the overhead of page traversal is reduced. Combined with multi-dimensional semantic value scoring to quantify the business value of each page, the processing of long documents is transformed from full scanning to value-driven screening. This significantly reduces invalid visual reasoning calculations while ensuring the recall rate of key information, thereby improving processing efficiency.

[0083] To improve the accuracy and key information recall rate of candidate page set screening, the step of performing multi-dimensional semantic value scoring on each page in the sampled page set to obtain the semantic value score of each page may include: performing keyword matching on each page in the sampled page set based on a preset semantic anchor point set to obtain the keyword semantic score of each page; determining the positional prior weight of each page according to the page number position of each page in the document to be identified; performing optical character recognition on each page, and determining the recognition quality coefficient and text density feature of each page based on the optical character recognition results; and weightedly fusing the keyword semantic score, the positional prior weight, the recognition quality coefficient, and the text density feature to obtain the semantic value score of each page.

[0084] It should be explained that the aforementioned set of preset semantic anchors can refer to a pre-defined set of keywords or key phrases that have high informational value in business documents. These semantic anchors typically correspond to the core identifiers of the business fields to be extracted. For example, in contract documents, the set of preset semantic anchors may include words with clear business meanings such as "Party B," "Party A," "Account Number," "Bank," "Account Name," "Contract Amount," and "Signing Date."

[0085] The keyword matching mentioned above refers to the operation of comparing and searching the text content on the page with each word in the preset semantic anchor set, in order to detect whether the page contains and which semantic anchors it contains.

[0086] Understandably, the aforementioned keyword semantic score can refer to a numerical score calculated based on the number or hit rate of semantic anchors on the page, used to reflect the richness of information on the page in terms of keywords.

[0087] The aforementioned prior weights can refer to the weight values ​​pre-assigned based on the page number position of a page in a document, used to reflect the differences in the prior probability of pages in different positions carrying key information. For example, the first page of a document usually contains information about the contracting entity, and the last page of a document usually contains signature information; these positions have higher prior weights.

[0088] It should be noted that the aforementioned optical character recognition can refer to the process of detecting text regions in an image and converting them into machine-readable text characters, often simply referred to as OCR.

[0089] Accordingly, the aforementioned recognition quality coefficient can refer to a numerical parameter determined based on the text recognition confidence information output from the optical character recognition process, used to measure the reliability of text recognition on a page. The aforementioned text density feature can refer to a feature value calculated based on the number of characters or the density of character distribution on a page, used to measure the richness of text on the page.

[0090] Understandably, the aforementioned weighted fusion can refer to the mathematical process of summing up scores or weights from multiple different dimensions according to their respective preset weight coefficients to obtain a comprehensive score.

[0091] In its implementation, after obtaining the sampled page set, the system performs multi-dimensional semantic value scoring on each page in the sampled page set. Specifically, the system first performs keyword matching on each page in the sampled page set based on a preset set of semantic anchors. The system compares the text content of each page with each anchor word in the preset set of semantic anchors one by one, counting the number of anchor words matched or the specific anchor words matched on each page. The system calculates the keyword semantic score for each page based on the match results; for example, a preset reward score is added for each matched preset semantic anchor word, and a score cap is set. The system iterates through all pages in the sampled page set to obtain the keyword semantic score for each page.

[0092] Simultaneously, the system determines the prior position weight of each page based on its position within the document to be identified. The system obtains the page number of each page and assigns a prior position weight to each page according to pre-defined position weight rules. For example, the first page (page number 1) is assigned the highest prior position weight, subsequent pages have their prior position weights linearly decreasing with increasing page numbers, and the last page receives additional fixed prior position weight compensation. The system calculates the prior position weight value for each page based on its page number.

[0093] Simultaneously, the system performs optical character recognition (OCR) on each page. For each page in the sampled page set, the system uses an OCR engine to detect and recognize text in the page's image, outputting the recognized text corresponding to all text boxes on each page, along with a confidence score for each text box. Based on the OCR recognition results, the system determines the recognition quality coefficient and text density features for each page. The recognition quality coefficient can be determined based on the average confidence score of all text boxes on the page, for example, by multiplying the average confidence score by a preset proportional coefficient. The text density features can be determined based on the total number of characters or the number of text boxes recognized on the page; a reward feature value is given when the number of characters is within a preset moderate range, and a penalty feature value is given when the number of characters is too few or too many.

[0094] After obtaining the keyword semantic score, positional prior weight, recognition quality coefficient, and text density feature for each page, the system performs a weighted fusion of these four dimensions. Specifically, the system assigns corresponding preset weight coefficients to the keyword semantic score, positional prior weight, recognition quality coefficient, and text density feature, then multiplies each page's keyword semantic score by the first weight coefficient, the positional prior weight by the second weight coefficient, the recognition quality coefficient by the third weight coefficient, and the text density feature by the fourth weight coefficient. Finally, the four products are summed to obtain the semantic value score for that page. The system performs the above weighted fusion calculation for each page in the sampled page set to obtain the semantic value score for each page.

[0095] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, assume that the sampled page set includes four pages: page 1, page 3, page 10, and page 22. The preset semantic anchor set includes "Party B", "Account", "Bank", and "Contract Amount". The system performs keyword matching on page 1 and finds that the page contains two anchor words: "Party B" and "Account". Each anchor word is awarded 15 points, with a maximum score of 50 points. Therefore, the keyword semantic score is 30 points. Page 1 has a page number of 1, and the prior weight for the homepage position is 20 points. The system performs OCR recognition on page 1 and obtains an average confidence score of 0.92 for all text boxes. Multiplying the average confidence score by 15 and rounding it down gives a recognition quality coefficient of 14 points. The total number of characters recognized by the OCR on this page is 280, which is within a moderate range, and the text density feature is awarded 5 points. The system described above performs a weighted fusion based on preset weight coefficients (keyword semantic score weight α=1, location prior weight β=1, recognition quality coefficient weight γ=15, and text density feature weight δ=1), resulting in a semantic value score of 30×1 + 20×1 + 14×15 + 5×1 = 30 + 20 + 210 + 5 = 265 points. The system performs the same calculation on pages 3, 10, and 22, ultimately obtaining semantic value scores of 265, 180, 95, and 210 points for each page, respectively.

[0096] It should be noted that by using a preset set of semantic anchors for keyword matching to accurately locate key business information, combining prior position weights to utilize document structure rules, and integrating optical character recognition quality coefficients and text density features for multi-dimensional weighted scoring, the page value measurement becomes more comprehensive and objective, effectively improving the accuracy of candidate page set selection and key information recall rate.

[0097] This embodiment discloses a method for identifying a document of the type of multi-page contract. The method involves semantic value filtering of each page to obtain a candidate page set; inputting each candidate page into a multimodal large language model for structured information extraction to obtain a single-page structured recognition result; and performing cross-page aggregation processing on the single-page structured recognition results to obtain the recognition result of the document to be identified. This embodiment, for multi-page contract documents, determines the candidate page set through semantic value filtering, avoiding the computational waste and response delay caused by full-page visual reasoning. After the candidate pages are extracted by the multimodal large language model, cross-page aggregation processing is used to achieve the fusion and conflict resolution of multi-page structured results, effectively improving the consistency and stability of field extraction from long documents.

[0098] refer to Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the document intelligent recognition method of the present invention.

[0099] Based on the above embodiments, in this embodiment, step S103 further includes steps S1031 to S1034: Step S1031: Obtain the single-page structured recognition result of each candidate page, wherein the single-page structured recognition result includes the extracted value and confidence level corresponding to the field.

[0100] Step S1032: For the same target field, cluster the same extracted values ​​to obtain at least one extracted value cluster.

[0101] Step S1033: Determine the comprehensive score of each extracted value cluster based on the sum of the confidence scores corresponding to each extracted value in each extracted value cluster and the frequency of occurrence of the extracted value.

[0102] Step S1034: Select the extraction value corresponding to the cluster with the highest comprehensive score as the final recognition result of the target field in the document to be recognized.

[0103] It should be clarified that the extracted values ​​mentioned above refer to the specific text content extracted from a target field in the single-page structured recognition result. For example, for the "lessee name" field, the extracted content for a candidate page might be "XX Leasing Co., Ltd." The confidence level mentioned above refers to the degree of confidence given by the multimodal large language model for the extracted values ​​of each field when outputting the single-page structured recognition result. This value reflects the model's self-assessment of the accuracy of the extraction result; a higher confidence level indicates greater certainty from the model regarding the extraction result.

[0104] It is understandable that the target fields mentioned above can refer to the preset business fields that need to be extracted from the document to be identified, such as "Party A Name", "Party B Name", "Total Contract Amount" and "Signing Date".

[0105] It should be noted that the clustering mentioned above refers to the operation of grouping and classifying multiple extracted values ​​for the same target field from multiple single-page structured recognition results according to whether the specific content of the extracted values ​​is the same. The extracted value clustering mentioned above can refer to a group consisting of several records with the same extracted value after the clustering operation.

[0106] Understandably, the frequency mentioned above can refer to the number of single-page structured recognition results with the same extracted value within an extracted value cluster. The comprehensive score mentioned above can refer to the comprehensive evaluation score calculated for an extracted value cluster, which is used to measure the reliability of the extracted values ​​corresponding to that extracted value cluster as the final recognition result. The higher the comprehensive score, the more reliable the extracted value.

[0107] In its implementation, after obtaining the single-page structured recognition results for each candidate page, the system performs cross-page aggregation processing on these results. Specifically, the system first obtains the single-page structured recognition results for each candidate page. Each single-page structured recognition result contains the extracted values ​​for each preset target field and the corresponding confidence level. For the same target field, the system collects all extracted values ​​and their associated confidence levels from all single-page structured recognition results.

[0108] Next, the system clusters all collected extracted values ​​for the target field based on whether their content is identical. Extracted values ​​with completely identical text content are grouped into the same cluster, resulting in at least one cluster, with each cluster corresponding to a single extracted value content. After clustering, the system calculates the sum of all confidence scores for each extracted value within that cluster, based on the confidence scores of each extracted value, and counts the number of records contained in that cluster as the frequency of that extracted value.

[0109] Subsequently, the system calculates the comprehensive score for each extracted value cluster based on the sum of confidence levels and frequency. The system can obtain the comprehensive score for each extracted value cluster by multiplying the sum of confidence levels and frequency by the corresponding weight coefficients according to a preset weighted calculation formula.

[0110] The system then compares the overall scores of all extracted value clusters and selects the cluster with the highest overall score. The extracted value content corresponding to this cluster is then used as the final recognition result for the target field in the document to be recognized. The system performs this operation for each preset target field to obtain the final recognition result for all target fields.

[0111] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, assume that the candidate page set contains three candidate pages: page 1, page 5, and page 18, and the target field is "lessee name". The system obtains the following extracted values ​​for "lessee name" from the single-page structured recognition results: "XX Technology Co., Ltd." for page 1 (confidence level 0.95); "XX Technology Co., Ltd." for page 5 (confidence level 0.92); and "XX Technology Co., Ltd." for page 18 (confidence level 0.78). The system then clusters the three extracted values ​​according to their content for the target field "lessee name", resulting in two clusters: the first cluster contains the two extracted values ​​corresponding to "XX Technology Co., Ltd." (from pages 1 and 5), and the second cluster contains the one extracted value corresponding to "XX Technology Co., Ltd." (from page 18). For the first cluster, the sum of confidence scores is 0.95 + 0.92 = 1.87, and the frequency is 2. Assuming a frequency reward coefficient of 5, the overall score is 1.87 + 5 × 2 = 11.87. For the second cluster, the sum of confidence scores is 0.78, and the frequency is 1, resulting in an overall score of 0.78 + 5 × 1 = 5.78. The system compares the overall scores of the two clusters and selects the extracted value "XX Technology Co., Ltd." from the first cluster with the highest overall score as the final recognition result for the "Lessee Name" field in the document to be identified.

[0112] This embodiment discloses the acquisition of single-page structured recognition results for each candidate page. The single-page structured recognition results include the extracted values ​​and confidence levels corresponding to the fields. For the same target field, identical extracted values ​​are clustered to obtain at least one extracted value cluster. Based on the sum of the confidence levels corresponding to each extracted value in each extracted value cluster and the frequency of occurrence of the extracted values, a comprehensive score for each extracted value cluster is determined. The extracted value corresponding to the extracted value cluster with the highest comprehensive score is selected as the final recognition result of the target field in the document to be recognized. Because this embodiment uses field-level clustering and a weighted voting mechanism to fuse independent inference results across multiple pages, and utilizes confidence accumulation and frequency consensus for comprehensive decision-making, compared to existing technologies, this embodiment effectively suppresses single-page inference illusions, resolves cross-page field conflicts, achieves information complementarity and consistent decision-making, and improves the stability and reliability of long document structured extraction.

[0113] Furthermore, this embodiment of the invention also proposes a storage medium storing a document intelligent recognition program, which, when executed by a processor, implements the steps of the document intelligent recognition method described above.

[0114] Reference Figure 4 , Figure 4This is a structural block diagram of the first embodiment of the document intelligent recognition device of the present invention.

[0115] like Figure 4 As shown, the document intelligent recognition device proposed in this embodiment of the invention includes: a type determination module 501, a visual enhancement module 502, an image processing module 503, and a result output module 504.

[0116] The type determination module 501 is used to acquire the document to be identified and determine the document type of the document to be identified.

[0117] The visual enhancement module 502 is used to perform visual enhancement processing on the document to be identified and determine the region of interest when the document type is a single-page certificate.

[0118] The image processing module 503 is used to perform standardization processing on the region of interest to obtain a standardized image.

[0119] The result output module 504 is used to input the standardized image into a multimodal large language model for structured information extraction to obtain the recognition result of the document to be recognized.

[0120] The visual enhancement module 502 is further configured to perform grayscale processing on the document to be identified to obtain a grayscale image, and construct a gradient energy field of the document to be identified based on the grayscale image; perform smoothing processing on the gradient energy field to obtain a smooth energy distribution map; perform iterative boundary optimization based on the smooth energy distribution map to determine the initial region of interest of the document to be identified; and expand the initial region of interest by a preset ratio to obtain the region of interest.

[0121] The image processing module 503 is further configured to obtain the long side dimension of the region of interest, and construct a square canvas with the long side dimension as the side length; fill the square canvas with a preset background color, and place the region of interest in the center within the square canvas; and scale the square canvas proportionally according to a preset target size to obtain a standardized image.

[0122] This device embodiment discloses acquiring a document to be identified and determining its document type. When the document type is a single-page certificate, visual enhancement processing is performed on the document to be identified to determine the region of interest. The region of interest is then standardized to obtain a standardized image. The standardized image is input into a multimodal large language model for structured information extraction to obtain the recognition result of the document to be identified. Because this device embodiment achieves differentiated processing through document type discrimination, and performs visual enhancement and standardization processing on single-page certificate documents first, it effectively suppresses the interference of shooting distortion and background noise on the multimodal large language model. Compared with the prior art, this device embodiment improves the recognition accuracy of certificate fields in complex shooting scenarios.

[0123] Based on the first embodiment of the document intelligent recognition device of the present invention described above, a second embodiment of the document intelligent recognition device of the present invention is proposed.

[0124] In this embodiment, the type determination module 501 is further configured to, when the document type is a multi-page contract, perform semantic value filtering on each page of the document to be identified to obtain a candidate page set; input each candidate page in the candidate page set into a multimodal large language model for structured information extraction to obtain a single-page structured recognition result for each candidate page; and perform cross-page aggregation processing on each single-page structured recognition result to obtain the recognition result of the document to be identified.

[0125] The type determination module 501 is further configured to perform non-uniform sampling on the document to be identified to obtain a set of sampled pages; perform multi-dimensional semantic value scoring on each page in the set of sampled pages to obtain a semantic value score for each page; and select a set of candidate pages from the set of sampled pages based on the semantic value scores of each page.

[0126] The type determination module 501 is further configured to perform keyword matching on each page in the sampled page set based on a preset set of semantic anchor points to obtain a keyword semantic score for each page; determine a priori position weight for each page based on its page number position in the document to be identified; perform optical character recognition on each page and determine a recognition quality coefficient and text density feature for each page based on the optical character recognition results; and perform weighted fusion of the keyword semantic score, the priori position weight, the recognition quality coefficient, and the text density feature to obtain a semantic value score for each page.

[0127] Other embodiments or specific implementations of the document intelligent recognition device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0128] This application provides a document intelligent recognition device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the document intelligent recognition method in Embodiment 1 above.

[0129] The following is for reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the document intelligent recognition device in the embodiments of this application. The document intelligent recognition device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The document intelligent recognition device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0130] like Figure 5 As shown, the document intelligent recognition device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the document intelligent recognition device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the document intelligent recognition device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows document intelligent recognition devices with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0131] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0132] The document intelligent recognition device provided in this application, employing the document intelligent recognition method in the above embodiments, can solve the technical problem of insufficient recognition accuracy of certificate fields under complex shooting conditions in the prior art. Compared with the prior art, the beneficial effects of the document intelligent recognition device provided in this application are the same as those of the document intelligent recognition method provided in the above embodiments, and other technical features in this document intelligent recognition device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0133] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0134] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0135] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0136] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0138] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A document intelligent recognition method, characterized in that, The method includes: Obtain the document to be identified and determine the document type of the document to be identified; When the document type is a single-page certificate, the document to be identified is subjected to visual enhancement processing to determine the region of interest; The region of interest is standardized to obtain a standardized image; The standardized image is input into a multimodal large language model for structured information extraction to obtain the recognition result of the document to be recognized.

2. The document intelligent recognition method as described in claim 1, characterized in that, The step of performing visual enhancement processing on the document to be identified and determining the region of interest includes: The document to be identified is converted to grayscale to obtain a grayscale image, and the gradient energy field of the document to be identified is constructed based on the grayscale image. The gradient energy field is smoothed to obtain a smooth energy distribution map; Iterative boundary optimization is performed based on the smooth energy distribution map to determine the initial region of interest of the document to be identified; The initial region of interest is expanded by a preset ratio to obtain the region of interest.

3. The document intelligent recognition method as described in claim 2, characterized in that, The step of standardizing the region of interest to obtain a standardized image includes: Obtain the length of the long side of the region of interest, and construct a square canvas with the length of the long side as the side length; The square canvas is filled with a preset background color, and the region of interest is placed in the center within the square canvas. The square canvas is scaled proportionally to a preset target size to obtain a standardized image.

4. The document intelligent recognition method as described in claim 1, characterized in that, After the steps of obtaining the document to be identified and determining the document type of the document to be identified, the method further includes: When the document type is a multi-page contract, semantic value filtering is performed on each page of the document to be identified to obtain a candidate page set; Each candidate page in the candidate page set is input into a multimodal large language model for structured information extraction to obtain the single-page structured recognition result of each candidate page; The single-page structured recognition results are aggregated across pages to obtain the recognition results of the document to be recognized.

5. The document intelligent recognition method as described in claim 4, characterized in that, The step of performing semantic value filtering on each page of the document to be identified to obtain a candidate page set includes: The document to be identified is subjected to non-uniform sampling to obtain a set of sampled pages; Multidimensional semantic value scoring is performed on each page in the sampled page set to obtain the semantic value score of each page; Based on the semantic value score of each page, a candidate page set is selected from the sampled page set.

6. The document intelligent recognition method as described in claim 4, characterized in that, The step of performing cross-page aggregation processing on the single-page structured recognition results to obtain the recognition result of the document to be recognized includes: Obtain the single-page structured recognition result for each candidate page, wherein the single-page structured recognition result includes the extracted values ​​and confidence scores corresponding to the fields; For the same target field, cluster the same extracted values ​​to obtain at least one extracted value cluster; The comprehensive score of each extracted value cluster is determined based on the sum of the confidence scores corresponding to each extracted value in each extracted value cluster and the frequency of occurrence of the extracted value. The extraction value corresponding to the cluster with the highest comprehensive score is selected as the final recognition result of the target field in the document to be recognized.

7. The document intelligent recognition method as described in claim 5, characterized in that, The step of performing multi-dimensional semantic value scoring on each page in the sampled page set to obtain the semantic value score of each page includes: Based on a preset set of semantic anchor points, keyword matching is performed on each page in the sampled page set to obtain the keyword semantic score of each page; The positional prior weight of each page is determined based on its page number position in the document to be identified. Optical character recognition is performed on each of the pages, and the recognition quality coefficient and text density features of each page are determined based on the optical character recognition results; The semantic value score of each page is obtained by weighted fusion of the keyword semantic score, the location prior weight, the recognition quality coefficient and the text density feature.

8. A document intelligent recognition device, characterized in that, The device includes: A type determination module is used to acquire the document to be identified and determine the document type of the document to be identified; The visual enhancement module is used to perform visual enhancement processing on the document to be identified and determine the region of interest when the document type is a single-page certificate. The image processing module is used to perform standardization processing on the region of interest to obtain a standardized image; The result output module is used to input the standardized image into a multimodal large language model for structured information extraction to obtain the recognition result of the document to be recognized.

9. A document intelligent recognition device, characterized in that, The device includes: a memory, a processor, and a document intelligent recognition program stored in the memory and executable on the processor, the document intelligent recognition program being configured to implement the steps of the document intelligent recognition method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a document intelligent recognition program, which, when executed by a processor, implements the steps of the document intelligent recognition method as described in any one of claims 1 to 7.

Images