Multi-modal large model-based seal detection and identification method and system

Abstract

The invention discloses a multi-modal large model-based seal detection and identification method and system, and the method comprises the steps: carrying out the size standardization processing of obtained document image data through an image preprocessing module, and generating a standardized image which accords with the input specification of a multi-modal large model; the standardized image and the structured recognition instruction are combined and then input into a multi-modal large model subjected to fine tuning training for end-to-end reasoning, and a recognition result character string conforming to the JSON format specification is output; and analyzing the recognition result character string to extract the bounding box coordinate, the type label, the text content and the text recognition confidence coefficient of each seal object, performing hierarchical decision processing based on the text recognition confidence coefficient, and finally outputting a structured recognition result. According to the invention, the accuracy and the automation level of seal identification are obviously improved, the normalization and the reliability of an output result are ensured, and the problems of low seal identification precision and non-standardized output in a complex scene are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and image recognition processing, in particular to a seal detection and recognition method and system based on a multi-modal large model. BACKGROUND

[0002] In financial, tax, legal and administrative approval business scenarios, seals are the core evidence of document authenticity and legality, and their accurate identification and automatic processing are of great significance to risk prevention and business process efficiency improvement. Traditional seal identification methods mainly rely on traditional image processing techniques based on rules or special deep learning models, such as combining edge detection, shape matching and optical character recognition (OCR) techniques to realize seal positioning and text extraction. However, such methods generally have limited generalization ability, are sensitive to image quality, and rely on a large amount of manually annotated data. In particular, when faced with blurred, incomplete, overlapping or non-standard seals, the recognition accuracy significantly decreases, making it difficult to meet the strict requirements of reliable identification results and structured output in high-compliance business scenarios. In recent years, multi-modal large models have shown strong cross-modal understanding and generation capabilities, providing a new technical path for seal identification tasks. However, existing general multi-modal models have not been specifically optimized for seal semantics, business specifications and output formats, and their free-text output form is difficult to directly integrate into automated business processes. Moreover, they lack effective modeling and expression of uncertainties in seal images, leading to large identification biases, unreliable results and unstructured output in practical applications. In summary, existing technologies have significant shortcomings in seal inspection and identification, especially for high-compliance business scenarios such as finance, which urgently need a multi-modal identification system and method that can accurately identify seal content, ensure strict format output and meet high-precision comparison requirements, while effectively handling blurred and incomplete seals. SUMMARY

[0003] In view of the above problems, the present application provides a seal detection and recognition method and system based on a multi-modal large model, which realizes high-precision end-to-end identification through multi-stage fine-tuning and structured output control, and solves the problems of poor adaptability to blurred seals and non-standardized output.

[0004] To achieve the above purpose, in a first aspect, the present application provides a seal detection and recognition method based on a multi-modal large model, comprising:

[0005] Obtaining document image data to be processed, and inputting the document image data to an image preprocessing module;

[0006] Performing size standardization processing on the input document image data through the image preprocessing module to generate a standardized image that meets the input specification of the multi-modal large model;

[0007] combine the standardized image and the predefined structured recognition instruction into multi-modal input data, and input the multi-modal input data into the fine-tuned multi-modal large model;

[0008] perform end-to-end inference on the multi-modal input data by the multi-modal large model, and output a recognition result string containing at least one seal object, the recognition result string conforming to a predefined JSON format specification;

[0009] parse the recognition result string, and extract a bounding box coordinate, a type label, text content, and text recognition confidence of each seal object;

[0010] perform hierarchical decision processing on each seal object according to the text recognition confidence, and output a final structured recognition result.

[0011] Further, the fine-tuned multi-modal large model is configured to be obtained by the following steps:

[0012] construct a multi-modal training dataset containing seal images, corresponding bounding box coordinates, type labels, text content, and text recognition confidence;

[0013] use the multi-modal training dataset to perform instruction fine-tuning training on the multi-modal large model as a base model;

[0014] use a reinforcement learning algorithm based on preference optimization to perform alignment training on the instruction fine-tuned model to improve the format specification and content accuracy of the output result.

[0015] Further, the multi-modal training dataset containing seal images, corresponding bounding box coordinates, type labels, text content, and text recognition confidence includes:

[0016] generate training seal images by a data synthesis engine based on physical simulation, the data synthesis engine being configured to simulate ink distribution characteristics generated under different stamping pressures, angles, and medium conditions;

[0017] automatically generate corresponding text recognition confidence labels according to the occlusion parameters and blur parameters applied during the synthesis process;

[0018] combine the synthesized seal images with the corresponding bounding box coordinates, type labels, text content, and text recognition confidence labels into the multi-modal training dataset.

[0019] Further, the fine-tuned multi-modal large model is configured to be obtained by the following steps:

[0020] A format correctness preference dataset is constructed, which contains format-correct responses as preferred samples and format-incorrect responses as dispreferred samples;

[0021] A content accuracy preference dataset is constructed, which contains responses that correctly express recognition uncertainty as preferred samples and responses that are overconfident or incorrectly derived as dispreferred samples;

[0022] Using the format correctness preference dataset and the content accuracy preference dataset, the model fine-tuned by the instruction is trained by multi-objective reinforcement learning through a direct preference optimization algorithm;

[0023] And during the training process, the multi-modal large model learns the dual constraints of output format correctness and content accuracy by optimizing the preference objective function.

[0024] Further, the image preprocessing module is used to perform size standardization processing on the input document image data to generate standardized images that meet the input specifications of the multi-modal large model, including:

[0025] The input document image data is subjected to size analysis to obtain the original width parameter and the original height parameter of the original image, and the original aspect ratio is obtained;

[0026] The image size of the original image is adjusted based on a predetermined integer factor, so that the adjusted image width parameter and image height parameter can be divided by the integer factor;

[0027] Under the constraint condition of maintaining the original aspect ratio, the total number of image pixels is adjusted to a predetermined numerical range;

[0028] An adaptive interpolation algorithm is used to resample the image pixels during the adjustment process to maintain the visual quality of the image content;

[0029] The size-standardized standardized image is output.

[0030] Further, the multi-modal large model is used to perform end-to-end inference on the multi-modal input data to output a recognition result string containing at least one seal object, including:

[0031] An iterative inference mechanism is used to process the multi-modal input data, first performing a seal area positioning operation to generate a set of potential regions;

[0032] A seal type discrimination operation is performed on each region in the set of potential regions to filter out target regions that are confirmed to be seals;

[0033] A text recognition operation is performed on each target region to generate corresponding text content and confidence evaluation results;

[0034] The bounding box coordinates of the target region, type discrimination results, text content, and confidence evaluation results are combined into structured data;

[0035] The structured data is converted into an identification result string conforming to the JSON format specification and is output.

[0036] Further, the identification result string is parsed to extract the bounding box coordinates, type label, text content, and text recognition confidence of each seal object, including:

[0037] The JSON parsing algorithm is used to perform syntax analysis and structure verification on the identification result string to obtain a data structure;

[0038] Seal objects are extracted from the data structure, and each seal object includes a bounding box coordinate array, a type label string, a text content string, and a text recognition confidence level;

[0039] The seal objects are verified, including:

[0040] The bounding box coordinate array is subjected to coordinate format verification and numerical range checking to ensure that the coordinate values are within the valid image range;

[0041] The text recognition confidence level is subjected to enumeration value verification to confirm that it belongs to the preset confidence level set;

[0042] The verification results are converted into final parsing results, and the final parsing results include the text recognition confidence.

[0043] Further, each seal object is subjected to hierarchical decision processing based on the text recognition confidence to generate and output the final structured identification result, including:

[0044] Based on the preset confidence grading rules, the text recognition confidence of each seal object is divided into one of the high, medium, and low levels;

[0045] For seal objects with high text recognition confidence, an automatic comparison process is performed to match their text content with a pre-stored institution name database, and a consistency verification result, i.e., an automatic comparison result, is generated;

[0046] For seal objects with medium or low text recognition confidence, a manual review marking process is triggered, and the location information of the corresponding seal region in the original image is marked as a marking result;

[0047] Based on the automatic comparison result and the marking result, a structured identification result containing the verification status and the confidence level is generated;

[0048] The structured identification result is serialized and output in the preset output format.

[0049] Further, the method further comprises:

[0050] constructing new training samples according to the manual review results or historical matching results;

[0051] performing online incremental learning on the multi-modal large model using the new training samples.

[0052] In a second aspect, the present application also provides a seal detection and recognition system based on a multi-modal large model, which is suitable for the method of the first aspect. The system comprises an image input module, an image preprocessing module, an instruction generation module, a multi-modal large model inference module, a result analysis module, and a decision output module. The image input module is used to obtain document image data to be processed. The image preprocessing module is used to perform size normalization processing on the input document image data to generate standardized images that meet the input specifications of the multi-modal large model. The instruction generation module is used to generate pre-defined structured recognition instructions. The multi-modal large model inference module is used to receive multi-modal input data composed of standardized images and structured recognition instructions, and perform end-to-end inference to output recognition result strings that meet the JSON format specifications. The result analysis module is used to analyze the recognition result strings and extract the bounding box coordinates, type labels, text content, and text recognition confidence of each seal object. The decision output module is used to perform hierarchical decision processing on each seal object based on the text recognition confidence, generate the final structured recognition result, and output it.

[0053] Unlike the prior art, the above technical solution performs size normalization processing on the obtained document image data through the image preprocessing module to generate standardized images that meet the input specifications of the multi-modal large model. The standardized images and structured recognition instructions are combined and input into the fine-tuned multi-modal large model for end-to-end inference to output recognition result strings that meet the JSON format specifications. The bounding box coordinates, type labels, text content, and text recognition confidence of each seal object are extracted by analyzing the recognition result strings, and hierarchical decision processing is performed based on the text recognition confidence to finally output the structured recognition result. The present application significantly improves the accuracy and automation level of seal recognition, ensures the standardization and reliability of the output results, and effectively solves the problems of low seal recognition accuracy and non-standard output in complex scenarios.

[0054] The above summary of the invention is only a summary of the technical solutions of the present application. In order for those skilled in the art to more clearly understand the technical solutions of the present application, and then can be implemented according to the content of the description and the drawings, and in order to make the above-mentioned purposes and other purposes, features and advantages of the present application more easily understood, the following will be described in conjunction with the specific embodiments of the present application and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings are only used to illustrate the principles, implementation manners, application, characteristics and effects of the specific embodiments and other related contents of the present application, and cannot be considered as limitations to the present application.

[0056] In the drawings:

[0057] Figure 1 A step schematic diagram of steps S101 to S106 of the method described in the specific embodiment;

[0058] Figure 2 A step schematic diagram of steps S201 to S203 of the method described in the specific embodiment;

[0059] Figure 3 A step schematic diagram of steps S301 to S303 of the method described in the specific embodiment;

[0060] Figure 4 A step schematic diagram of steps S401 to S403 of the method described in the specific embodiment;

[0061] Figure 5 A structure schematic diagram of the system described in the specific embodiment.

[0062] The reference signs involved in the above drawings are explained as follows:

[0063] 1. A seal detection and identification system;

[0064] 11. An image input module;

[0065] 12. An image preprocessing module;

[0066] 13. An instruction generation module;

[0067] 14. A multimodal large model inference module;

[0068] 15. A result analysis module;

[0069] 16. A decision output module. DETAILED DESCRIPTION

[0070] In order to make the possible application scenarios, technical principles, specific schemes that can be implemented, purposes and effects that can be achieved of the present application clear, the following will be described in detail in combination with the specific embodiments listed and the accompanying drawings. The embodiments recorded in the present text are only used to more clearly illustrate the technical schemes of the present application, and therefore only serve as examples, and cannot be considered as limitations to the protection scope of the present application.

[0071] The term "embodiment" is mentioned herein means that the specific features, structures, or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit the independence or association between other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, each technical feature mentioned in each embodiment can be combined in any way to form a corresponding implementable technical solution.

[0072] Unless otherwise defined, the meanings of the technical terms used herein are the same as those commonly understood by those skilled in the art to which the present application belongs; the use of related terms herein is only for the purpose of describing specific embodiments, and is not intended to limit the present application.

[0073] In the description of the present application, the phrase "and / or" is a description of the logical relationship between the objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases: A exists, B exists, and A and B exist at the same time. In addition, the character " / " herein generally represents that the associated objects before and after are a "or" logical relationship.

[0074] In the present application, the terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, primary and secondary or order relationship between the entities or operations.

[0075] In the present application, without more limitation, the "includes", "contains", "has" or other similar open expressions used in the sentence are intended to cover non-exclusive inclusion, and these expressions do not exclude the presence of other elements in the process, method or product including the described elements, so that the process, method or product including a series of elements can not only include those limited elements, but also include other elements not explicitly listed, or also include the elements inherent to such process, method or product.

[0076] As the same as the understanding in the "Guidelines for Examination", in the present application, the expressions such as "greater than", "less than", "exceed" are understood as not including the number; the expressions such as "above", "below", "within" are understood as including the number. In addition, in the description of the embodiments of the present application, the meaning of "multiple" is more than two (including two), and similar expressions related to "multiple" are also understood in this way, for example, "multiple groups", "multiple times", etc., unless otherwise explicitly limited.

[0077] In the description of the embodiments of the present application, the spatial relative expressions used, such as "central", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", and the like, indicate the orientation or positional relationship shown in the specific embodiments or the drawings, and are only used for the convenience of describing the specific embodiments of the present application or for the convenience of the reader to understand, and do not indicate or imply that the indicated device or component must have a particular position, a particular orientation, or be constructed or operated in a particular orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0078] The processor described in the embodiments of the present application can be implemented by hardware, firmware, software or a combination thereof, and can use at least one of circuit, single or multiple application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), central processing units (CPU), controllers, microcontrollers, microprocessors, and other physical, biological or chemical structures that can realize the same or equivalent functions as the above-mentioned processors, such as biological neurons, quantum computing units, DNA computing units, etc., so that the processor can execute part or all of the steps or any combination of the steps mentioned in the computer programs or methods of the embodiments of the present application.

[0079] The computer program involved in the embodiments can be stored in a computer device readable storage medium, including but not limited to magnetic disk, magnetic tape, magnetic card, floppy disk, flash memory, optical disk, optical card, read-only memory (ROM), random access memory (RAM), erasable programmable ROM (EPROM) and electrically erasable programmable ROM (EEPROM) and the like, and also includes other biological, physical or chemical structures that can realize similar or equivalent functions as the above-mentioned storage media, such as DNA, RNA, protein and the like units with information storage ability. In specific embodiments, the storage medium involved can be one of the above-mentioned medium types, or a combination of the above-mentioned medium types. In different embodiments, the computer program involved in the embodiments can be stored in a single medium in a centralized manner, or stored in multiple media in a distributed manner. The storage medium containing the computer device readable storage medium can be a non-volatile memory or a random access memory. These computer device readable storage media can be built into the device, or connected to the device as an external device or part of the external device. In some embodiments, the storage medium with the computer device readable storage medium is deployed locally; in other embodiments, the storage medium can also be deployed remotely from the processor, such as network-attached storage accessed via RF circuitry or external port and communication network, wherein the communication network can be the Internet, one or more intranets, local area networks (LANs), wide area networks (WANs), storage area networks (SANs) and the like, or appropriate combinations thereof, as long as the computer device can access the storage medium. In addition, the computer program involved in the embodiments can be stored in plaintext / ciphertext, or can be designed as training data, and integrated and reorganized by model training to be implicitly saved in the parameter state of the deep neural network or other machine learning model.

[0080] Please refer to Figure 1 In the first aspect, the embodiment provides a seal detection and recognition method based on a multi-modal large model, comprising:

[0081] S101, acquiring document image data to be processed, and inputting the document image data to an image preprocessing module;

[0082] S102, performing size normalization processing on the input document image data by the image preprocessing module to generate a normalized image conforming to the input specification of the multi-modal large model;

[0083] S103, combining the normalized image with a pre-defined structured recognition instruction into multi-modal input data, and inputting the multi-modal input data to the fine-tuned multi-modal large model;

[0084] S104, performing end-to-end inference on the multi-modal input data by the multi-modal large model, outputting a recognition result string containing at least one seal object, the recognition result string conforming to a predefined JSON format specification;

[0085] S105, parsing the recognition result string to extract the bounding box coordinates, type label, text content and text recognition confidence of each seal object;

[0086] S106, performing hierarchical decision processing on each seal object according to the text recognition confidence, generating a final structured recognition result and outputting.

[0087] In step S101, the document image data refers to the electronic document image containing the seal obtained by a scanner, a digital camera or a file upload interface, and its format includes but is not limited to common image formats such as JPEG, PNG or PDF. This data is used as the original input of the whole recognition process for subsequent preprocessing and analysis. The acquisition of document image data can be realized by automatic file collection system or manual uploading method to ensure the digitalization of the original material.

[0088] In step S102, the image preprocessing module refers to a special functional unit for size standardization processing of the input document image data. Size standardization processing adjusts the image size to meet the input requirements of the multi-modal large model, including ensuring that the image width and height meet the divisibility condition, and controlling the total number of image pixels within the preset range while maintaining the original aspect ratio. Preferably, the processing uses bilinear interpolation or Lanczos resampling algorithm to maintain image quality and avoid geometric distortion. The standardized image is the preprocessing result that meets the model input specification, providing a unified specification visual input for subsequent multi-modal processing.

[0089] In step S103, the predefined structured recognition instruction refers to a pre-defined natural language instruction text, which is used to clearly indicate the multi-modal large model to perform seal detection and recognition tasks. Preferably, the structured recognition instruction contains specific requirements for seal positioning, type identification, text extraction and confidence evaluation, and is combined with the standardized image to form multi-modal input data. The multi-modal input data is sent to the multi-modal large model through the model input interface, triggering the inference process of the model.

[0090] In step S104, the multi-modal large model is a visual-linguistic model trained by instruction fine-tuning and preference optimization, which has cross-modal understanding and structured generation capabilities. The multi-modal large model processes multi-modal input data at one time and directly produces recognition results to realize end-to-end inference. The recognition result string adopts strict JSON format specification, containing the structured description of one or more seal objects, ensuring that the output can be directly parsed and processed by the downstream system.

[0091] In step S105, the parsing of the recognition result string is implemented by a JSON parser, and the extraction operation includes syntax analysis, structure verification, and data extraction. The bounding box coordinates represent the seal position in the form of a pixel coordinate array, the type label uses a predefined classification identifier, the text content contains the recognized text string, and the text recognition confidence uses a hierarchical identifier to represent the recognition reliability. This step ensures accurate extraction of all required seal information from the model output.

[0092] In step S106, the hierarchical decision processing classifies the seal object based on the text recognition confidence, and uses a pre-set confidence threshold rule for automated decision-making. Preferably, high-confidence results trigger an automated comparison process, medium-confidence results and low-confidence results are marked for manual review. The final structured recognition result is output in a standard data format, containing complete seal information and processing status identification, ensuring that the recognition result can be directly integrated into a business system.

[0093] This embodiment realizes end-to-end automation of seal detection and recognition through systematic image preprocessing, multi-modal fusion reasoning, and structured output processing. This method significantly improves recognition accuracy and output standardization, effectively solving the problems of poor adaptability to complex seals and non-standardized output in traditional methods, while ensuring the reliability of business processing through a confidence level classification mechanism, balancing processing efficiency and result quality, and being suitable for large-scale application in actual business scenarios.

[0094] Referring to Figure 2 In some embodiments, the fine-tuned multi-modal large model is configured to obtain by the following steps:

[0095] S201, constructing a multi-modal training data set containing seal images, corresponding bounding box coordinates, type labels, text content, and text recognition confidence;

[0096] S202, using a multi-modal large model as a base model, and using the multi-modal training data set to fine-tune the base model;

[0097] S203, using a reinforcement learning algorithm based on preference optimization to perform alignment training on the fine-tuned model to improve the format standardization and content accuracy of the output results.

[0098] In this embodiment, the multi-modal large model is preferably based on a high-performance visual-language architecture, such as the Qwen-2.5-VL-3B model. This base model has strong visual understanding and text generation capabilities, and has a moderate parameter size, providing an excellent starting point for subsequent instruction fine-tuning and reinforcement alignment for seal recognition tasks. In actual applications, other multi-modal large models with similar capabilities can also be used as the base.

[0099] In step S201, the multi-modal training dataset refers to a collection of training samples containing seal images and their corresponding annotation information. This dataset is constructed through manual annotation or synthetic data generation methods. The seal image serves as the visual input, the bounding box coordinates accurately mark the seal position, the type label identifies the seal category, the text content records the seal text information, and the text recognition confidence level represents the recognition reliability level. This dataset provides comprehensive learning samples for model training, ensuring that the model can accurately understand various features and variations of seals.

[0100] In step S202, the instruction fine-tuning training is a supervised fine-tuning process of the base model using the multi-modal training dataset. The base model uses a pre-trained multi-modal large model, and by inputting image-instruction pairs and corresponding standard outputs, the model learns the specific tasks of seal detection and recognition. During training, the cross-entropy loss function is used to optimize the model parameters, enabling the model to gradually master the ability to generate JSON format outputs that meet the requirements.

[0101] In step S203, the reinforcement learning algorithm based on preference optimization refers to the use of the direct preference optimization (DPO) method to further train the model. The reinforcement learning algorithm constructs a preference dataset for format specification and content accuracy, where preferred samples meet strict JSON format and correct confidence expression, and non-preferred samples contain format errors or inaccurate content. Through aligned training, the model simultaneously learns the dual constraints of output format specification and content accuracy, significantly improving the reliability and usability of the output results.

[0102] This embodiment achieves precise optimization of multi-modal large models in seal recognition tasks through systematic training data construction and multi-stage fine-tuning strategies. This method effectively solves the problems of non-standard output format and inaccurate content of general models, and through the combination of instruction fine-tuning and preference optimization, ensures that the model output not only meets strict JSON format specifications, but also accurately expresses recognition confidence, providing a reliable technical foundation for seal recognition.

[0103] Please refer to Figure 3 In some embodiments, the multi-modal training dataset containing seal images, corresponding bounding box coordinates, type labels, text content, and text recognition confidence is constructed, including:

[0104] S301, generate training seal images through a data synthesis engine based on physical simulation, the data synthesis engine is configured to simulate the ink distribution characteristics generated under different sealing pressures, angles, and medium conditions;

[0105] S302, automatically generate corresponding text recognition confidence labels according to the occlusion parameters and blur parameters applied during the synthesis process;

[0106] S303, combine the synthesized seal image with the corresponding bounding box coordinates, type label, text content, and text recognition confidence label into a multi-modal training dataset.

[0107] In step S301, the data synthesis engine based on physical simulation refers to an image generation system that simulates the real seal stamping process through computer graphics technology. The engine simulates the changes in ink density caused by different stamping pressures, the deformation effects caused by different angles, and the effects of different paper media on ink penetration and diffusion through algorithms, generating training images with various real seal characteristics. This simulation method can generate a large number of seal samples covering various extreme cases, effectively solving the problem of scarcity of real labeled data.

[0108] In step S302, the occlusion parameter and the blur parameter refer to control parameters that artificially introduce the degree of seal damage and blur in real-world scenarios during image synthesis. The occlusion parameter controls the proportion and location of the seal covered by text, table lines, or other objects, and the blur parameter controls the degree of image clarity degradation. According to the setting values of these parameters, the system automatically generates corresponding text recognition confidence labels, such as marking low confidence for highly blurred or severely occluded areas, and marking medium confidence for slightly affected areas.

[0109] Preferably, the generation of the text recognition confidence label is based on quantifiable image quality evaluation criteria. Specifically, by calculating the sharpness index (such as Laplacian variance) of the synthesized image and setting a threshold, areas with a sharpness lower than the predetermined threshold are automatically marked as low confidence. At the same time, according to the set character occlusion area ratio during synthesis, for example, if the occlusion area of a character exceeds a certain proportion (such as 30%), the overall confidence of the seal where the character is located will be limited to no higher than medium confidence. This automated process ensures the objective relevance of the confidence label to the visual quality of the image.

[0110] In step S303, the combination of the multi-modal training dataset refers to the process of structurally integrating the synthesized seal image with the corresponding labeled information. The bounding box coordinates accurately mark the seal position information, the type label identifies the seal category attribute, the text content provides accurate text labeling, and the text recognition confidence label reflects the recognition reliability level under different quality conditions. The alignment and integration of such multi-modal data provide high-quality supervision signals for model training.

[0111] The embodiment realizes large-scale automatic generation of training data through physical simulation technology, effectively solving the problems of high cost and low coverage of traditional manual annotation. The confidence annotation mechanism based on parameterized control ensures the quality consistency of the training data, providing a reliable data foundation for the model to learn accurate seal recognition and confidence evaluation. This method significantly improves the diversity and quality of training data, laying a solid data support for subsequent model optimization.

[0112] Please refer to Figure 4 In some embodiments, a reinforcement learning algorithm based on preference optimization is used to align train the instruction fine-tuned model to improve the format specification and content accuracy of its output results, including:

[0113] S401, a format specification preference data set is constructed, which contains correct format responses as preferred samples and incorrect format responses as inferior samples;

[0114] S402, a content accuracy preference data set is constructed, which contains responses that correctly express recognition uncertainty as preferred samples and responses that are overconfident or incorrectly derived as inferior samples;

[0115] S403, using the format specification preference data set and the content accuracy preference data set, the instruction fine-tuned model is trained by multi-objective reinforcement learning algorithm through direct preference optimization algorithm;

[0116] And in the training process, by optimizing the preference objective function, the multimodal large model simultaneously learns the dual constraints of output format specification and content accuracy.

[0117] In step S401, the format specification preference data set refers to a set of comparison samples used to train the model output format specification. The preferred sample is a pure output that strictly conforms to the JSON syntax specification and has no redundant characters, and the inferior sample contains responses with format errors, structure confusion or non-JSON content. This data set is constructed by automatic verification tools combined with manual review, ensuring that the model learns strict format output requirements.

[0118] In step S402, the content accuracy preference data set refers to a set of comparison samples used to train the model content expression accuracy. The preferred sample correctly reflects the image content and appropriately expresses the recognition uncertainty, such as labeling low confidence for ambiguous areas; the inferior sample contains overconfident incorrect inference or omission of uncertainty expression. This data set ensures that the model remains honest and accurate when generating content.

[0119] In step S403, the direct preference optimization algorithm adopts a multi-objective reinforcement learning method, and simultaneously uses two preference data sets for training. By optimizing the preference objective function, the algorithm updates the model parameters in the direction of both format specification and content accuracy, and learns the output that meets both the JSON format and the content accuracy.

[0120] This embodiment realizes the simultaneous improvement of the format specification and content accuracy of the model output by constructing a dual-preference data set and adopting multi-objective direct preference optimization. This method effectively solves the common problems of format errors and content inaccuracies in the stamp recognition task of multi-modal large models, ensures that the output results meet the specification requirements of machine readability and maintain the reliability of content expression, and provides high-quality structured output for downstream automated processing.

[0121] In some embodiments, the image preprocessing module performs size normalization processing on the input document image data to generate a standardized image that meets the input specification of the multi-modal large model, including:

[0122] The input document image data is subjected to size analysis to obtain the original width parameter and the original height parameter of the original image, and the original aspect ratio is obtained.

[0123] The image size of the original image is adjusted based on a preset integer factor, so that the adjusted image width parameter and image height parameter can be divided by the integer factor.

[0124] Under the constraint condition of maintaining the original aspect ratio, the total number of image pixels is adjusted to a preset value range.

[0125] An adaptive interpolation algorithm is used to resample the image pixels during the adjustment process to maintain the visual quality of the image content.

[0126] The size-normalized standardized image is output.

[0127] In this embodiment, the image preprocessing module performs size normalization processing to uniformly adjust document images of different sources and sizes to meet the input requirements of the multi-modal large model. The original width parameter and the original height parameter are obtained by the image decoder, representing the initial size information of the input image. The original aspect ratio is calculated by the ratio of the width to the height, and is used to maintain the geometric proportion of the image content unchanged during the size adjustment process.

[0128] The integer factor is an integer divisor preset according to the architecture characteristics of the multi-modal large model, which is usually an integer multiple of the model patch size. By adjusting the image width and height to a size that can be divided by the integer factor, it is ensured that the model will not have boundary padding or information truncation problems during processing, improving the accuracy and efficiency of model inference.

[0129] The preset numerical range is set according to model calculation complexity and display memory limitations, the minimum number of pixels ensures that image details can be identified, and the maximum number of pixels avoids overloading of computing resources. Under the constraint of maintaining the original aspect ratio, the total number of image pixels is adjusted to be within the range through equal scaling, achieving a balance between calculation efficiency and identification accuracy.

[0130] The adaptive interpolation algorithm dynamically selects the most suitable resampling method according to the scaling ratio, such as bilinear interpolation for moderate scaling and Lanczos algorithm for larger scaling. The adaptive interpolation algorithm calculates the pixel neighborhood by weighting, maximizing the edge sharpness and overall visual quality of the seal text during the resizing process.

[0131] The standardized image is the final output of the size standardization process, with uniform size specifications, maintaining the original ratio, moderate total number of pixels, and optimized visual quality, providing standardized input data for subsequent multi-modal large model accurate inference.

[0132] This embodiment solves the problem of inconsistent model input caused by different source document image sizes through systematic size standardization processing. The size adjustment based on the integer factor ensures the efficiency of model processing, maintaining the aspect ratio and adaptive resampling ensures the image quality, and the preset pixel range optimizes the use of computing resources. This preprocessing method provides standardized and high-quality input for multi-modal large models, laying a solid foundation for subsequent seal detection and recognition.

[0133] In some embodiments, the multi-modal large model performs end-to-end inference on multi-modal input data, outputting a recognition result string containing at least one seal object, including:

[0134] An iterative inference mechanism is used to process multi-modal input data. First, a seal area positioning operation is performed to generate a set of potential regions;

[0135] A seal type discrimination operation is performed on each region in the set of potential regions to filter out target regions confirmed as seals;

[0136] A text recognition operation is performed on each target region to generate corresponding text content and confidence evaluation results;

[0137] The bounding box coordinates, type discrimination results, text content, and confidence evaluation results of the target regions are combined into structured data;

[0138] The structured data is converted into a recognition result string conforming to the JSON format specification and output.

[0139] In this embodiment, the end-to-end inference by the multi-modal large model is implemented by an iterative mechanism within the model. This mechanism breaks down the complex recognition task into multiple logical steps for sequential execution, but to the outside system, it still appears as a one-time input-output process. In the specific implementation steps, first, the visual attention mechanism is used to locate all possible seal areas in the image, generating a potential region set containing position information, effectively avoiding the problem of missing targets in single inference, and improving the recall rate of seal detection; the seal area positioning operation uses the visual encoder of the model to extract image features, and identifies candidate regions that may contain seals through a region proposal network or an attention heat map; the potential region set, as the preliminary detection result, provides candidate targets for subsequent fine processing, ensuring that potential seal objects are not missed.

[0140] The seal type identification operation classifies each candidate region in a fine-grained manner, distinguishes real seals from interference (such as circular icons, decorative patterns, etc.) by analyzing the visual features within the region, filters out the target regions confirmed as seals, effectively eliminates false positives, and improves recognition accuracy.

[0141] The text recognition operation uses the text decoding capability of the model to extract seal text content for each confirmed seal region, while generating confidence assessment results based on image clarity, text completeness, and other visual features, ensuring the accuracy of text recognition and quantitatively evaluating the reliability of recognition.

[0142] The structured data combination integrates the position information (bounding box coordinates), category information (type discrimination results), text information (text content), and reliability information (confidence assessment results) of the target region into a unified data structure, providing a complete information foundation for subsequent format conversion.

[0143] The JSON format conversion serializes the structured data into a string format according to the predefined specification, ensuring that the output results meet the standard format requirements for machine readability, facilitating direct parsing and processing by downstream systems.

[0144] This embodiment decomposes the complex seal recognition task into three hierarchical steps of positioning, classification, and recognition through the iterative inference mechanism, significantly improving processing accuracy and reliability. The step-by-step filtering mechanism effectively reduces the false positive rate, multi-dimensional information integration ensures the completeness of the output data, and standardized format conversion guarantees the usability of the results. This inference method not only maintains the efficiency advantage of end-to-end processing, but also achieves more accurate recognition results through step-by-step optimization, especially suitable for seal detection and recognition tasks that have specific shape and text characteristics.

[0145] In some embodiments, the recognition result string is parsed to extract the bounding box coordinates, type label, text content, and text recognition confidence of each seal object, including:

[0146] The JSON parsing algorithm is used to perform syntax analysis and structure verification on the recognition result string to obtain a data structure.

[0147] The seal objects are extracted from the data structure, each containing a bounding box coordinate array, a type label string, a text content string, and a text recognition confidence level.

[0148] The seal objects are verified, including:

[0149] The bounding box coordinate array is verified for coordinate format and numerical range to ensure that the coordinate values are within the valid image range.

[0150] The text recognition confidence level is enumerated and verified to confirm that it belongs to the preset confidence level set.

[0151] The verification results are converted into the final parsing results, which include the text recognition confidence.

[0152] In this embodiment, the JSON parsing algorithm is a processing method for syntax analysis and structure verification of the recognition result string. The string is converted into a token sequence by a lexical analyzer, and an abstract syntax tree is constructed by a syntax parser to finally generate a data structure that can be directly operated by a program. This parsing method ensures accurate extraction of structured information in the string, providing a reliable basis for subsequent data extraction.

[0153] The data structure refers to the tree or object form of data organization formed after parsing, which contains complete seal information hierarchical relationships. When extracting seal objects from this structure, each object contains a bounding box coordinate array, a type label string, a text content string, and a text recognition confidence level. The bounding box coordinate array represents the seal position, the type label string identifies the seal category, the text content string records the recognized text, and the text recognition confidence level assesses the recognition reliability. This structured extraction ensures the integrity and accuracy of the information.

[0154] The verification of the seal objects includes coordinate format verification and numerical range checking to ensure that the bounding box coordinates are within the image size range and are correctly formatted. Enumerated value verification is also performed to confirm that the text recognition confidence level belongs to the preset set. The verification process is implemented through a rule engine, effectively filtering abnormal data and ensuring the reliability of subsequent processing.

[0155] The final parsing result converts the verified seal object information into a standardized data format, with the text recognition confidence as a key field. This result provides a reliable input for subsequent hierarchical decision processing, ensuring the data quality of the entire recognition process.

[0156] This embodiment ensures the accuracy and reliability of the conversion process from model output to structured data through strict JSON parsing and multiple verification mechanisms. Syntax analysis and structure verification prevent parsing errors, and coordinate and confidence verification filters abnormal data. The final standardized parsing result provides a high-quality data basis for subsequent processing, effectively improving the stability of the entire system and the reliability of the output results, especially suitable for high-accuracy data recognition applications.

[0157] In some embodiments, each seal object is processed according to the text recognition confidence to generate the final structured recognition result and output, including:

[0158] Based on the pre-set confidence level rules, the text recognition confidence of each seal object is divided into one of high, medium, and low levels.

[0159] For seal objects with high text recognition confidence, an automated comparison process is performed to match their text content with a pre-stored agency name database, generating a consistency verification result, which is the automated comparison result.

[0160] For seal objects with medium or low text recognition confidence, an artificial review marking process is triggered, and the location information of the corresponding seal area in the original image is marked as a marking result.

[0161] According to the automated comparison result and the marking result, a structured recognition result is generated, including verification status and confidence level.

[0162] The structured recognition result is serialized and output according to the pre-set output format.

[0163] In this embodiment, the pre-set confidence level rules refer to the standard specification for dividing seal objects into different processing levels according to the text recognition confidence level. This rule divides the confidence level into three levels: high, medium, and low, corresponding to different subsequent processing processes. High level represents clear and complete; medium level represents partial blur / defect, with most recognizable; low level represents high blur / defect, difficult to recognize complete content. This grading mechanism realizes differentiated processing strategies based on confidence, ensuring optimal resource allocation and processing efficiency.

[0164] The automatic comparison process is an automatic verification process performed on high-confidence seal objects by accurately matching the recognized text content with a pre-stored agency name database. This process uses a string similarity algorithm for comparison, generating consistency verification results to provide further verification assurance for high-confidence recognition results.

[0165] The manual review marking process is a manual intervention mechanism activated for medium and low-confidence seal objects, which generates a review mark by marking the location information of the corresponding seal area in the original image. This process ensures that low-quality recognition results can be manually reviewed to avoid errors caused by automatic processing.

[0166] The structured recognition result is the final output data generated after integrating the automatic comparison result and the manual review mark, containing the verification status and confidence level information of each seal object. This result uses a unified data structure to provide complete and reliable decision-making basis for subsequent business processing.

[0167] Serialization processing is the process of converting structured recognition results into standardized data formats according to pre-set output formats, ensuring that output results can be directly parsed and used by downstream systems. This processing method ensures the standardization of data exchange and the convenience of system integration.

[0168] This embodiment realizes intelligent processing flow distribution through a confidence level classification mechanism. High-confidence results use automatic comparison to ensure efficiency, medium-confidence results and low-confidence results introduce manual review to ensure accuracy. This hierarchical decision-making processing method not only takes full advantage of the efficiency of automated processing, but also effectively controls risks through manual intervention. The final structured recognition result contains both original recognition information and verification status information, providing a more comprehensive and reliable data foundation for subsequent business applications, ensuring processing efficiency while maximizing the accuracy of recognition results.

[0169] In some embodiments, the method further comprises:

[0170] Constructing new training samples according to manual review results or historical matching results;

[0171] Using new training samples for online incremental learning of the multi-modal large model.

[0172] In this embodiment, the manual review result refers to the correct recognition result confirmed through manual review, and the historical matching result refers to the accurate matching data verified in the system history record. These results serve as high-quality labeled data for constructing new training samples, providing real and reliable training materials for continuous model optimization.

[0173] Online incremental learning refers to the process of fine-tuning and updating the knowledge of a multi-modal large model using new training samples without retraining the entire model. By incrementally updating the model parameters, the model can continuously learn new knowledge and patterns, improving recognition accuracy and adaptability. Preferably, the online incremental learning adopts a continuous learning strategy to effectively avoid catastrophic forgetting. Specifically, it can be implemented through a method based on a replay buffer, which mixes a portion of old training samples with new samples to train together when updating the model. Alternatively, the Elastic Weight Consolidation (EWC) algorithm can be used, which calculates the importance of old task parameters to protect parameters critical to old tasks when optimizing new tasks, thereby maximizing the preservation of original performance while absorbing new knowledge.

[0174] The embodiment realizes the self-evolution ability of the system by constructing a feedback closed loop, converts high-quality recognition results generated in actual business into training data, and continuously optimizes the model performance through online incremental learning. This method effectively solves the problems of data distribution changes and new scene adaptation that the model may encounter in actual application, ensuring that the system can continuously maintain excellent recognition performance, while reducing the cost of continuous labeling and retraining.

[0175] Please refer to Figure 5 In a second aspect, the embodiment also provides a seal detection and recognition system 1 based on a multi-modal large model, which is suitable for the method of the first aspect. The system includes an image input module 11, an image preprocessing module 12, an instruction generation module 13, a multi-modal large model inference module 14, a result analysis module 15, and a decision output module 16. The image input module 11 is used to obtain document image data to be processed. The image preprocessing module 12 is used to perform size normalization processing on the input document image data to generate standardized images that meet the input specifications of the multi-modal large model. The instruction generation module 13 is used to generate pre-defined structured recognition instructions. The multi-modal large model inference module 14 is used to receive multi-modal input data composed of standardized images and structured recognition instructions, and perform end-to-end inference to output recognition result strings that meet the JSON format specifications. The result analysis module 15 is used to analyze the recognition result strings and extract the bounding box coordinates, type labels, text content, and text recognition confidence of each seal object. The decision output module 16 is used to perform hierarchical decision processing on each seal object based on the text recognition confidence, generate the final structured recognition result, and output it.

[0176] In this embodiment, the image input module 11 is used to receive document image data in various formats, including scanned documents and electronic files, providing the system with raw input data; the image preprocessing module 12 processes the input image through a size normalization algorithm to generate a standardized image that meets the input requirements of the multi-modal large model; the instruction generation module 13 automatically generates structured recognition instructions to guide the model to perform specific seal recognition tasks.

[0177] The multi-modal large model inference module 14 is the core processing unit of the system, which receives multi-modal input data composed of images and instructions, and outputs a recognition result string in JSON format through end-to-end inference; the result analysis module 15 analyzes and verifies the output string to extract accurate seal information data; the decision output module 16 implements hierarchical decision processing according to the confidence level to generate the final structured recognition result.

[0178] The system realizes a complete seal detection and recognition process through modular design, and the modules work together to form an efficient processing pipeline. Image input and preprocessing ensure data standardization, multi-modal large models provide powerful recognition capabilities, and result analysis and decision output ensure the accuracy and usability of the results. The system deeply integrates advanced multi-modal artificial intelligence technology with seal recognition business needs, realizes end-to-end automated processing from image input to structured output, significantly improves the efficiency and accuracy of seal recognition, and provides a reliable seal recognition solution for various business scenarios.

[0179] By adopting the above technical scheme, the present application is different from the prior art and has the following beneficial effects: the seal detection and recognition method and system based on a multi-modal large model effectively solve the problems of low accuracy, non-standard output format, and poor adaptability to blurred and incomplete seals in the prior art. The method realizes input data standardization through an image preprocessing module and uses a multi-modal large model trained through fine-tuning for end-to-end inference, outputting structured recognition results strictly conforming to the JSON format specification. The system uses an iterative inference mechanism to improve detection accuracy and realizes intelligent process shunting through confidence level hierarchical decision processing, ensuring that high-confidence results are automatically processed and low-confidence results are manually reviewed. At the same time, data synthesis based on physical simulation and preference optimization training ensure the format standardization and content accuracy of the model output. The present application realizes complete automated processing from image input to structured output, significantly improves the accuracy and reliability of seal recognition, and provides an efficient and reliable seal recognition solution for financial, tax, and other business scenarios. The system also has online incremental learning capability, which can continuously optimize model performance and adapt to changing business needs.

[0180] Finally, it should be noted that the above embodiments have been described in the specification and drawings of the present application, but this does not limit the patent protection scope of the present application. Any equivalent structure or equivalent process replacement or modification based on the essential concept of the present application, using the content described in the specification and drawings of the present application, and directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, etc., are all included in the patent protection scope of the present application.

Claims

1. A multi-modal large model-based seal detection and identification method, characterized in that, The method comprises: acquiring document image data to be processed and inputting the document image data to an image preprocessing module; performing size normalization processing on the input document image data by the image preprocessing module to generate a standardized image conforming to the input specification of a multimodal large model; combining the standardized image with a predefined structured recognition instruction into multimodal input data and inputting the multimodal input data to a fine-tuned multimodal large model; performing end-to-end inference on the multimodal input data by the multimodal large model to output a recognition result string containing at least one seal object, the recognition result string conforming to a predefined JSON format specification; parsing the recognition result string to extract the bounding box coordinates, type label, text content and text recognition confidence of each seal object; performing hierarchical decision processing on each seal object according to the text recognition confidence to generate a final structured recognition result and output the final structured recognition result.

2. The multi-modal large model-based seal detection and identification method according to claim 1, wherein, The fine-tuned multimodal large model is configured to be obtained by the following steps: constructing a multimodal training data set containing seal images, corresponding bounding box coordinates, type labels, text content and text recognition confidence; using the multimodal training data set to fine-tune the multimodal large model as a base model; using a reinforcement learning algorithm based on preference optimization to perform alignment training on the fine-tuned model to improve the format specification and content accuracy of the output result.

3. The multi-modal large model-based seal detection and identification method according to claim 2, characterized in that, Constructing a multimodal training data set containing seal images, corresponding bounding box coordinates, type labels, text content and text recognition confidence, comprises: generating training seal images by a data synthesis engine based on physical simulation, the data synthesis engine being configured to simulate ink distribution characteristics generated under different stamping pressures, angles and medium conditions; automatically generating corresponding text recognition confidence labels according to the occlusion parameters and blur parameters applied during the synthesis process; combining the synthesized seal images with the corresponding bounding box coordinates, type labels, text content and text recognition confidence labels into the multimodal training data set.

4. The multi-modal large model-based seal detection and identification method according to claim 2, characterized in that, Using a reinforcement learning algorithm based on preference optimization to perform alignment training on the fine-tuned model to improve the format specification and content accuracy of the output result, comprises: constructing a format specification preference data set, the format specification preference data set containing correct format responses as preferred samples and incorrect format responses as less preferred samples; constructing a content accuracy preference data set, the content accuracy preference data set containing responses that correctly express recognition uncertainty as preferred samples and responses that are overconfident or incorrectly derived as less preferred samples; using the format specification preference data set and the content accuracy preference data set to perform multi-objective reinforcement learning training on the fine-tuned model by a direct preference optimization algorithm; and, during the training process, optimizing the preference objective function to enable the multimodal large model to simultaneously learn the dual constraints of output format specification and content accuracy.

5. The multi-modal large model-based seal detection and identification method according to claim 1, wherein, Performing size normalization processing on the input document image data by the image preprocessing module to generate a standardized image conforming to the input specification of a multimodal large model, comprises: Performing size analysis on the input document image data to obtain an original width parameter and an original height parameter of the original image, and obtaining an original aspect ratio; Adjusting the image size of the original image based on a preset integer factor, so that the adjusted image width parameter and the image height parameter can be divided by the integer factor; Adjusting the total number of image pixels to be within a preset numerical range while maintaining the constraint of the original aspect ratio; Using an adaptive interpolation algorithm to resample the image pixels during the adjustment process to maintain the visual quality of the image content; Outputting the standardized image after size standardization.

6. The multi-modal large model-based seal detection and identification method according to claim 1, wherein, The multi-modal large model performs end-to-end inference on the multi-modal input data, and outputs an identification result string containing at least one seal object, including: Using an iterative inference mechanism to process the multi-modal input data, first performing a seal area positioning operation to generate a set of potential regions; Performing a seal type discrimination operation on each region in the set of potential regions to filter out target regions confirmed as seals; Performing a text recognition operation on each target region to generate corresponding text content and confidence evaluation results; Combining the bounding box coordinates, type discrimination results, text content, and confidence evaluation results of the target regions into structured data; Converting the structured data into an identification result string that conforms to the JSON format specification and outputting it.

7. The multi-modal large model-based seal detection and identification method according to claim 1, wherein, Parsing the identification result string to extract the bounding box coordinates, type label, text content, and text recognition confidence of each seal object, including: Using a JSON parsing algorithm to perform syntax analysis and structure verification on the identification result string to obtain a data structure; Extracting seal objects from the data structure, each seal object containing a bounding box coordinate array, a type label string, a text content string, and a text recognition confidence level; Verifying the seal objects, including: Performing coordinate format verification and numerical range checking on the bounding box coordinate array to ensure that the coordinate values are within the valid image range; Performing enumeration value verification on the text recognition confidence level to confirm that it belongs to the preset confidence level set; Converting the verification results into a final parsing result, which includes the text recognition confidence.

8. The multi-modal large model-based seal detection and identification method according to claim 1, wherein, Performing hierarchical decision processing on each seal object based on the text recognition confidence to generate and output a final structured identification result, including: Based on the preset confidence grading rules, the text recognition confidence of each seal object is divided into one of the high, medium, and low levels; For seal objects with high text recognition confidence, perform an automated comparison process to match their text content with a pre-stored institution name database, generating a consistency verification result, which is the automated comparison result; For seal objects with medium or low text recognition confidence, trigger a manual review marking process and mark the location information of the corresponding seal region in the original image as a marking result; Based on the automated comparison result and the marking result, generate a structured identification result containing the verification status and confidence level; Serializing and outputting the structured identification result according to the preset output format. 9.The multi-modal large model based stamp detection and identification method according to claim 1, wherein, The method further includes: According to the manual review result or the historical matching result, a new training sample is constructed; The new training sample is used for online incremental learning of the multi-modal large model.

10. A multi-modal large model based seal detection and identification system, characterized in that, The system is suitable for the method in any one of claims 1 to 9, and the system comprises: An image input module configured to acquire document image data to be processed; An image preprocessing module configured to perform size normalization processing on the input document image data to generate normalized images conforming to input specifications of the multi-modal large model; An instruction generation module configured to generate pre-defined structured recognition instructions; A multi-modal large model inference module configured to receive multi-modal input data composed of the normalized images and the structured recognition instructions, and perform end-to-end inference to output a recognition result string conforming to a JSON format specification; A result analysis module configured to analyze the recognition result string to extract a bounding box coordinate, a type label, text content, and text recognition confidence of each seal object; A decision output module configured to perform hierarchical decision processing on each seal object according to the text recognition confidence, generate a final structured recognition result, and output the final structured recognition result.

Images