Multi-bill intelligent segmentation identification method and system

Abstract

The invention discloses a multi-bill intelligent segmentation identification method and system, and relates to the technical field of computer vision and artificial intelligence. The method comprises the following steps: acquiring a mixed pasting image and prompt information; visual features of the mixed pasting image are extracted and fused with prompt information, a fusion prompt vector is generated, and a preliminary segmentation mask is generated based on the fusion prompt vector; geometric topology analysis and adaptive optimization are carried out on the preliminary segmentation mask to obtain an accurate segmentation mask, so that a single bill image is segmented; after each single bill image is enhanced, a dual-path OCR collaborative recognition framework is adopted for recognition, and a field recognition result is obtained; performing logic verification on the identification result, and determining a risk score according to verification and a subsequent anomaly detection result of the single bill image; and finally outputting a segmentation identification result containing the single bill image, the field identification result and the risk score. According to the method, accurate segmentation can be realized through learning prompt fusion and adaptive mask optimization.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method and system for intelligent segmentation and recognition of multiple tickets. Background Technology

[0002] In medical insurance claims, to improve the convenience of document submission, users often paste multiple medical invoices, expense lists, and other different types of receipts onto a single sheet of paper and upload them as electronic images by taking photos or scanning them. However, existing optical character recognition (OCR) technologies and image processing methods have significant technical limitations and processing bottlenecks when dealing with such mixed receipt images.

[0003] In the invoice segmentation stage, due to the complexities often present in mixed invoices, such as overlapping pasting, paper wrinkles, and shadows from photography, the boundary detection accuracy of traditional rule-based or ordinary image segmentation algorithms drops significantly, making it difficult to achieve pixel-level accurate separation. While deep learning-based segmentation methods have some effectiveness, they typically rely on large amounts of labeled data. The diverse types and complex formats of medical invoices lead to high annotation costs and limited model generalization ability. Furthermore, the fusion of prompt information in existing methods is relatively simple and fails to fully incorporate semantic priors related to the business scenario, affecting the robustness and adaptability of the segmentation.

[0004] During the invoice recognition stage, medical invoices typically include various formats such as printed text, handwritten text, stamped areas, and thermally printed text, often exhibiting issues like blurry fonts, low resolution, and localized glare. General-purpose OCR systems lack specific optimizations for the format and field structure of medical invoices, resulting in limited accuracy in recognizing handwritten content and blurry handwriting, making it difficult to meet the high-precision requirements for extracting key information in claims processing.

[0005] In terms of verification, existing processes often handle segmentation, identification, and anomaly detection independently, lacking a closed-loop collaborative mechanism. The system cannot automatically verify the logical consistency of invoice content, such as the relevance of amounts, consecutive invoice numbers, and the reasonableness of appointment times. It also lacks comprehensive detection capabilities for anomalies such as image tampering, blurring, damage, and the authenticity of seals. This makes claims verification still highly dependent on manual review, with low automation and difficulty in improving processing efficiency.

[0006] In addition, existing methods mostly adopt image post-processing strategies with fixed parameters, which cannot adaptively cope with changes in actual scenes such as different lighting conditions, folding deformation, and shooting angles, resulting in unstable overall performance in complex real-world environments. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a multi-document intelligent segmentation and recognition method and system, which can achieve accurate segmentation through learnable prompt fusion and adaptive mask optimization, and combine dual-path OCR collaboration and closed-loop multi-task processing to complete efficient and automated processing from extraction, recognition, verification to risk assessment of mixed document images.

[0008] To achieve the above objectives, the present invention provides a multi-document intelligent segmentation and recognition method, comprising: Acquire a mixed image containing multiple receipts and corresponding prompts; the prompts include user text prompts and pre-trained business semantic prompts. Visual features are extracted from the composite image, and the visual features are fused with the prompt information to generate a fused prompt vector; Based on the fused cue vector and visual features, a preliminary segmentation mask is generated; Geometric topological analysis is performed on the preliminary segmentation mask, and mask optimization is performed based on the analysis results to obtain the accurate segmentation mask; Based on the precise segmentation mask, multiple single ticket images are segmented from the mixed image; Image enhancement processing is performed on each of the single ticket images, and then optical character recognition is performed using a dual-path OCR collaborative recognition framework to obtain field recognition results; By performing logical verification on the field recognition results, the final single ticket image and the final field recognition results are determined, and the logical verification results are obtained at the same time. If an abnormal region is found during the logical verification process, a re-segmentation and / or re-recognition operation is performed on the abnormal region to update the single ticket image and / or field recognition results. Anomaly detection is performed on the final single ticket image to obtain the anomaly detection results; Based on the logical verification results and anomaly detection results, a risk score is determined; Generate and output the final single-ticket image, the final field recognition result, and the segmentation and recognition result of the risk score.

[0009] Optionally, re-segmentation and / or re-identification operations are performed on the abnormal regions to update the single ticket image and / or field recognition results, including: Based on the anomaly type corresponding to the anomaly region, it is determined to perform a re-segmentation operation and / or a re-identification operation; the anomaly type is a segmentation anomaly or an identification anomaly. If the exception type is a segmentation exception, then perform a resegmentation operation: Based on the location information of the abnormal region in the composite image, the fusion parameters used to generate the fusion prompt vector are adjusted; Based on the adjusted fusion parameters, local visual features at corresponding positions in the mixed image are extracted, and the local visual features are fused with the adjusted prompt information to generate a local fusion prompt vector. Based on the local fusion cue vector and the local visual features, a new preliminary segmentation mask for the abnormal region is generated; Perform mask optimization on the new preliminary segmentation mask to obtain a new precise segmentation mask for the abnormal region; The new precise segmentation mask is used to replace or correct the corresponding part in the original precise segmentation mask to generate an updated single ticket image. Image enhancement processing is performed on the updated single ticket image, and then optical character recognition is performed using the dual-path OCR collaborative recognition framework to obtain the updated field recognition result. If the anomaly type is an identification anomaly, then a re-identification operation is performed: Extract the image block corresponding to the abnormal region from the single ticket image; Image enhancement processing is performed on the image block, and then optical character recognition is performed using the dual-path OCR collaborative recognition framework to obtain the text content of the new field. The corresponding part of the original field recognition result is replaced with the new field text content to form the updated field recognition result.

[0010] Optionally, the analysis results include the coordinates of the four corner points of the ticket, the geometric distortion measure characterizing the degree of boundary deformation, the shadow discrimination information characterizing whether there is a boundary shadow, and the overlap discrimination information characterizing whether there is regional overlap. Based on the analysis results, mask optimization is performed, including: Based on the geometric distortion metric, one of the preset geometric correction strategies is dynamically selected, and the boundary of the preliminary segmentation mask is geometrically corrected according to the coordinates of the four corner points of the ticket. If the shadow discrimination information indicates the presence of a boundary shadow, then the local area where the boundary is located is subjected to contrast enhancement and edge repair processing. If the overlap discrimination information indicates that there is regional overlap, then image segmentation processing is performed on the overlapping ticket regions to separate the different ticket regions.

[0011] Optionally, the method further includes: Calculate the intersection-union ratio (IU / U) between the optimized mask obtained after mask optimization and the initial segmentation mask; If the intersection-union ratio is lower than a preset threshold, the fusion cue vector is updated based on the visual features of the optimized mask and the mixed image. Based on the updated fusion hint vector, the initial segmentation mask is regenerated, and the geometric topology analysis and mask optimization are performed again. The above steps are executed iteratively until the intersection-union ratio between the optimized mask obtained in the current round and the corresponding preliminary segmentation mask reaches or exceeds the preset threshold, and the current optimized mask is used as the accurate segmentation mask.

[0012] Optionally, image enhancement processing is performed on each of the single ticket images, including: For each of the aforementioned single ticket images, perform the following steps: The layout of the single ticket image is recognized to determine the corresponding ticket category; Based on the ticket type and the business semantic prompts, dynamically select one enhancement strategy from multiple preset image enhancement strategies; The selected enhancement strategy is used to enhance the image of the single ticket.

[0013] Optionally, the dual-path OCR collaborative recognition framework includes a main recognition path and a secondary recognition path; optical character recognition is performed using the dual-path OCR collaborative recognition framework to obtain field recognition results, including: The image enhancement process of the single ticket image is globally recognized through the main recognition path to obtain the first recognition result and the corresponding first confidence level of each image region. The auxiliary recognition path is used to enhance the recognition of image regions where the first confidence level is lower than a preset value, thereby obtaining a second recognition result and a corresponding second confidence level. Based on the first confidence level, the second confidence level, and business semantic prompts, the first recognition result and the second recognition result are dynamically fused to generate the field recognition result.

[0014] Optionally, before employing a dual-path OCR collaborative recognition framework for optical character recognition, the method further includes: The image of a single invoice after image enhancement is used to detect key field regions and locate the bounding box position information of the key fields; the key fields include at least one of amount, date, name and invoice code; the bounding box position information is used to provide attention focus for the dual-path OCR collaborative recognition framework.

[0015] Optionally, the logical verification includes at least one of intra-field verification, inter-field logical verification, and cross-document logical verification.

[0016] Optionally, the anomaly detection includes at least one of the following: tampering trace detection, blur and defect detection, and seal authenticity detection.

[0017] This invention also provides a multi-document intelligent segmentation and recognition system, comprising: The information acquisition unit is used to acquire a mixed image containing multiple tickets and corresponding prompt information; the prompt information includes user text prompts and pre-trained business semantic prompts. The fusion unit is used to extract visual features from the mixed images and fuse the visual features with the prompt information to generate a fused prompt vector; Segmentation unit, used for: Based on the fused cue vector and visual features, a preliminary segmentation mask is generated; Geometric topological analysis is performed on the preliminary segmentation mask, and mask optimization is performed based on the analysis results to obtain the accurate segmentation mask; Based on the precise segmentation mask, multiple single ticket images are segmented from the mixed image; The recognition unit is used to perform image enhancement processing on each of the single ticket images, and then use a dual-path OCR collaborative recognition framework to perform optical character recognition to obtain field recognition results. The logic verification unit is used to perform logic verification on the field recognition results to determine the final single ticket image and the final field recognition results, and to obtain the logic verification results. During the logic verification process, if an abnormal region is found, the abnormal region is re-segmented and / or re-recognized to update the single ticket image and / or field recognition results. An anomaly detection unit is used to perform anomaly detection on the final single ticket image and obtain anomaly detection results; The risk assessment unit is used to determine a risk score based on the logic verification results and anomaly detection results. The result generation and output unit is used to generate and output the final single ticket image, the final field recognition result, and the segmentation and recognition result of the risk score.

[0018] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: The intelligent segmentation and recognition method for multiple invoices provided by this invention introduces multi-source prompting information, including user text prompts and pre-trained business semantic prompts, and deeply fuses them with image visual features to generate a guiding vector. This effectively integrates semantic prior knowledge of the business scenario into the segmentation process, solving the problem that existing technologies use prompting information in a simple way and fail to fully integrate business semantics. This significantly improves the understanding and adaptability of the segmentation model in specific scenarios such as medical invoices. By generating a preliminary segmentation mask based on the fused prompting vector and further performing adaptive mask optimization combined with geometric topology analysis, dynamic, pixel-level boundary correction and repair can be performed on complex situations such as wrinkles, shadows, and overlaps commonly found on invoices. This eliminates the need for post-processing with fixed parameters, effectively solving the problem of insufficient boundary segmentation accuracy caused by physical deformation of invoices and shooting interference, and achieving high-precision segmentation of single invoice images from mixed images.

[0019] By enhancing the segmented single-document images and employing a dual-path OCR collaborative recognition framework for optical character recognition, the enhancement process improves image quality. The dual-path collaborative mechanism balances overall efficiency with focused recognition of challenging areas, thereby enhancing the robustness and accuracy of recognizing various forms of text, including printed, handwritten, and blurred fonts. Crucially, this invention performs logical verification of the recognition results and anomaly detection on the document images. Based on the anomalies detected during verification and detection, it triggers a closed-loop optimization operation for re-segmenting or re-recognizing related regions. For the first time, this invention organically integrates multiple stages such as segmentation, recognition, verification, and detection into a collaborative system with self-feedback and correction capabilities. This breaks down the barriers of independent operation and lack of linkage in existing technologies, achieving automated verification and closed-loop processing of document logical consistency and image authenticity, significantly reducing the need for manual review. Ultimately, it can output complete results including segmented images, structured recognition results, and comprehensive risk scores, providing clear decision support. This enables a high-precision and highly automated process from the submission of mixed invoice images to the generation of intelligent review conclusions, significantly improving the processing efficiency and reliability of medical insurance claims and other business operations. Attached Figure Description

[0020] The above and other objects, features and advantages of the present invention will become more apparent from the more detailed description of exemplary embodiments of the invention in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same parts.

[0021] Figure 1 This is a schematic diagram of the method flow of the intelligent segmentation and recognition method for multiple tickets shown in an embodiment of the present invention; Figure 2 The flowchart illustrates the adaptive mask optimization algorithm according to an embodiment of the present invention. Figure 3 This is a comparison chart of the processing effects in actual application scenarios shown in the embodiments of the present invention; Figure 4 This is a microservice architecture diagram of a multi-ticket intelligent segmentation and recognition system shown in an embodiment of the present invention; Figure 5 This is a schematic diagram of the module structure of a multi-document intelligent segmentation and recognition system according to an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figure 1 , Figure 1 This is a flowchart illustrating the method for intelligent segmentation and recognition of multiple invoices.

[0024] A multi-document intelligent segmentation and recognition method includes: S101: Obtain a mixed image containing multiple tickets and corresponding prompts.

[0025] The prompts include user text prompts and pre-trained business semantic prompts.

[0026] In practical medical insurance claims applications, to improve the convenience of document submission, users often paste multiple medical invoices, expense lists, diagnostic certificates, and other different types of documents onto a single sheet of paper and upload them as electronic images by taking photos or scanning them. Therefore, the input data acquired by this invention is the mixed image containing multiple documents. Depending on actual business needs and hardware processing capabilities, the file format of this mixed image can be, but is not limited to, JPG, PNG, or PDF formats, with an image resolution typically between 300 dpi and 1200 dpi, and the data size of a single image generally not exceeding 10 MB. Furthermore, this method supports batch acquisition and processing of data; for example, up to 50 mixed images can be acquired in a single batch to meet the needs of large-scale claims review.

[0027] While acquiring the mixed-image data, this invention also simultaneously acquires prompting information corresponding to the image and subsequent processing tasks. This prompting information is a key element guiding subsequent accurate segmentation and recognition, and mainly includes two parts: user text prompts and pre-trained business semantic prompts. By combining these two different dimensions of prompting information, the technical problems of existing technologies—such as simple prompt fusion methods, insufficient integration of business rule semantics, and lack of adaptability to the special format of medical invoices—can be effectively solved.

[0028] In this context, user text prompts refer to descriptive text instructions entered by users when submitting mixed images, based on their actual needs. To transform the user-input natural language into a computer-understandable vector form, this invention utilizes a text prompt encoder to process the user text prompts. For example, this text prompt encoder can employ a bidirectional Transformer structure, capable of bidirectional context modeling of the input text sequence, thereby capturing deep semantic dependencies. In the specific processing, the user-input text instruction (e.g., segmenting the top-left corner of an outpatient invoice) first undergoes word segmentation (e.g., BERT segmentation) to generate a corresponding text sequence. This text sequence is typically limited to a certain maximum length (e.g., a maximum length of 64) to ensure computational efficiency and processing speed. Subsequently, this text sequence is input to the text prompt encoder, which, through its internal deep neural network layers, converts the segmented text sequence into a semantically rich vector representation. This process transforms the user's high-level intent into a specific, computable processing-oriented vector, ensuring that subsequent image feature extraction and segmentation decoding accurately respond to the user's specific segmentation or recognition instructions. This method supports incremental prompts via API, allowing users to supplement or correct segmentation and recognition instructions in real time during processing, thereby enhancing the flexibility and accuracy of human-computer interaction.

[0029] The pre-trained business semantic prompts are prior knowledge vectors obtained through self-supervised pre-training on a large amount of medical invoice data. These prompts are generated by a business semantic prompt learner, which learns and masters the unique layout semantic structure and spatial distribution patterns of medical invoices. For example, the learned prior knowledge includes positional relationships such as "the invoice title is usually at the top" and "the amount area is mostly in the lower right corner," as well as key business rule keywords such as "invoice code" and "total amount." During pre-training, a contrastive learning strategy is employed, causing invoices with similar layout features to cluster together in the semantic space, thus forming pluggable business knowledge vectors that can represent the general business rules of medical invoices. This pre-trained business semantic prompt vector can be deeply integrated with user text prompts, providing accurate business semantic guidance for subsequent visual feature extraction, prompt fusion, and precise segmentation. This compensates for the shortcomings of single user text prompts in terms of professional domain knowledge, ensuring that the segmentation and recognition results conform to the business logic of medical claims.

[0030] Furthermore, during the process of obtaining prompts, if the user does not actively input specific user text prompts, this invention will automatically trigger default business semantic prompts, that is, execute the processing logic of segmenting all medical invoices and identifying key fields by default. This mechanism ensures that, without specific user intervention, the method can still automatically and efficiently complete the segmentation and recognition of multiple invoices using pre-trained business knowledge, thereby improving the overall level of automation and robustness.

[0031] S102: Extract visual features from the composite image and fuse the visual features with the prompt information to generate a fused prompt vector.

[0032] After acquiring the composite images and corresponding prompts, visual features are extracted and multimodal fusion is performed to generate a fused prompt vector that can deeply guide subsequent processing. This process aims to unify the pixel-level information of the image, the user's text commands, and internalized business knowledge through a learnable encoding mechanism, thereby solving the problems of superficial use of prompts and disconnection from business scenarios in existing methods, and providing powerful guidance for the entire process that combines visual accuracy and semantic relevance.

[0033] Regarding visual feature extraction, this invention employs a hierarchical visual encoder based on an improved ResNet50 architecture to capture multi-scale features of mixed images. While retaining the powerful feature extraction capabilities of the standard ResNet50 core residual blocks (such as conv1 to conv5_x), this encoder innovatively introduces a local and global feature interaction block. It focuses on capturing fine local details such as document edges and text through parallel convolutional branches with small kernels, while simultaneously utilizing dilated convolutional branches with large receptive fields to understand the overall layout and structural relationships of the document. The features extracted by these two pathways—local and global features—are dynamically fused through a learnable weight allocation module (e.g., by calculating weights using a multilayer perceptron and then performing a weighted sum), ensuring that the output visual feature map retains both discriminative details and rich semantic context.

[0034] To further construct a multi-layered and robust feature representation, this invention builds a multi-scale feature pyramid (FPN) on top of the visual encoder. This pyramid fuses feature layers of different depths (e.g., C2 to C5) from the encoder through top-down paths and lateral connections, generating a series of feature maps with decreasing resolution but progressively enhanced semantic information. Crucially, this invention also introduces a prompt-guided dynamic feature selection mechanism. This mechanism calculates the cosine similarity between the features of each layer in the pyramid and the current task intent (represented by the prompt information to be fused) in real time, and assigns appropriate weights to each layer accordingly. Finally, through weighted fusion, an optimal multi-scale visual feature representation focused on the key regions of the task is obtained.

[0035] Simultaneously, the two types of prompts are encoded into dense semantic vectors. The user-input text prompts are processed by a text prompt encoder based on a bidirectional Transformer structure. This encoder performs bidirectional context modeling on the segmented text sequence, converting it into a text semantic vector that accurately reflects the user's intent. The encoder may also include a gating mechanism to adaptively adjust the contribution strength of text features in subsequent fusion. The other type of input is a pre-trained business semantic prompt, generated offline by a dedicated business semantic prompt learner. This learner typically employs a multilayer perceptron structure and utilizes self-supervised tasks such as contrastive learning and mask reconstruction to learn inherent formatting patterns and spatial priors (such as common positions of invoice titles and the distribution of amount regions) from massive amounts of medical invoice data, thereby forming a pluggable business semantic vector rich in domain knowledge.

[0036] Finally, deep fusion is achieved through cross-modal dynamic alignment and iterative refinement. First, multi-head attention and other mechanisms are used to calibrate features between modalities. The concatenated text and business semantic vectors are used as queries, and multi-scale visual features are used as keys and values ​​for attention calculation, thereby aligning the high-level semantic intent to the specific spatial context of the image. Then, the initially aligned features are integrated with the original visual and cue vectors to form the initial fusion state. This state is fed into a lightweight refinement network for multiple rounds of iterative updates. In each round, the network (e.g., a small Transformer) continuously refines the internal representation of the fusion vector based on the interactions between all input information. After several rounds of refinement, a unified high-dimensional fusion cue vector is output.

[0037] S103: Generate a preliminary segmentation mask based on the fused cue vector and visual features.

[0038] After obtaining the deeply fused cue vectors and multi-scale visual features, the initial segmentation mask generation step is performed. The core of this step is to design an efficient and accurate segmentation decoder that can fully understand the multimodal guidance information and map the abstract semantic intent and specific visual features into pixel-level region segmentation predictions, thereby initially outlining the contours of each ticket in the mixed image.

[0039] To achieve this goal, this invention constructs a segmentation decoder based on a dynamic sparse interactive attention mechanism. This decoder typically consists of a multi-layered structure, with each layer containing multiple attention heads to balance the demands of high-precision segmentation with real-time processing efficiency. The key innovation of the decoder lies in its dynamic sparsity. Specifically, the decoder calculates the activation probability of each attention head in real time based on the currently input fused cue vector rich in task semantics. Only when the activation probability of an attention head exceeds a preset threshold (e.g., 0.5) will that head be activated and participate in the current feature interaction calculation; otherwise, it is temporarily suppressed. This dynamic filtering mechanism driven by cue content allows the decoder to intelligently allocate limited computational resources, focusing on the feature dimensions most critical to the current segmentation task. Statistics show that an average of approximately 75% of attention heads are activated, significantly reducing computational complexity by about 25% while maintaining model expressiveness and improving processing speed. Within the activated attention heads, the fused cue vector serves as the query, and multi-scale visual features simultaneously serve as both keys and values. Through cross-attention calculation, the semantic guidance of the cue on the image spatial region and feature focusing are achieved.

[0040] In the mask generation path, the decoder fully utilizes the multi-scale information provided by the feature pyramid. It works in parallel on visual features at different levels, generating preliminary segmentation mask predictions corresponding to different resolutions. These predictions from different scales each have their advantages: deep, low-resolution predictions focus on the overall semantic integrity and consistency of the document region; shallow, high-resolution predictions focus on capturing fine boundary details and local shapes. Subsequently, this invention integrates these preliminary predictions through a multi-scale mask fusion module. This module first uses a transposed convolution operation to progressively upsample the lower-resolution mask predictions to the size of the original image. Then, based on fixed weights obtained through extensive experimental verification or learning (e.g., assigning weights from high to low to high, medium, and low-resolution features), the mask predictions at all scales are weighted, summed, and fused. This process effectively combines the accuracy of high-level semantics with the precision of low-level details, ultimately outputting a pixel-level preliminary segmentation mask consistent with the spatial size of the input image. This mask provides an initial probability that each pixel in the composite image belongs to the foreground (ticket) or the background, clearly defining the approximate region and boundaries of each ticket, and providing a high-quality and reliable initial input for subsequent geometric topology analysis and adaptive mask optimization.

[0041] S104: Perform geometric topology analysis on the preliminary segmentation mask, and perform mask optimization based on the analysis results to obtain the accurate segmentation mask.

[0042] See Figure 2 After obtaining the initial segmentation mask, mask optimization is performed to refine the preliminary results and eliminate boundary errors caused by factors such as physical deformation of the document, shooting conditions, and mutual occlusion, thereby obtaining a pixel-level accurate segmentation mask. This approach abandons traditional fixed-parameter image post-processing methods and replaces them with an adaptive mask optimization algorithm. Its core lies in conducting in-depth geometric and topological analysis of the initial segmentation mask and dynamically selecting and combining multiple optimization strategies based on the analysis results, effectively solving the boundary segmentation challenges caused by document folds, wrinkles, shadows, and mutual overlap.

[0043] Specifically, the analysis results include the coordinates of the four corner points of the ticket, the geometric distortion measure representing the degree of boundary deformation, the shadow discrimination information representing whether there is a boundary shadow, and the overlap discrimination information representing whether there is regional overlap. Based on the analysis results, mask optimization is performed, including: Based on geometric distortion measurement, one of the preset geometric correction strategies is dynamically selected, and the boundary of the preliminary segmentation mask is geometrically corrected according to the coordinates of the four corner points of the ticket. If the shadow discrimination information indicates the existence of a boundary shadow, then the local area where the boundary is located will be subjected to contrast enhancement and edge repair processing; If the overlap discrimination information indicates that there is regional overlap, then image segmentation processing is performed on the overlapping ticket regions to separate the different ticket regions.

[0044] In this application, the initial segmentation mask is first subjected to geometric topological analysis to extract key structured information. A crucial step is the detection and optimization of the four corner points of the ticket, guided by prompts. This invention utilizes the semantically rich fusion prompt vector generated in the preceding steps to predict the boundary prior coordinates of the four corner points of the ticket using a lightweight multilayer perceptron. To improve the accuracy and robustness of the localization, a random consistency sampling algorithm is further employed for fitting optimization. This algorithm combines the predicted corner point priors with the contour information extracted from the initial segmentation mask, minimizing the distance error between the corner points and the true contours to solve for the optimal and stable four corner point coordinates. Based on this, the geometric distortion metric of the ticket can be calculated, for example, by calculating the ratio of the quadrilateral area to the total image area to quantify the severity of the distortion.

[0045] Simultaneously, the analysis process detects local contrast features in the boundary region to determine the presence of shadow interference; and by analyzing the topological connectivity of the mask region and the consistency of its internal color and texture, it determines whether there are overlapping areas between documents. The above analysis will output a comprehensive result including precise corner coordinates, quantified geometric distortion measurements, shadow discrimination information, and overlap discrimination information.

[0046] Subsequently, based on the aforementioned multi-dimensional analysis results, a series of targeted mask optimization operations are dynamically executed. For geometric deformation, the optimal solution is adaptively selected from a pre-defined correction strategy library based on the calculated geometric distortion metric. If the distortion level is determined to be slight bending (e.g., the metric value is below the threshold of 0.05), a thin-plate spline interpolation algorithm is used for smooth geometric correction. This algorithm utilizes radial basis functions to achieve natural surface deformation recovery. If the distortion is determined to be severe bending or folding, a piecewise polynomial fitting strategy is adopted, for example, using a cubic polynomial to fit and correct each boundary curve to handle more complex local deformations.

[0047] To address image quality issues, if shadow discrimination information indicates shadow interference in the boundary region, the contrast index of that local region is first calculated. Then, a contrast-limited adaptive histogram equalization algorithm is used to enhance the contrast of the affected area to suppress the shadow's influence. Simultaneously, an edge-connection algorithm is used to repair boundary breaks and blurring caused by shadows or low contrast, connecting broken edge segments to obtain a clear and continuous document outline.

[0048] For cases where tickets are stuck together, if the overlap detection information indicates regional overlap, an improved watershed algorithm is activated for fine separation. The innovation of this algorithm lies in its seed point weight calculation method: it not only considers the color similarity and local binary pattern texture features of the overlapping areas, but also innovatively incorporates the matching degree of business semantic prompts. The cosine similarity between local image features and pre-trained business semantic prompt vectors is calculated as the prompt matching degree, and this is weighted and combined with color and texture features to jointly determine the seed point for the watershed transformation. This separation strategy, which incorporates semantic priors, can more accurately distinguish pixel regions with similar colors and textures but belonging to different tickets.

[0049] Furthermore, the above methods also include: Calculate the intersection-union ratio (IU / U) between the optimized mask obtained after mask optimization and the initial segmentation mask; If the intersection-union ratio is lower than the preset threshold, the fusion cue vector is updated based on the visual features of the optimized mask and the mixed image. Based on the updated fusion cue vector, the initial segmentation mask is regenerated, and geometric topology analysis and mask optimization are performed again. The above steps are executed iteratively until the intersection-union ratio between the optimized mask obtained in the current round and the corresponding preliminary segmentation mask reaches or exceeds a preset threshold, and the current optimized mask is used as the accurate segmentation mask.

[0050] To ensure optimization quality and achieve intelligent iteration, this invention introduces a refinement judgment mechanism based on the intersection-union ratio (IU / R) evaluation. After completing one round of optimization, the IU / R between the current optimized mask and the initial preliminary segmentation mask is calculated. If this value is lower than a preset high-precision threshold (e.g., 0.95), it indicates that there is still room for improvement. At this point, the system will calculate and update the fusion cue vector based on the features of the current optimized mask using a cross-attention mechanism. Then, the updated cue vector is used to re-trigger the part of the process from feature extraction to mask optimization for the next round of refinement. This iterative process continues until the IU / R between the currently generated optimized mask and the corresponding preliminary segmentation mask reaches or exceeds the set threshold. At this point, the current optimization result is output as the final accurate segmentation mask.

[0051] S105: Based on a precise segmentation mask, segment multiple single ticket images from a mixed image.

[0052] In practice, the precise segmentation mask output from the preceding steps serves as the authoritative spatial index and extraction blueprint. This mask is a pixel-level labeled map, where each connected component (or a set of pixels with a unique label) precisely corresponds to a ticket in the original composite image. First, connected component analysis or instance decoding is performed on the precise segmentation mask to identify all independent ticket targets and obtain geometric information describing the location and extent of each target, such as the bounding box or a polygonal representation of its precise contour.

[0053] Based on this geometric information, the corresponding image regions are extracted from the original composite image. An efficient and commonly used method is cropping based on the bounding box. To improve robustness and prevent loss of edge information, the cropping range can be slightly expanded beyond the bounding box. A more precise method is to directly utilize the alpha channel of the mask: the region corresponding to a specific ticket in the precisely segmented mask is used as the alpha channel template, and a pixel-level AND operation is performed with the original composite image to directly extract the ticket image with its irregular, realistic shape. The background is then set to transparent or a uniform background color. This method perfectly preserves the true outline of the ticket after geometric correction, including any non-rectangular edges.

[0054] Through the above processing, the originally stuck and overlapping mixed images are deconstructed into a set of clear and independent single ticket images. Each output image contains the complete ticket content from the original image, while minimizing interference from other tickets and complex backgrounds. Because the mask on which the extraction depends has been adaptively optimized for wrinkles, shadows, and overlaps, the resulting single ticket images have accurate boundaries, complete content, and very little noise.

[0055] S106: Perform image enhancement processing on each single ticket image, and then use a dual-path OCR collaborative recognition framework to perform optical character recognition to obtain field recognition results.

[0056] After segmenting the mixed-image data and obtaining a series of independent single-document images, adaptive image enhancement processing is first applied to these images to improve image quality and create optimal conditions for subsequent high-precision character recognition. Then, an innovative dual-path collaborative mechanism is employed for optical character recognition, ultimately outputting structured field recognition results. This process aims to overcome the common quality degradation problem in medical document images and significantly improve the recognition accuracy for complex fonts and handwritten characters.

[0057] Specifically, image enhancement processing is performed on each individual ticket image, including: For each invoice image, perform the following steps: Perform layout recognition on single invoice images to determine the corresponding invoice category; Based on the bill type and business semantics, dynamically select one enhancement strategy from multiple preset image enhancement strategies; The selected enhancement strategy is used to enhance the image of the single ticket.

[0058] Image enhancement processing is a customized process for each individual document image. First, precise document format recognition is performed on the image, a process implemented by a cascaded convolutional neural network. The first level is a coarse classification network, which quickly classifies the input image into broad categories, such as invoices, expense lists, or medical certificates. The second level is a fine classification network, which, based on the features extracted by the coarse classification network, further combines pre-trained "business semantic cues" vectors. Through a "cue semantic injection layer," visual features and semantic priors are fused, thereby accurately identifying more specific document subcategories, such as outpatient invoices, inpatient invoices, or designated pharmacy invoices. This cascaded structure, combined with domain knowledge injection, significantly improves the accuracy of document format classification.

[0059] Based on the accurately identified document category and the domain knowledge implied by "business semantic cues," this invention dynamically selects the optimal processing strategy from a pre-defined image enhancement strategy library. The strategy library contains specialized algorithms designed for different degradation scenarios. For example, for documents identified as "thermal paper invoices," the system triggers a "reflection suppression strategy." This strategy employs an adaptive threshold determination method based on mean and standard deviation, combined with the Retinex algorithm for reflection suppression, and supplemented by grayscale stretching to eliminate highlight areas and enhance overall contrast. For "low-quality scanned documents," a "super-resolution enhancement strategy" is triggered, employing an improved generative adversarial network (such as ESRGAN) generator. Its network structure includes a dedicated module specifically designed for document text features, and it is optimized by combining perceptual loss and adversarial loss, aiming to recover clear text details from blurred images. For regular documents of acceptable quality, a "conventional enhancement strategy" may be selected, such as using a contrast-limited adaptive histogram equalization algorithm to enhance contrast, combined with Gaussian filtering for noise reduction. This context-aware dynamic strategy selection mechanism ensures that each ticket receives the most suitable preprocessing, maximizing the recovery of critical information.

[0060] Furthermore, the aforementioned dual-path OCR collaborative recognition framework includes a main recognition path and a secondary recognition path; using the dual-path OCR collaborative recognition framework for optical character recognition yields field recognition results, including: The image of a single ticket after image enhancement is globally recognized by the main recognition path to obtain the first recognition result and the corresponding first confidence level of each image region. By using an auxiliary recognition path, image regions with a first confidence level lower than a preset value are enhanced for recognition, resulting in a second recognition result and a corresponding second confidence level. Based on the first confidence level, the second confidence level, and business semantic prompts, the first recognition result and the second recognition result are dynamically fused to generate the field recognition result.

[0061] After image enhancement, the ticket image is fed into a dual-path OCR collaborative recognition framework. This framework comprises two recognition paths, a primary and a secondary one, which dynamically collaborate to improve overall recognition performance. The primary recognition path employs a document understanding model based on VisionTransformer. It segments the enhanced image into fixed-size image blocks and converts them into vector sequences, which are then combined with positional encoding and input into a multi-layer Transformer encoder. This model captures the global contextual dependencies between image blocks through a self-attention mechanism, understanding the overall layout of the ticket and the semantic relationships between its fields. Finally, it outputs the first recognition result and its corresponding first confidence score by connecting to a temporal classification decoder.

[0062] The auxiliary recognition path serves as a supplement and enhancement to the main path, specifically handling difficult regions where the main path's recognition confidence is low. When the first confidence score output by the main path is lower than a preset threshold, it indicates uncertainty in the recognition of the corresponding region (such as handwritten text or blurred stamps). In this case, the system automatically extracts image patches corresponding to these low-confidence regions and hands them over to the auxiliary recognition path for processing. The auxiliary path typically employs a temporal convolutional network structure, effectively extracting sequence features of local regions through multiple layers of dilated convolutions, and performing secondary recognition on these difficult regions, outputting a second recognition result and a corresponding second confidence score.

[0063] The final field recognition result is intelligently fused from the outputs of the two paths through a meta-learning dynamic weight adjustment mechanism. A lightweight weight prediction network (such as a multilayer perceptron) comprehensively analyzes the confidence of the main path, the confidence of the secondary path, and the matching degree between the business semantic cues and the current invoice format features (cue confidence), dynamically calculating the fusion weights of the main and secondary path recognition results and prior business rules (such as invoice code length and amount format) in the final result. Based on these weights, the three information sources are weighted and fused to generate the final, accurate field recognition result. This mechanism creatively combines global semantic understanding, local detail tackling, and business rule constraints, significantly improving the accuracy and robustness of recognizing complex fonts, handwritten content, and blurred handwriting while ensuring processing efficiency.

[0064] In one embodiment, the method further includes the following steps before employing a dual-path OCR collaborative recognition framework for optical character recognition: Key field region detection is performed on the single invoice image after image enhancement processing to locate the bounding box position information of the key fields; the key fields include at least one of the following: amount, date, name and invoice code; the bounding box position information is used to provide attention focus for the dual-path OCR collaborative recognition framework.

[0065] In practice, this can be accomplished using an efficient and lightweight object detection model, such as a specifically improved YOLOv8-nano network. This network is specifically trained to detect several key information fields commonly found in medical invoices. The network input is a single invoice image that has undergone the aforementioned adaptive enhancement (usually uniformly scaled to a fixed size, such as 512×512 pixels). To better adapt to the typical size and aspect ratio of invoice fields, the anchor box size of the detection network has been specifically optimized to better match the actual proportions of fields such as amount, date, name, and invoice code in the image.

[0066] This detection model performs forward inference on the input image and outputs a series of predicted bounding boxes. Each predicted box contains the coordinates of its boundary in the image (usually represented by center point coordinates, width, and height), a class label, and a confidence score. The target categories it locates include at least fields crucial for insurance claim verification, such as amount, date, name, and invoice code. After being trained on large-scale labeled invoice data, the model can stably locate these field regions in complex layouts, typically achieving a high Intersection over Union (IOU) threshold (e.g., IOU ≥ 0.85).

[0067] The bounding box location information obtained during the detection process has clear practical significance. This bounding box information is directly utilized in the subsequent dual-path OCR recognition stage. For the main recognition path, regional feature extraction or attention masking can be performed on the image based on the positions of these boxes, making the model focus more on these known key regions. For the secondary recognition path, when the main path's recognition confidence is insufficient in certain regions (especially these predefined key field regions), image patches can be more accurately extracted for secondary recognition by combining the pre-detected box positions. This pre-localization, focused recognition strategy effectively narrows the image range that the OCR model needs to process in detail, significantly improving overall processing speed. Furthermore, by reducing background interference and irrelevant text, it directly improves the recognition accuracy and reliability of key fields such as amounts and dates, thus providing crucial assurance for the efficiency and robustness of the entire automated process.

[0068] S107: By performing logical verification on the field recognition results, the final single ticket image and the final field recognition results are determined, and the logical verification results are obtained at the same time.

[0069] During the logical verification process, if an abnormal region is detected, a re-segmentation and / or re-recognition operation is performed on the abnormal region to update the single-document image and / or field recognition results. The logical verification includes at least one of intra-field verification, inter-field logical verification, and cross-document logical verification.

[0070] In the application, logical verification can be systematically implemented through a structured "multi-layer logical verification decision diagram". The first layer is the in-field verification, which verifies the format and inherent logic of a single field. For example, use regular expressions to strictly match the date format, such as standard formats like "YYYY-MM-DD" or "YYYY年MM月DD日"; for the 18-digit ID number, calculate and verify the accuracy of the last check digit according to national standards; for the invoice code, verify whether its length is 10 digits and check whether its first two provincial codes and the 3rd - 4th industry codes (such as the medical industry code "11") comply with the specifications. The second layer is the inter-field logical verification, aiming to discover contradictions between related fields within the same bill. The core verifications include checking whether the numerical values of the lowercase amount and the uppercase amount are exactly the same, and verifying whether the sum of all item amounts in the expense list is equal to the total amount. The third layer is the cross-bill logical verification, which is used to review the logical relevance between multiple bills under the same batch or the same medical record. For example, check whether the numbers of multiple invoices are consecutive to detect missing numbers; more deeply, through a semantic vector model pre-trained based on a large-scale medical dictionary, calculate the cosine similarity between the expense item description and the clinical diagnosis terms. If the similarity is too low (for example, lower than the threshold of 0.3), it indicates that the reasonableness of the expenses is in doubt. These verifications together constitute a rigorous automated audit network, which can effectively identify various anomalies caused by OCR misrecognition, problems with the bills themselves, or logical contradictions.

[0071] Once an anomaly is detected in any layer of verification, the intelligent closed-loop optimization mechanism is immediately activated instead of just making a mark. First, the content of the abnormal field and its spatial position in the image (i.e., the abnormal area) will be accurately recorded, and it will be determined whether the root cause of the anomaly belongs to segmentation anomaly (such as incorrect region boundaries including content of other bills) or recognition anomaly (such as incorrect text content recognition). Subsequently, this information together with the anomaly confidence level (e) is fed back to the previous processing module. One of the key feedback actions is to dynamically adjust the core parameter that guides the entire process, that is, the business semantic hint vector (P_{biz}). Its weight will be updated according to the formula P_{biz}' = P_{biz} (1 + 0.5×e) to enhance the model's attention to the problem area.

[0072] Based on the updated guidance information, targeted reprocessing operations are performed on abnormal regions. If a segmentation anomaly is determined, local visual features of the mixed images are re-extracted based on the region's location. Combined with the adjusted cue vector, a local re-segmentation process, from feature fusion and decoding to mask optimization, is executed to correct boundaries and generate an updated single-document image. If a recognition anomaly is determined, the corresponding image patch is precisely extracted from the current single-document image and re-entered into the dual-path OCR collaborative recognition framework for re-recognition to correct text errors and update field recognition results. The updated results are then re-entered into the logical verification stage for validation, forming a closed-loop cycle until all verifications pass or the preset number of iterations is reached.

[0073] This closed-loop, multi-task anomaly detection and collaborative optimization design places verification, segmentation, and recognition tasks within a unified framework, enabling them to share features and provide mutual supervision signals. It not only automates result verification but also endows the method with powerful autonomous error correction and iterative optimization capabilities. Through the closed loop of "perception-recognition-verification-re-perception," it effectively corrects errors accumulated in the early processing stages, significantly improving the quality, consistency, and reliability of the final output single-document image and field recognition results.

[0074] Specifically, re-segmentation and / or re-identification operations are performed on abnormal regions to update the single-ticket image and / or field recognition results, including: Based on the anomaly type corresponding to the abnormal region, determine whether to perform a re-segmentation operation and / or a re-identification operation; the anomaly type is either a segmentation anomaly or an identification anomaly. If the exception type is a splitting exception, then perform a re-splitting operation: Based on the location information of abnormal regions in the composite image, the fusion parameters used to generate the fusion cue vector are adjusted; Based on the adjusted fusion parameters, local visual features at corresponding positions in the mixed image are extracted, and the local visual features are fused with the adjusted prompt information to generate a local fusion prompt vector. Based on local fusion cue vectors and local visual features, a new preliminary segmentation mask for the abnormal region is generated. Perform mask optimization on the new initial segmentation mask to obtain a new precise segmentation mask for the abnormal regions; The corresponding parts in the original precise segmentation mask are replaced or corrected using the new precise segmentation mask to generate an updated single ticket image. Image enhancement processing is performed on the updated single ticket image, and then optical character recognition is performed using a dual-path OCR collaborative recognition framework to obtain the updated field recognition results. If the exception type is an identification exception, then perform a re-identification operation: Extract the image patch corresponding to the abnormal region from the single ticket image; Image enhancement processing is performed on the image block, and then optical character recognition is performed using a dual-path OCR collaborative recognition framework to obtain the text content of the new field; The corresponding part of the original field recognition result is replaced with the new field text content to form the updated field recognition result.

[0075] Once the logic verification process identifies an anomaly and determines its type, a targeted closed-loop optimization operation is initiated. The specific execution process varies depending on whether the anomaly originates from segmentation or identification, aiming to achieve the most effective correction at the lowest cost.

[0076] If an anomaly is determined to be a segmentation anomaly, meaning the boundary region of the current single-ticket image may be deviated (e.g., including portions of adjacent tickets or being missing), a re-segmentation operation is performed. First, based on the recorded precise location information of the anomaly region in the original composite image, the key fusion parameters used to generate the fusion cue vector are adjusted. For example, a lightweight parameter adjustment network is used to recalculate the weight allocation of text cue and business semantic cue in local fusion based on the anomaly location. Then, based on the aforementioned improved ResNet50 architecture visual encoder, multi-scale local visual features corresponding to the anomaly region are specifically extracted. These local features and the adjusted cue information are interacted and aligned through a cue fusion network incorporating a multi-head attention mechanism to generate a local fusion cue vector focused on the local context. Based on this vector and local visual features, a lightweight segmentation decoder (which can be a dynamic sparse attention decoder) is used to generate a new preliminary segmentation mask for the anomaly region. This mask then undergoes the same adaptive mask optimization process as global segmentation, including geometric topology analysis, targeted geometric correction, and shadow repair, resulting in a new, accurate local segmentation mask for the region. Finally, this new locally precise segmentation mask is seamlessly replaced or merged into the original globally precise segmentation mask, thereby re-extracting and generating the updated single-ticket image. Because the image content has changed, the updated single-ticket image needs to undergo a complete post-processing chain, including adaptive image enhancement based on its layout and optical character recognition performed by a dual-path OCR collaborative recognition framework, to obtain completely updated field recognition results.

[0077] If an anomaly is identified as a recognition anomaly—that is, the segmentation boundary is correct but the text content recognized by OCR contains errors (such as deviation in recognizing monetary amounts)—a more efficient re-recognition operation is performed. First, an image patch containing the anomalous field is accurately extracted from the current single-document image. This image patch also undergoes necessary image enhancement processing based on its document category (such as targeted contrast enhancement or super-resolution reconstruction). Then, this image patch is directly fed into a dual-path OCR collaborative recognition framework for secondary recognition. During this process, the framework can focus more on applying the capabilities of its auxiliary recognition path (such as a model based on a temporal convolutional network) for this local region, or adjust the meta-learning fusion weights to rely more on the high-confidence path. Through this round of recognition, the correct new field text content for that region is obtained. Finally, simply replacing the corresponding erroneous part in the original field recognition result with this new text content generates an updated, corrected field recognition result, without needing to re-segment and generate the entire document image.

[0078] S108: Perform anomaly detection on the final single ticket image to obtain the anomaly detection results.

[0079] Anomaly detection includes at least one of the following: tampering trace detection, blur and defect detection, and seal authenticity detection.

[0080] After completing the logical verification and determining the final single-document image, a comprehensive anomaly detection step is performed on the image itself. This step aims to conduct in-depth verification of the document from the perspectives of image authenticity, integrity, and quality, in order to identify potential issues such as forgery, tampering, and image degradation, providing crucial evidence for subsequent risk assessment.

[0081] The anomaly detection network employs a high-efficiency convolutional neural network as a shared feature extraction backbone, such as EfficientNet-B2, which can extract robust visual features of document images at multiple scales (from high-resolution detail layer C1 to semantically rich high-level layer C5). To tightly integrate high-level business intent with image analysis, this invention innovatively introduces a "hint-anomaly feature interaction layer." Specifically, the fused hint vector generated in the preceding steps, which integrates user instructions and business knowledge, is concatenated with specific-level visual features extracted by the backbone network (such as C4 layer features for tamper detection and C3 layer features for blur detection), and then fed into the convolutional block attention module for processing. This allows the anomaly detection network to be guided by high-level task semantics when making anomaly judgments, such as focusing more on easily tampered key fields (such as amount and date) or specific seal locations in business rules, thereby significantly improving the accuracy and targeting of anomaly localization.

[0082] Based on these shared and semantically enhanced features, multiple dedicated anomaly detection branches are deployed in parallel to collaboratively complete the comprehensive detection task. The tampering detection branch primarily focuses on whether the image has been manually modified. This branch takes cue-enhanced high-level features as input, further focuses on suspicious areas through a convolutional block attention module, and after processing through several layers of convolutional networks, outputs a pixel-level tampering probability heatmap corresponding to the input image space, used to identify areas that may contain traces of Photoshop manipulation, smearing, copy-pasting, etc. The blur and defect detection branch aims to evaluate the overall quality and integrity of the image. This branch fuses low-level features containing details (such as C3) from the backbone network with high-level features containing global semantics (such as C5), enabling the anomaly detection network to simultaneously perceive local sharpness and overall structural coherence, ultimately outputting a probability map identifying blurred, missing corners, or defective areas in the image. The seal authenticity analysis branch is specifically designed to verify the authenticity of seals. This branch first locates and extracts candidate regions of the seal from high-level features, uses ROI Align operation to obtain aligned region features, and then combines traditional image analysis features (such as color histograms and HOG features) with depth features. A multilayer perceptron classifier is used to determine the authenticity of the seal. The analysis dimensions cover the regularity of the shape, the uniformity of the color distribution, and the sharpness of the edges.

[0083] To ensure the synergy and feature consistency of multi-task learning, the training of the anomaly detection network is optimized through a comprehensive multi-task loss function. This total loss function not only includes the specific losses (L_tamper, L_blur, L_seal) of each branch (tamper detection, blur and defect detection, and seal authenticity analysis), but also innovatively incorporates a feature consistency contrast loss (L_shared) to constrain the features extracted by the shared backbone network to maintain a reasonable distribution consistency across different anomaly tasks; and a cue consistency loss (L_prompt) to constrain the features of detected anomaly regions to maintain a certain semantic correlation with the attention-grabbing fusion cue vector. The total loss function is typically combined in a weighted sum form: L_total = L_tamper + λ1×L_blur + λ2×L_seal + λ3×L_shared + λ4×L_prompt, where λ1, λ2, λ3, and λ4 are all balancing weights.

[0084] S109: Determine the risk score based on the logic verification results and anomaly detection results.

[0085] After completing logical verification and anomaly detection, an integrated intelligent review decision engine calculates a quantitative risk score for each individual invoice image based on the results of these verifications and anomaly detections. This score aims to comprehensively and objectively reflect the overall credibility and risk level of the invoice across three dimensions: information accuracy, logical consistency, and image authenticity, providing a clear and actionable basis for subsequent decisions on whether manual review is necessary.

[0086] Risk scoring can be achieved through a well-designed, quantifiable scoring function that intelligently weights and fuses heterogeneous evidence from multiple sources. Specifically, the scoring comprehensively considers the core output indicators of the entire preceding process: First, the overall reliability of the Optical Character Recognition (OCR) stage, characterized by the average recognition confidence (S_ocr). The lower the confidence, the higher the fundamental risk arising from recognition uncertainty. Second, the scale of substantial errors discovered in the logic verification stage, quantified by the proportion of the number of anomalous fields to the total number of detected key fields (N_err / N_total). This indicator directly reflects the degree of internal contradiction in the extracted information and is therefore given a high weight in the scoring. Finally, the physical risk indicators for the image itself output from the anomaly detection stage include the detection confidence of tampering traces (P_tamper), the detection confidence of blur and defects (P_blur), and the detection confidence of the authenticity of the seal (P_seal). These three together assess the authenticity and integrity of the document image carrier.

[0087] This invention employs a validated linear weighted formula to integrate the aforementioned factors. For example, the score can be specifically calculated as: Score = 0.3 × (1 - S_ocr) + 0.4 × (N_err / N_total) + 0.1 × P_tamper + 0.1 × P_blur + 0.1 × P_seal. The weighting coefficients in the formula (0.3, 0.4, 0.1, etc.) reflect the relative importance of different risk dimensions. Their setting is based on validation using a large amount of business data and auditing experience, ensuring the rationality and effectiveness of the scoring model results. This calculation process maps identification uncertainties, logical contradictions, and various physical image anomalies to a unified scalar risk value, achieving the quantification and integration of complex auditing information.

[0088] Based on the calculated risk score, the intelligent review decision engine automatically executes tiered decisions, forming clear review conclusions and processing strategies. Typically, specific decision thresholds are preset for comparison. For example, if the risk score is greater than or equal to 0.6, it is judged as high risk, indicating significant problems in multiple dimensions of the invoice, requiring marking and submission for manual review. If the risk score is between 0.3 (inclusive) and 0.6, it is judged as medium risk, and the output will precisely highlight specific abnormal fields, logical contradictions, and abnormal image areas to help reviewers quickly locate problems for focused examination. If the risk score is below 0.3, it is judged as low risk, indicating that the automated processing result is highly credible and can be automatically approved. This precise triage mechanism based on comprehensive quantitative scoring maximizes the efficiency of automated processing while ensuring that various risks receive control measures commensurate with their risk levels.

[0089] S110: Generate and output the final single-ticket image, the final field recognition results, and the segmentation and recognition results of the risk score.

[0090] The final segmentation and recognition result is a multimodal, structured data packet, whose core contains three dimensions of output. The first part is the final single-ticket image. (Example...) Figure 3 As shown, the single-document image is an independent document image accurately extracted from the original mixed-image based on a precise segmentation mask obtained through geometric topology analysis and adaptive optimization. These images not only effectively eliminate interference from overlap, shadows, and complex backgrounds, but also undergo adaptive enhancement processing for their specific format (such as thermal paper or scanned documents), ensuring the integrity, high definition, and good visibility of the document content, and can be directly used for archiving or visual verification.

[0091] The second part is the final field recognition result. This result is the accurate and reliable key business data formed by extracting text information from the dual-path OCR collaborative recognition framework, and then verifying it through a multi-layered logical decision graph and making possible closed-loop corrections. This data is organized according to a preset structured format, such as generating JSON objects, XML documents, Excel spreadsheets, or CSV files, clearly recording the core fields of each invoice, such as "amount," "date," "name," and "invoice code," and their corresponding text content. This structured output allows business systems to directly call and integrate this information, seamlessly connecting to subsequent automated processes such as claims calculation and data entry.

[0092] The third part is a comprehensive risk score and review conclusion. This score, based on the quantitative output of previous steps, is calculated using a specific formula within the intelligent review decision engine (e.g., a weighted average calculation of the average OCR confidence level, the proportion of logically abnormal fields, tampering confidence, fuzzy confidence, and the authenticity of the seal). It objectively reflects the overall credibility and risk level of the document processing. The output not only includes this numerical score but also automatically generates explicit review strategy labels based on preset thresholds (such as 0.3 and 0.6), such as "Low Risk: Automatic Approval," "Medium Risk: Review Recommended," or "High Risk: Review Required." For medium- and high-risk documents, the results include specific anomaly location information, such as highlighting tampered areas, blurred or damaged areas, or questionable seal locations on the image, and indicating specific fields where logical verification failed in the structured data, greatly assisting manual review for quick and accurate focus.

[0093] To meet diverse business integration needs, this invention supports flexible output configuration. The entire segmentation and recognition result can be provided in a unified data packet (such as one containing image files, structured data files, and review reports), or it can be returned in real time as a structured response through a standard application programming interface. In a typical optimized deployment environment (such as using an engine optimized with TensorRT INT8 quantization and model pruning, deployed on an Nvidia GPU), the entire process from receiving mixed images to generating the complete output result can typically be controlled within hundreds of milliseconds, fully meeting the requirements of high concurrency and real-time business processing.

[0094] The following provides a further illustrative example of the network architecture used to implement the above method.

[0095] The learnable cue encoding network for achieving learnable cue fusion includes a hierarchical visual encoder, a multi-scale feature pyramid (FPN), a text cue encoder, and a business semantic cue learner.

[0096] Hierarchical visual encoder: Improved ResNet50 architecture, adding local-global feature interaction blocks; basic structure is conv1 (64 kernels 3×3, stride 1, ReLU) → maxpool (2×2) → conv2_x (3 residual blocks) → conv3_x (4 residual blocks) → conv4_x (6 residual blocks) → conv5_x (3 residual blocks). Feature extraction: 3×3 convolution (64 kernels, stride 1, padding=1) to capture document edge details; global feature extraction: 7×7 dilated convolution (dilation=4, 64 kernels stride 1, padding=6); capture layout structure features. Fusion: F_{vis}=\alpha\cdotF_{local}+(1-\alpha)\cdotF_{global}, where \alpha=\text{Softmax}(\text{MLP}(F_{local}\oplusF_{global})), dynamically allocating local / global feature weights.

[0097] Multi-Scale Feature Pyramid (FPN): A cue-guided dynamic feature selection mechanism: Top-down fusion: C5 → 1×1 convolution to 256 dimensions → upsampling (2×) → concatenated with C4, and C3 and C2 are fused sequentially to output a 4-scale feature map (C2: 1 / 4 resolution, C3: 1 / 8, C4: 1 / 16, C5: 1 / 32) - Feature selection: Calculate the cosine similarity S_k=\text{cos}(P_{fuse},F_k) between each FPN feature and the fused cue vector, with weight W_k=\text{Softmax}(S_k / \tau) (\tau=0.1), and the weighted fusion output F_{fpn}=\sum_{k=1}^4W_k\cdotF_k.

[0098] Text prompt encoder: bidirectional Transformer + prompt adaptive encoding layer; Transformer structure: 6-layer encoder + 6-layer decoder, 8 attention heads, hidden layer dimension 512, input is the text sequence after BERT word segmentation (maximum length 64); encoding layer: P_{text}=\text{GELU}(W_2\cdot\text{LayerNorm}(W_1\cdotX+b_1)+b_2), gating coefficient g=\sigma(W_g\cdotX)(W_g\in\mathbb{R}^{512×512},b_g\in\mathbb{R}^{512}), controlling the text feature contribution (g∈[0,1]); output: 256-dimensional text semantic vector.

[0099] Business semantic prompt learner: 3-layer MLP + contrastive learning + mask reconstruction pre-training; structure is input 256-dimensional → 512-dimensional (ReLU) → 256-dimensional (GELU) learning: InfoNCE loss, temperature parameter tau=0.07, positive and negative sample ratio 3:1; the mask reconstruction task is to randomly occlude key areas of the invoice (amount, title, invoice code), and predict the occluded content through business semantics.

[0100] The loss is L_{recon}=\text{MSE}(P_{biz},P_{recon}); the total loss is L_{biz}=L_{InfoNCE}+0.5\cdotL_{recon}; the pre-training data consists of 150,000 medical bills (including 100,000 labeled data and 50,000 unlabeled data), supporting self-supervised learning on unlabeled data.

[0101] A cascaded CNN layout classification network for achieving adaptive recognition of invoice layouts, comprising a coarse classification network and a fine classification network.

[0102] Coarse classification network: The input is a segmented single invoice image (512×512), and the structure is conv1 (64 kernels 3×3, stride 1, ReLU) → maxpool (2×2) → conv2 (128 kernels 3×3) → maxpool (2×2) → conv3 (256 kernels 3×3) → maxpool (2×2) → globallavgpool → FC (128-dimensional) → Softmax (3 categories: invoice / invoice / diagnostic certificate). Function: L_{coarse}=\text{CrossEntropyLoss}.

[0103] Fine-grained classification network: Reuse features from coarse-grained classification networks conv1-conv3 → conv4 (512 kernels, 3×3, stride 1, ReLU) → maxpool (2×2) → globalagpool → FC (256-dimensional) → semantic injection layer → Softmax (8 categories: outpatient invoices / inpatient invoices / designated pharmacy invoices, etc.); Inject F_{cls}=\text{Concat}(F_{cnn},P_{biz})\cdotW_{fusion}+b_{fusion}(W_{fusion}\in\mathbb{R}^{512×256},b_{fusion}\in\mathbb{R}^{256}); The loss function is L_{fine}=\text{CrossEntropyLoss}.

[0104] A dual-path OCR collaborative recognition framework for achieving highly robust character recognition, comprising a primary recognition path (ViT) and a secondary recognition path (TCN).

[0105] Main recognition path (ViT): The input is the enhanced ticket image (512×512); the structure is 32×32 16×16 patches → 768-dimensional patch vector + position encoding → 12-layer Transformer (8-head attention) → CTC decoding; the output is the field text sequence and confidence s (s∈[0,1]).

[0106] Auxiliary recognition path (TCN): Input is the local region (256×64) of the main path confidence s.9; Structure: 4 layers of dilated convolution (dilation=1,2,4,8, 64 kernels 3×3) → GELU activation → layer normalization → CTC; Decode the local region text sequence and confidence t (t∈[0,1]).

[0107] The meta-learning dynamic weight adjustment mechanism for dynamically fusing dual-path OCR recognition results is implemented as follows: (1) Weight prediction network Input: Confidence of the main path s, confidence of the secondary path t, and confidence of the prompt c_{prompt} (c_{prompt}=\text{cos}(P_{biz},V_{layout}), reflecting the degree of matching between the prompt and the ticket layout); Network structure: 2-layer MLP (3-dimensional input → 64-dimensional (ReLU) → 1-dimensional (Sigmoid)); Weight constraints: w1=\text{clip}(\text{MLP}(s,t,c_{prompt}),0.2,0.8), w2=1-w1, w3=0.1×c_{prompt} (w3∈[0,0.1]); The fusion formula is: P_{final}=w1×P_{main}+w2×P_{aux}+w3×P_{prompt}, where P_{prompt} is the business prompt field prior (such as the invoice code length of 10 digits and the amount format of numbers + decimal point).

[0108] (2) Meta-learning training process Step 1: Support set construction - 5000 low-quality tickets (including handwritten text and blurred areas), divided into 50 tasks (100 tickets per task); Step 2: Inner Loop Training – For each task, calculate the loss L_{support}=\text{CTCLoss}(P_{final},Y_{true}) using the support set, and update the network parameters \theta'=\theta-\alpha\nablaL_{support}(\theta) (\alpha=0.001); Step 3: Outer loop training - Calculate the meta-loss L_{meta}=\text{CTCLoss}(P_{final}(\theta'),Y_{true}) using the query set (20 images per task), and update the meta-parameters \theta=\theta-\beta\nablaL_{meta}(\theta) (\beta=0.0001). Step 4: Training epochs - 100 epochs, optimizer AdamW (lr=5e-5, weight_decay=1e-5).

[0109] Corresponding to the aforementioned application function implementation method embodiments, the present invention also provides a multi-document intelligent segmentation and recognition system and corresponding embodiments.

[0110] Please see Figure 4 and Figure 5 , Figure 5 This is a schematic diagram of the module structure of a multi-document intelligent segmentation and recognition system.

[0111] A multi-document intelligent segmentation and recognition system includes: The information acquisition unit 51 is used to acquire a mixed image containing multiple tickets and corresponding prompt information; the prompt information includes user text prompts and pre-trained business semantic prompts. The fusion unit 52 is used to extract visual features from the mixed images and fuse the visual features with the prompt information to generate a fused prompt vector; Segmentation unit 53, used for: A preliminary segmentation mask is generated based on the fused cue vectors and visual features; Geometric topological analysis is performed on the initial segmentation mask, and mask optimization is performed based on the analysis results to obtain the accurate segmentation mask; Based on accurate segmentation masks, multiple single ticket images are segmented from mixed images; The recognition unit 54 is used to perform image enhancement processing on each single ticket image, and then use a dual-path OCR collaborative recognition framework to perform optical character recognition to obtain field recognition results. The logic verification unit 55 is used to perform logic verification on the field recognition results to determine the final single ticket image and the final field recognition results, and at the same time obtain the logic verification results; if an abnormal region is found during the logic verification process, the abnormal region is re-segmented and / or re-recognized to update the single ticket image and / or field recognition results. Anomaly detection unit 56 is used to perform anomaly detection on the final single ticket image and obtain anomaly detection results; Risk assessment unit 57 is used to determine the risk score based on the logic verification results and anomaly detection results; The result generation and output unit 58 is used to generate and output the final single ticket image, the final field recognition result, and the segmentation and recognition result of the risk score.

[0112] In one embodiment, the logic verification unit 55 is specifically used for performing resegmentation and / or re-identification operations on abnormal regions to update the single ticket image and / or field recognition results: Based on the anomaly type corresponding to the abnormal region, determine whether to perform a re-segmentation operation and / or a re-identification operation; the anomaly type is either a segmentation anomaly or an identification anomaly. If the exception type is a splitting exception, then perform a re-splitting operation: Based on the location information of abnormal regions in the composite image, the fusion parameters used to generate the fusion cue vector are adjusted; Based on the adjusted fusion parameters, local visual features at corresponding positions in the mixed image are extracted, and the local visual features are fused with the adjusted prompt information to generate a local fusion prompt vector. Based on local fusion cue vectors and local visual features, a new preliminary segmentation mask for the abnormal region is generated. Perform mask optimization on the new initial segmentation mask to obtain a new precise segmentation mask for the abnormal regions; The corresponding parts in the original precise segmentation mask are replaced or corrected using the new precise segmentation mask to generate an updated single ticket image. Image enhancement processing is performed on the updated single ticket image, and then optical character recognition is performed using a dual-path OCR collaborative recognition framework to obtain the updated field recognition results. If the exception type is an identification exception, then perform a re-identification operation: Extract the image patch corresponding to the abnormal region from the single ticket image; Image enhancement processing is performed on the image block, and then optical character recognition is performed using a dual-path OCR collaborative recognition framework to obtain the text content of the new field; The corresponding part of the original field recognition result is replaced with the new field text content to form the updated field recognition result.

[0113] In one embodiment, the analysis results include the coordinates of the four corner points of the ticket, a geometric distortion measure characterizing the degree of boundary deformation, shadow discrimination information characterizing whether there is a boundary shadow, and overlap discrimination information indicating whether there is regional overlap. When performing mask optimization based on the analysis results, segmentation unit 53 is specifically used for: Based on geometric distortion measurement, one of the preset geometric correction strategies is dynamically selected, and the boundary of the preliminary segmentation mask is geometrically corrected according to the coordinates of the four corner points of the ticket. If the shadow discrimination information indicates the existence of a boundary shadow, then the local area where the boundary is located will be subjected to contrast enhancement and edge repair processing; If the overlap discrimination information indicates that there is regional overlap, then image segmentation processing is performed on the overlapping ticket regions to separate the different ticket regions.

[0114] In one embodiment, the segmentation unit 53 is further configured to: Calculate the intersection-union ratio (IU / U) between the optimized mask obtained after mask optimization and the initial segmentation mask; If the intersection-union ratio is lower than the preset threshold, the fusion cue vector is updated based on the visual features of the optimized mask and the mixed image. Based on the updated fusion cue vector, the initial segmentation mask is regenerated, and geometric topology analysis and mask optimization are performed again. The above steps are executed iteratively until the intersection-union ratio between the optimized mask obtained in the current round and the corresponding preliminary segmentation mask reaches or exceeds a preset threshold, and the current optimized mask is used as the accurate segmentation mask.

[0115] In one embodiment, when performing image enhancement processing on each individual ticket image, the recognition unit 54 is specifically used for: For each invoice image, perform the following steps: Perform layout recognition on single invoice images to determine the corresponding invoice category; Based on the bill type and business semantics, dynamically select one enhancement strategy from multiple preset image enhancement strategies; The selected enhancement strategy is used to enhance the image of the single ticket.

[0116] In one embodiment, the dual-path OCR collaborative recognition framework includes a main recognition path and a secondary recognition path; in terms of using the dual-path OCR collaborative recognition framework for optical character recognition to obtain field recognition results, the recognition unit 54 is specifically used for: The image of a single ticket after image enhancement is globally recognized by the main recognition path to obtain the first recognition result and the corresponding first confidence level of each image region. By using an auxiliary recognition path, image regions with a first confidence level lower than a preset value are enhanced for recognition, resulting in a second recognition result and a corresponding second confidence level. Based on the first confidence level, the second confidence level, and business semantic prompts, the first recognition result and the second recognition result are dynamically fused to generate the field recognition result.

[0117] In one embodiment, prior to optical character recognition using a dual-path OCR collaborative recognition framework, the recognition unit 54 is further configured to: Key field region detection is performed on the single invoice image after image enhancement processing to locate the bounding box position information of the key fields; the key fields include at least one of the following: amount, date, name and invoice code; the bounding box position information is used to provide attention focus for the dual-path OCR collaborative recognition framework.

[0118] Regarding the system in the above embodiments, the specific manner in which each unit module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0119] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for intelligent segmentation and recognition of multiple invoices, characterized in that, include: Acquire a mixed image containing multiple receipts and corresponding prompts; the prompts include user text prompts and pre-trained business semantic prompts. Visual features are extracted from the composite image, and the visual features are fused with the prompt information to generate a fused prompt vector; Based on the fused cue vector and visual features, a preliminary segmentation mask is generated; Geometric topological analysis is performed on the preliminary segmentation mask, and mask optimization is performed based on the analysis results to obtain the accurate segmentation mask; Based on the precise segmentation mask, multiple single ticket images are segmented from the mixed image; Image enhancement processing is performed on each of the single ticket images, and then optical character recognition is performed using a dual-path OCR collaborative recognition framework to obtain field recognition results; By performing logical verification on the field recognition results, the final single ticket image and the final field recognition results are determined, and the logical verification results are obtained at the same time. If an abnormal region is found during the logical verification process, a re-segmentation and / or re-identification operation is performed on the abnormal region to update the single ticket image and / or field recognition results. Anomaly detection is performed on the final single ticket image to obtain the anomaly detection results; Based on the logical verification results and anomaly detection results, a risk score is determined; Generate and output the final single-ticket image, the final field recognition result, and the segmentation and recognition result of the risk score.

2. The intelligent segmentation and recognition method for multiple invoices according to claim 1, characterized in that, Perform resegmentation and / or re-identification operations on the abnormal regions to update the single-ticket image and / or field recognition results, including: Based on the anomaly type corresponding to the anomaly region, it is determined to perform a re-segmentation operation and / or a re-identification operation; the anomaly type is a segmentation anomaly or an identification anomaly. If the exception type is a segmentation exception, then perform a resegmentation operation: Based on the location information of the abnormal region in the composite image, the fusion parameters used to generate the fusion prompt vector are adjusted; Based on the adjusted fusion parameters, local visual features at corresponding positions in the mixed image are extracted, and the local visual features are fused with the adjusted prompt information to generate a local fusion prompt vector. Based on the local fusion cue vector and the local visual features, a new preliminary segmentation mask for the abnormal region is generated; Perform mask optimization on the new preliminary segmentation mask to obtain a new precise segmentation mask for the abnormal region; The new precise segmentation mask is used to replace or correct the corresponding part in the original precise segmentation mask to generate an updated single ticket image. Image enhancement processing is performed on the updated single ticket image, and then optical character recognition is performed using the dual-path OCR collaborative recognition framework to obtain the updated field recognition result. If the anomaly type is an identification anomaly, then a re-identification operation is performed: Extract the image block corresponding to the abnormal region from the single ticket image; Image enhancement processing is performed on the image block, and then optical character recognition is performed using the dual-path OCR collaborative recognition framework to obtain the text content of the new field. The corresponding part of the original field recognition result is replaced with the new field text content to form the updated field recognition result.

3. The intelligent segmentation and recognition method for multiple invoices according to claim 1, characterized in that, The analysis results include the coordinates of the four corner points of the ticket, the geometric distortion measure representing the degree of boundary deformation, the shadow discrimination information representing whether there is a boundary shadow, and the overlap discrimination information representing whether there is regional overlap. Based on the analysis results, mask optimization is performed, including: Based on the geometric distortion metric, one of the preset geometric correction strategies is dynamically selected, and the boundary of the preliminary segmentation mask is geometrically corrected according to the coordinates of the four corner points of the ticket. If the shadow discrimination information indicates the presence of a boundary shadow, then the local area where the boundary is located is subjected to contrast enhancement and edge repair processing. If the overlap discrimination information indicates that there is regional overlap, then image segmentation processing is performed on the overlapping ticket regions to separate the different ticket regions.

4. The intelligent segmentation and recognition method for multiple tickets according to claim 3, characterized in that, The method further includes: Calculate the intersection-union ratio (IU / U) between the optimized mask obtained after mask optimization and the initial segmentation mask; If the intersection-union ratio is lower than a preset threshold, the fusion cue vector is updated based on the visual features of the optimized mask and the mixed image. Based on the updated fusion hint vector, the initial segmentation mask is regenerated, and the geometric topology analysis and mask optimization are performed again. The above steps are executed iteratively until the intersection-union ratio between the optimized mask obtained in the current round and the corresponding preliminary segmentation mask reaches or exceeds the preset threshold, and the current optimized mask is used as the accurate segmentation mask.

5. The intelligent segmentation and recognition method for multiple tickets according to claim 1, characterized in that, Image enhancement processing is performed on each of the single ticket images, including: For each of the aforementioned single ticket images, perform the following steps: The layout of the single ticket image is recognized to determine the corresponding ticket category; Based on the ticket type and the business semantic prompts, dynamically select one enhancement strategy from multiple preset image enhancement strategies; The selected enhancement strategy is used to enhance the image of the single ticket.

6. The intelligent segmentation and recognition method for multiple invoices according to claim 1, characterized in that, The dual-path OCR collaborative recognition framework includes a main recognition path and an auxiliary recognition path; A dual-path OCR collaborative recognition framework is used for optical character recognition to obtain field recognition results, including: The image enhancement process of the single ticket image is globally recognized through the main recognition path to obtain the first recognition result and the corresponding first confidence level of each image region. The auxiliary recognition path is used to enhance the recognition of image regions where the first confidence level is lower than a preset value, thereby obtaining a second recognition result and a corresponding second confidence level. Based on the first confidence level, the second confidence level, and business semantic prompts, the first recognition result and the second recognition result are dynamically fused to generate the field recognition result.

7. The intelligent segmentation and recognition method for multiple invoices according to claim 1, characterized in that, Before employing a dual-path OCR collaborative recognition framework for optical character recognition, the method further includes: The image of a single invoice after image enhancement is used to detect key field regions and locate the bounding box position information of the key fields; the key fields include at least one of amount, date, name and invoice code; the bounding box position information is used to provide attention focus for the dual-path OCR collaborative recognition framework.

8. The intelligent segmentation and recognition method for multiple tickets according to claim 1, characterized in that, The logical verification includes at least one of intra-field verification, inter-field logical verification, and cross-document logical verification.

9. The intelligent segmentation and recognition method for multiple tickets according to claim 1, characterized in that, The anomaly detection includes at least one of the following: tampering trace detection, blur and defect detection, and seal authenticity detection.

10. A multi-document intelligent segmentation and recognition system, characterized in that, include: The information acquisition unit is used to acquire mixed images containing multiple tickets and corresponding prompt information; The prompt information includes user text prompts and pre-trained business semantic prompts; The fusion unit is used to extract visual features from the mixed images and fuse the visual features with the prompt information to generate a fused prompt vector; Segmentation unit, used for: Based on the fused cue vector and visual features, a preliminary segmentation mask is generated; Geometric topological analysis is performed on the preliminary segmentation mask, and mask optimization is performed based on the analysis results to obtain the accurate segmentation mask; Based on the precise segmentation mask, multiple single ticket images are segmented from the mixed image; The recognition unit is used to perform image enhancement processing on each of the single ticket images, and then use a dual-path OCR collaborative recognition framework to perform optical character recognition to obtain field recognition results. The logic verification unit is used to perform logic verification on the field recognition results to determine the final single ticket image and the final field recognition results, and at the same time obtain the logic verification results. If an abnormal region is found during the logical verification process, a re-segmentation and / or re-identification operation is performed on the abnormal region to update the single ticket image and / or field recognition results. An anomaly detection unit is used to perform anomaly detection on the final single ticket image and obtain anomaly detection results; The risk assessment unit is used to determine a risk score based on the logic verification results and anomaly detection results. The result generation and output unit is used to generate and output the final single ticket image, the final field recognition result, and the segmentation and recognition result of the risk score.

Images