A data processing method and system based on financial information bills

Abstract

The present invention relates to the field of financial data processing technology, and in particular to a data processing method and system based on financial information bills. The data processing method comprises: obtaining multimodal financial information bill data; performing bill modal decomposition on the multimodal financial information bill data to generate financial information bill modal decomposition data. Bond coupon price yield prediction requires a large amount of financial data, such as market interest rates, bond maturity dates, coupon rates, etc. These data are similar to the data form of bill modal decomposition. By standardizing and extracting features from multimodal data, a more accurate model can be obtained, thereby providing valuable input for bond price and yield prediction. The present invention improves the security and accuracy of financial bill identification through multimodal data fusion, forgery feature comparison modeling, abnormal bill clustering analysis and dynamic risk assessment.

Description

Technical Field

[0001] The present invention relates to the technical field of financial data processing, and in particular to a data processing method and system based on financial information bills. Background Art

[0002] In the early days, financial information bills mainly existed in paper form. This processing method mainly relied on manual inspection of the authenticity, integrity and compliance of the bills. Although there were no computerized auxiliary tools at the time, price and yield forecasts similar to the bond market would also rely on relatively basic interest rate data and market trends. With the popularization of computer technology, the financial industry gradually introduced automated processing systems. At this stage, bill processing began to rely on basic computer programs that could automatically read bill information and perform some simple verification. The application of this technology has also led to the development of automation trends related to price and yield analysis in the bond market, especially in the prediction model of bond coupon yields. Computer algorithms help analyze the relationship between variables such as market interest rates, face rates, and bond maturity dates, and then predict bond price changes and yield trends. The application of artificial intelligence (AI) and big data technology has brought new opportunities for the processing of financial information bills. Machine learning algorithms can intelligently classify and analyze massive bill information, thereby improving the automation and intelligence level of data processing. However, there are a large number of non-standardized bills on the market, especially customized bill templates used by small and medium-sized enterprises or individuals. Their formats are diverse, the seal styles are complex, and there are often problems with poor printing quality or missing information. At the same time, the seals and handwritten signatures on the bills are easy to forge. The existing technology lacks efficient and accurate identification methods to distinguish real seals and signatures from forgeries, resulting in low security and accuracy in financial bill identification. Summary of the invention

[0003] Based on this, it is necessary to provide a data processing method and system based on financial information bills to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a data processing method based on financial information bills is provided, the method comprising the following steps:

[0005] Step S1: Acquire multimodal financial information bill data; perform bill modal decomposition on the multimodal financial information bill data to generate financial information bill modal decomposition data; perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multimodal hierarchical data;

[0006] Step S2: extract the multi-core key features of bills from the standardized multi-modal hierarchical data to obtain the multi-core key feature data of bills; perform dynamic forgery feature comparison modeling on the multi-core key feature data of bills to generate forgery feature comparison model data; perform dynamic consistency verification of seals and signatures on the multi-core key feature data of bills through the forgery feature comparison model data to generate verification feature matching data; perform abnormal bill clustering analysis on the multi-modal financial information bill data according to the verification feature matching data to generate abnormal bill classification data;

[0007] Step S3: Initialize the graph structure of the multi-core key feature data of the bill to generate an initial abnormal correlation relationship graph; perform full-graph abnormal risk scoring on the initial abnormal correlation relationship graph to generate full-graph abnormal risk scoring data; embed the full-graph abnormal risk scoring data into the initial abnormal correlation relationship graph to optimize the correlation graph and generate a bill abnormal correlation dynamic graph;

[0008] Step S4: dynamically render the bill abnormality association relationship dynamic graph to generate a bill abnormality association relationship rendering graph; transmit the bill abnormality association relationship rendering graph to the system front end for real-time push to generate a bill abnormality push graph; make automated risk decision recommendations based on the bill abnormality push graph to execute abnormal financial information bill processing operations.

[0009] The present invention can integrate data from different sources into standardized multimodal hierarchical data through the decomposition of multimodal financial information bill data and the fusion of cross-modal data, providing a more accurate and comprehensive data basis for subsequent analysis and decision-making. By extracting the multi-core key features of bills and dynamically modeling the forged features, the signs of forgery on bills can be effectively identified, and the recognition accuracy of forged bills can be improved. The consistency of the seal and signature can be verified to help accurately identify the authenticity of bills. Cluster analysis of multimodal bill data based on verification feature matching data can identify and classify abnormal bills, providing clear clues and data support for further risk processing. By initializing the graph structure and scoring the abnormal risk of the whole graph, the correlation and potential risks between bills can be comprehensively evaluated, a dynamic graph of the correlation relationship can be generated, the risk assessment can be continuously updated, and the accuracy and efficiency of the analysis process can be improved. By dynamically rendering the bill abnormal correlation relationship graph and pushing it to the system front end in real time, the risk status of abnormal bills can be monitored in real time, and countermeasures can be taken in time to reduce the losses caused by potential risks. Using the bill abnormal push graph to make automated risk decision suggestions, it is possible to quickly respond to and process abnormal financial information bills, achieve efficient abnormal bill processing, reduce human errors, and improve decision-making efficiency. Combining full-graph risk scoring with graph structure optimization, an effective risk management system can be established within the entire financial system, providing financial institutions with more efficient and accurate risk warning and decision support tools. Therefore, the present invention improves the security and accuracy of financial bill identification through multimodal data fusion, forgery feature comparison modeling, abnormal bill clustering analysis and dynamic risk assessment.

[0010] Preferably, step S1 comprises the following steps:

[0011] Step S11: Acquire multimodal financial information bill data;

[0012] Step S12: performing data preprocessing on the multimodal financial information bill data to generate standard multimodal financial information bill data, wherein the data preprocessing includes data cleaning, data denoising, data missing value filling and data standardization;

[0013] Step S13: performing bill modality decomposition on the standard multimodal financial information bill data to generate financial information bill modality decomposition data, wherein the financial information bill modality decomposition data includes financial information bill image modality, financial information bill text modality and financial information bill layout modality;

[0014] Step S14: Perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multi-modal hierarchical data.

[0015] The present invention can effectively improve data quality, eliminate noise and outliers, and ensure the reliability of subsequent analysis by preprocessing multimodal financial information bill data, including cleaning, denoising, missing value filling and standardization. By performing modal decomposition on standard multimodal financial information bill data, extracting image modality, text modality and layout modality, not only the characteristics of each modality are retained, but also a data basis is provided for cross-modal analysis. Integrating the decomposed modal data to generate standardized multimodal hierarchical data helps to fully explore the correlation between modalities, thereby improving the accuracy and efficiency of financial bill information processing. Standardized multimodal hierarchical data provides a unified data structure for subsequent classification, recognition and prediction tasks, so that the system can adapt to the needs of diverse financial scenarios. Through high-quality preprocessing and fusion processes, data redundancy and noise interference are reduced, thereby improving the performance and operating efficiency of the overall data processing system. The generated standardized multimodal hierarchical data lays a solid foundation for subsequent intelligent analysis and automated processing based on machine learning or deep learning, and helps to realize automated processing and intelligent decision support for financial information bills.

[0016] Preferably, step S13 includes the following steps:

[0017] Step S131: performing spectral separation on the standard multimodal financial information bill data to generate an image bill RGB layer and a grayscale layer; performing image bill edge extraction on the standard multimodal financial information bill data according to the image bill RGB layer and the grayscale layer to obtain image modality extraction data;

[0018] Step S132: performing optical character recognition on the standard multimodal financial information bill data to generate bill text content data; performing field type annotation on the standard multimodal financial information bill data based on the bill text content data to generate text modality extraction data;

[0019] Step S133: performing geometric structure analysis on the standard multimodal financial information bill data to generate bill layout distribution data; performing key area spatial decomposition on the standard multimodal financial information bill data according to the bill layout distribution data to generate layout modal extraction data;

[0020] Step S134: Perform modal hierarchical feature optimization on the image modal extraction data, the text modal extraction data and the layout modal extraction data to generate financial information bill modal decomposition data, wherein the financial information bill modal decomposition data includes financial information bill image modality, financial information bill text modality and financial information bill layout modality.

[0021] The present invention achieves more detailed processing of image information by separating the RGB layer and the grayscale layer of the image bill, and combines the edge extraction technology to capture the key features of the image, generate high-quality image modality extraction data, and lay the foundation for subsequent image analysis. The OCR technology is used to extract the text content of the bill with high precision, and combined with the field type annotation, the semantic accuracy of the text modality is improved, providing support for the classification and matching of bill information. By analyzing the geometric layout of the bill, the spatial distribution characteristics of the bill structure are captured, and the key areas are further decomposed to provide high-resolution spatial data for the extraction and analysis of the layout modality. The image modality, text modality and layout modality are hierarchically optimized, which not only retains the unique features of each modality, but also enhances the correlation between modalities through optimization. This method ensures the efficiency and consistency of modal decomposition data. Through spectral separation, OCR and geometric analysis technology, the extraction of images, texts and layouts is more accurate, and information loss and extraction errors are reduced. Modal hierarchical feature optimization accelerates the processing speed of subsequent cross-modal fusion and reduces the computational complexity. The standardized design of modal decomposition data makes it suitable for a variety of financial scenarios, including but not limited to invoice processing, bill verification and bill classification. The decomposed modal data can be directly used in machine learning or deep learning models, facilitating subsequent model training and deployment.

[0022] Preferably, performing modal hierarchical feature optimization on the image modality extraction data, the text modality extraction data, and the layout modality extraction data includes:

[0023] Performing Hough transform on the image modal extraction data to generate image modal tilt data; performing perspective correction on the image modal extraction data according to the image modal tilt data to generate image modal perspective correction data; performing light balance on the image modal perspective correction data to generate financial information bill image modality;

[0024] Performing context analysis on the text modal extracted data to generate text modal context data; performing text segmentation on the text modal extracted data according to the text modal context data to generate text modal segmented data; performing field association on the text modal extracted data according to the text modal segmented data to generate text modality of financial information bills;

[0025] Based on the layout modality extraction data, dynamic graph structure is generated to obtain the bill layout structure dynamic graph; the bill layout structure dynamic graph is hierarchically divided to generate the financial information bill layout modality; the financial information bill image modality, financial information bill text modality and financial information bill layout modality are modally optimized and integrated to generate financial information bill modal decomposition data.

[0026] The present invention uses Hough transform to extract the tilt angle information in the image, and combines perspective correction technology to correct image deformation and offset, ensure the correct position of the bill image in the standardized coordinate system, and improve the accuracy of subsequent information extraction. Light balance eliminates the uneven brightness and darkness in the image, making the image modal data clearer and more unified, which helps to improve the stability of visual feature analysis. The optimized financial information bill image modality provides high-quality input for image classification, visual detection and automatic recognition. The semantic relationship between bill text is mined through context analysis technology to ensure accurate understanding of the content and avoid ambiguity caused by isolated character extraction. Block technology logically divides the text area, and further organizes the data in combination with field association, providing efficient support for structured information extraction and semantic analysis. The generated financial information bill text modality has the characteristics of clear logic and reasonable structure, and is suitable for a variety of information processing tasks, such as text verification and data entry. Generate a dynamic graph structure based on bill layout data to capture the dynamic changes and spatial topological relationships of the layout, and improve the ability to express complex layout information. Through hierarchical division technology, the layout structure is graded according to importance and logical levels, which is convenient for subsequent analysis and matching tasks. The generated financial information bill layout mode has highly abstract spatial hierarchical characteristics, providing an accurate data basis for layout analysis, field positioning and key area extraction. The data of image mode, text mode and layout mode are integrated into unified modal decomposition data, while optimizing the correlation between modes to achieve deep fusion of multimodal features. The generated financial information bill modal decomposition data has high quality, high consistency and cross-modal collaboration capabilities, providing standardized input for multimodal machine learning and deep learning models.

[0027] Preferably, step S2 comprises the following steps:

[0028] Step S21: extracting multi-core key features of bills from the standardized multi-modal hierarchical data to obtain multi-core key feature data of bills;

[0029] Step S22: Dynamically perform forgery feature comparison modeling on the multi-core key feature data of the bill to generate forgery feature comparison model data;

[0030] Step S23: Performing dynamic consistency verification of the seal and signature on the multi-core key feature data of the bill by forging feature comparison model data to generate verification feature matching data;

[0031] Step S24: Screen the multimodal financial information bill data for abnormal bill information based on the verification feature matching data to obtain abnormal financial information bills; perform abnormal bill clustering analysis on the abnormal financial information bills to generate abnormal bill classification data.

[0032] The present invention uses multi-core key feature extraction technology to comprehensively capture the core features of image modality, text modality and layout modality, ensuring that the feature data covers the key elements of the bill (such as seals, signatures, field positions, etc.). The extracted multi-core key feature data not only retains the uniqueness of single-modal information, but also improves the comprehensive expression ability of multi-modal features through fusion. The generated multi-core key feature data of the bill provides high-precision basic data for subsequent dynamic verification and abnormal analysis. Through dynamic forgery feature comparison modeling technology, multi-dimensional forgery feature data is generated to capture potential forgery patterns (such as seal imitation, signature tampering, field tampering, etc.). Dynamic modeling realizes the real-time comparison capability of bill features, ensuring the sensitivity and applicability of forgery detection. The generated forgery feature comparison model data significantly improves the discrimination of abnormal bills, and is particularly suitable for the detection of complex forgery behaviors. Based on the forgery feature comparison model, the seal and signature features in the bill are dynamically checked for consistency to detect whether there are signs of forgery or tampering. The information of image modality (seal position, signature style), text modality (signature content) and layout modality (relative position of signature and seal) is integrated to ensure the comprehensiveness and reliability of the verification results. The generated verification feature matching data has the characteristics of high precision and high consistency, laying the foundation for subsequent abnormal screening and classification. According to the verification feature matching data, the suspected abnormal bills (such as forged, tampered, missing information, etc.) are effectively screened out. Through the abnormal bill clustering analysis technology, the abnormal bills are classified according to the forgery type and abnormal feature pattern, providing reference data for further processing and analysis. The generated abnormal bill classification data clearly reveals different abnormal types and their feature patterns, which helps to quickly identify and respond to forgery.

[0033] Preferably, step S22 includes:

[0034] Step S221: extracting the edge texture feature of the bill seal from the standardized multimodal layered data to obtain the edge texture feature data of the bill seal; performing texture frequency analysis on the edge texture feature data of the bill seal to generate the edge texture frequency data of the bill seal;

[0035] Step S222: Perform high-frequency texture feature screening on the edge texture frequency data of the bill seal to generate high-frequency texture feature data of the edge of the bill seal; perform imprinting area segmentation on the standardized multimodal hierarchical data through the high-frequency texture feature data of the edge of the bill seal to generate imprinting area data of the bill seal;

[0036] Step S223: performing regional pressure estimation on the bill stamp imprinting area data to generate imprinting force estimation data; performing bill handwritten signature feature data on the standardized multimodal layered data to generate bill handwritten signature feature data; performing handwriting change analysis on the bill handwritten signature feature data to generate handwriting signature handwriting change data;

[0037] Step S224: Dynamically compare and learn the model through the imprint force estimation data and the handwritten signature handwriting change data to generate forgery feature comparison model data.

[0038] The invention realizes accurate capture of bill seal details through edge texture feature extraction, high-frequency texture feature screening and stamping area segmentation of bill seals, and provides a high-precision basis for bill authenticity identification. Combining the estimation of seal stamping force and the analysis of handwritten signature handwriting changes, the forged feature comparison model data is generated through dynamic contrast learning modeling, which can quickly identify the slight feature changes in forged bills and greatly improve the accuracy and reliability of forged feature detection. By using standardized multimodal hierarchical data, combining the seal texture features and the handwritten signature handwriting features, a variety of bill elements are comprehensively analyzed to improve the comprehensiveness and credibility of bill identification. Automatic processing of bill seal and handwritten signature features, from texture analysis to area segmentation, and then to dynamic modeling, reduces manual operation, reduces identification costs, and improves efficiency. Through texture frequency analysis and high-frequency texture feature screening, the detailed features of bill seals are focused on, and the resolution ability of complex bill data is improved, especially in low-quality scans or complex bill backgrounds. Combining the stamping force with the handwritten signature handwriting changes, and establishing a linkage analysis mechanism through dynamic contrast learning modeling, forgery behavior can be detected more accurately and the risk of single feature misjudgment can be reduced. Based on the forged feature comparison model data, it supports the subsequent continuous optimization of the bill identification model and improves the model's generalization ability on different bill types.

[0039] Preferably, the dynamic comparison learning modeling based on the imprint force estimation data and the handwritten signature handwriting change data includes:

[0040] Acquire forged bill sample data and real bill sample data based on a preset historical bill database; calculate the minimum Euclidean distance in the modal feature space between the forged bill sample data and the real bill sample data, thereby obtaining comparison bill sample pair data;

[0041] Based on the stamping force estimation data and the handwritten signature handwriting change data, the forgery comparison loss of the comparison bill sample data is calculated to generate the bill forgery risk score; the stamping force estimation data and the handwritten signature handwriting change data are compared and verified for multiple rounds using the bill forgery risk score to generate the forgery risk score stability index;

[0042] The forgery risk score stability index is used to learn and model the data of the comparison bill samples to generate forgery feature comparison model data.

[0043] The present invention can effectively extract the features of counterfeit bills by comparing and learning the real and counterfeit samples in the historical bill library, and combine the imprint force estimation data and the handwriting change data of the handwritten signature, so as to realize high-precision counterfeit feature detection. The minimum Euclidean distance calculation of the modal feature space and the counterfeit comparison loss calculation are used to generate the bill counterfeit risk score, thereby providing a quantitative index for the authenticity identification of the bill, making the risk assessment more intuitive and scientific. The counterfeit risk score stability index is generated through multiple rounds of comparison verification, which helps to ensure the consistency and reliability of the counterfeit risk score and provide a guarantee for the robustness of the model. The multiple rounds of verification and modeling learning mechanism of the counterfeit risk score enable the model to dynamically adapt to the characteristics of different bills and improve the adaptability to diversified bill counterfeiting behaviors. The real and counterfeit sample data in the historical bill library are introduced as the training basis to provide the model with a wide range of counterfeit feature data sources, thereby improving the recognition breadth and accuracy of the model. The minimum Euclidean distance calculation of the modal feature space can optimize the distinction between counterfeit bills and real bills, reduce the false positive rate and missed positive rate, and improve the identification efficiency. The counterfeit feature comparison model can automatically generate counterfeit risk scores and feature learning modeling, greatly reduce manual participation, and realize an automated and efficient bill identification process. The application of the counterfeit feature comparison model can significantly reduce the circulation risk of counterfeit bills and ensure the security and credibility of bill management. It is especially suitable for important bill scenarios such as finance and law.

[0044] Preferably, step S23 includes the following steps:

[0045] Step S231: Calculate the strength of the bill seal on the bill multi-core key feature data to generate a bill seal strength distribution curve; calculate the seal color saturation on the bill multi-core key feature data to generate bill seal color saturation data; calculate the feature comprehensive similarity of the bill seal strength distribution curve and the bill seal color saturation data according to the forgery feature comparison model data to generate seal authenticity verification data;

[0046] Step S232: extract the signature handwriting time series from the multi-core key feature data of the bill to obtain handwriting time series data; perform dynamic time regularization matching on the handwriting time series data to generate signature handwriting error data; perform handwriting authenticity scoring on the signature handwriting error data through forged feature comparison model data to generate signature authenticity verification data;

[0047] Step S233: Verify the seal authenticity verification data and the signature authenticity verification data and integrate them into one to obtain verification feature matching data.

[0048] The present invention uses the time series characteristics of the force distribution, color saturation and signature handwriting of the bill to verify the authenticity of the bill in multiple dimensions, thereby improving the comprehensiveness and scientificity of the verification. Based on the counterfeit feature comparison model, the feature comprehensive similarity calculation is performed on the seal force distribution curve and the color saturation data, so as to accurately determine the authenticity of the seal and effectively identify the counterfeit seal. The dynamic time regularization matching of the signature handwriting time series can effectively capture the subtle differences of the handwritten signature and improve the ability to distinguish the authenticity of the signature. Through the seal and signature authenticity scoring and quantification verification process, clear verification data is generated to provide objective and intuitive results for bill identification and reduce the error of subjective judgment. Applying the counterfeit feature comparison model to the authenticity verification process of different features helps to optimize the adaptability of the model and improve the detection efficiency of new counterfeiting methods. By integrating the seal and signature verification data to form verification feature matching data, a comprehensive evaluation of the overall authenticity of the bill can be achieved to ensure the consistency and integrity of the verification results. The dual verification mode of seal and signature effectively improves the efficiency of counterfeit bill identification and reduces the misjudgment or missed judgment caused by single feature verification. Combined with the processing capability of multi-core key feature data, it is applicable to a variety of bill scenarios, including different seal types and signature methods, which enhances the wide applicability of the verification method. Through high-precision authenticity verification technology, the circulation risk of counterfeit bills is reduced, providing important security guarantees for the financial, insurance and other fields. Using intelligent verification models and data integration capabilities, the authenticity of bills can be automatically detected, reducing manual participation, improving verification efficiency, and providing technical support for smart bill management.

[0049] Preferably, step S3 comprises the following steps:

[0050] Step S31: generating node data for abnormal financial information bills according to abnormal bill classification data to obtain abnormal bill node data; performing potential correlation analysis on the abnormal bill node data to generate potential correlation data;

[0051] Step S32: marking the potential association relationship data as edge data, and initializing the graph structure of the abnormal bill node data through the edge data to generate an initial abnormal association relationship graph;

[0052] Step S33: Perform node relevance scoring on the initial abnormal association relationship graph to generate node relevance scoring data; perform full-graph abnormal risk scoring on the initial abnormal association relationship graph using the node relevance scoring data to generate full-graph abnormal risk scoring data;

[0053] Step S34: embed the whole-graph abnormal risk score data into the initial abnormal association relationship graph to optimize the association graph and generate a bill abnormal association relationship dynamic graph.

[0054] The present invention can comprehensively identify the potential risks in bills and form a comprehensive risk assessment of bills by analyzing and scoring the potential association relationships of abnormal bill nodes. The potential association relationship data is converted into edge data, and the initial abnormal association relationship graph is formed by initializing the graph structure, which can accurately reflect the relationship between bills and provide strong structural support for subsequent correlation analysis. Through node association scoring and full-graph abnormal risk scoring, the system can quickly evaluate the bill correlation and overall risk, provide accurate risk indications, and help decision makers quickly identify high-risk bills. The association graph is optimized based on the full-graph abnormal risk scoring data to form a bill abnormal association relationship dynamic graph, which can dynamically update the relationship between bills and improve the system's adaptability to abnormal changes. Through the graph structure and potential association relationship analysis, valuable association patterns can be extracted from complex financial information, the ability to identify abnormal bills can be improved, and the probability of missed judgment and misjudgment can be reduced. The preliminary abnormal association graph and the subsequent optimization process can dig out the abnormal patterns hidden in the financial bills and present them in a graphical manner, providing visual support for further analysis. The dynamically optimized bill abnormal association relationship graph provides a more accurate abnormal prediction function. With the continuous updating and optimization of data, the system can adapt to the changing bill risk pattern. Through automated graph structure generation and abnormal risk scoring, the reliance on manual operations and judgments is reduced, and the overall processing speed and accuracy are improved.

[0055] In this specification, a data processing system based on financial information notes is provided, which is used to execute the above-mentioned data processing method based on financial information notes. The data processing system based on financial information notes includes:

[0056] The bill modal decomposition module is used to obtain multimodal financial information bill data; perform bill modal decomposition on the multimodal financial information bill data to generate financial information bill modal decomposition data; perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multimodal hierarchical data;

[0057] The bill verification module is used to extract the multi-core key features of bills from the standardized multi-modal hierarchical data to obtain the multi-core key feature data of bills; to perform dynamic forgery feature comparison modeling on the multi-core key feature data of bills to generate forgery feature comparison model data; to perform dynamic consistency verification of seals and signatures on the multi-core key feature data of bills through the forgery feature comparison model data to generate verification feature matching data; to perform abnormal bill clustering analysis on the multi-modal financial information bill data based on the verification feature matching data to generate abnormal bill classification data;

[0058] The abnormal bill association module is used to initialize the graph structure of the multi-core key feature data of the bill to generate an initial abnormal association relationship graph; perform full-graph abnormal risk scoring on the initial abnormal association relationship graph to generate full-graph abnormal risk scoring data; embed the full-graph abnormal risk scoring data into the initial abnormal association relationship graph to optimize the association graph and generate a dynamic graph of bill abnormal association relationships;

[0059] The risk decision module is used to dynamically render the bill abnormality association relationship dynamic graph to generate a bill abnormality association relationship rendering graph; transmit the bill abnormality association relationship rendering graph to the system front end for real-time push to generate a bill abnormality push graph; make automated risk decision recommendations based on the bill abnormality push graph to execute abnormal financial information bill processing operations.

[0060] The beneficial effect of the present invention is that by acquiring multimodal data (images, texts, layouts, etc.), comprehensive and detailed collection of bill information is achieved, providing a rich source of data for subsequent processing. Through modal decomposition, the independent features of each modality are clarified, which helps to extract accurate bill information, avoid information mixing, and improve data quality. Through cross-modal data fusion, the information of each modality is integrated into a unified standardized structure, providing a clear and neat data framework for subsequent analysis and reducing information redundancy. Through multi-core key feature extraction, key features are extracted for each modality to ensure the comprehensiveness and accuracy of information extraction. Through dynamic forgery feature comparison modeling, it is possible to accurately detect forgery traces (such as forgery of seals and signatures) in bills, thereby enhancing the ability to verify the authenticity of bills. Through consistency verification of seals and signatures, forged bills can be effectively checked and the accuracy of risk identification can be improved. Through abnormal bill clustering analysis, abnormal bills can be quickly identified, the efficiency of abnormal identification can be improved, and missed detection can be avoided. Converting the key feature data of bills into a graph structure can better capture the relationship between bills and their potential risk associations. The full-graph abnormal risk scoring helps to comprehensively evaluate the overall data, identify bills with high potential risks, and improve the accuracy of risk warning. By optimizing the association graph, multi-level information is interconnected, which enhances the dynamic adaptability and accuracy of the risk assessment model. Dynamic rendering of the abnormal association relationship graph can display the relationship changes and risk evolution between bills in real time, helping analysts to respond promptly. Through the real-time push function, risk information and processing results are promptly delivered to the front-end system or decision makers to ensure that abnormal bills are processed in a timely manner and reduce potential losses. Automated risk decision suggestions are generated based on the abnormal push graph, which can quickly provide processing solutions, reduce manual intervention, and improve processing efficiency and decision accuracy. Therefore, the present invention improves the security and accuracy of financial bill identification through multimodal data fusion, forgery feature comparison modeling, abnormal bill clustering analysis and dynamic risk assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the steps of a data processing method based on financial information notes;

[0062] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0063] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0064] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0065] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0066] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0067] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0068] To achieve this, please refer to Figures 1 to 3 , a data processing method based on financial information bills, the method comprising the following steps:

[0069] Step S1: Acquire multimodal financial information bill data; perform bill modal decomposition on the multimodal financial information bill data to generate financial information bill modal decomposition data; perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multimodal hierarchical data;

[0070] Step S2: extract the multi-core key features of bills from the standardized multi-modal hierarchical data to obtain the multi-core key feature data of bills; perform dynamic forgery feature comparison modeling on the multi-core key feature data of bills to generate forgery feature comparison model data; perform dynamic consistency verification of seals and signatures on the multi-core key feature data of bills through the forgery feature comparison model data to generate verification feature matching data; perform abnormal bill clustering analysis on the multi-modal financial information bill data according to the verification feature matching data to generate abnormal bill classification data;

[0071] Step S3: Initialize the graph structure of the multi-core key feature data of the bill to generate an initial abnormal correlation relationship graph; perform full-graph abnormal risk scoring on the initial abnormal correlation relationship graph to generate full-graph abnormal risk scoring data; embed the full-graph abnormal risk scoring data into the initial abnormal correlation relationship graph to optimize the correlation graph and generate a bill abnormal correlation dynamic graph;

[0072] Step S4: dynamically render the bill abnormality association relationship dynamic graph to generate a bill abnormality association relationship rendering graph; transmit the bill abnormality association relationship rendering graph to the system front end for real-time push to generate a bill abnormality push graph; make automated risk decision recommendations based on the bill abnormality push graph to execute abnormal financial information bill processing operations.

[0073] The present invention can integrate data from different sources into standardized multimodal hierarchical data through the decomposition of multimodal financial information bill data and the fusion of cross-modal data, providing a more accurate and comprehensive data basis for subsequent analysis and decision-making. By extracting the multi-core key features of bills and dynamically modeling the forged features, the signs of forgery on bills can be effectively identified, and the recognition accuracy of forged bills can be improved. The consistency of the seal and signature can be verified to help accurately identify the authenticity of bills. Cluster analysis of multimodal bill data based on verification feature matching data can identify and classify abnormal bills, providing clear clues and data support for further risk processing. By initializing the graph structure and scoring the abnormal risk of the whole graph, the correlation and potential risks between bills can be comprehensively evaluated, a dynamic graph of the correlation relationship can be generated, the risk assessment can be continuously updated, and the accuracy and efficiency of the analysis process can be improved. By dynamically rendering the bill abnormal correlation relationship graph and pushing it to the system front end in real time, the risk status of abnormal bills can be monitored in real time, and countermeasures can be taken in time to reduce the losses caused by potential risks. Using the bill abnormal push graph to make automated risk decision suggestions, it is possible to quickly respond to and process abnormal financial information bills, achieve efficient abnormal bill processing, reduce human errors, and improve decision-making efficiency. Combining full-graph risk scoring with graph structure optimization, an effective risk management system can be established within the entire financial system, providing financial institutions with more efficient and accurate risk warning and decision support tools. Therefore, the present invention improves the security and accuracy of financial bill identification through multimodal data fusion, forgery feature comparison modeling, abnormal bill clustering analysis and dynamic risk assessment.

[0074] In the embodiment of the present invention, reference Figure 1 The above is a schematic flow chart of the steps of a data processing method based on financial information bills of the present invention. In this example, the data processing method based on financial information bills includes the following steps:

[0075] Step S1: Acquire multimodal financial information bill data; perform bill modal decomposition on the multimodal financial information bill data to generate financial information bill modal decomposition data; perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multimodal hierarchical data;

[0076] In an embodiment of the present invention, multimodal financial information bill data is collected from multiple sources, and the data types include but are not limited to bill image data, bill text data (such as bill text recognized by OCR), bill geometry data, and bill metadata (such as time, amount, invoice type, etc.). The data collection method can be a scanning device, a picture upload interface, or directly extracting bill data from a financial system. The collected bill data is preprocessed, including noise removal, image correction (such as perspective correction), text recognition (OCR processing), field alignment, etc. The preprocessed data provides clear and accurate input for subsequent modal decomposition and fusion. The bill data is decomposed according to different modalities (such as image modality, text modality, layout modality, etc.) to ensure that the data of each modality can be processed independently. Use image processing technology (such as edge detection, image segmentation, etc.) to decompose the image part of the bill into multiple sub-areas, such as the edge, text area, seal area, etc. of the bill. The text information in the bill is extracted by OCR technology, and combined with the image modality to generate detailed text content data, including field name, number, date, amount, etc. Based on the geometric structure analysis technology, the layout of the bill is analyzed to identify the spatial position of each part of the bill (such as the invoice number, signature area, amount area, etc.) to obtain the layout data of the bill. The data of each modality (such as image modality, text modality and layout modality) will be stored separately, and the modal decomposition data of the financial information bill will be generated. Each modal data includes the feature information of the modality and the other modal information associated with it. The data of the image modality, text modality and layout modality are integrated together using cross-modal data fusion technology to form a unified data format. The key to cross-modal fusion lies in the alignment and association between different modalities. For the fusion between image and text modalities, alignment can be performed through the correspondence between the text area and the image area; for the fusion of layout and image, the complete bill information can be synthesized based on spatial position matching. The fused data will be standardized to ensure that the features between different modalities are compatible. For example, all data are converted into a unified format, unit or scale for further analysis and calculation. Data of different modalities are processed in layers according to their importance and structural levels. For example, image modal data (such as RGB data, grayscale data, edge features, etc. of bill images) form an image layer. Text modal data (such as text information extracted by OCR) form a text layer. Layout modal data (such as geometric structure information of bills) form a layout layer. Each layer of data is standardized to form standardized multimodal hierarchical data, and finally a complete and structured data set is generated, containing information of all modalities, and ready to enter the subsequent data analysis, verification or model training stage.

[0077] Step S2: extract the multi-core key features of bills from the standardized multi-modal hierarchical data to obtain the multi-core key feature data of bills; perform dynamic forgery feature comparison modeling on the multi-core key feature data of bills to generate forgery feature comparison model data; perform dynamic consistency verification of seals and signatures on the multi-core key feature data of bills through the forgery feature comparison model data to generate verification feature matching data; perform abnormal bill clustering analysis on the multi-modal financial information bill data according to the verification feature matching data to generate abnormal bill classification data;

[0078] In the embodiment of the present invention, by selecting multiple core features from the multimodal data of the bill, these features can effectively characterize the important parts of the bill, for example: extracting the geometric shape, edge texture, color saturation and other features of the seal; extracting the handwriting features of the signature, such as the continuity, curvature, and thickness of the strokes; extracting the field position, text format, amount field, date field, etc.; extracting the geometric structure features of the bill layout, such as the relative position and spatial distribution of elements. For image data, a deep learning method (such as a convolutional neural network CNN) can be used to extract high-level features in the image, especially for areas such as seals and signatures in the bill; for text data, OCR (optical character recognition) technology is used to extract text data, and the text fields are further identified and analyzed; for layout data, image segmentation and layout analysis technology are used to obtain the spatial structure information of the bill. The above-mentioned extracted features are integrated together to form a feature vector containing multi-dimensional information, namely "multi-core key feature data of bills". Through the preset historical bill library, real bill samples and forged bill samples are collected. Forged bill samples are usually generated by artificial imitation or forgery technology. The feature space of the real bill samples and the forged bill samples is compared. The distance of modal features is calculated using methods such as minimum Euclidean distance and cosine similarity, thereby obtaining the similarity in the feature space. Based on these comparison results, a forged feature comparison model is established, and the key feature differences between real bills and forged bills are learned through the training process. Forged feature modeling is performed using machine learning techniques (such as support vector machine SVM, random forest RF, deep learning, etc.). The feature comparison results are used as input to dynamically adjust the model weights and parameters to optimize the recognition ability of forged bills and generate forged feature comparison model data. The model can determine whether the bill data is forged when input and output the forged risk score. The multi-core feature data of the bill is input into the trained forged feature comparison model to perform dynamic consistency verification of the seal and signature. The verification content includes: comparing the seal area on the bill with the real seal features in the model to determine the authenticity and integrity of the seal; comparing the handwritten signature on the bill with the real signature data in the historical data to check the consistency and personalized features of the signature. The verification feature matching data is generated through the results output by the model. The data can reflect the authenticity of the seal and signature and provide a basis for subsequent abnormal bill analysis. Based on the results of the verification feature matching data, the bills that failed the verification are screened out and considered to have a higher risk of counterfeiting. The main basis is: the score threshold of the verification feature matching data; the bills with poor verification consistency. Cluster analysis methods (such as K-means, DBSCAN, hierarchical clustering, etc.) are used to group abnormal bills. Cluster analysis will classify similar abnormal bills according to the counterfeit features of the bills (such as changes in seals and signatures) to better identify counterfeit patterns and potential risks.Bills with similar counterfeit features are grouped into the same category to further study the counterfeiting methods and generate corresponding anti-counterfeiting strategies. Based on the cluster analysis results, abnormal bill classification data is generated, which contains the counterfeit feature category, risk score, cluster label and other information of each abnormal bill, which is convenient for subsequent processing and analysis.

[0079] Step S3: Initialize the graph structure of the multi-core key feature data of the bill to generate an initial abnormal correlation relationship graph; perform full-graph abnormal risk scoring on the initial abnormal correlation relationship graph to generate full-graph abnormal risk scoring data; embed the full-graph abnormal risk scoring data into the initial abnormal correlation relationship graph to optimize the correlation graph and generate a bill abnormal correlation dynamic graph;

[0080] In an embodiment of the present invention, the multi-core key feature data of the bill includes multiple information dimensions on the bill, such as seal, signature, amount, date, layout features of the bill, etc. These features can be regarded as nodes in the graph. Each node represents an independent feature or component of the bill. For example, the seal area, signature area, field content, amount field, etc. can all be used as independent nodes. The edges between the nodes represent the association relationship between them, for example, the mutual association between the seal and the signature, the relationship between the amount field and other fields of the bill, etc. The weight of the edge can be determined based on the similarity, association, or matching degree between the features. A graph construction algorithm (such as K-nearest neighbor based on similarity, graph convolution network GCN, etc.) is used to generate a preliminary association graph of the bill according to the feature data. Based on the nodes and edges of the multi-core key features of the bill, an initial abnormal association relationship graph is constructed. In this graph, the nodes represent the core features of the bill, the edges represent the relationships between the features (such as similarity or dependency), and the structure of the graph reflects the interaction and association between different features in the bill. In the initial abnormal association relationship graph, the abnormal risk of the entire graph structure is evaluated based on the features of each node and the edges between the nodes. The abnormal risk of each node can be calculated based on the following criteria: such as the texture of the seal, the change of the signature, the abnormal fluctuation of the amount, etc.; such as the connection between the nodes does not conform to the regular pattern (for example, the time sequence or space between the seal and the signature is inconsistent). Use graph algorithms (such as graph embedding, graph neural network (GNN), graph convolutional network (GCN), etc.) to score the abnormality of the entire graph. By weighted aggregation of the abnormality scores of nodes and edges, the abnormal risk score of the entire graph is obtained. The calculated abnormal risk score data of the entire graph provides a risk value for each node and edge in the graph, so that the counterfeit or abnormal risk in the bill can be evaluated. The abnormal risk score data of the entire graph is embedded in the initial abnormal association relationship graph as an additional attribute of the nodes and edges. The score can be used as the weight of the node and edge to indicate its importance in anomaly detection. For example, the risk score of a node can be used as the weight value of the node in the graph, and the risk score of the edge can be used as the weight of the edge connecting the two nodes. Through graph optimization algorithms, such as graph convolutional network (GCN) or graph attention network (GAT), the structure of the graph is further optimized, making the abnormal relationship more prominent and the key features of abnormal bills easier to identify. Graph optimization technology is used to adjust the weights of nodes and edges to improve the recognition accuracy of abnormal bills. The optimized graph structure is converted into a dynamic graph of bill anomaly associations. This dynamic graph can be updated and optimized in real time as more data is continuously input, analyzed, and modeled, thereby improving the ability to detect bill anomalies. The dynamic graph nature of the graph allows it to reflect dynamic changes in bills (such as changes in seals, changes in dynamic features of signatures, etc.) and make adaptive adjustments based on real-time data.

[0081] Step S4: dynamically render the bill abnormality association relationship dynamic graph to generate a bill abnormality association relationship rendering graph; transmit the bill abnormality association relationship rendering graph to the system front end for real-time push to generate a bill abnormality push graph; make automated risk decision recommendations based on the bill abnormality push graph to execute abnormal financial information bill processing operations.

[0082] In an embodiment of the present invention, the bill abnormality association dynamic graph generated in step S3 is dynamically rendered. The rendering process generates a dynamic graph through a visualization algorithm according to the risk score of each node and edge in the graph and the structural characteristics of the graph. The visualization of this graph will help analysts or automated systems better understand the relationship and risk of bill abnormalities. Using graph visualization technology, such as tools such as D3.js, Graphviz, Gephi or Cytoscape, the nodes, edges and their risk scores in the graph are dynamically displayed. The nodes and edges in the graph can be represented by color, size or transparency according to the level of abnormal risk score. For example, nodes and edges with higher risk scores can be displayed in red and larger size, while those with low risk are displayed in green and smaller size. The rendering result should include the following content: each node represents the core feature of the bill (such as a seal, signature, etc.), and each edge represents the association between nodes. The thickness and color of the edge can be dynamically adjusted according to its risk score. Each element (node ​​and edge) in the graph should be equipped with a corresponding risk indicator to express the intensity of the bill abnormality risk through a diagram. The rendering should have a certain degree of interactivity. For example, the user can hover the mouse to view the detailed information of the node or click on the node to view the relationship between the node and other nodes. The generated bill abnormality association rendering is an interactive, dynamically updated graph that reflects the abnormal patterns and risk trends in the bill in real time. This rendering can not only provide visual support for manual analysis, but also provide real-time data input for the system's automated decision-making. The bill abnormality association rendering generated in step S4.1 is transmitted to the system front-end interface through the network protocol. At this time, the back-end data system pushes the rendering data to the front-end interface in real time through the API interface or WebSocket to realize the display of the bill abnormality push graph. At the front end of the system, the bill abnormality push graph is displayed through a web page or desktop application. The front-end technology can select HTML5, JavaScript (such as React, Vue.js), WebGL, Canvas, etc. to display dynamic graphics and interact with users. Real-time push mechanism: The push graph can use WebSocket or long polling technology to ensure real-time update of data, and reflect it to the front-end interface immediately when each data is updated. The bill abnormality push graph will be a real-time updated visual chart that displays the risk score of each feature in the bill and updates the abnormal pattern of the bill at any time. Through color changes, node changes, dynamic updates, etc., front-end users can intuitively view the abnormal risk of bills. Using the real-time generated bill abnormality push graph, the system can automatically analyze the nodes and edges in the graph to identify bills with higher abnormal risks. The system will automatically generate risk decision recommendations based on the abnormal patterns, risk scores, and their changing trends in the bills. For example, the system can automatically recommend that a certain type of bill be prioritized based on the abnormal risk score, or recommend in-depth investigation of certain abnormal features.Based on the risk threshold, the system can set up an alarm mechanism. When the risk score of a bill exceeds the set threshold, an alarm is automatically generated and pushed to relevant decision makers.

[0083] Preferably, step S1 comprises the following steps:

[0084] Step S11: Acquire multimodal financial information bill data;

[0085] Step S12: performing data preprocessing on the multimodal financial information bill data to generate standard multimodal financial information bill data, wherein the data preprocessing includes data cleaning, data denoising, data missing value filling and data standardization;

[0086] Step S13: performing bill modality decomposition on the standard multimodal financial information bill data to generate financial information bill modality decomposition data, wherein the financial information bill modality decomposition data includes financial information bill image modality, financial information bill text modality and financial information bill layout modality;

[0087] Step S14: Perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multi-modal hierarchical data.

[0088] In an embodiment of the present invention, multimodal financial information bill data is obtained through a bank, a financial institution or a third-party data provider. These data include scanned bill images, text content (such as OCR recognition results) and layout structure information of bills (such as field positions and hierarchical relationships). The acquired data should include multiple formats, such as image files (JPEG, PNG), text files (CSV, TXT), and structured data (JSON, XML), to ensure that all relevant bill information is covered. Identify and remove invalid or damaged bill data files (for example, damaged images or unreadable text). Mark and remove abnormal data (such as field errors, inconsistent formats, etc.). Apply denoising algorithms (such as median filtering, Gaussian filtering, etc.) to the image data to remove image noise. Remove noise from the text data to remove irrelevant symbols, meaningless characters, etc. For missing image areas, use image restoration algorithms to supplement to ensure the integrity of the image. For missing information in text data, use text inference, template filling or context filling methods to fill in the gaps. The image data is normalized and adjusted to a uniform size and resolution for subsequent processing. Standardize text data to ensure consistent character encoding and handle character sets in different languages. Normalize layout data to unify standards for field labels and hierarchical structures for unified parsing. Apply computer vision algorithms (such as convolutional neural networks (CNNs)) to extract visual information from bills, including elements such as the background, pattern, and icon of bills. Use image segmentation methods to distinguish various information areas in bills (such as invoice number, date, amount, etc.). Use OCR (optical character recognition) technology to extract text information on bills, including bill number, date, amount, payee, etc. Perform natural language processing (NLP) on the extracted text information, such as word segmentation, named entity recognition (NER), keyword extraction, etc., to help subsequent data understanding. Use deep learning technology to analyze the layout of bills and identify the location, size, and hierarchy of fields. This can usually be achieved through a layout analysis model that can identify the regional divisions on the bill (for example, columns and rows in a table). Align the image modality with the text modality to correspond to the extracted text information. Use region recommendation algorithms or image-text alignment models to ensure that each region in the image corresponds to the correct text content. Extract features from each modality (image, text, layout), and use multimodal learning techniques (such as deep neural networks and attention mechanisms) to fuse data from different modalities. Ensure that the features of each modality are fully utilized and can work together to enhance data representation capabilities. Standardize the fused multimodal data to ensure that data from different modalities have consistent formats and scales in the same data structure.Classify and process information at different levels. For example, store and process image information, text information, and layout information separately at different levels to ensure that data at each level is used in the appropriate context and generate standardized multimodal financial information bill data as input data for subsequent analysis, query, or machine learning model training.

[0089] Preferably, step S13 includes the following steps:

[0090] Step S131: performing spectral separation on the standard multimodal financial information bill data to generate an image bill RGB layer and a grayscale layer; performing image bill edge extraction on the standard multimodal financial information bill data according to the image bill RGB layer and the grayscale layer to obtain image modality extraction data;

[0091] Step S132: performing optical character recognition on the standard multimodal financial information bill data to generate bill text content data; performing field type annotation on the standard multimodal financial information bill data based on the bill text content data to generate text modality extraction data;

[0092] Step S133: performing geometric structure analysis on the standard multimodal financial information bill data to generate bill layout distribution data; performing key area spatial decomposition on the standard multimodal financial information bill data according to the bill layout distribution data to generate layout modal extraction data;

[0093] Step S134: Perform modal hierarchical feature optimization on the image modal extraction data, the text modal extraction data and the layout modal extraction data to generate financial information bill modal decomposition data, wherein the financial information bill modal decomposition data includes financial information bill image modality, financial information bill text modality and financial information bill layout modality.

[0094] In the embodiment of the present invention, the image is split into RGB three-color channels (red, green, and blue) and a grayscale channel by spectrally separating the bill image. The RGB layers represent the distribution of different colors in the image, while the grayscale layer represents the brightness information of the image. The color separation function in the image processing library (such as OpenCV) is used to extract each pixel value in the image as an RGB component and generate a corresponding grayscale image. Based on the RGB layer and the grayscale layer, an edge detection algorithm (such as Canny edge detection, Sobel operator, etc.) is used to extract the edge of the image. The purpose of edge extraction is to identify key boundaries in the image, such as the borders, numbers, text, and contours of other visual elements of the bill. After edge extraction, image modality extraction data can be obtained to clarify the main elements and regions in the bill image, providing a basis for subsequent analysis. Use OCR technology (such as Tesseract, EasyOCR, etc.) to recognize the text in the bill image and extract the text content on the bill, such as bill number, date, amount, payee, etc. Post-process the text recognized by OCR to remove noise characters and ensure the accuracy of the recognition results. Based on the recognized text content of the bill, field type annotation is performed. This step requires classifying and labeling text fields according to the structure and domain knowledge of the bill, such as "invoice number", "invoice date", "amount", etc. Automatic annotation can be performed through rule matching, machine learning algorithms or domain knowledge bases. Based on the results of OCR and field type annotation, text modal data is extracted and generated. Text modal extraction data includes the text content of each field and its type annotation information, such as text field position, field content, field label, etc. The structural layout of the bill is analyzed using a geometric analysis algorithm to determine the overall structure of the bill (such as tables, paragraphs, titles, etc.) and the spatial relationship between the various parts. Image processing technology can be used to identify the various areas on the bill and their arrangement. The various blocks in the bill (such as tables, text areas, amount boxes, etc.) are divided to generate bill layout distribution data, recording the location information, size and arrangement of each area. Based on the bill layout distribution data, the key areas of the bill are spatially decomposed. These key areas include areas related to important information such as bill verification, amount recognition, and timestamps. Use spatial analysis algorithms (such as segmentation, cluster analysis, etc.) to separate these areas from the overall bill image and generate layout modality extraction data to ensure that the information of each area is accurately identified. Based on the image modality, text modality, and layout modality extraction data, deep learning techniques (such as convolutional neural networks, recurrent neural networks, Transformer, etc.) are used to optimize the features of each modality. Perform convolution operations on the image modality to extract the detailed features of the bill image; perform sequence modeling on the text modality to extract the semantic information in the text; perform graphic analysis on the layout modality to extract the spatial relationship features of the bill layout.The optimized image, text and layout modal features are fused, and a multimodal neural network or feature cascade method can be used to combine the advantages of different modal features to enhance the information representation capability. The fused features are further optimized to generate standardized financial information bill modal decomposition data, ensuring that the data of each modality is highly representative and consistent, and generating the final financial information bill modal decomposition data, including the image modality, text modality and layout modality of the financial information bill, providing multi-level and multi-dimensional information support for subsequent financial bill analysis, verification and processing tasks.

[0095] Preferably, performing modal hierarchical feature optimization on the image modality extraction data, the text modality extraction data, and the layout modality extraction data includes:

[0096] Performing Hough transform on the image modal extraction data to generate image modal tilt data; performing perspective correction on the image modal extraction data according to the image modal tilt data to generate image modal perspective correction data; performing light balance on the image modal perspective correction data to generate financial information bill image modality;

[0097] Performing context analysis on the text modal extracted data to generate text modal context data; performing text segmentation on the text modal extracted data according to the text modal context data to generate text modal segmented data; performing field association on the text modal extracted data according to the text modal segmented data to generate text modality of financial information bills;

[0098] Based on the layout modality extraction data, dynamic graph structure is generated to obtain the bill layout structure dynamic graph; the bill layout structure dynamic graph is hierarchically divided to generate the financial information bill layout modality; the financial information bill image modality, financial information bill text modality and financial information bill layout modality are modally optimized and integrated to generate financial information bill modal decomposition data.

[0099] In the embodiment of the present invention, the Hough transform is an algorithm for detecting straight lines in an image, which can identify tilted or distorted parts in the image. First, the Hough transform is used to detect straight line elements in the image, which represent the edges or key information lines of the bill. By analyzing the angles of these straight lines, the tilt angle of the image can be calculated. The Hough transform will generate tilt data in the image, including polar coordinate representations of each detected straight line. Perspective correction is the process of correcting the tilted part in the image. Using the tilt data in the image (angle information obtained by the Hough transform), the distortion in the image is corrected by an affine transform or a perspective transform. The perspective correction operation usually involves selecting four corner points in the image (such as the four corners of the bill), calculating the transformation matrix, and applying it to the image to ensure the linearity and alignment of the bill information. Light balance is used to adjust the brightness and contrast of the image to ensure that the bill information is clearly visible, especially when there are shadows or overexposed areas. By using histogram equalization, local illumination adjustment or color balance technology of the image, the visual effect of the image is optimized, the image details are enhanced, and the final financial information bill image modality is generated. Context analysis is used to understand the relationship between texts, and natural language processing (NLP) methods are usually used for analysis. In the text mode of the bill, context analysis can help determine the order of fields, semantic relationships, and dependencies between fields. Use context analysis techniques (such as word vector models, LSTM networks, etc.) to process the text content of the bill, identify the semantics and logical structure of each field in the bill, and generate text mode context data. Text segmentation is to divide the text content in the bill into logical blocks. The text content can be divided into multiple blocks according to the format of the text (such as tables, paragraphs, etc.) to ensure that each block is independent and semantically clear. The text segmentation algorithm can automatically segment the text according to the layout of the bill, combined with the data recognized by OCR, through text position and space analysis. Field association analyzes the field types in the text mode (such as invoice number, amount, date, etc.) and their positions in the bill to perform semantic association of the fields. Use field type annotation and context analysis results to associate the content of each field with the corresponding label to ensure that the generated bill information has the correct structure, and finally generate the financial information bill text mode. Dynamic graph structure generation analyzes the layout mode of the bill and uses graph neural network (GNN) or other graph algorithms to represent the structure of the bill layout. In this process, the various parts of the bill (such as tables, text areas, amount boxes, etc.) are regarded as nodes in the graph, and a dynamic graph of the bill layout structure is generated by analyzing the spatial relationship between these areas. Hierarchical division is to stratify different areas according to the structure of the layout graph. For example, the header of the bill can be used as the top layer, the main content as the middle layer, and the signature and other information at the bottom as the bottom layer. Hierarchical clustering algorithms or graph analysis methods can be used to stratify the various areas of the bill according to their spatial or semantic relationships, and finally generate the financial information bill layout mode.Modal optimization integration is to fuse and optimize the data of image modality, text modality and layout modality. Deep learning models (such as multimodal neural networks) can be used to fuse the features of different modalities to ensure that the information of each modality can complement and strengthen each other. By processing image data through convolutional neural networks (CNN), processing text data through RNN or Transformer, and processing layout information using graph neural networks, the information of the three modalities is integrated into a unified data representation, and finally the modal decomposition data of financial information bills is generated.

[0100] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0101] Step S21: extracting multi-core key features of bills from the standardized multi-modal hierarchical data to obtain multi-core key feature data of bills;

[0102] Step S22: Dynamically perform forgery feature comparison modeling on the multi-core key feature data of the bill to generate forgery feature comparison model data;

[0103] Step S23: Performing dynamic consistency verification of the seal and signature on the multi-core key feature data of the bill by forging feature comparison model data to generate verification feature matching data;

[0104] Step S24: Screen the multimodal financial information bill data for abnormal bill information based on the verification feature matching data to obtain abnormal financial information bills; perform abnormal bill clustering analysis on the abnormal financial information bills to generate abnormal bill classification data.

[0105] In an embodiment of the present invention, multiple important features of the bill are extracted from standardized multimodal hierarchical data. Each modality (image, text, layout) has different features, which may include: extracting image features on the bill, such as image texture, color mode, seal and signature features. Extracting field content, such as text information and layout features of fields such as date, amount, invoice number, and payee. Extracting structural features of the bill, including table layout, field arrangement, border information, etc. Image features can be extracted by convolutional neural network (CNN). Text features can be extracted by optical character recognition (OCR) technology combined with natural language processing (NLP) technology to extract key information. Layout features are extracted by analyzing the geometric shape, position and relative relationship in the image. Finally, the feature data extracted from different modalities are fused to generate "multi-core key feature data of bills". The purpose of forged feature comparison is to identify forged features (such as forged seals, signatures, modified text information, etc.) in the bill. This requires the construction of a forged feature comparison model to compare and analyze the features of real bills with those of forged bills. Use deep learning (such as convolutional neural networks, generative adversarial networks GAN, etc.) to train models, and perform forged feature recognition based on the feature differences between historical real bills and forged bills. Forged features include: seal forgery: features such as the shape, color, and position of the seal; signature forgery: features such as the handwriting, pressure, and speed of the signature; text forgery: the spacing, font, and font size of text characters. By continuously updating the model to adapt to different forgery methods, improve the accuracy of forged feature comparison, and generate forged feature comparison model data. Use the forged feature comparison model data to perform dynamic consistency verification on the seal and signature in the multi-core key feature data of the bill. Detect the similarity between the seal on the bill and the sample seal, including the seal's outline, color, texture, and other attributes. Verify whether the signature conforms to the predetermined signature mode, including the smoothness of the strokes, speed features, and the pressure of the handwriting. Use deep neural networks for image feature matching, and match signatures and seals based on image similarity metrics (such as cosine similarity, Euclidean distance, etc.). For text signatures, text comparison algorithms (such as character-level comparison and word vector-based matching) can also be used to verify the authenticity of the signature. The "verification feature matching data" is generated from the verification results to determine the legality of the bill. The verification feature matching data is used to screen the multimodal financial information bill data to screen out bills with forgery risks or inconsistencies. If the seal, signature or text information of the bill is inconsistent with the comparison results of the standard template or historical data, the bill is marked as "abnormal". Machine learning algorithms (such as support vector machines SVM, random forests, etc.) or deep learning methods (such as autoencoders AE, generative adversarial networks GAN, etc.) can be used to determine the abnormality of the bill. Cluster analysis is performed on the screened abnormal bills to find potential abnormal patterns.Common clustering algorithms include: grouping according to the characteristics of abnormal bills (such as signature consistency, seal consistency, etc.). The clustering analysis results will generate "abnormal bill classification data" that can help identify and distinguish different types of counterfeit bills or problematic bills.

[0106] Preferably, step S22 includes:

[0107] Step S221: extracting the edge texture feature of the bill seal from the standardized multimodal layered data to obtain the edge texture feature data of the bill seal; performing texture frequency analysis on the edge texture feature data of the bill seal to generate the edge texture frequency data of the bill seal;

[0108] Step S222: Perform high-frequency texture feature screening on the edge texture frequency data of the bill seal to generate high-frequency texture feature data of the edge of the bill seal; perform imprinting area segmentation on the standardized multimodal hierarchical data through the high-frequency texture feature data of the edge of the bill seal to generate imprinting area data of the bill seal;

[0109] Step S223: performing regional pressure estimation on the bill stamp imprinting area data to generate imprinting force estimation data; performing bill handwritten signature feature data on the standardized multimodal layered data to generate bill handwritten signature feature data; performing handwriting change analysis on the bill handwritten signature feature data to generate handwriting signature handwriting change data;

[0110] Step S224: Dynamically compare and learn the model through the imprint force estimation data and the handwritten signature handwriting change data to generate forgery feature comparison model data.

[0111] In the embodiment of the present invention, the edge area of ​​the bill seal is extracted by an image processing algorithm (such as Canny edge detection, Sobel operator, etc.). Then, the texture features of the edge area are extracted by a texture analysis method (such as gray level co-occurrence matrix, local binary pattern LBP, etc.), and the texture pattern in the image is obtained. The extracted seal edge texture data is subjected to frequency analysis, and frequency analysis techniques such as Fourier transform or wavelet transform are used to obtain the distribution of different frequency components in the seal edge texture. High-frequency texture components are identified, which are usually closely related to the authenticity of the seal. For example, forged seals differ in detail frequency. "Bill seal edge texture frequency data" is generated by frequency analysis to provide a basis for subsequent seal feature screening and region segmentation. High-frequency components are screened out from the bill seal edge texture frequency data. Usually, these high-frequency components reflect detail features, such as the texture and depth of the seal. In the screening process, a filter (such as a high-pass filter) can be used to remove low-frequency components and retain those high-frequency features related to forged seals. Through high-frequency texture feature data, the seal area is accurately segmented using image segmentation techniques (such as threshold segmentation, region growing method, watershed algorithm, etc.). This segmentation not only helps to identify the location of the seal, but also can mark the boundary of the imprinted area. Obtain "invoice seal imprinting area data", which provides a data basis for further estimation of the imprinting force. Estimate the imprinting force based on the invoice seal imprinting area data. The imprinting force and the depth of the imprinting can be evaluated by image depth information (if any) or based on the illumination changes in the image (for example, shadows of the imprinted area, reflected light, etc.). Specific methods include estimating the imprinting force of the area using an illumination model or a deep learning model. Through the imprinting force data, different imprinting depths or unnatural seal indentations in counterfeit bills can be identified. Use OCR technology and handwriting feature extraction algorithms (such as stroke extraction, handwriting trajectory analysis, etc.) to extract the handwritten signature features in the bill. The extracted features can include the number of strokes of the signature, the coherence of the handwriting, pressure changes, etc. Capture the details of the handwritten signature and identify the authenticity of the signature. By analyzing the characteristics of handwritten signatures, the changing trend of handwriting can be identified. For example, by detecting the speed of change and pressure of the signature, it can be determined whether the signature is forged or plagiarized. Unnatural changes in signatures can be revealed to help identify forged signatures. Based on the analysis of handwriting changes in handwritten signatures, "handwritten signature handwriting change data" is generated, which describes the change pattern and characteristics of the signature. Using "imprinting force estimation data" and "handwritten signature handwriting change data", modeling is performed through contrastive learning algorithms (such as contrastive loss function, generative adversarial network GAN, etc.). The model learns the feature differences between real bills and forged bills, thereby improving the ability to identify forged bills. Use data from real bills and forged bills for training, and optimize the model through contrastive learning. The model will learn to identify the differences between real and forged features of seals and signatures.Through the training process, forgery feature comparison model data is generated, which can accurately describe the forgery features of seals and signatures in forged bills, thus providing a basis for subsequent forgery detection.

[0112] Preferably, the dynamic comparison learning modeling based on the imprint force estimation data and the handwritten signature handwriting change data includes:

[0113] Acquire forged bill sample data and real bill sample data based on a preset historical bill database; calculate the minimum Euclidean distance in the modal feature space between the forged bill sample data and the real bill sample data, thereby obtaining comparison bill sample pair data;

[0114] Based on the stamping force estimation data and the handwritten signature handwriting change data, the forgery comparison loss of the comparison bill sample data is calculated to generate the bill forgery risk score; the stamping force estimation data and the handwritten signature handwriting change data are compared and verified for multiple rounds using the bill forgery risk score to generate the forgery risk score stability index;

[0115] The forgery risk score stability index is used to learn and model the data of the comparison bill samples to generate forgery feature comparison model data.

[0116] In an embodiment of the present invention, sample data of counterfeit bills and real bills are extracted from the collected historical bill library. These data should cover various bill types and include information such as bill images, seals, signatures, dates, amounts, etc. Each bill sample must be clearly marked whether it is counterfeit for subsequent training and comparative analysis. The sample data extracted from the historical bill library is standardized to ensure data consistency and comparability. Standardization processing includes image resolution adjustment, signal smoothing, etc. The seal imprint force features of each bill are extracted through deep learning or traditional image processing technology. These features include the depth of the imprint, the sharpness of the contour, etc. The dynamic features of the handwritten signature are extracted using machine learning algorithms (such as LSTM, CNN, etc.), and the stroke changes, speed changes, and pressure changes in the signature are analyzed. All bills (counterfeit and real) are mapped to a common feature space through the extracted imprint force features and signature handwriting change features. The minimum Euclidean distance in the modal feature space is calculated for each pair of bills (counterfeit and real bills) to obtain a similarity measure between the two in the feature space. The Euclidean distance calculation formula is: in and They are the first dimensional eigenvalue. Through the minimum Euclidean distance calculation, the forged bills with high similarity are paired with the real bills to form a "comparison bill sample pair data" set for subsequent modeling. Based on the imprinting force estimation data and the handwritten signature handwriting change data, a loss function is designed to quantify the difference between the forged bills and the real bills. The loss function can be calculated by contrast loss. There are three commonly used contrast loss functions: Calculate the distance difference between the two. The smaller the distance, the more similar they are. Measure the similarity of similar bill pairs and the difference between dissimilar bill pairs. Loss function form:

[0117]

[0118] in, It is expressed as the quantitative value of the difference between the counterfeit bill and the real bill. Expressed as the sample size, Represented as a label (1 for match, 0 for no match), Expressed as Euclidean distance, Expressed as a set boundary, it represents the maximum tolerance distance of different categories of bills. The "forgery risk score" of each pair of bills (forgery bills and real bills) is calculated through the forgery comparison loss function. The risk score reflects the possibility of bill forgery. The higher the score, the greater the risk of forgery of the bill. The bill forgery risk score is used to perform multiple rounds of verification on the imprinting force estimation data and handwritten signature handwriting change data. The risk score is calculated for each round of comparison data, and the result is compared with the labels (forgery and real) in the historical bill library to evaluate the accuracy of forgery detection. The risk score is updated after each round of verification to improve the recognition ability of the model. Each round of verification can be performed by adjusting the threshold, optimizing the loss function, etc. The stability index of the forgery risk score is calculated through multiple rounds of verification. The stability index can be measured based on statistics such as the standard deviation and variance of the score to evaluate the stability of the model on different data. The stability index formula is as follows: in, Expressed as The risk score of the second verification, Expressed as the average of the risk scores, Expressed as the number of verifications. Based on the forgery risk score stability index, a deep learning method (such as convolutional neural network CNN, recurrent neural network RNN, or Transformer, etc.) is used to learn and model the data of the comparison bill samples. Using the contrastive learning method, the model is trained by forgery risk score and stability index to optimize its ability to distinguish forged bills. After multiple rounds of learning and training, "forgery feature comparison model data" is generated. This data contains the forgery marks of seals, signatures and other features in the bill, which is used for subsequent bill detection and verification.

[0119] Preferably, step S23 includes the following steps:

[0120] Step S231: Calculate the strength of the bill seal on the bill multi-core key feature data to generate a bill seal strength distribution curve; calculate the seal color saturation on the bill multi-core key feature data to generate bill seal color saturation data; calculate the feature comprehensive similarity of the bill seal strength distribution curve and the bill seal color saturation data according to the forgery feature comparison model data to generate seal authenticity verification data;

[0121] Step S232: extract the signature handwriting time series from the multi-core key feature data of the bill to obtain handwriting time series data; perform dynamic time regularization matching on the handwriting time series data to generate signature handwriting error data; perform handwriting authenticity scoring on the signature handwriting error data through forged feature comparison model data to generate signature authenticity verification data;

[0122] Step S233: Verify the seal authenticity verification data and the signature authenticity verification data by integrating the verification data to obtain verification feature matching data.

[0123] In the embodiment of the present invention, the seal area in the bill is detected and extracted by image processing technology (such as image graying and edge detection). The stamping force of the seal area is calculated by using the change of pixel values ​​in the image to obtain the strength distribution of the seal. The specific calculation method can use an algorithm based on brightness difference, for example, quantifying the stamping force by the change rate of pixel values ​​in the grayscale image. Seal force calculation formula: in It is expressed as the strength of the seal. Represented as the pixel value of the stamp area, Represented as the pixel value of the background area, Expressed as the total number of pixels, Expressed as the horizontal coordinate of the pixel point, Expressed as the vertical coordinate of the pixel point; according to the strength calculation results of different stamp areas, the distribution curve of the stamp strength is generated to show the strength of different areas. The color information is extracted in the stamp area, and the HSV (hue, saturation, brightness) color model is used to calculate the color saturation of the stamp. The higher the saturation, the brighter the color of the stamp. Saturation calculation formula: In the formula, Expressed as the stamp color saturation, The pixel values ​​of the red, green and blue channels in the seal area are expressed as the pixel values ​​of the red, green and blue channels. The seal force distribution curve and color saturation data are calculated for comprehensive similarity using the forged feature comparison model data. The authenticity of the seal is judged by calculating the comprehensive similarity of the two. The higher the feature similarity, the more authentic the seal is. The dynamic time series data of the signature is extracted using the image information of the handwritten signature. Time series data usually includes dynamic information such as the drawing order of the signature, the starting and ending points of the strokes, and the drawing speed. This information can be extracted through an electronic pen, a tablet input device, or from video data. The extracted signature time series data is matched with the standard signature template using dynamic time warping (DTW) to calculate the error data of the signature. The DTW algorithm minimizes the distance between the sequences by dynamically adjusting the time axis, thereby maximizing the similarity of the two sequences. The error data obtained after DTW matching reflects the degree of change in the handwriting of the signature. The larger the error, the more forged the signature is. The signature handwriting error data is analyzed using the forged feature comparison model data to generate a signature authenticity score. The higher the score, the more authentic the signature is. The seal authenticity verification data and signature authenticity verification data are integrated to form a complete verification feature matching data set. This data set will integrate the authenticity results of the seal and signature to provide a basis for subsequent bill verification. According to the integrated feature matching data, the overall verification result of the bill is output. If the authenticity of the seal and signature are both verified to pass, the bill is considered to be credible, otherwise it is considered a forged bill.

[0124] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0125] Step S31: generating node data for abnormal financial information bills according to abnormal bill classification data to obtain abnormal bill node data; performing potential correlation analysis on the abnormal bill node data to generate potential correlation data;

[0126] Step S32: marking the potential association relationship data as edge data, and initializing the graph structure of the abnormal bill node data through the edge data to generate an initial abnormal association relationship graph;

[0127] Step S33: Perform node relevance scoring on the initial abnormal association relationship graph to generate node relevance scoring data; perform full-graph abnormal risk scoring on the initial abnormal association relationship graph using the node relevance scoring data to generate full-graph abnormal risk scoring data;

[0128] Step S34: embed the whole-graph abnormal risk score data into the initial abnormal association relationship graph to optimize the association graph and generate a bill abnormal association relationship dynamic graph.

[0129] In the embodiment of the present invention, the features of each abnormal bill are extracted according to the abnormal bill classification data, including the seal, signature, field information, etc. of the bill. The node data of each bill is generated by using these features, and the basic information of the node is constructed, such as the bill ID, bill type, seal feature, signature feature, field data, etc. Potential association analysis is performed based on the bill node data to identify the association relationship, for example, finding potential associated bills by the frequency of similar seals, signatures or fields. Potential association relationship data is identified and generated by data mining algorithms (such as cluster analysis, similarity calculation, etc.) to represent the potential connection between bills. The potential association relationship data is converted into edge data, that is, the association relationship between bill nodes is represented as an edge. For example, if two bills have similar seals or signatures, an edge is formed between them in the graph. The graph structure of the abnormal bill node data is initialized by the edge data, and a preliminary association relationship graph (i.e., the initial abnormal association relationship graph) is constructed, in which the node represents the abnormal bill and the edge represents the potential association between the bills. Each node in the initial abnormal association relationship graph is scored for association, and the scoring basis includes the similarity between the bills, the potential fraud risk, and other association factors (such as time series, amount, etc.). Use an algorithm (such as a weighted graph algorithm) to calculate the association of each node with other nodes and generate node association score data. Use the node association score data to score the abnormal risk of the entire graph of the initial abnormal association relationship graph. Calculate the abnormal risk of the entire graph through the association of all nodes in the graph to generate abnormal risk score data for the entire graph. This score reflects the risk level of abnormal bills in the entire graph, and is usually calculated by factors such as the connection strength and path length between nodes. Optimize the initial abnormal association relationship graph based on the abnormal risk score data of the entire graph. This process can be done in an iterative manner to adjust the structure of the graph according to the risk score and association of the nodes, so that the connection between the bill nodes with higher risks is tighter. The optimization algorithm can use graph optimization methods such as graph convolutional networks (GCN) to enhance the performance of the graph by updating the edge connection weights between nodes. Generate a dynamic graph of abnormal bill association relationships through graph structure optimization. The dynamic graph can be dynamically updated, and the structure of the graph is continuously adjusted over time and with the addition of new data to help identify new potential abnormal bills. The resulting abnormal association relationship dynamic graph visually displays the association between bills, which is convenient for subsequent risk analysis and decision support.

[0130] In this specification, a data processing system based on financial information notes is provided, which is used to execute the above-mentioned data processing method based on financial information notes. The data processing system based on financial information notes includes:

[0131] The bill modal decomposition module is used to obtain multimodal financial information bill data; perform bill modal decomposition on the multimodal financial information bill data to generate financial information bill modal decomposition data; perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multimodal hierarchical data;

[0132] The bill verification module is used to extract the multi-core key features of bills from the standardized multi-modal hierarchical data to obtain the multi-core key feature data of bills; to perform dynamic forgery feature comparison modeling on the multi-core key feature data of bills to generate forgery feature comparison model data; to perform dynamic consistency verification of seals and signatures on the multi-core key feature data of bills through the forgery feature comparison model data to generate verification feature matching data; to perform abnormal bill clustering analysis on the multi-modal financial information bill data based on the verification feature matching data to generate abnormal bill classification data;

[0133] The abnormal bill association module is used to initialize the graph structure of the multi-core key feature data of the bill to generate an initial abnormal association relationship graph; perform full-graph abnormal risk scoring on the initial abnormal association relationship graph to generate full-graph abnormal risk scoring data; embed the full-graph abnormal risk scoring data into the initial abnormal association relationship graph to optimize the association graph and generate a dynamic graph of bill abnormal association relationships;

[0134] The risk decision module is used to dynamically render the bill abnormality association relationship dynamic graph to generate a bill abnormality association relationship rendering graph; transmit the bill abnormality association relationship rendering graph to the system front end for real-time push to generate a bill abnormality push graph; make automated risk decision recommendations based on the bill abnormality push graph to execute abnormal financial information bill processing operations.

[0135] The beneficial effect of the present invention is that by acquiring multimodal data (images, texts, layouts, etc.), comprehensive and detailed collection of bill information is achieved, providing a rich source of data for subsequent processing. Through modal decomposition, the independent features of each modality are clarified, which helps to extract accurate bill information, avoid information mixing, and improve data quality. Through cross-modal data fusion, the information of each modality is integrated into a unified standardized structure, providing a clear and neat data framework for subsequent analysis and reducing information redundancy. Through multi-core key feature extraction, key features are extracted for each modality to ensure the comprehensiveness and accuracy of information extraction. Through dynamic forgery feature comparison modeling, it is possible to accurately detect forgery traces (such as forgery of seals and signatures) in bills, thereby enhancing the ability to verify the authenticity of bills. Through consistency verification of seals and signatures, forged bills can be effectively checked and the accuracy of risk identification can be improved. Through abnormal bill clustering analysis, abnormal bills can be quickly identified, the efficiency of abnormal identification can be improved, and missed detection can be avoided. Converting the key feature data of bills into a graph structure can better capture the relationship between bills and their potential risk associations. The full-graph abnormal risk scoring helps to comprehensively evaluate the overall data, identify bills with high potential risks, and improve the accuracy of risk warning. By optimizing the association graph, multi-level information is interconnected, which enhances the dynamic adaptability and accuracy of the risk assessment model. Dynamic rendering of the abnormal association relationship graph can display the relationship changes and risk evolution between bills in real time, helping analysts to respond promptly. Through the real-time push function, risk information and processing results are promptly delivered to the front-end system or decision makers to ensure that abnormal bills are processed in a timely manner and reduce potential losses. Automated risk decision suggestions are generated based on the abnormal push graph, which can quickly provide processing solutions, reduce manual intervention, and improve processing efficiency and decision accuracy. Therefore, the present invention improves the security and accuracy of financial bill identification through multimodal data fusion, forgery feature comparison modeling, abnormal bill clustering analysis and dynamic risk assessment.

[0136] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0137] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A data processing method based on financial information bills, characterized in that: The following steps are involved: Step S1: Acquire multimodal financial information bill data; Performing bill modal decomposition on multimodal financial information bill data to generate financial information bill modal decomposition data; Perform cross-modal data fusion on the modal decomposition data of financial information bills to generate standardized multi-modal hierarchical data; Step S2: extract the multi-core key features of the bill from the standardized multi-modal hierarchical data to obtain the multi-core key feature data of the bill; perform dynamic forgery feature comparison modeling on the multi-core key feature data of the bill to generate forgery feature comparison model data; perform dynamic consistency verification of the seal and signature on the multi-core key feature data of the bill through the forgery feature comparison model data to generate verification feature matching data; Perform abnormal bill clustering analysis on multimodal financial information bill data based on verification feature matching data to generate abnormal bill classification data; Step S3: Initialize the graph structure of the multi-core key feature data of the bill to generate an initial abnormal association relationship graph; perform full-graph abnormal risk scoring on the initial abnormal association relationship graph to generate full-graph abnormal risk scoring data; embed the full-graph abnormal risk scoring data into the initial abnormal association relationship graph to optimize the association graph and generate a bill abnormal association relationship dynamic graph; Step S3 includes the following steps: Step S31: generating node data for abnormal financial information bills according to abnormal bill classification data to obtain abnormal bill node data; performing potential correlation analysis on the abnormal bill node data to generate potential correlation data; Step S32: marking the potential association relationship data as edge data, and initializing the graph structure of the abnormal bill node data through the edge data to generate an initial abnormal association relationship graph; Step S33: Perform node relevance scoring on the initial abnormal association relationship graph to generate node relevance scoring data; perform full-graph abnormal risk scoring on the initial abnormal association relationship graph using the node relevance scoring data to generate full-graph abnormal risk scoring data; Step S34: embedding the full-graph abnormal risk score data into the initial abnormal association relationship graph to optimize the association graph and generate a bill abnormal association relationship dynamic graph; Step S4: dynamically render the bill abnormality association relationship dynamic graph to generate a bill abnormality association relationship rendering graph; transmit the bill abnormality association relationship rendering graph to the system front end for real-time push to generate a bill abnormality push graph; make automated risk decision recommendations based on the bill abnormality push graph to execute abnormal financial information bill processing operations.

2. The data processing method based on financial information bills according to claim 1 is characterized in that: Step S1 includes the following steps: Step S11: Acquire multimodal financial information bill data; Step S12: performing data preprocessing on the multimodal financial information bill data to generate standard multimodal financial information bill data, wherein the data preprocessing includes data cleaning, data denoising, data missing value filling and data standardization; Step S13: performing bill modality decomposition on the standard multimodal financial information bill data to generate financial information bill modality decomposition data, wherein the financial information bill modality decomposition data includes financial information bill image modality, financial information bill text modality and financial information bill layout modality; Step S14: Perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multi-modal hierarchical data.

3. The data processing method based on financial information bills according to claim 2 is characterized in that: Step S13 includes the following steps: Step S131: performing spectral separation on the standard multimodal financial information bill data to generate an image bill RGB layer and a grayscale layer; performing image bill edge extraction on the standard multimodal financial information bill data according to the image bill RGB layer and the grayscale layer to obtain image modality extraction data; Step S132: performing optical character recognition on the standard multimodal financial information bill data to generate bill text content data; performing field type annotation on the standard multimodal financial information bill data based on the bill text content data to generate text modality extraction data; Step S133: performing geometric structure analysis on the standard multimodal financial information bill data to generate bill layout distribution data; performing key area spatial decomposition on the standard multimodal financial information bill data according to the bill layout distribution data to generate layout modal extraction data; Step S134: Perform modal hierarchical feature optimization on the image modal extraction data, the text modal extraction data and the layout modal extraction data to generate financial information bill modal decomposition data, wherein the financial information bill modal decomposition data includes financial information bill image modality, financial information bill text modality and financial information bill layout modality.

4. The data processing method based on financial information bills according to claim 3 is characterized in that: Modal hierarchical feature optimization of image modal extraction data, text modal extraction data, and layout modal extraction data includes: Performing Hough transform on the image modal extraction data to generate image modal tilt data; performing perspective correction on the image modal extraction data according to the image modal tilt data to generate image modal perspective correction data; performing light balance on the image modal perspective correction data to generate financial information bill image modality; Performing context analysis on the text modal extracted data to generate text modal context data; performing text segmentation on the text modal extracted data according to the text modal context data to generate text modal segmented data; performing field association on the text modal extracted data according to the text modal segmented data to generate text modality of financial information bills; Based on the layout modality extraction data, dynamic graph structure is generated to obtain the bill layout structure dynamic graph; the bill layout structure dynamic graph is hierarchically divided to generate the financial information bill layout modality; the financial information bill image modality, financial information bill text modality and financial information bill layout modality are modally optimized and integrated to generate financial information bill modal decomposition data.

5. The data processing method based on financial information bills according to claim 1 is characterized in that: Step S2 includes the following steps: Step S21: extracting multi-core key features of bills from the standardized multi-modal hierarchical data to obtain multi-core key feature data of bills; Step S22: Dynamically perform forgery feature comparison modeling on the multi-core key feature data of the bill to generate forgery feature comparison model data; Step S23: Performing dynamic consistency verification of the seal and signature on the multi-core key feature data of the bill by forging feature comparison model data to generate verification feature matching data; Step S24: Screen the multimodal financial information bill data for abnormal bill information based on the verification feature matching data to obtain abnormal financial information bills; perform abnormal bill clustering analysis on the abnormal financial information bills to generate abnormal bill classification data.

6. The data processing method based on financial information bills according to claim 5 is characterized in that: Step S22 includes: Step S221: extracting the edge texture feature of the bill seal from the standardized multimodal layered data to obtain the edge texture feature data of the bill seal; performing texture frequency analysis on the edge texture feature data of the bill seal to generate the edge texture frequency data of the bill seal; Step S222: Perform high-frequency texture feature screening on the edge texture frequency data of the bill seal to generate high-frequency texture feature data of the edge of the bill seal; perform imprinting area segmentation on the standardized multimodal hierarchical data through the high-frequency texture feature data of the edge of the bill seal to generate imprinting area data of the bill seal; Step S223: performing regional pressure estimation on the bill stamp imprinting area data to generate imprinting force estimation data; performing bill handwritten signature feature extraction on the standardized multimodal layered data to generate bill handwritten signature feature data; performing handwriting change analysis on the bill handwritten signature feature data to generate handwriting signature handwriting change data; Step S224: Dynamically compare and learn the model through the imprint force estimation data and the handwritten signature handwriting change data to generate forgery feature comparison model data.

7. The data processing method based on financial information bills according to claim 6 is characterized in that: Dynamic contrast learning modeling based on the impression force estimation data and handwritten signature handwriting change data includes: Acquire forged bill sample data and real bill sample data based on a preset historical bill database; calculate the minimum Euclidean distance in the modal feature space of the forged bill sample data and the real bill sample data, thereby obtaining comparison bill sample pair data; Based on the stamping force estimation data and the handwritten signature handwriting change data, the forgery comparison loss of the comparison bill sample data is calculated to generate the bill forgery risk score; the stamping force estimation data and the handwritten signature handwriting change data are compared and verified for multiple rounds using the bill forgery risk score to generate the forgery risk score stability index; The forgery risk score stability index is used to learn and model the data of the comparison bill samples to generate forgery feature comparison model data.

8. The data processing method based on financial information bills according to claim 5 is characterized in that: Step S23 includes the following steps: Step S231: Calculate the strength of the bill seal on the bill multi-core key feature data to generate a bill seal strength distribution curve; calculate the seal color saturation on the bill multi-core key feature data to generate bill seal color saturation data; calculate the feature comprehensive similarity of the bill seal strength distribution curve and the bill seal color saturation data according to the forgery feature comparison model data to generate seal authenticity verification data; Step S232: extract the signature handwriting time series from the multi-core key feature data of the bill to obtain handwriting time series data; perform dynamic time regularization matching on the handwriting time series data to generate signature handwriting error data; perform handwriting authenticity scoring on the signature handwriting error data through forged feature comparison model data to generate signature authenticity verification data; Step S233: Verify the seal authenticity verification data and the signature authenticity verification data by integrating the verification data to obtain verification feature matching data.

9. A data processing system based on financial information bills, characterized in that: For executing the data processing method based on financial information bills as claimed in claim 1, the data processing system based on financial information bills comprises: The bill modal decomposition module is used to obtain multimodal financial information bill data; perform bill modal decomposition on the multimodal financial information bill data to generate financial information bill modal decomposition data; perform cross-modal data fusion on the financial information bill modal decomposition data to generate standardized multimodal hierarchical data; The bill verification module is used to extract the multi-core key features of bills from the standardized multi-modal hierarchical data to obtain the multi-core key feature data of bills; to perform dynamic forgery feature comparison modeling on the multi-core key feature data of bills to generate forgery feature comparison model data; to perform dynamic consistency verification of seals and signatures on the multi-core key feature data of bills through the forgery feature comparison model data to generate verification feature matching data; to perform abnormal bill clustering analysis on the multi-modal financial information bill data based on the verification feature matching data to generate abnormal bill classification data; The abnormal bill association module is used to initialize the graph structure of the multi-core key feature data of the bill to generate an initial abnormal association relationship graph; perform full-graph abnormal risk scoring on the initial abnormal association relationship graph to generate full-graph abnormal risk scoring data; embed the full-graph abnormal risk scoring data into the initial abnormal association relationship graph to optimize the association graph and generate a dynamic graph of bill abnormal association relationships; The risk decision module is used to dynamically render the bill abnormality association relationship dynamic graph to generate a bill abnormality association relationship rendering graph; transmit the bill abnormality association relationship rendering graph to the system front end for real-time push to generate a bill abnormality push graph; make automated risk decision recommendations based on the bill abnormality push graph to execute abnormal financial information bill processing operations.

Images