Document data determination method and device and electronic equipment

Abstract

The invention discloses a document data determination method and device and electronic equipment, and relates to the technical field of image recognition and the field of artificial intelligence, and the method comprises the steps: analyzing a document image of a target document, and obtaining region types and region coordinates corresponding to M image regions; performing feature extraction on the document image through a first model to obtain X first features; determining a second model corresponding to each image region based on the region type of the image region, and performing feature extraction on the image region through the second model corresponding to each image region to obtain M second features corresponding to the M image regions; and determining target data based on the region coordinates corresponding to the M image regions, the X first features and the M second features. The technical problem that in the prior art, the accuracy of data recognition is low due to the fact that the content of the unstructured data document is recognized in a manual mode or a single large model is solved.

Description

Technical Field

[0001] This application relates to the fields of image recognition technology and artificial intelligence, and more specifically, to a method, apparatus, and electronic device for determining document data. Background Technology

[0002] With the deepening application of enterprise financial informatization and digitalization, there is an increasing amount of unstructured knowledge data in the enterprise financial domain. How to quickly transform unstructured financial data into structured data required by business is one of the problems that enterprises need to solve.

[0003] In existing technologies, technicians transcribe raw unstructured data into structured data manually or using a single large model. However, unstructured documents (such as invoices, contracts, bank statements, etc.) typically contain complex layouts, seals, handwritten signatures, tables, and other diverse information. Manually identifying and transcribing these formats requires a significant amount of manpower, and relying entirely on manual operation during the information processing carries the risk of errors. Furthermore, single-model solutions also face the following problems:

[0004] (1) Due to the diversity of layouts of unstructured documents, the layout of different documents varies greatly. Large models are limited by the number of training samples and the degree of fit, resulting in limited recognition accuracy.

[0005] (2) Unstructured documents include multimodal content. Large models need to process text, tables, seals, signatures and other elements at the same time, resulting in low recognition accuracy.

[0006] (3) Large models cannot accurately identify financial terms and numerical formats (such as amounts and dates) that are subject to domain specialization.

[0007] There is currently no effective solution to the problem of low accuracy in recognizing document content, including unstructured data, in the existing technologies mentioned above. Summary of the Invention

[0008] This application provides a method, apparatus, and electronic device for determining document data, in order to at least solve the technical problem of low accuracy in data recognition caused by the prior art's method of manually identifying the content of unstructured data documents / using a single large model.

[0009] According to one aspect of this application, a method for determining document data is provided, comprising: analyzing a document image of a target document to obtain region types and region coordinates corresponding to M image regions, where M is a positive integer; extracting features from the document image using a first model to obtain X first features, wherein the X first features include text features and image features extracted from the document image, the first model being a large language model trained based on training images corresponding to U historical documents, where X and U are both positive integers; determining a second model corresponding to each image region based on its region type, and extracting features from the image region using the second model corresponding to each image region to obtain M second features corresponding to the M image regions; and determining target data based on the region coordinates of the M image regions, the X first features, and the M second features, wherein the target data is used to represent the document content of the target document in a structured preset format.

[0010] Optionally, before analyzing the document image of the target document, the method for determining the document data further includes: scanning the target document, which includes unstructured data, to obtain an original image; performing tilt correction on the original image using a preset perspective algorithm to obtain a first image; performing image denoising on the first image to obtain a second image; and performing image enhancement on the second image to obtain the document image of the target document.

[0011] Optionally, the step of extracting features from the document image using the first model includes: inputting the document image into the first model; extracting features from the text content in the document image using the text encoder in the first model to obtain text features from X first features; and extracting features from and enhancing the visualization of the image content in the document image using the convolutional neural network layer in the first model to obtain image features from X first features.

[0012] Optionally, the step of extracting features from each image region using a second model corresponding to that image region to obtain M second features corresponding to the M image regions includes: when the region type of the i-th image region among the M image regions is financial text, extracting features from the financial text information in the i-th image region using the second model corresponding to the i-th image region to obtain initial text features corresponding to the i-th image region; performing semantic correction on the initial text features based on preset financial rules to obtain first text features corresponding to the i-th image region; and normalizing the first text features to obtain second features corresponding to the i-th image region.

[0013] Optionally, the step of extracting features from the image region using the second model corresponding to each image region to obtain M second features corresponding to the M image regions includes: when the region type of the j-th image region among the M image regions is a table type, extracting features from the table structure in the j-th image region using the second model corresponding to the j-th image region to obtain the second features corresponding to the j-th image region.

[0014] Optionally, the step of extracting features from each image region using a second model corresponding to each image region to obtain M second features corresponding to M image regions includes: when the region type of the kth image region among the M image regions is a stamp type, extracting features from the stamp graphic in the kth image region using the second model corresponding to the kth image region to obtain initial stamp features corresponding to the kth image region; obtaining the similarity between the initial stamp features and preset stamp features; generating prompt information when the similarity is less than the preset similarity; and using the initial stamp features as the second features corresponding to the kth image region when the similarity is greater than or equal to the preset similarity.

[0015] Optionally, the step of determining target data based on the region coordinates corresponding to M image regions, X first features, and M second features includes: determining the correspondence between the X first features and the M second features based on the region coordinates corresponding to the M image regions; performing weighted feature fusion on the X first features and the M second features based on the correspondence between the X first features and the M second features to obtain target features; and performing format conversion on the target features based on a preset format to obtain target data.

[0016] Optionally, after determining the target data based on the region coordinates corresponding to M image regions, X first features, and M second features, the method for determining the document data further includes: using a first model to perform logical rule verification on the target data based on preset prompt words and the context information of the target document; if the verification result corresponding to the logical rule verification is an abnormal result, generating prompt information, wherein the prompt information is used to prompt the business personnel corresponding to the target document to correct the target data.

[0017] According to another aspect of this application, a document data determination apparatus is also provided, comprising: an analysis unit for analyzing a document image of a target document to obtain region types and region coordinates corresponding to M image regions, wherein M is a positive integer; a first extraction unit for extracting features from the document image using a first model to obtain X first features, wherein the X first features include text features and image features extracted from the document image, the first model being a large language model trained based on training images corresponding to U historical documents, wherein X and U are both positive integers; a second extraction unit for determining a second model corresponding to each image region based on the region type of each image region, and extracting features from the image region using the second model corresponding to each image region to obtain M second features corresponding to the M image regions; and a first determination unit for determining target data based on the region coordinates corresponding to the M image regions, the X first features, and the M second features, wherein the target data is used to represent the document content of the target document using a structured preset format.

[0018] According to another aspect of this application, a computer program product is also provided, which stores a computer program, wherein a method for determining document data that controls the computer program product to execute any of the above-mentioned items is provided when the computer program is running.

[0019] According to another aspect of this application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining document data as described above.

[0020] In this application, the document image of the target document is first analyzed to obtain the region type and region coordinates corresponding to M image regions, where M is a positive integer. A first model is used to extract features from the document image to obtain X first features, which include text features and image features extracted from the document image. The first model is a large language model trained on training images corresponding to U historical documents, where X and U are both positive integers. Then, based on the region type of each image region, this application determines the second model corresponding to that image region. Features are extracted from each image region using the second model corresponding to that image region to obtain M second features corresponding to M image regions. Finally, based on the region coordinates of the M image regions, the X first features, and the M second features, this application determines the target data, which is used to represent the document content of the target document using a structured preset format.

[0021] As can be seen from the above, this application adopts a collaborative application of large and small models (the large model is the first model and the small model is the second model). By integrating the global analysis capabilities of the large model and the refined detection methods of the small model, it achieves the goal of automatically and quickly identifying and converting unstructured document content in the financial domain into structured data, thereby solving the dilemma of unstructured financial data conversion caused by the diversity of layouts, multimodal content processing, and domain professional requirements.

[0022] Specifically, this application first performs image analysis on the target document to identify M key regions and their types; then, it uses a large language model trained on U historical documents to extract features, obtaining X comprehensive features; next, this application selects the corresponding second model according to the region type to perform deep feature extraction on each region; finally, this application combines the text / image comprehensive features extracted by the large model with the professional features extracted by the small model to perform feature fusion, ultimately obtaining structured financial data, thereby achieving the technical effect of improving the recognition accuracy of document data, and thus solving the technical problem of low data recognition accuracy caused by the recognition of unstructured data document content by manual methods / a single large model in the prior art. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0024] Figure 1 This is a flowchart of an optional method for determining document data according to an embodiment of this application;

[0025] Figure 2 This is a flowchart of an optional method for converting unstructured document data into structured data according to an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of an optional document data determination device according to an embodiment of this application;

[0027] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] It should also be noted that all information and data (including but not limited to information used for display and analysis) and data (including but not limited to document images of the target document) involved in this application are information and data authorized by the user or fully authorized by all parties. For example, if there is an interface between this system and the relevant user or organization, before obtaining the relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent from the aforementioned user or organization.

[0031] Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of relevant information and data involved in this application all comply with the relevant laws, regulations, and standards of the relevant regions, and necessary confidentiality measures have been taken. This application does not violate public order and good morals. In addition, this application provides a corresponding operation entry point for users to choose to agree to or refuse authorization. If the user chooses to refuse authorization, the corresponding expert decision-making process will be initiated.

[0032] According to an embodiment of this application, an embodiment of a method for determining document data is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This application provides a document data determination system (hereinafter referred to as the determination system) for performing the document data determination method of this application. Figure 1 This is a flowchart of an optional method for determining document data according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0034] Step S101: Analyze the document image of the target document to obtain the region type and region coordinates corresponding to M image regions, where M is a positive integer.

[0035] Optionally, the system is capable of analyzing document images of target documents (e.g., financial invoices or contract documents requiring data recognition) using image processing techniques (e.g., algorithms based on OpenCV (Open Source Computer Vision Library, an open-source computer vision and machine learning software library) or PyTorch (an open-source machine learning framework)). The aim is to identify and locate different types of image regions included in the document (e.g., text regions, table regions, seal regions, and signature regions, etc.), where each image region corresponds to a region type and region coordinates.

[0036] Optionally, by performing layout analysis on the target document, the system can selectively choose the corresponding second model to perform more refined analysis on each image region based on the region type of each image region, rather than blindly processing the entire document image uniformly through a single large model. In this case, the region type and region coordinates identified in step S101 provide a data foundation for the subsequent second model to perform more refined image data recognition, thereby improving the accuracy of document data recognition.

[0037] Step S102: Extract features from the document image using the first model to obtain X first features, where the X first features include text features and image features extracted from the document image. The first model is a large language model trained based on training images corresponding to U historical documents, where X and U are both positive integers.

[0038] Optionally, in step S102, the system-driven large model (i.e., the first model) is determined to perform a global analysis of the document image, thereby obtaining multimodal data features (i.e., X first features) including text features and image features.

[0039] Optionally, the system can extract features through a large language model, which can capture the complex semantic and visual information contained in the document image. Since target documents in the financial field usually contain a large number of professional terms and complex table structures, the large language model can understand the meaning of these professional terms and the logical meaning of the tables to a certain extent, thereby accurately identifying and classifying the multimodal data features that represent the content of the document.

[0040] Step S103: Determine the second model corresponding to each image region based on the region type of each image region, and extract features from the image region using the second model corresponding to each image region to obtain M second features corresponding to M image regions.

[0041] Optionally, in step S103, the system selects a corresponding second model for each image region's region type. This selective processing method allows the system to use a more targeted second model for feature extraction for different types of image regions (such as seals or signatures), thereby obtaining more accurate specific features corresponding to the M image regions, i.e., M second features.

[0042] Optionally, the use of a second model ensures that the determination system can process specific types of image regions with high precision. For example, for the detection of seals and signatures, the determination system can use a specially trained "small model" (i.e., the second model). Specific types of "small models" perform better on these specific tasks and can accurately identify the authenticity of seals and the style of signatures, thereby improving the accuracy of document image region data recognition. Furthermore, the use of the second model makes up for the shortcomings of the large model in processing specific details of document images, thereby improving the accuracy of the final recognized document data.

[0043] Step S104: Determine target data based on the region coordinates corresponding to M image regions, X first features, and M second features. The target data is used to represent the document content of the target document through a structured preset format.

[0044] Optionally, after integrating the region coordinates, X first features, and M second features obtained in the above steps, the system determines that, through logical processing and feature fusion technology, it performs feature fusion on the first and second features of the corresponding image region based on the region coordinates. Then, the fused data features are converted into structured target data in a preset format, such as JSON (JavaScript Object Notation, a lightweight data exchange format) or XML (eXtensible Markup Language), which facilitates direct connection to the enterprise's financial business system and facilitates subsequent business processing.

[0045] Optionally, step S104 determines that the system can convert unstructured information in financial documents into a structured data format. This not only facilitates information storage and retrieval but also greatly improves data readability and usability, enabling enterprises to quickly apply this information to the financial system for data analysis, report generation, and other operations. Simultaneously, the structured data output also facilitates subsequent data verification and auditing processes, reducing the need for manual intervention and increasing the degree of automation.

[0046] As can be seen from the above, this application adopts a collaborative application of large and small models (the large model is the first model and the small model is the second model). By integrating the global analysis capabilities of the large model and the refined detection methods of the small model, it achieves the goal of automatically and quickly identifying and converting unstructured document content in the financial domain into structured data, thereby solving the dilemma of unstructured financial data conversion caused by the diversity of layouts, multimodal content processing, and domain professional requirements.

[0047] Specifically, this application first performs image analysis on the target document to identify M key regions and their types; then, it uses a large language model trained on U historical documents to extract features, obtaining X comprehensive features; next, this application selects the corresponding second model according to the region type to perform deep feature extraction on each region; finally, this application combines the text / image comprehensive features extracted by the large model with the professional features extracted by the small model to perform feature fusion, ultimately obtaining structured financial data, thereby achieving the technical effect of improving the recognition accuracy of document data, and thus solving the technical problem of low data recognition accuracy caused by the recognition of unstructured data document content by manual methods / a single large model in the prior art.

[0048] In one optional embodiment, before analyzing the document image of the target document, the determination system first scans the target document, which includes unstructured data, to obtain an original image. Then, the determination system performs tilt correction on the original image using a preset perspective algorithm to obtain a first image. Next, the determination system performs image denoising on the first image to obtain a second image. Finally, the determination system performs image enhancement on the second image to obtain the document image of the target document.

[0049] Optionally, target documents refer to various unstructured documents in the enterprise's financial field that need to be converted into structured data, such as invoices, contracts, bank statements, etc. Unstructured data refers to information without a fixed format or predefined data model, and often exists in the form of free text, images, audio, etc.

[0050] For example, the system can automatically detect the edges of a document image and obtain a perspective matrix using a preset perspective transformation algorithm (e.g., Homography matrix transformation). Then, the angle and perspective effect of the document image are adjusted based on the perspective matrix.

[0051] Optionally, tilt correction is used to eliminate the tilt angle that may occur in the target document during scanning or shooting, thereby ensuring that the document image used for subsequent data recognition is aligned in the horizontal and vertical directions. In other words, the system improves the image quality of the document image through tilt correction, making subsequent character recognition and information extraction more accurate. At the same time, through automated image processing, the need for technicians to manually adjust the document and rescan it can be avoided, thereby improving the processing efficiency and accuracy of batch financial documents.

[0052] Optionally, image denoising refers to removing random and messy pixels from an image during image processing to reduce noise interference and improve the clarity of the preprocessed document image. This process can employ filter techniques, such as Gaussian filtering, median filtering, and nonlocal mean filtering. Optionally, the system can also utilize a pre-trained deep learning model for denoising.

[0053] Optionally, the system improves image quality through noise reduction, especially for documents with complex backgrounds or low-quality scans, significantly enhancing the accuracy of OCR (Optical Character Recognition). By reducing irrelevant information and interference in the image, the system ensures the clear presentation of key information, thereby improving the efficiency and reliability of the entire document parsing process.

[0054] For example, the system can be determined to enhance the text contrast in document images by combining an adaptive binarization algorithm with the adaptive threshold in OpenCV.

[0055] Optionally, image enhancement not only improves the visual clarity of text images, but also enables the OCR engine in the model to more effectively recognize and parse characters. By enhancing the text portion of the image, the system reduces noise interference, thereby improving the accuracy of character recognition. This ensures that even when the scanning quality of the original image is poor, the system can still extract high-quality information through subsequent preprocessing techniques, thus providing a reliable business data foundation for the enterprise's intelligent financial applications.

[0056] In one optional embodiment, the determining system first inputs a document image into a first model. Then, the determining system extracts features from the text content in the document image using a text encoder in the first model to obtain text features from X first features. Then, the determining system extracts features from the image content in the document image and enhances its visualization using a convolutional neural network layer in the first model to obtain image features from X first features.

[0057] Optionally, after inputting the document image into the first model, the system drives the large model (i.e., the first model) to perform a global analysis of the document image. Through the text encoder in the first model (e.g., BERT (Bidirectional Encoder Representations from Transformers)), the text content and its semantic information in the document image can be effectively extracted, thereby obtaining the text features among the X first features. Through the convolutional neural network (CNN) layer, the features of non-text elements such as seals, signatures, and tables can be extracted from the document image. Furthermore, the system can also obtain the image features among the X first features by visually enhancing the features of non-text elements, thereby further improving the quality of the extracted X first features. Thus, by calling the first model to extract features from the text content and non-text content in the document image respectively, the system improves the accuracy and feature extraction efficiency of the overall X first features obtained.

[0058] Optionally, the text features in the X first features are w1, w2, ..., w in the following formula. n As shown, V t The first model extracts the overall text features from the document image.

[0059] V t =BERT(w1, w2, ..., w n );

[0060] Optionally, the overall image features extracted from the document image by the first model are processed by V. c express.

[0061] In one optional embodiment, when the region type of the i-th image region in the M image regions is financial text, the determination system extracts features from the financial text information in the i-th image region using the second model corresponding to the i-th image region to obtain the initial text features corresponding to the i-th image region. Then, the determination system performs semantic error correction on the initial text features based on preset financial rules to obtain the first text features corresponding to the i-th image region. Finally, the determination system normalizes the first text features to obtain the second features corresponding to the i-th image region.

[0062] Optionally, i is a positive integer less than or equal to M, and the i-th image region refers to one of the specific image regions identified in the document image that contains financial text information.

[0063] Optionally, the image region of financial text type refers to the text region containing financial terms and numerical formats (such as amount and date) related to finance and accounting. The second model corresponding to the i-th image region refers to a model trained on training images corresponding to multiple historical documents containing financial text information, which is used to process and recognize financial text information in a specific region.

[0064] Optionally, the system improves the targeting and efficiency of document image processing by segmenting the document image into multiple image regions and using the corresponding second model only on those regions identified as "financial text type". By using a specific second model to perform deep feature extraction on financial text information, it can more accurately identify and understand complex financial terms and numerical formats, thereby improving the recognition accuracy and domain adaptability of document data.

[0065] Optionally, the system determines the accuracy of the extracted key financial data such as numbers and dates by performing semantic correction on the initial text features according to preset financial rules and introducing professional knowledge and detection rules in the financial field.

[0066] For example, semantic error correction can correct "1000 yuan" in the initial text features to "1000￥".

[0067] Optionally, the system determines that the first text feature is normalized, that is, the fields in the first text feature are normalized, and the format of amount and date is matched by regular expression, for example, "2023-12-01" is converted to "20231201".

[0068] Optionally, normalization ensures consistency of different types of financial data (such as currency units and date formats) in subsequent processing, avoiding the problem of low processing efficiency caused by inconsistent data formats. The second feature is the final feature representation obtained after normalization of the first text feature, which is more suitable for subsequent unified data analysis and integration. This processing makes structured data easier for the financial system to understand and use, facilitating tasks such as financial accounting, report generation, and data integration, thereby improving the efficiency and automation level of subsequent financial business processing.

[0069] In one optional embodiment, if the region type of the j-th image region in the M image regions is a table type, the system determines the second feature corresponding to the j-th image region by extracting the features of the table structure in the j-th image region through the second model corresponding to the j-th image region.

[0070] Optionally, j is a positive integer less than or equal to M, and the j-th image region refers to one of the specific image regions identified in the document image that contains table information.

[0071] Optionally, the second model corresponding to the j-th image region refers to a model trained on training images corresponding to multiple historical documents containing table information, used to process and recognize table information within a specific region.

[0072] Optionally, the second feature corresponding to the j-th image region describes the internal structure of the table in detail, including columns, rows, cell boundaries, table content information, and cross-page table structure.

[0073] Optionally, feature extraction can be performed on image regions of table type using a specific type of second model, thereby accurately identifying the table location and internal structure of the table in the document, including the boundary information of rows, columns, and cells. This avoids misprocessing of non-table content, ensures the accuracy of information extraction, and for financial documents containing a large number of tables, using a dedicated second model to process table regions can speed up the overall processing speed, improve processing accuracy, and reduce unnecessary waste of computing resources.

[0074] In one optional embodiment, when the region type of the kth image region in the M image regions is a stamp type, the system first extracts features from the stamp graphic in the kth image region using the second model corresponding to the kth image region to obtain the initial stamp features corresponding to the kth image region. Then, the system obtains the similarity between the initial stamp features and the preset stamp features. If the similarity is less than the preset similarity, a prompt message is generated. Otherwise, if the similarity is greater than or equal to the preset similarity, the system uses the initial stamp features as the second feature corresponding to the kth image region.

[0075] Optionally, k is a positive integer less than or equal to M, and the k-th image region refers to one of the specific image regions identified in the document image that contains stamp information.

[0076] Optionally, the second model corresponding to the k-th image region refers to a model trained on training images corresponding to multiple historical documents containing seal information, used to process and identify seal information within a specific region.

[0077] Optionally, the second feature corresponding to the k-th image region provides detailed information about the shape, texture, and size of the seal.

[0078] Optionally, by using a second model of a specific type to extract features from image regions of the seal type, subtle features of the seal can be captured more accurately, providing a basis for subsequent seal authenticity judgment and reducing the possibility of false alarms and missed detections.

[0079] Optionally, preset seal features refer to the standard seal features pre-entered into the system, which are usually derived from seal templates provided by enterprises. These features are used to compare with the extracted initial seal features to determine the degree of matching (i.e., similarity) between the initial seal features and the preset seal features through methods such as SIFT feature matching and feature matching based on convolutional neural networks.

[0080] Optionally, when a seal is verified as invalid (i.e., the similarity is less than the preset similarity), the system determines that a warning message can be generated to warn the user, thus preventing potential fraudulent behavior; when a seal is verified as valid (i.e., the similarity is greater than or equal to the preset similarity), the extracted seal features will become part of the document's structured information, thereby ensuring that only verified seal features will be officially recorded and used, further guaranteeing the security of subsequent financial business processing.

[0081] In one optional embodiment, the determination system first determines the correspondence between X first features and M second features based on the region coordinates corresponding to M image regions. Then, based on the correspondence between the X first features and M second features, the determination system performs weighted feature fusion on the X first features and M second features to obtain target features. Finally, the determination system performs format conversion on the target features based on a preset format to obtain target data.

[0082] Optionally, region coordinates refer to the specific location information of each image region in the document, usually represented by coordinate pairs, which are used for subsequent processing to locate and associate features.

[0083] Optionally, when processing financial documents, the system determines that X primary features and M secondary features are accurately associated using regional coordinates, providing the necessary logical basis for subsequent feature fusion, thereby ensuring the continuity and consistency of information processing.

[0084] Optionally, the calculation process for the target feature V is shown in the following formula:

[0085] αv t +βV c +γV g ;

[0086] In the above formula, V t V represents the overall text features extracted from the document image by the first model. c V represents the overall image features extracted from the document image by the first model. g V represents the local image features formed by fusing M second features extracted from the document image by the second model. t V c and V g It is the basic information carrier in the system, used to provide representations from different dimensions and perspectives, and to help identify and capture document image data from multiple dimensions.

[0087] In the above formula, α, β and γ are all weight coefficients of vector features, used to determine the "contribution value" of different modal features in feature fusion. The above weight coefficients can be optimized by grid search to find the best combination of weight coefficients.

[0088] Optionally, while determining the system output target feature V, the system also determines the classification decision based on the probability distribution P output by Softmax (an activation function). That is, if max(P) (the maximum value in the probability distribution P) is greater than or equal to 0.9, the target data is directly determined based on the target feature; if max(P) is less than 0.9, an interactive process is triggered to update the target feature through the first model and the second model until max(P) is greater than or equal to 0.9.

[0089] Optionally, feature fusion can combine the general understanding of a large model with the specialized detection capabilities of a small model. Through weighted fusion, the system can fuse the key regional information (such as the authenticity of the seal, amount, etc.) emphasized by the document image with the features extracted by the general large model. This fusion strategy ensures the comprehensiveness and accuracy of the final target features, which can more accurately reflect the true content of the document, thereby improving the accuracy of document information extraction.

[0090] Optionally, the default format refers to a standard format defined for structured data output, such as JSON and XML formats.

[0091] Optionally, format conversion serves as a bridge to transform extracted features into business-usable data. By predefining preset formats, it ensures that the target data is seamlessly received and parsed by the enterprise's internal financial system, simplifying the post-processing workflow of document data and improving the processing efficiency of document data. At the same time, the output of standard formats also facilitates the unified management of data, making it easier to compare and analyze document information from different sources, thereby supporting more efficient and accurate financial business decisions.

[0092] In one optional embodiment, after determining the target data based on the region coordinates corresponding to M image regions, X first features, and M second features, the determination system first performs logical rule verification on the target data using a first model based on preset prompt words and context information of the target document. Then, if the verification result corresponding to the logical rule verification is an abnormal result, the determination system generates prompt information, wherein the prompt information is used to prompt the business personnel corresponding to the target document to correct the target data.

[0093] Optionally, the system determines that the corrected data (x′, y′) will be stored in a circular buffer, where x′ is a new input feature (such as an image, text, structured data, etc.), which is a "new knowledge carrier" for model learning and is used to supplement the model with data of new scenarios and new patterns, and y′ is the label corresponding to x′ (the category for classification tasks, the target value for regression, etc.), which is a "target guide" for model learning and lets the model know what result to fit for the new data.

[0094] Furthermore, it is determined that the EWC (Elastic Weight Consolidation) loss function is triggered every 1000 accumulated data points, and the update process is shown in the following formula:

[0095]

[0096] In the above formula, L ce (Cross-entropy loss) represents the core loss value for classification tasks, measuring the difference between the "predicted result on new data" and the "label y", thus driving the model to learn patterns in new data (such as new categories or distributions); λ is the weight coefficient of the EWC regularization term. The larger λ is, the stronger the influence on the old model parameters θ. * The constraints of λ prevent the model from overfitting new data and forgetting old knowledge; conversely, the smaller λ is, the more emphasis is placed on learning new knowledge. i (Fisher information matrix), based on the old model parameters θ * The calculations show that different parameters contribute to the fitting of the old data; the old model parameter θ * It represents the parameter state of the model before incremental learning (i.e., the previous stage), and represents the "old knowledge" that the model has learned. In EWC loss, it is used as a constraint anchor point so that when the new parameters are updated, there will be no large deviation, thus achieving a balance between "new knowledge learning + old knowledge retention".

[0097] Optionally, the system can perform field logic validation on keywords in the target data using the rule engine integrated in the first model. For example, it can verify whether the invoice amount equals the sum of the tax-inclusive amount and the tax amount. Then, if the validation result corresponding to the logical rule validation is an abnormal result, the system can generate a prompt message based on the contradictory field. For example, "Amount inconsistent, please check".

[0098] Optionally, the system can perform deep logical rule verification on structured target data based on preset prompts and contextual information of the target document. This ensures that the system can detect and mark abnormal data that violates logical rules. This method greatly improves the accuracy and efficiency of data verification, reduces the workload of subsequent manual review, and generates prompts immediately when logical rule verification detects anomalies. This allows business personnel to quickly locate the problem and accelerates the correction efficiency of target data through an instant feedback mechanism. In this way, enterprises can effectively reduce business delays caused by inaccurate data, thereby improving the automation level and accuracy of financial document processing, and ultimately enhancing the transparency and controllability of the entire financial data management.

[0099] As can be seen from the above, this application adopts a collaborative application of large and small models (the large model is the first model and the small model is the second model). By integrating the global analysis capabilities of the large model and the refined detection methods of the small model, it achieves the goal of automatically and quickly identifying and converting unstructured document content in the financial domain into structured data, thereby solving the dilemma of unstructured financial data conversion caused by the diversity of layouts, multimodal content processing, and domain professional requirements.

[0100] Specifically, this application first performs image analysis on the target document to identify M key regions and their types; then, it uses a large language model trained on U historical documents to extract features, obtaining X comprehensive features; next, this application selects the corresponding second model according to the region type to perform deep feature extraction on each region; finally, this application combines the text / image comprehensive features extracted by the large model with the professional features extracted by the small model to perform feature fusion, ultimately obtaining structured financial data, thereby achieving the technical effect of improving the recognition accuracy of document data, and thus solving the technical problem of low data recognition accuracy caused by the recognition of unstructured data document content by manual methods / a single large model in the prior art.

[0101] In one alternative embodiment, Figure 2 This is a flowchart illustrating an optional method for converting unstructured document data into structured data according to an embodiment of this application, such as... Figure 2 As shown, this method, based on artificial intelligence technology, provides a data model that can convert unstructured document data that needs to be entered into the financial system into structured data, and provides it to the corresponding business system through an interface. The main implementation process adopts a pipeline architecture of "unstructured financial data collection → preprocessing → multimodal analysis → hierarchical recognition → result fusion → data output → standard API (Application Programming Interface) output". Combining the semantic understanding ability of the large model with the professional detection ability of the small model, it realizes end-to-end intelligent parsing of unstructured financial data.

[0102] exist Figure 2First, the image / PDF (Portable Document Format) files corresponding to the unstructured documents are uploaded to the system. A preprocessing small model preprocesses the original images corresponding to the uploaded image / PDF files, including image correction, layout analysis, and noise reduction. Then, based on the prompt word engineering, a multimodal large model (i.e., the first model) is called to perform a global analysis of the preprocessed document images. A vertical category small model (i.e., the second model) performs fine-grained detection on the preprocessed document images, including seal recognition, signature detection, text detection, table recognition, and title detection. Finally, the system fuses the results extracted from the large and small models to obtain structured data. The specific implementation steps are as follows:

[0103] (1) Collection of unstructured financial data:

[0104] 1. Collect unstructured financial documents, including invoice images, PDF documents, etc., based on actual business needs.

[0105] 2. Upload unstructured documents.

[0106] (2) Document preprocessing and image enhancement:

[0107] 1. Image correction:

[0108] Input: Original image of scanned financial invoices / financial contract documents (PDF).

[0109] Output: Aligned front view.

[0110] 2. Noise Reduction and Enhancement:

[0111] Input: The corrected front view.

[0112] Output: Denoising and enhanced unstructured documents and images.

[0113] (3) Multimodal layout analysis:

[0114] 1. Large-scale model-driven global analysis:

[0115] Input: The image after document preprocessing and image enhancement.

[0116] Output: JSON format layout structure, including area type, coordinates, and confidence level.

[0117] 2. Refined detection for small models:

[0118] Seal detection based on a small model; signature detection based on a small model; differentiation between printed and handwritten signatures based on a small model.

[0119] (4) Layered content recognition:

[0120] Input: Text or table content.

[0121] Output: Structured data in CSV (Comma-Separated Values) or HTML (Hyper Text Markup Language) format.

[0122] (5) Results fusion and output:

[0123] Input: Structured data in CSV or HTML format.

[0124] Output: Data format is JSON / XML, and it supports integration with financial business systems through a unified standard data model.

[0125] (6) Provide a unified standard API:

[0126] Based on the needs of various business systems, we provide APIs that follow the principle of unified interface design for invocation.

[0127] This application embodiment can also provide a document data determination device. It should be noted that the document data determination device in this application embodiment can be used to execute the document data determination method provided in this application embodiment. The document data determination device provided in this application embodiment is described below.

[0128] According to embodiments of this application, an apparatus for implementing the above-described method for determining document data is also provided. Figure 3 This is a schematic diagram of an optional document data determination device according to an embodiment of this application, such as... Figure 3 As shown, the device includes: an analysis unit 301, a first extraction unit 302, a second extraction unit 303, and a first determination unit 304.

[0129] Optionally, the analysis unit 301 is used to analyze the document image of the target document to obtain the region type and region coordinates corresponding to M image regions, where M is a positive integer; the first extraction unit 302 is used to extract features from the document image using a first model to obtain X first features, where the X first features include text features and image features extracted from the document image, and the first model is a large language model trained based on training images corresponding to U historical documents, where X and U are both positive integers; the second extraction unit 303 is used to determine the second model corresponding to each image region based on the region type of each image region, and extract features from the image region using the second model corresponding to each image region to obtain M second features corresponding to M image regions; the first determination unit 304 is used to determine the target data based on the region coordinates corresponding to the M image regions, the X first features, and the M second features, where the target data is used to represent the document content of the target document in a structured preset format.

[0130] In one optional embodiment, the document data determination device further includes a scanning unit, a correction unit, a noise reduction unit, and an enhancement unit.

[0131] Optionally, the scanning unit is used to scan the target document including unstructured data to obtain an original image; the correction unit is used to perform tilt correction on the original image using a preset perspective algorithm to obtain a first image; the denoising unit is used to perform image denoising on the first image to obtain a second image; and the enhancement unit is used to perform image enhancement on the second image to obtain a document image of the target document.

[0132] In one optional embodiment, the first extraction unit includes an input subunit, a first extraction subunit, and a second extraction subunit.

[0133] Optionally, the input subunit is used to input the document image into the first model; the first extraction subunit is used to extract features from the text content in the document image through the text encoder in the first model to obtain text features from X first features; the second extraction subunit is used to extract features from the image content in the document image and enhance visualization through the convolutional neural network layer in the first model to obtain image features from X first features.

[0134] In one optional embodiment, the second extraction unit includes: a third extraction subunit, a semantic error correction subunit, and a normalization processing subunit.

[0135] Optionally, the third extraction subunit is used to extract features from the financial text information in the i-th image region using the second model corresponding to the i-th image region when the region type of the i-th image region in the M image regions is financial text, to obtain the initial text features corresponding to the i-th image region; the semantic correction subunit is used to perform semantic correction on the initial text features based on preset financial rules, to obtain the first text features corresponding to the i-th image region; and the normalization processing subunit is used to normalize the first text features to obtain the second features corresponding to the i-th image region.

[0136] In an optional embodiment, the second extraction unit further includes a fourth extraction subunit.

[0137] Optionally, the fourth extraction subunit is used to extract features from the table structure in the j-th image region using the second model corresponding to the j-th image region when the region type of the j-th image region in the M image regions is a table type, so as to obtain the second feature corresponding to the j-th image region.

[0138] In one optional embodiment, the second extraction unit further includes a fifth extraction subunit, an information generation subunit, and a first determination subunit.

[0139] Optionally, the fifth extraction subunit is used to extract features from the stamp image in the kth image region using the second model corresponding to the kth image region when the region type of the kth image region in the M image regions is a stamp type, thereby obtaining the initial stamp features corresponding to the kth image region; the information generation subunit is used to obtain the similarity between the initial stamp features and the preset stamp features, and generate prompt information when the similarity is less than the preset similarity; the first determination subunit is used to use the initial stamp features as the second feature corresponding to the kth image region when the similarity is greater than or equal to the preset similarity.

[0140] In one optional embodiment, the first determining unit includes: a second determining subunit, a feature fusion subunit, and a format conversion subunit.

[0141] Optionally, the second determining subunit is used to determine the correspondence between X first features and M second features based on the region coordinates corresponding to M image regions; the feature fusion subunit is used to perform weighted feature fusion on the X first features and M second features based on the correspondence between the X first features and M second features to obtain target features; and the format conversion subunit is used to perform format conversion on the target features based on a preset format to obtain target data.

[0142] In one optional embodiment, the document data determination device further includes a rule verification unit and a correction unit.

[0143] Optionally, the rule verification unit is used to perform logical rule verification on the target data based on preset prompt words and context information of the target document through the first model; the correction unit is used to generate prompt information when the verification result corresponding to the logical rule verification is an abnormal result, wherein the prompt information is used to prompt the business personnel corresponding to the target document to correct the target data.

[0144] As can be seen from the above, this application adopts a collaborative application of large and small models (the large model is the first model and the small model is the second model). By integrating the global analysis capabilities of the large model and the refined detection methods of the small model, it achieves the goal of automatically and quickly identifying and converting unstructured document content in the financial domain into structured data, thereby solving the dilemma of unstructured financial data conversion caused by the diversity of layouts, multimodal content processing, and domain professional requirements.

[0145] Specifically, this application first performs image analysis on the target document to identify M key regions and their types; then, it uses a large language model trained on U historical documents to extract features, obtaining X comprehensive features; next, this application selects the corresponding second model according to the region type to perform deep feature extraction on each region; finally, this application combines the text / image comprehensive features extracted by the large model with the professional features extracted by the small model to perform feature fusion, ultimately obtaining structured financial data, thereby achieving the technical effect of improving the recognition accuracy of document data, and thus solving the technical problem of low data recognition accuracy caused by the recognition of unstructured data document content by manual methods / a single large model in the prior art.

[0146] It should be noted that the analysis unit 301, the first extraction unit 302, the second extraction unit 303, and the first determination unit 304 mentioned above correspond to steps S101 to S104 in the method embodiment. The four units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments. It should be noted that the above modules or units can be hardware components or software components stored in memory and processed by one or more processors. The above modules can also be part of the device and run in the computer terminal provided in the embodiment.

[0147] Embodiments of this application can also provide an electronic device. Figure 4 This is a structural block diagram of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, the electronic device includes: one or more ( Figure 4 Only one of the following is shown: processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0148] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0149] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: analyze the document image of the target document to obtain the region type and region coordinates corresponding to M image regions, where M is a positive integer; extract features from the document image using a first model to obtain X first features, where the X first features include text features and image features extracted from the document image, and the first model is a large language model trained based on training images corresponding to U historical documents, where X and U are both positive integers; determine the second model corresponding to each image region based on the region type of each image region, and extract features from the image region using the second model corresponding to each image region to obtain M second features corresponding to M image regions; determine the target data based on the region coordinates corresponding to the M image regions, the X first features, and the M second features, where the target data is used to represent the document content of the target document in a structured preset format.

[0150] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: scanning the target document including unstructured data to obtain the original image; correcting the tilt of the original image using a preset perspective algorithm to obtain the first image; denoising the first image to obtain the second image; and enhancing the second image to obtain the document image of the target document.

[0151] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: inputting the document image into the first model; extracting features from the text content in the document image through the text encoder in the first model to obtain text features from X first features; and extracting features and enhancing the visualization of the image content in the document image through the convolutional neural network layer in the first model to obtain image features from X first features.

[0152] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: when the region type of the i-th image region in the M image regions is financial text, the financial text information in the i-th image region is extracted using the second model corresponding to the i-th image region to obtain the initial text features corresponding to the i-th image region; the initial text features are semantically corrected based on preset financial rules to obtain the first text features corresponding to the i-th image region; the first text features are normalized to obtain the second features corresponding to the i-th image region.

[0153] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: when the region type of the j-th image region in the M image regions is a table type, the second model corresponding to the j-th image region is used to extract features from the table structure in the j-th image region to obtain the second feature corresponding to the j-th image region.

[0154] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: when the region type of the kth image region in the M image regions is a seal type, extract the features of the seal graphic in the kth image region through the second model corresponding to the kth image region to obtain the initial seal features corresponding to the kth image region; obtain the similarity between the initial seal features and the preset seal features; if the similarity is less than the preset similarity, generate a prompt message; if the similarity is greater than or equal to the preset similarity, use the initial seal features as the second feature corresponding to the kth image region.

[0155] The processor can also call the information and application program stored in the memory through the transmission device to perform the following steps: determine the correspondence between X first features and M second features based on the region coordinates corresponding to M image regions; perform weighted feature fusion on the X first features and M second features based on the correspondence between the X first features and M second features to obtain target features; and perform format conversion on the target features based on a preset format to obtain target data.

[0156] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: using the first model, based on the preset prompt words and the context information of the target document, to perform logical rule verification on the target data; if the verification result corresponding to the logical rule verification is an abnormal result, to generate a prompt message, wherein the prompt message is used to prompt the business personnel corresponding to the target document to correct the target data.

[0157] As can be seen from the above, the embodiments of this application provide a method for determining document data. This application adopts a collaborative application of large and small models (the large model is the first model and the small model is the second model). By integrating the global analysis capabilities of the large model and the refined detection methods of the small model, it achieves the purpose of automatically and quickly identifying and converting unstructured document content in the financial domain into structured data, thereby solving the dilemma of unstructured financial data conversion caused by the diversity of layouts, multimodal content processing, and domain professional requirements.

[0158] Specifically, this application first performs image analysis on the target document to identify M key regions and their types; then, it uses a large language model trained on U historical documents to extract features, obtaining X comprehensive features; next, this application selects the corresponding second model according to the region type to perform deep feature extraction on each region; finally, this application combines the text / image comprehensive features extracted by the large model with the professional features extracted by the small model to perform feature fusion, ultimately obtaining structured financial data, thereby achieving the technical effect of improving the recognition accuracy of document data, and thus solving the technical problem of low data recognition accuracy caused by the recognition of unstructured data document content by manual methods / a single large model in the prior art.

[0159] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. The electronic device can also be a smartphone, tablet, PDA, or other mobile internet device. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0160] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0161] Embodiments of this application may also provide a storage medium.

[0162] Optionally, in this embodiment of the application, the storage medium can be used to store the program code executed by the method for determining document data provided in the above method embodiments.

[0163] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0164] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform method steps for determining document data.

[0165] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0166] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0170] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0171] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for determining document data, characterized in that, include: The document image of the target document is analyzed to obtain the region type and region coordinates corresponding to M image regions, where M is a positive integer; The document image is subjected to feature extraction by the first model to obtain X first features, wherein the X first features include text features and image features extracted from the document image, and the first model is a large language model trained based on training images corresponding to U historical documents, where X and U are both positive integers; Based on the region type of each image region, a second model corresponding to that image region is determined. The second model corresponding to each image region is used to extract features from the image region to obtain M second features corresponding to the M image regions. Target data is determined based on the region coordinates corresponding to the M image regions, the X first features, and the M second features, wherein the target data is used to characterize the document content of the target document through a structured preset format.

2. The method for determining document data according to claim 1, characterized in that, Before analyzing the document image of the target document, the method for determining the document data further includes: The target document, which includes unstructured data, is scanned to obtain the original image; The original image is tilted and corrected using a preset perspective algorithm to obtain the first image; The first image is denoised to obtain the second image; The second image is enhanced to obtain the document image of the target document.

3. The method for determining document data according to claim 1, characterized in that, Feature extraction of the document image using the first model includes: The document image is input into the first model; The text encoder in the first model is used to extract features from the text content in the document image to obtain the text features among the X first features; The convolutional neural network layer in the first model is used to extract features and enhance the visualization of the image content in the document image, thereby obtaining the image features among the X first features.

4. The method for determining document data according to claim 1, characterized in that, By extracting features from each image region using the second model corresponding to that image region, M second features corresponding to the M image regions are obtained, including: When the region type of the i-th image region in the M image regions is financial text, the financial text information in the i-th image region is extracted using the second model corresponding to the i-th image region to obtain the initial text features corresponding to the i-th image region. Based on preset financial rules, semantic error correction is performed on the initial text features to obtain the first text features corresponding to the i-th image region; The first text feature is normalized to obtain the second feature corresponding to the i-th image region.

5. The method for determining document data according to claim 1, characterized in that, By extracting features from each image region using the second model corresponding to that image region, M second features corresponding to the M image regions are obtained, including: When the region type of the j-th image region in the M image regions is a table type, the second feature corresponding to the j-th image region is obtained by extracting the features of the table structure in the j-th image region through the second model corresponding to the j-th image region.

6. The method for determining document data according to claim 1, characterized in that, By extracting features from each image region using the second model corresponding to that image region, M second features corresponding to the M image regions are obtained, including: When the region type of the kth image region in the M image regions is a seal type, the seal graphic in the kth image region is extracted using the second model corresponding to the kth image region to obtain the initial seal features corresponding to the kth image region; Obtain the similarity between the initial seal features and the preset seal features, and generate a prompt message if the similarity is less than the preset similarity. If the similarity is greater than or equal to the preset similarity, the initial stamp feature is used as the second feature corresponding to the k-th image region.

7. The method for determining document data according to claim 1, characterized in that, Target data is determined based on the region coordinates corresponding to the M image regions, the X first features, and the M second features, including: The correspondence between the X first features and the M second features is determined based on the region coordinates corresponding to the M image regions; Based on the correspondence between the X first features and the M second features, a weighted feature fusion is performed on the X first features and the M second features to obtain the target features; The target features are converted according to the preset format to obtain the target data.

8. The method for determining document data according to claim 1, characterized in that, After determining the target data based on the region coordinates corresponding to the M image regions, the X first features, and the M second features, the method for determining the document data further includes: Using the first model, logical rule verification is performed on the target data based on preset prompt words and the context information of the target document; If the verification result corresponding to the logical rule verification is an abnormal result, a prompt message is generated, wherein the prompt message is used to prompt the business personnel corresponding to the target document to correct the target data.

9. A device for determining document data, characterized in that, include: The analysis unit is used to analyze the document image of the target document to obtain the region type and region coordinates corresponding to M image regions, where M is a positive integer; The first extraction unit is used to extract features from the document image using a first model to obtain X first features, wherein the X first features include text features and image features extracted from the document image, and the first model is a large language model trained based on training images corresponding to U historical documents, where X and U are both positive integers; The second extraction unit is used to determine the second model corresponding to each image region based on the region type of each image region, and to extract features from the image region through the second model corresponding to each image region to obtain M second features corresponding to the M image regions. The first determining unit is used to determine target data based on the region coordinates corresponding to the M image regions, the X first features, and the M second features, wherein the target data is used to characterize the document content of the target document through a structured preset format.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for determining document data as described in any one of claims 1 to 8.

Images