A document file table recognition method and related device

Abstract

The present application discloses a table recognition method and related device for a document file, which relates to the fields of computer application and artificial intelligence. The method includes: converting a page of a document file to be processed into a first image; performing image enhancement on the first image to obtain a second image; using a target intelligent agent to perform table detection on the second image to obtain table area position information; the target intelligent agent includes a table detection model constructed based on dual-branch model tuning, the first branch of the target intelligent agent is used to learn and extract the structural features of the table, and the second branch is used to learn and extract the text semantic features of the table; identifying the table data in the table area indicated by the table area position information, and converting the table data into a structured form for output. The present application can realize high-precision, high-efficiency, and high-adaptability table recognition for document files by combining the use of image enhancement processing and dual-branch model tuning based on deep learning in table recognition.

Description

Technical Field

[0001] The present application relates to the field of computer application and artificial intelligence technology, and in particular to a table recognition method for a document file and a related device. Background Art

[0002] Among the known technologies, although the table recognition and structured processing technology for document files has made some progress and achieved certain results in simple scenarios, it still has the following problems when facing complex document files: 1) The table recognition accuracy is low, especially for tilted, curved or noisy tables; 2) The table recognition speed is slow and it is difficult to meet the needs of large-scale data processing; 3) The adaptability to table styles is poor and it is difficult to cope with diverse table layouts.

[0003] Therefore, the existing table recognition and processing technology still needs to be improved in terms of table recognition accuracy, speed and adaptability of complex document files. How to improve the table recognition and processing technology in one or more aspects such as recognition accuracy, speed and adaptability has become an issue worthy of attention in this field. Summary of the invention

[0004] In view of this, the present application provides a table recognition method and related devices for document files, which are used to achieve high-precision, high-efficiency and high-adaptability table recognition for document files by combining technologies such as image enhancement and deep learning model tuning.

[0005] The specific technical solutions are as follows:

[0006] A method for identifying a table in a document file, comprising:

[0007] converting a page of the document file to be processed into a first image;

[0008] Performing image enhancement processing on the first image to obtain a second image; the image enhancement processing is at least used to perform angle correction on the first image;

[0009] Using a target intelligent agent to perform table detection on the second image, obtaining table area position information of the table in the second image; the target intelligent agent is a table detection model constructed by dual-branch model tuning based on deep learning; the first branch of the target intelligent agent is used to learn and extract structural features of the table, and the second branch of the target intelligent agent is used to learn and extract text semantic features of the table;

[0010] The table data in the table area indicated by the table area position information is identified, and the identified table data is converted into a structured form for output.

[0011] Optionally, performing image enhancement processing on the first image includes:

[0012] The first image is subjected to at least part of size standardization, grayscale conversion, binarization, table contour detection, angle correction, and denoising and sharpening processing.

[0013] Optionally, the size standardization includes:

[0014] Determining an adaptive factor for controlling the degree of adjustment of the interpolation kernel based on geometric features of the table in the first image;

[0015] The size and shape of the interpolation kernel are adjusted based on the adaptive factor, and the adjusted interpolation kernel is used to perform interpolation-based scaling processing on the first image to obtain an image of a standard size.

[0016] Optionally, the grayscale conversion includes:

[0017] Performing color space analysis on the first image or the size-standardized first image to obtain a color space analysis result;

[0018] assigning a weight to each color according to the distribution and importance of each color in the first image or the size-normalized first image represented by the color space analysis result;

[0019] Based on the weight corresponding to each color in the color channel of the pixel on the first image or the size-normalized first image, weighted average is performed on the brightness of various colors involved in the color channel of the pixel to obtain a weighted average result of the pixel brightness;

[0020] Based on the weighted average result of pixel brightness, converting the first image or the size-normalized first image into a grayscale image;

[0021] The binarization includes:

[0022] Dividing the grayscale image into a plurality of local windows;

[0023] Calculating the local contrast of each pixel in the local window;

[0024] Determine a corresponding binarization threshold for each pixel in each of the local windows according to the local contrast of the pixel and a preset contrast threshold;

[0025] The grayscale image is converted into a binary image according to the binarization threshold corresponding to each pixel.

[0026] Optionally, the table outline detection includes:

[0027] A convolutional neural network is used to extract low-level features and high-level features of a binary image; the low-level features include basic image features of the binary image, and the high-level features include table structure information in the binary image; the binary image is a result of binarizing the first image;

[0028] Using an attention mechanism to perform feature fusion on the extracted low-level features and the high-level features to obtain a feature fusion result; the feature fusion is used to highlight features related to the table outline;

[0029] Based on the feature fusion result, generating a contour information map of the table in the binary image;

[0030] The angle correction includes:

[0031] Based on the contour information map, determining text rows and table rows in the binary image;

[0032] Determining the tilt angle of the binary image based on the voting results of the tilt angles of each text row and table row in the binary image through a regional voting mechanism;

[0033] Angle correction is performed on the binarized image based on the tilt angle of the binarized image.

[0034] Optionally, the denoising and sharpening process includes:

[0035] Determine a denoising parameter based on a noise level of an input image, and determine a sharpening parameter based on a local feature of the input image; the input image is an image obtained after the first image is processed accordingly;

[0036] The input image is subjected to denoising and sharpening processing based on the denoising parameter and the sharpening parameter respectively.

[0037] Optionally, the process of constructing the target intelligent agent includes at least part of the following:

[0038] Providing pre-trained model weights for the first deep learning model on the first branch and the second deep learning model on the second branch through transfer learning; wherein the pre-trained model weights provided for the first deep learning model are used to extract structural features of the table, and the pre-trained model weights provided for the second deep learning model are used to extract text semantic features of the table;

[0039] Using small samples to respectively tune the first deep learning model and the second deep learning model after transfer learning;

[0040] Performing data enhancement on the small samples, and using the enhanced samples obtained after the data enhancement to respectively tune the first deep learning model and the second deep learning model after transfer learning;

[0041] During the tuning process, the first deep learning model and the second deep learning model can be independently optimized on their respective branches and can share table information and features.

[0042] A table recognition device for a document file, comprising:

[0043] An image conversion module, used for converting a page of a document file to be processed into a first image;

[0044] An image enhancement module, configured to perform image enhancement processing on the first image to obtain a second image; the image enhancement processing is at least used to perform angle correction on the first image;

[0045] A table detection module, used to perform table detection on the second image using a target agent to obtain table area position information of the table in the second image; the target agent is a table detection model constructed by dual-branch model tuning based on deep learning; the first branch of the target agent is used to learn and extract structural features of the table, and the second branch of the target agent is used to learn and extract text semantic features of the table;

[0046] The table recognition module is used to recognize the table data in the table area indicated by the table area position information, and convert the recognized table data into a structured form for output.

[0047] An electronic device, comprising:

[0048] Memory, used to store computer programs;

[0049] The processor is used to implement the table recognition method of the document file as described in any one of the above items by calling and executing the computer program in the memory.

[0050] A computer-readable medium stores a computer program, which, when executed by a processor, can be used to implement the table recognition method for a document file as described in any one of the above.

[0051] According to the above scheme, the table recognition method and related devices for document files provided by the present application pre-construct a target intelligent agent for table detection based on deep learning dual-branch model tuning, and the target intelligent agent includes a first branch for learning and extracting structural features of the table and a second branch for learning and extracting text semantic features of the table. On this basis, for the document file to be subjected to table recognition, it is converted into a first image, the first image is subjected to image enhancement processing to obtain a second image, and the second image is subjected to table detection using the target intelligent agent, and then the table data is recognized based on the table area position information obtained by the detection, thereby realizing table recognition for document files by combining image enhancement and deep learning model tuning and other technologies. The present application can realize high-precision, high-efficiency, and high-adaptability table recognition for document files by combining the use of image enhancement processing and deep learning-based dual-branch model tuning and other technologies in table recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0053] Figure 1 It is a flow chart of the table recognition method of document files provided by this application;

[0054] Figure 2 It is a modular design example diagram of the table recognition process provided by this application;

[0055] Figure 3 It is a composition structure diagram of the table recognition device for document files provided by the present application;

[0056] Figure 4 It is a structural diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] The embodiment of the present application provides a table recognition method and related device for a document file, which is applicable to but not limited to the recognition of table data in a portable document format file of a literature document type.

[0059] See also Figure 1 As shown in the method flow chart, the table recognition method for document files provided in the embodiment of the present application may at least include the following steps 101 to 104, which are described in detail below.

[0060] Step 101: Convert a page of a document file to be processed into a first image.

[0061] The document file to be processed may be, but is not limited to, a portable document file in a format such as PDF, DOC, or DOCX.

[0062] For document files to be processed in formats such as PDF, DOC, DOCX, etc., optionally, the document file can be loaded, parsed, and each page of the document file can be converted into an image according to the parsing result to obtain an image file of each page of the document file. The embodiment of the present application refers to these image files as first images.

[0063] For example, taking a document file in PDF format as an example, the document file can be parsed through, but not limited to, Python's MuPDF interface implementation library, where MuPDF is a lightweight PDF parsing library that supports multiple platforms and is tailored for high-quality anti-aliased graphics, which renders text with metrics and spacing accurate to fractions of a pixel.

[0064] In addition, in actual applications, the page of the document file to be processed can be converted into a picture by screenshot, snapshot, scanning, shooting, etc., so as to obtain the first image. Compared with image conversion methods such as screenshot and snapshot, the image conversion method based on document parsing mentioned above can better guarantee image quality and conversion performance.

[0065] Step 102: Perform image enhancement processing on the first image to obtain a second image; the image enhancement processing is at least used to perform angle correction on the first image.

[0066] The image enhancement processing of the first image may include, but is not limited to, some or all of the processing of size standardization, grayscale conversion, binarization, table outline detection, angle correction, denoising and sharpening, etc. This step performs image enhancement processing on the first image to provide high-quality image data for subsequent table recognition.

[0067] The various image enhancement processes performed on the first image are respectively as follows:

[0068] Size normalization: By adjusting the image size, all input images are ensured to have a uniform resolution, which helps consistency and stability in subsequent processing steps;

[0069] Grayscale conversion: used to convert a color image (such as the first image or the first image after size standardization) into a grayscale image to simplify image data, reduce the amount of calculation, and retain key information of the table structure in the image;

[0070] Binarization: It is used to convert grayscale images into binary images with only black and white colors, which helps to highlight the text and table lines in the image, thus laying the foundation for table outline detection and text recognition;

[0071] Table outline detection: used to detect the outline of the table in the image (such as the binary image) by using image processing technology, so as to provide a basis for subsequent image angle correction;

[0072] Angle correction: used to automatically adjust the angle of the image (such as the binary image) according to the detected form outline to eliminate the tilt, form bending, etc. caused by image conversion such as scanning or shooting, so as to ensure the accuracy of form recognition;

[0073] Denoising and sharpening: used to reduce random noise in an image (such as an image after angle correction) through denoising processing, and to enhance edge information of an image (such as an image after angle correction) through sharpening processing to improve the accuracy of subsequent form recognition.

[0074] Step 103: Use the target intelligent agent to perform table detection on the second image to obtain the table area position information of the table in the second image; the target intelligent agent is a table detection model constructed by dual-branch model tuning based on deep learning; the first branch of the target intelligent agent is used to learn and extract the structural features of the table, and the second branch of the target intelligent agent is used to learn and extract the text semantic features of the table.

[0075] In order to achieve high-precision and high-efficiency table recognition in complex scenarios, the embodiment of the present application constructs a table detection model for table detection based on a dual-branch model tuning method, namely the target intelligent agent.

[0076] Furthermore, the embodiment of the present application is specifically based on an innovative dual-branch model tuning method, combining transfer learning, small sample training and data enhancement to construct the target intelligent agent, so that the model of the target intelligent agent can be efficiently trained on a small amount of labeled data for table recognition tasks, significantly improving the recognition ability and adaptability of the model. The core idea of ​​the dual-branch model tuning method of the embodiment of the present application is to simultaneously optimize the generalization ability of the model in different table layouts and styles through parallel processing of the dual-branch structure, thereby enhancing its adaptability to new tasks and new data.

[0077] The core concept of the dual-branch structure is to divide the model's processing flow into two parallel branches, each of which is optimized for different data processing tasks to better cope with the complexity of small sample learning and tabular data.

[0078] The first branch of the dual-branch structure is a branch for learning and extracting structural features of the table, and the first branch has a first deep learning model for learning and extracting structural features of the table based on deep learning. The structural features of the table include but are not limited to table lines, cell relationships, and geometric properties of the table.

[0079] The second branch is a semantic feature learning and extraction branch, and the second branch has a second deep learning model for learning and extracting text semantic features of the table based on deep learning. The text semantic features of the table include but are not limited to text content, data values, and header information.

[0080] Among them, the first deep learning model can be but is not limited to a deep learning model such as TabNet, and the second deep learning model can be but is not limited to a structured learning model such as StructEqTable.

[0081] The TabNet and StructEqTable models are described below.

[0082] 1. TabNet

[0083] TabNet is a deep learning model based on the attention mechanism, which aims to improve the interpretability and performance of the model through feature selection mechanism, mainly through feature selection, attention mechanism and multi-scale feature learning.

[0084] Feature Selection: TabNet uses a feature selection mechanism to decide which features to use in each layer, which helps the model focus on the most important information.

[0085] Attention mechanism: The model uses the attention mechanism to assign different weights to each feature so that it can learn more complex feature relationships.

[0086] Multi-scale feature learning: TabNet processes features through multiple decision steps and scales so that the model can capture feature information at different levels.

[0087] Based on the above feature selection and attention mechanism, TabNet can achieve high-accuracy table recognition in table recognition tasks; and the model can provide an explanation of feature importance, which helps to understand how the model makes decisions. In addition, the applicant found that TabNet can also show good generalization ability when dealing with unseen table structures.

[0088] 2. StructEqTable

[0089] StructEqTable is a deep learning model for structured data (such as tables) that improves prediction performance by learning the structural information of the data. It is mainly achieved through structured representation, equilateral mapping and hierarchical learning.

[0090] Structured representation: The model transforms tabular data into a structured representation that can be used to capture the relationships between cells.

[0091] Equivariant mapping: StructEqTable uses equivariant mapping to ensure that the model is invariant to changes in the data arrangement in the table, that is, the model can recognize tables with different arrangements but the same content.

[0092] Hierarchical learning: The model learns tabular data in a hierarchical manner, gradually building an understanding of the table from local features to global structure.

[0093] StructEqTable can effectively identify the structure of a table and is highly robust to changes in the arrangement and format of data in the table. By learning the structural information of the data, the model can be trained on less data to reduce the risk of overfitting; at the same time, the model can adapt to different types of table structures, including complex and irregular table layouts.

[0094] The detailed construction process of the target intelligent agent based on dual-branch model tuning will be described in detail in the following embodiments.

[0095] After obtaining the second image by performing image enhancement processing on the first image, the second image can be input into the target intelligent agent to extract table structural features and text semantic features based on the dual-branch structure provided by the target intelligent agent, and then accurate table detection can be achieved based on the extracted table structural features and text semantic features to obtain the table area position information of the table in the second image.

[0096] Step 104: Identify the table data in the table area indicated by the table area position information, and convert the identified table data into a structured form for output.

[0097] After obtaining the table area position information of the table in the second image, the table data in the table area indicated by the table area position information may be further identified.

[0098] Optionally, the table data can be obtained by performing OCR (Optical Character Recognition) on the table area on the page image, i.e., the second image. PaddleOCR can be preferably used to perform OCR on the table area image to achieve high-precision and high-efficiency table data recognition.

[0099] PaddleOCR is an open source OCR tool based on deep learning, with the characteristics of fast recognition speed and high accuracy; PaddleOCR supports recognition of multiple languages, which enables this application to cope with the table recognition task of portable document format files in different languages. And PaddleOCR can realize end-to-end text recognition, which can directly go from image input to text output, thus simplifying the processing flow and improving recognition efficiency.

[0100] According to the above description, it can be seen that based on the method provided in the embodiment of the present application, end-to-end efficient and accurate automatic table recognition can be achieved, thereby supporting the structured output results of table data directly based on the input portable document format file page. Optionally, the table data can be output in a JSON structured format. The embodiment of the present application is based on the end-to-end automated table recognition process, which reduces manual intervention and improves processing efficiency.

[0101] In practical applications, each processing step (such as image enhancement, dual-branch model optimization, table detection, OCR recognition, etc.) in the method provided in this application can be implemented based on a modular design approach. Figure 2 , provides a modular design example of each processing step in the method of the present application, which includes an image enhancement processing module, a table detection model optimization module, and a table detection and recognition module. The functions of each module are as follows: Figure 2 As shown in the figure. The modular design makes system maintenance and upgrades easier.

[0102] According to the above scheme, the table recognition method for document files provided by the present application pre-constructs a target intelligent agent for table detection based on deep learning dual-branch model tuning, and the target intelligent agent includes a first branch for learning and extracting structural features of the table and a second branch for learning and extracting text semantic features of the table. On this basis, for the document file to be subjected to table recognition, it is converted into a first image, the first image is subjected to image enhancement processing to obtain a second image, and the second image is subjected to table detection using the target intelligent agent, and then the table data is recognized based on the table area position information obtained by the detection, thereby realizing table recognition for document files by combining image enhancement and deep learning model tuning and other technologies. The present application can realize high-precision, high-efficiency, and high-adaptability table recognition for document files by combining image enhancement processing and deep learning-based dual-branch model tuning and other technologies in table recognition.

[0103] In particular, the embodiments of the present application provide high-quality image data for subsequent table recognition by performing image enhancement on the page images of document files, such as size standardization, grayscale conversion, binarization, contour detection, angle correction, and denoising and sharpening processing; by adopting advanced table detection models with dual-branch structures such as TabNet and StructEqTable, and through small sample tuning, the recognition ability of the model (target intelligent agent) in complex scenarios is improved; by using OCR technologies such as PaddleOCR to perform OCR recognition of table area images, high-precision and high-efficiency text recognition is achieved.

[0104] In an optional embodiment, the size normalization process in the image enhancement process can be implemented as the following steps 11)-12):

[0105] 11) Based on the geometric features of the table in the first image, determining an adaptive factor for controlling the degree of adjustment of the interpolation kernel.

[0106] The geometric features of a table may include, but are not limited to, row spacing, column spacing, cell size, and other features of the table.

[0107] 12) adjusting the size and shape of the interpolation kernel based on the adaptive factor, and performing interpolation-based scaling processing on the first image using the adjusted interpolation kernel to obtain an image of a standard size.

[0108] Optionally, this embodiment specifically performs image size standardization based on an interpolation algorithm.

[0109] In order to improve the quality of image size standardization, in the process of image size standardization based on the interpolation algorithm, first, the input image (such as the first image) is preliminarily analyzed to roughly identify the table area and non-table area in the image. Then, an adaptive factor is calculated based on the geometric characteristics of the table (such as row spacing, column spacing, cell size, etc.). During the interpolation process, this adaptive factor is used to dynamically adjust the size and shape of the interpolation kernel to ensure that the geometric proportion and relative position of the table area remain unchanged after scaling.

[0110] The adaptive factor is used to control the adjustment degree of the interpolation kernel to achieve the optimal scaling effect.

[0111] The embodiment of the present application performs size standardization processing on the image and scales the image to the target resolution to obtain a standard size image of the target resolution. The target resolution can be set according to actual application requirements, such as 800x600 pixels. Optionally, when setting the target resolution, the resolution of the original image to be size-standardized (such as the first image) and the complexity of the table can be considered to avoid image distortion caused by overscaling.

[0112] This embodiment determines an adaptive factor to adjust the interpolation kernel based on the geometric features of the table in the image, and uses the adjusted interpolation kernel to scale the image, which can ensure that the geometric proportions and relative positions of the table area remain unchanged after scaling, thereby achieving the optimal scaling effect, providing high-quality image data for subsequent table recognition, and is conducive to improving the efficiency and accuracy of subsequent table recognition.

[0113] In an optional embodiment, the grayscale conversion process in the image enhancement process can be implemented as the following steps 21)-24):

[0114] 21) Performing color space analysis on the first image or the size-standardized first image to obtain a color space analysis result.

[0115] The color space analysis results may include identified primary colors and secondary colors (such as text color, table color, background color, etc.), as well as the distribution and importance of these colors in the image.

[0116] 22) assigning a weight to each color according to the distribution and importance of each color in the first image or the size-normalized first image represented by the color space analysis result.

[0117] 23) Based on the weight corresponding to each color in the color channel of the pixel on the first image or the size-standardized first image, weighted average the brightness of various colors involved in the color channel of the pixel to obtain a weighted average result of the pixel brightness.

[0118] Among them, the weighted average result of the brightness of various colors involved in the color channel of the pixel can be used as the grayscale of the pixel.

[0119] 24) Based on the weighted average result of pixel brightness, convert the first image or the size-standardized first image into a grayscale image.

[0120] This step 24) can specifically realize grayscale conversion of the image by taking the weighted average result of the brightness of various colors involved in the color channel of the pixel as the pixel grayscale, so as to obtain the corresponding grayscale image.

[0121] Specifically, this embodiment proposes and adopts a grayscale conversion method based on adaptive weighting to perform grayscale conversion processing on an image.

[0122] Traditional grayscale conversion usually adopts a fixed weight allocation method, which easily leads to the loss of image color information. To solve this problem, the embodiment of the present application proposes an adaptive weighted grayscale conversion method, in which the image is first analyzed in color space to identify the main color and secondary color of the image (such as text color, table color, background color, etc.). Then, a weight is assigned to each color according to the distribution and importance of each color in the image. In the grayscale conversion process, these weights are used to weighted average the brightness of each color in the pixel color channel to retain more key information, and then the weighted average is used as the grayscale value of the pixel, thereby realizing the grayscale conversion of each pixel of the image.

[0123] In practical applications, a color weight matrix can be obtained through a large number of sample training to adapt to different types of document images. During the training process, the optimal color weight matrix can be generated by learning the relationship between different colors and image content.

[0124] This embodiment uses an adaptive weighted grayscale conversion method to perform grayscale conversion processing on the image, which can retain more key information of the image and avoid the loss of color information, thereby further providing high-quality image data for subsequent table recognition, which is conducive to further improving the accuracy and efficiency of subsequent table recognition.

[0125] In an optional embodiment, the binarization process in the image enhancement process can be implemented as the following steps 31)-34):

[0126] 31) Divide the grayscale image into a plurality of local windows.

[0127] The grayscale image may be divided into a plurality of local windows based on a traversal segmentation method (such as a traversal segmentation method from left to right and from top to bottom) according to a certain local window size but is not limited to it.

[0128] Optionally, the local window size can be automatically adjusted according to the image resolution and table density. The local window size obtained by division should be large enough to contain useful local information, but not too large to introduce too much noise.

[0129] 32) Calculate the local contrast of each pixel in the local window.

[0130] The local contrast of the pixels in the local window may be, but is not limited to, the local mean or standard deviation of the pixel values ​​in the local window.

[0131] 33) Determine a corresponding binarization threshold for each pixel based on the local contrast of the pixel and a preset contrast threshold.

[0132] The contrast threshold may be dynamically calculated based on image statistical characteristics. During the calculation process, factors such as the overall contrast of the image and the local contrast distribution may be considered to generate an adaptive contrast threshold.

[0133] In this step 33), the comparison result between the local contrast of the pixel and the contrast threshold may be used as a reference to determine a corresponding binarization threshold for each pixel.

[0134] 34) Convert the grayscale image into a binary image according to the binarization threshold corresponding to each pixel.

[0135] Traditional binarization uses a fixed threshold value, which is easily affected by illumination and noise. To address this problem, the embodiment of the present application dynamically determines the optimal threshold value for each pixel based on the local contrast of the image. First, the image is divided into multiple local windows, and in each window, the local contrast of the pixel (such as the local mean and standard deviation) is calculated. Then, based on the local contrast and the preset contrast threshold, a binarization threshold is determined for each pixel, and then based on the binarization threshold value dynamically determined for each pixel, the image pixels are binarized.

[0136] The binarization processing based on the dynamic binarization threshold provided in the embodiment of the present application can effectively adapt to different lighting and noise conditions, and correspondingly improve the accuracy of the image binarization processing.

[0137] In an optional embodiment, the table outline detection process in the image enhancement process can be implemented as the following steps 41)-43):

[0138] 41) Using a convolutional neural network to extract low-level features and high-level features of the binary image; the low-level features include basic image features of the binary image, and the high-level features include table structure information in the binary image.

[0139] The low-level features may specifically include but are not limited to basic image features such as edges and corners; the high-level features may specifically include but are not limited to complex table structure information such as table lines, cell relationships, and geometric attributes.

[0140] 42) Using an attention mechanism, the extracted low-level features and the high-level features are subjected to feature fusion to obtain a feature fusion result; the feature fusion is used to highlight features related to the table outline.

[0141] Specifically, based on the attention weight, feature fusion may be performed on the extracted low-level features and the high-level features, so as to highlight features related to the table outline through the attention weight.

[0142] 43) Based on the feature fusion result, a contour information map of the table in the binary image is generated.

[0143] This step 43) can generate a complete table outline information graph through the fully connected layer in the convolutional neural network structure.

[0144] The table contour detection in this embodiment is contour detection based on multi-level feature fusion.

[0145] Specifically, the contour detection of the table in the image based on multi-level feature fusion can be achieved through the CNN structure of the Convolutional Neural Network (CNN).

[0146] The CNN structure can include multiple convolutional layers, pooling layers, and fully connected layers, and its specific structure can be optimized according to experiments to achieve the best feature extraction and fusion effects for the table.

[0147] Specifically, when performing table contour detection, a convolutional neural network is first used to extract low-level and high-level features of the image. Through multiple convolutional layers and pooling layers, CNN can gradually extract these features; then, the extracted features are fused using the attention mechanism to highlight the features related to the table contour; finally, a complete table contour information map is generated through the fully connected layer.

[0148] Among them, the attention weight can be automatically learned through training. During the training process, the optimal attention weight can be generated by learning the relationship between different features and the table outline.

[0149] This embodiment can provide a basis for subsequent angle correction by performing preliminary table outline detection (different from the subsequent precise table position detection based on the target intelligent body).

[0150] In an optional embodiment, the angle correction process in the image enhancement process can be implemented as the following steps 51)-53):

[0151] 51) Based on the contour information map, determine text rows and table rows in the binary image.

[0152] Optionally, based on the contour information graph of the table, a minimum bounding rectangle fitting method may be used to detect text rows and table rows in the image.

[0153] 52) Determine the tilt angle of the binary image based on the voting results of the tilt angles of each text row and table row in the binary image through a regional voting mechanism.

[0154] During the voting process, each text line or table line will cast a vote according to its tilt angle, and accordingly the tilt angle of the binary image can be determined according to the voting result of the tilt angle of each text line and table line.

[0155] 53) Performing angle correction on the binarized image based on the tilt angle of the binarized image.

[0156] Specifically, the embodiment of the present application proposes an adaptive angle correction method based on geometric transformation and regional voting to perform angle correction of the image. During the angle correction process, first, the minimum enclosing rectangle fitting method can be used to detect the text lines and table lines in the image; then, the tilt angle of the image is determined through the regional voting mechanism. During the voting process, each text line or table line will cast a vote based on its tilt angle; finally, the overall tilt angle of the image is determined based on the voting results, and the affine transformation method can be used to perform angle correction on the image based on the overall tilt angle of the image.

[0157] Optionally, for complex documents, a recursive strategy can be further introduced to perform angle detection and correction on the image block by block.

[0158] In practical applications, the size of voting areas (such as text lines, table rows, or voting areas divided based on other strategies) can be automatically adjusted according to the table size and density. The voting area size should be large enough to contain useful voting information, but not too large to introduce too much noise.

[0159] The affine transformation parameters based on the affine transformation can be automatically adjusted by the optimization function for angle correction. During the optimization / correction process, factors such as the alignment of text lines and the distortion of the image can be considered to achieve the correction effect of minimizing the tilt of the text / table.

[0160] This embodiment uses a regional voting mechanism to perform image angle correction, which can effectively eliminate factors such as image tilt and curvature that affect table recognition, thereby providing support for subsequent efficient and accurate table recognition.

[0161] In an optional embodiment, the denoising and sharpening process in the image enhancement process can be implemented as the following steps 61)-62):

[0162] 61) Denoising parameters are determined based on the noise level of the input image, and sharpening parameters are determined based on the local features of the input image.

[0163] The input image is an image obtained after corresponding processing is performed on the first image, such as an image obtained after size standardization, grayscale processing, binarization, and angle correction.

[0164] The local features may include local features of edge regions and local features of non-edge regions of the input image.

[0165] De-noising parameters may include but are not limited to denoising intensity, and sharpening parameters may include but are not limited to sharpening intensity.

[0166] 62) Performing denoising and sharpening processing on the input image based on the denoising parameter and sharpening parameter respectively.

[0167] Specifically, this embodiment proposes to perform image denoising based on a deep learning adaptive denoising network. The adaptive denoising network uses deep learning technology to learn the noise pattern in the image and automatically adjust denoising parameters such as denoising strength. During the training process of the denoising network, the network will learn denoising strategies under different noise types and degrees to generate an adaptable denoising model network. During the denoising process, the network automatically adjusts denoising parameters such as denoising strength according to the noise level of the input image to ensure that important image details are retained while removing noise.

[0168] The network structure of the adaptive denoising network can include multiple convolutional layers, residual connections and activation functions. Its specific structure can be optimized according to the type and degree of noise to effectively remove the noise and maintain the clarity of the image edges, thereby ensuring the highest denoising performance.

[0169] Among them, the denoising strength is used to control the degree of network denoising to avoid image distortion caused by excessive denoising. In practical applications, the denoising strength can also be automatically adjusted based on the complexity of the image content. The complexity of the image content can be evaluated by analyzing the image's noise level, texture complexity and other features.

[0170] Traditional Laplace sharpening may over-enhance noise, resulting in image quality degradation. To overcome this problem, this embodiment automatically adjusts sharpening parameters such as sharpening intensity according to local features of the image, such as edge strength and texture density, to achieve adaptive Laplace sharpening and avoid image quality degradation.

[0171] In adaptive Laplace sharpening, first, the edge area and non-edge area of ​​the image are detected. For the edge area, a stronger sharpening parameter (such as a stronger sharpening strength) is used to highlight the edge details; for the non-edge area, a weaker sharpening parameter (such as a weaker sharpening strength) is used to avoid noise enhancement.

[0172] Specifically, the sharpening parameters such as sharpening strength can be dynamically calculated based on the local features of the image. During the calculation process, local features such as edge strength and texture density can be considered, but are not limited to, to generate adaptive sharpening parameters such as sharpening strength.

[0173] In addition, the size of the Laplacian operator can be automatically adjusted according to the image resolution and table density to achieve a more natural sharpening effect. The Laplacian operator itself is not a sharpening parameter, but it can be used for image sharpening.

[0174] The size of the Laplacian operator can be automatically adjusted based on the image resolution and table density. The operator size should be large enough to capture useful edge information, but not so large that it introduces too much noise.

[0175] The embodiment of the present application can provide a high-quality image foundation for subsequent fast and high-precision table recognition through a series of meticulous image enhancement processing technologies described above. Among them, size standardization ensures the consistency of the image, grayscale conversion and binarization simplify the image data and retain key information, contour detection extracts the table outline, angle correction corrects factors such as image tilt and table bending that are not conducive to table recognition, and denoising and sharpening improve image clarity. The combined application of these technologies significantly improves the accuracy and robustness of the final table recognition.

[0176] In an optional embodiment, a process for constructing a target agent is further provided.

[0177] The process of constructing the target intelligent agent can be implemented as the following steps 71)-73):

[0178] 71) Through transfer learning, pre-trained model weights are provided for the first deep learning model on the first branch and the second deep learning model on the second branch; the pre-trained model weights provided for the first deep learning model are used to extract structural features of the table, and the pre-trained model weights provided for the second deep learning model are used to extract text semantic features of the table.

[0179] The first deep learning model can be, but is not limited to, a deep learning model such as TabNet.

[0180] The second deep learning model can be, but is not limited to, a structured learning model such as StructEqTable.

[0181] 72) Using small samples, respectively, fine-tune the first deep learning model and the second deep learning model after transfer learning.

[0182] 73) Performing data enhancement on the small samples, and using the enhanced samples obtained after the data enhancement to respectively tune the first deep learning model and the second deep learning model after transfer learning.

[0183] During the tuning process, the first deep learning model and the second deep learning model can be independently optimized on their respective branches and can share table information and features.

[0184] The above implementation process is further described in detail below.

[0185] This embodiment adopts a dual-branch model tuning method, combined with transfer learning, small sample training and data enhancement techniques, so that the model can be efficiently trained on a small amount of labeled data for table recognition tasks, so as to significantly improve the recognition ability and adaptability of the model. The core idea of ​​this method is to optimize the generalization ability of the model on different table layouts and styles through parallel processing of the dual-branch structure, so as to enhance its adaptability to new tasks and new data.

[0186] In implementation, it is possible but not limited to using a dual-branch structure based on TabNet and StructEqTable to build an advanced deep learning table detection model (i.e., target agent), and through small sample tuning, improve the model's recognition ability in complex scenarios, and through transfer learning, data enhancement and other methods, improve the model's generalization ability and recognition accuracy on limited labeled data.

[0187] The core concept of the dual-branch structure is to divide the model's processing flow into two parallel branches, each of which is optimized for different data processing tasks to better cope with the complexity of small sample learning and tabular data. Specifically, the two branches perform the following tasks:

[0188] The first branch: learning the structural features of the table

[0189] This branch focuses on learning the structural features of tabular data, especially table lines, cell relationships, and geometric properties of the table. By using deep learning models such as TabNet, the first branch can extract key geometric information and structural features from the table to identify the basic layout and data arrangement of the table.

[0190] In the first branch, the model focuses on deep learning of the structural features of the table to establish an understanding of the spatial structure and data distribution of the table, ensuring that the table area can be accurately detected and marked. After the model (target agent) is built, the first branch will be used to extract the structural features of the table in the image to perform table detection based on the extracted table structural features.

[0191] Second branch: semantic information learning branch

[0192] This branch focuses on capturing the text information features in the table, such as text content, data values, and header information features. Through structured learning models such as StructEqTable, the second branch can learn the semantic features of table data and understand the relationship between texts.

[0193] The second branch can learn the text in the table through a deep learning network and extract important text information in the table based on natural language processing (NLP) technology. In this way, even if the table structure is complex or the text position changes, the model can accurately identify the meaning of the text and the structure of the table through dual branches.

[0194] The dual-branch structure proposed in this application provides strong support for model tuning and small sample training. Under this framework, the model can be efficiently trained on limited labeled data, thereby improving performance in new scenarios and new tasks. The specific tuning process includes the following three key steps: transfer learning, small sample training, and data enhancement.

[0195] Among them, through transfer learning, the knowledge acquired by the pre-trained model on a large-scale data set can be transferred to a small sample task, thereby reducing the time and cost of training the target agent from scratch. Transfer learning can provide pre-trained model weights for the two branches in the dual-branch structure of this application, helping the model to quickly adapt to new tasks.

[0196] On the basis of transfer learning, the model is further tuned through small sample learning so that the model can still maintain a high recognition accuracy when data samples are scarce. Through small sample training, the model can extract common features from limited samples and adapt them, thereby improving its recognition ability for different table formats.

[0197] The dual-branch model can share information and features while optimizing each branch independently, so as to promote efficient training of the model on a small number of samples. Through this strategy, the two branches can achieve information fusion while ensuring personalized learning, thereby further improving the performance of the final table detection model (target agent).

[0198] In implementation, for example, a certain number of, for example, 5,000 PDF files can be randomly selected from a portable document format file data source as sample data, a portion of which is used as training samples and the remaining portion is used as verification samples, for example, 4,000 PDF files are used as training samples and the remaining 1,000 PDF files are used as verification samples, etc. Since the table detection model performs target detection on page images, it is necessary to first perform page-to-image conversion processing on the PDF sample data, and then perform image enhancement processing such as size standardization, grayscale conversion, binarization, contour detection, angle correction, denoising and sharpening on the page images.

[0199] On this basis, we can further use open source deep learning image annotation tools, such as labelme, to annotate the table area of ​​the enhanced image data to complete the small sample preparation, and then perform model tuning based on the small sample. The annotation type can include the following labels: table.

[0200] Optionally, due to the lack of annotated data, this embodiment further constructs enhanced samples by performing different enhancement processing (such as table rotation, noise addition, random cropping, etc.) on the original small sample data, so that the model can expand its learning feature space by simulating different table formats and layouts. The enhanced image data provides more training samples for the model, thereby further improving the generalization ability of the model.

[0201] Each branch in the dual-branch structure can independently apply data enhancement strategies to ensure that each branch can effectively learn under different data transformations and obtain diverse feature information.

[0202] Afterwards, dual-branch optimization can be performed based on the constructed samples to improve the table recognition accuracy of the model.

[0203] Among them, for the first branch (the structural feature branch of the table), it focuses on the spatial structure of the table, and can be but not limited to using the TabNet model through feature selection mechanism and attention mechanism to optimize its ability to extract the structural features of the table. During the optimization process of this branch, the model can understand the geometric layout of the table through a multi-level decision-making process to automatically capture the arrangement and key elements of the table.

[0204] For the second branch (semantic feature branch), the learning of text information, especially the text and data in the table, can be enhanced through the StructEqTable model, but is not limited to it. This branch can improve the model's understanding of the relationship between data, headers, and cells by understanding the context of the table, so as to ensure that the structure and text information of the table can be accurately recognized even when the table content is complex.

[0205] Through the dual-branch structure, the model can simultaneously optimize the feature learning and extraction capabilities of the table in terms of geometric structure features and semantic features, thereby obtaining more comprehensive and detailed table understanding results.

[0206] After the dual-branch model is tuned, the tuned model can be further verified based on the validation sample. Experimental results show that when only a small amount of labeled data is used, the dual-branch tuned model improves the table recognition accuracy by about 12%-18% compared to the traditional method. In particular, the adaptability of the model is significantly enhanced when processing tables of different formats or diverse layouts.

[0207] This embodiment trains the table detection model (target agent) through a dual-branch structure and a small sample method, which enables the model to efficiently learn the geometric structure and semantic information features of the table on a small amount of data samples, and improves the generalization ability of the model under diverse table formats and layouts.

[0208] Corresponding to the above method, the present application embodiment also provides a table recognition device for a document file, see Figure 3 The schematic diagram of the composition structure of the device comprises:

[0209] An image conversion module 301 is used to convert a page of a document file to be processed into a first image;

[0210] An image enhancement module 302 is used to perform image enhancement processing on the first image to obtain a second image; the image enhancement processing is at least used to perform angle correction on the first image;

[0211] A table detection module 303 is used to perform table detection on the second image using a target agent to obtain table area position information of the table in the second image; the target agent is a table detection model constructed by dual-branch model tuning based on deep learning; the first branch of the target agent is used to learn and extract structural features of the table, and the second branch of the target agent is used to learn and extract text semantic features of the table;

[0212] The table identification module 304 is used to identify the table data in the table area indicated by the table area position information, and convert the identified table data into a structured form for output.

[0213] In an optional implementation, the image enhancement module 302 is specifically configured to:

[0214] The first image is subjected to at least part of size standardization, grayscale conversion, binarization, table contour detection, angle correction, and denoising and sharpening processing.

[0215] In an optional implementation, the image enhancement module 302, when performing size standardization, is specifically configured to:

[0216] Determining an adaptive factor for controlling the degree of adjustment of the interpolation kernel based on geometric features of the table in the first image;

[0217] The size and shape of the interpolation kernel are adjusted based on the adaptive factor, and the adjusted interpolation kernel is used to perform interpolation-based scaling processing on the first image to obtain an image of a standard size.

[0218] In an optional implementation, the image enhancement module 302, when performing grayscale conversion, is specifically used to:

[0219] Performing color space analysis on the first image or the size-standardized first image to obtain a color space analysis result;

[0220] assigning a weight to each color according to the distribution and importance of each color in the first image or the size-normalized first image represented by the color space analysis result;

[0221] Based on the weight corresponding to each color in the color channel of the pixel on the first image or the size-standardized first image, weighted average is performed on the brightness of various colors involved in the color channel of the pixel to obtain a weighted average result of the pixel brightness;

[0222] Based on the weighted average result of pixel brightness, converting the first image or the size-normalized first image into a grayscale image;

[0223] The image enhancement module 302, when performing binarization, is specifically used for:

[0224] Dividing the grayscale image into a plurality of local windows;

[0225] Calculating the local contrast of each pixel in the local window;

[0226] Determine a corresponding binarization threshold for each pixel in each of the local windows according to the local contrast of the pixel and a preset contrast threshold;

[0227] The grayscale image is converted into a binary image according to the binarization threshold corresponding to each pixel.

[0228] In an optional implementation, the image enhancement module 302, when performing table outline detection, is specifically used to:

[0229] A convolutional neural network is used to extract low-level features and high-level features of a binary image; the low-level features include basic image features of the binary image, and the high-level features include table structure information in the binary image; the binary image is a result of binarizing the first image;

[0230] Using an attention mechanism to perform feature fusion on the extracted low-level features and the high-level features to obtain a feature fusion result; the feature fusion is used to highlight features related to the table outline;

[0231] Based on the feature fusion result, generating a contour information map of the table in the binary image;

[0232] The image enhancement module 302, when performing angle correction, is specifically used to:

[0233] Based on the contour information map, determining text rows and table rows in the binary image;

[0234] Determining the tilt angle of the binary image based on the voting results of the tilt angles of each text row and table row in the binary image through a regional voting mechanism;

[0235] Angle correction is performed on the binarized image based on the tilt angle of the binarized image.

[0236] In an optional implementation manner, the image enhancement module 302, when performing denoising and sharpening processing, is specifically used to:

[0237] Determine a denoising parameter based on a noise level of an input image, and determine a sharpening parameter based on a local feature of the input image; the input image is an image obtained after the first image is processed accordingly;

[0238] The input image is subjected to denoising and sharpening processing based on the denoising parameter and the sharpening parameter respectively.

[0239] In an optional embodiment, the above device further includes a model building module, which is used to:

[0240] Providing pre-trained model weights for the first deep learning model on the first branch and the second deep learning model on the second branch through transfer learning; wherein the pre-trained model weights provided for the first deep learning model are used to extract structural features of the table, and the pre-trained model weights provided for the second deep learning model are used to extract text semantic features of the table;

[0241] Using small samples to respectively tune the first deep learning model and the second deep learning model after transfer learning;

[0242] Performing data enhancement on the small samples, and using the enhanced samples obtained after the data enhancement to respectively tune the first deep learning model and the second deep learning model after transfer learning;

[0243] During the tuning process, the first deep learning model and the second deep learning model can be independently optimized on their respective branches and can share table information and features.

[0244] As for the table recognition device for document files disclosed in the embodiments of the present application, since it corresponds to the table recognition method for document files disclosed in the above method embodiments, the description is relatively simple. For relevant similarities, please refer to the description of the above method embodiments, which will not be described in detail here.

[0245] The present application also discloses an electronic device, the composition structure of the electronic device, such as Figure 4 As shown, including at least:

[0246] The memory 10 is used to store computer programs.

[0247] The processor 20 is configured to implement the table recognition method for a document file provided in any one of the above method embodiments by calling and executing the computer program in the memory.

[0248] The processor 20 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a neural network processor (NPU), a deep learning processor (DPU) or other programmable logic devices, etc.

[0249] In addition, the electronic device may also include components such as a communication interface and a communication bus. The memory, the processor and the communication interface communicate with each other through the communication bus.

[0250] The communication interface is used for communication between electronic devices and other devices. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0251] In addition, the present application also provides a computer-readable medium having a computer program stored thereon, wherein the computer program includes program code for executing the table recognition method for a document file as disclosed in any of the method embodiments above, and when executed by a processor, can be used to implement the table recognition method for a document file as disclosed in any of the method embodiments above.

[0252] In the context of the present application, a computer readable medium (machine readable medium) may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine readable medium may be a machine readable signal medium or a machine readable storage medium. A machine readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0253] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0254] For the convenience of description, the above system or device is described as being divided into various modules or units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0255] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0256] Finally, it should be noted that, in this article, relational terms such as first, second, third and target are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0257] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying a table in a document file, characterized in that: include: converting a page of the document file to be processed into a first image; Performing image enhancement processing on the first image to obtain a second image; The image enhancement processing is at least used to perform angle correction on the first image; Using a target intelligent agent to perform table detection on the second image, obtaining table area position information of the table in the second image; the target intelligent agent is a table detection model constructed by dual-branch model tuning based on deep learning; the first branch of the target intelligent agent is used to learn and extract structural features of the table, and the second branch of the target intelligent agent is used to learn and extract text semantic features of the table, and the first branch and the second branch are parallel; Identifying table data in the table area indicated by the table area position information, and converting the identified table data into a structured form for output; The process of constructing the target intelligent agent includes at least part of the following: Providing pre-trained model weights for the first deep learning model on the first branch and the second deep learning model on the second branch through transfer learning; wherein the pre-trained model weights provided for the first deep learning model are used to extract structural features of the table, and the pre-trained model weights provided for the second deep learning model are used to extract text semantic features of the table; Using small samples to respectively tune the first deep learning model and the second deep learning model after transfer learning; Performing data enhancement on the small samples, and using the enhanced samples obtained after the data enhancement to respectively tune the first deep learning model and the second deep learning model after transfer learning; During the tuning process, the first deep learning model and the second deep learning model can be independently optimized on their respective branches and can share table information and features.

2. The table recognition method of a document file according to claim 1, characterized in that: The performing image enhancement processing on the first image includes: The first image is subjected to at least part of size standardization, grayscale conversion, binarization, table contour detection, angle correction, and denoising and sharpening processing.

3. The table recognition method of a document file according to claim 2, characterized in that: The size standardization includes: Determining an adaptive factor for controlling the degree of adjustment of the interpolation kernel based on geometric features of the table in the first image; The size and shape of the interpolation kernel are adjusted based on the adaptive factor, and the adjusted interpolation kernel is used to perform interpolation-based scaling processing on the first image to obtain an image of a standard size.

4. The table recognition method of a document file according to claim 2, characterized in that: The grayscale conversion includes: Performing color space analysis on the first image or the size-standardized first image to obtain a color space analysis result; assigning a weight to each color according to the distribution and importance of each color in the first image or the size-normalized first image represented by the color space analysis result; Based on the weight corresponding to each color in the color channel of the pixel on the first image or the size-normalized first image, weighted average is performed on the brightness of various colors involved in the color channel of the pixel to obtain a weighted average result of the pixel brightness; Based on the weighted average result of pixel brightness, converting the first image or the size-normalized first image into a grayscale image; The binarization includes: Dividing the grayscale image into a plurality of local windows; Calculating the local contrast of each pixel in the local window; Determine a corresponding binarization threshold for each pixel in each of the local windows according to the local contrast of the pixel and a preset contrast threshold; The grayscale image is converted into a binary image according to the binarization threshold corresponding to each pixel.

5. The table recognition method of a document file according to claim 2, characterized in that: The table outline detection comprises: A convolutional neural network is used to extract low-level features and high-level features of a binary image; the low-level features include basic image features of the binary image, and the high-level features include table structure information in the binary image; the binary image is a result of binarizing the first image; Using an attention mechanism to perform feature fusion on the extracted low-level features and the high-level features to obtain a feature fusion result; the feature fusion is used to highlight features related to the table outline; Based on the feature fusion result, generating a contour information map of the table in the binary image; The angle correction includes: Based on the contour information map, determining text rows and table rows in the binary image; Determining the tilt angle of the binary image based on the voting results of the tilt angles of each text row and table row in the binary image through a regional voting mechanism; Angle correction is performed on the binarized image based on the tilt angle of the binarized image.

6. The table recognition method of a document file according to claim 2, characterized in that: The denoising and sharpening process comprises: Determine a denoising parameter based on a noise level of an input image, and determine a sharpening parameter based on a local feature of the input image; the input image is an image obtained after the first image is processed accordingly; The input image is subjected to denoising and sharpening processing based on the denoising parameter and the sharpening parameter respectively.

7. A table recognition device for a document file, characterized in that: include: An image conversion module, used for converting a page of a document file to be processed into a first image; An image enhancement module, used for performing image enhancement processing on the first image to obtain a second image; the image enhancement processing is at least used for performing angle correction on the first image; A table detection module, used to perform table detection on the second image using a target agent to obtain table area position information of the table in the second image; the target agent is a table detection model constructed by dual-branch model tuning based on deep learning; the first branch of the target agent is used to learn and extract structural features of the table, and the second branch of the target agent is used to learn and extract text semantic features of the table, and the first branch and the second branch are parallel; A table recognition module, used to recognize the table data in the table area indicated by the table area position information, and convert the recognized table data into a structured form for output; A model building module, used to provide pre-trained model weights for the first deep learning model on the first branch and the second deep learning model on the second branch through transfer learning; wherein the pre-trained model weights provided for the first deep learning model are used to extract structural features of the table, and the pre-trained model weights provided for the second deep learning model are used to extract text semantic features of the table; Using small samples to respectively tune the first deep learning model and the second deep learning model after transfer learning; Performing data enhancement on the small samples, and using the enhanced samples obtained after the data enhancement to respectively tune the first deep learning model and the second deep learning model after transfer learning; During the tuning process, the first deep learning model and the second deep learning model can be independently optimized on their respective branches and can share table information and features.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; The processor is used to implement the table recognition method of the document file according to any one of claims 1 to 6 by calling and executing the computer program in the memory.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can be used to implement the table recognition method for a document file as described in any one of claims 1 to 6.

Images