Intelligent analysis method for scanned professional title declaration documents based on deep feature extraction

Abstract

The present application belongs to the technical field of document intelligent analysis, and particularly relates to a scanned professional title declaration document intelligent analysis method based on deep feature extraction. First, the uploaded scanned document image is subjected to adaptive two-stage tilt correction and quality perception enhancement, and a standardized image is output. Then, the layout features are extracted through a network that fuses feature pyramids and global multi-head self-attention mechanisms, and the document type is automatically classified. Next, the text blocks are cut based on the table lines, and the text content is extracted using a fine-grained re-identification mechanism that enhances spatial attention and is driven by confidence. Then, the pre-trained language model is used to convert the text content into semantic feature vectors. Finally, the semantic features are matched with the pre-set template rule feature library, the compliance is calculated, and the judgment result is output. The present application realizes the full-link intelligent processing from the original scanned image to the compliance judgment result, and significantly improves the automation level, accuracy and auditing efficiency of the professional title declaration document analysis.

Description

Technical Field

[0001] This invention belongs to the field of document intelligent parsing technology, and in particular relates to an intelligent parsing method for scanned professional title application documents based on deep feature extraction. Background Technology

[0002] In the professional and technical personnel title evaluation and management system, personnel management departments at all levels need to accept and review a large number of title application documents submitted by applicants every year. With the in-depth promotion of digital archive management, most historical documents and newly submitted application documents are stored and archived in scanned form.

[0003] In current practices for reviewing application materials for professional titles, reviewers mostly rely on manually flipping through scanned images page by page, judging whether the document content is complete, the format is standardized, and key information items are filled in based on visual observation and subjective experience. This purely manual review model suffers from low efficiency; a single reviewer can typically process only a limited number of application files per day, easily leading to a large backlog of documents during peak application periods. The industry is also exploring the use of traditional optical character recognition (OCR) technology to extract text from scanned documents. However, scanned documents for professional title applications have unique characteristics that distinguish them from ordinary printed documents: First, original scanned documents generally carry varying degrees of slant distortion, which significantly reduces the recognition accuracy of traditional OCR engines for slanted text; second, the layout of professional title documents is complex, containing a mix of various layout elements such as tables, signatures, and barcodes, making it difficult for traditional methods to accurately distinguish different functional blocks during the layout analysis stage, leading to text positioning errors; third, traditional OCR only completes text extraction at the character level, lacking the ability to perform deep semantic understanding and automatic compliance judgment of the extracted text, requiring reviewers to still manually verify the OCR output results, failing to fundamentally solve the efficiency bottleneck problem. Furthermore, although some researchers have proposed document analysis solutions based on deep learning in recent years, most existing deep learning document analysis technologies are designed for general document scenarios, and their feature extraction network structures and training strategies have not been specifically optimized and adapted for the layout features and semantic content of scanned professional title application documents. When general document analysis models are directly applied to the professional title application scenario, due to factors such as dense table lines in professional title documents, text obscured by seal areas, and large differences in the layout of different application materials, the models have obvious shortcomings in terms of layout area segmentation accuracy, text recognition accuracy, and semantic compliance judgment reliability. Summary of the Invention

[0004] To address the technical problems existing in the background art described above, this invention proposes an intelligent parsing method for scanned professional title application documents based on deep feature extraction.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] S1. Receive the uploaded scanned image of the professional title document, perform preprocessing operations on the image, and output a standardized document image; the preprocessing includes tilt correction and image enhancement operations.

[0007] S2. Extract the global layout feature map from the standardized document image and output the document type probability distribution vector;

[0008] S3. Crop text blocks from standardized document images based on table lines, extract visual content features from the blocks, and obtain the text content of the blocks.

[0009] S4. Perform feature extraction on the text content to obtain semantic feature vectors;

[0010] S5. Match the semantic feature vector with the preset template rule feature library; calculate the matching score and compare it with the preset compliance threshold, output the judgment result, and if qualified, output the text content to obtain the document parsing result.

[0011] Preferably, the tilt correction operation in step S1 specifically includes:

[0012] The original scanned document image is binarized, and the main line segment features in the image are detected.

[0013] Based on the detected straight line segment features, the global dominant tilt angle of the document layout is calculated using the probabilistic Hough transform.

[0014] The entire image is initially rotated and corrected using an affine transformation based on the global dominant tilt angle.

[0015] On the image after initial rotation correction, locate and crop a reference area with standard rectangular features that is preset in the document. The reference area includes the title bar, the stamp area, or the fixed-format table lines.

[0016] Perform local edge refinement detection on the cropped reference region image and calculate the local residual tilt angle of its boundary relative to the horizontal axis of the image;

[0017] Based on the local residual tilt angle, a second fine rotation transformation is performed on the entire image to complete the tilt correction, and the corrected image is output.

[0018] Preferably, the image enhancement operation in step S1 specifically includes:

[0019] The tilt-corrected image is converted to grayscale to obtain a grayscale image G, which is then divided into M×N non-overlapping local blocks. Calculate the variance of the global grayscale histogram and the local block grayscale distribution, respectively.

[0020] Based on the global grayscale histogram and local variance, the image's illumination uniformity, overall contrast, and noise level are evaluated, and a quality feature vector is generated; the illumination uniformity is: ,in, For local blocks The average gray level, The global grayscale average; the overall contrast ratio ,in, The global standard deviation of the grayscale image G; the noise level ,in, For local blocks Noise standard deviation estimated based on the Laplacian operator, To prevent extremely small positive numbers from being divided by zero;

[0021] The indicators in the quality feature vector are normalized. Based on the comparison results between the normalized quality feature vector and each preset threshold, operators are adaptively selected and processed from the preset enhancement strategy library to generate an enhancement processing sequence. The selection rules are as follows: when the illumination uniformity indicator is lower than the corresponding illumination uniformity threshold, illumination normalization processing based on homomorphic filtering is selected; when the overall contrast indicator is lower than the overall contrast threshold, contrast-limited adaptive histogram equalization processing is selected; when the noise level indicator is higher than the noise level threshold, filtering and denoising processing based on non-local means is selected.

[0022] The operators in the processing sequence are topologically sorted according to their dependencies. The enhancement operators are then applied to the grayscale image G in sequence according to the sorting results, and the final output is a standardized document image. The dependencies are: denoising before illumination normalization, denoising before contrast enhancement, and illumination normalization before contrast enhancement.

[0023] As a preferred embodiment, step S2, which extracts the global layout feature map from the standardized document image and outputs the document type probability distribution vector, is specifically implemented as follows:

[0024] Standardized document images are input into a backbone feature extraction network, which employs a multi-level residual convolutional structure. At different depth levels, the spatial resolution is progressively reduced while the channel dimension is increased, extracting multiple feature maps at different scales to form a multi-scale feature map set. A top-down feature pyramid fusion operation is then performed on this multi-scale feature map set. The feature map of the previous level is sequentially convolved laterally to unify the channel dimension, and then added element-wise with the higher-level fused feature map that has undergone bilinear upsampling. This process gradually fuses feature information from different scales, resulting in a fused feature pyramid set with unified channel dimensions across all layers.

[0025] The feature maps of each layer in the fused feature pyramid set are flattened into one-dimensional feature sequences along the spatial dimension. The feature sequences of each layer are spliced ​​along the sequence length dimension to form a global layout feature sequence representing global layout information. Learnable hierarchical position codes and spatial position codes are added to each position element in the global layout feature sequence. The two sets of position codes are added to the original feature sequence element by element to obtain an input sequence with position awareness.

[0026] The feature sequence with positional encoding is input into a multi-layered global multi-head self-attention encoder. In each encoder layer: the input sequence is transformed into a query matrix, a key matrix, and a value matrix through three independent linear projection transformations; the dot product similarity between the query and the key is calculated and scaled; the attention weight matrix is ​​obtained by normalization using the Softmax function; and the attention weights are multiplied by the value matrix to achieve global information interaction between any two positions in the sequence. After the multi-head self-attention operation, each encoder layer outputs through residual connections and layer normalization, then passes through a feedforward network, residual connections, and layer normalization to output the encoding result of that layer, which serves as the input for the next layer. The output of the final encoder layer is the global layout feature map.

[0027] The global layout feature map output by the final encoder layer is taken, and a global average pooling operation is performed along the sequence length dimension to compress the sequence information into a global layout feature vector of fixed dimension. The global layout feature vector is input into a document type classification head composed of a fully connected layer and a Softmax activation function, and mapped to a probability distribution vector with a dimension equal to the total number of predefined document types. The category with the highest probability value is taken as the document type prediction result. The document type information is used for subsequent matching by selecting the corresponding template rule feature library.

[0028] Preferably, step S3 involves extracting visual content features from the block image to obtain the text content of the block image, specifically including:

[0029] For each cropped block image, a grayscale projection histogram is calculated along the horizontal direction. The upper and lower boundaries of each text line are determined based on the start and end positions of continuous non-zero pixel segments in the projection histogram, thus segmenting the block image into multiple text line images. For each text line image, a grayscale projection histogram is calculated along the vertical direction. The segmentation points between characters are identified based on the comparison between the gap between adjacent non-zero pixel segments and a preset character spacing threshold. Connected components of adjacent characters with a spatial distance less than a preset merging threshold are clustered to form a complete text line image sequence.

[0030] Each text line image is scaled to a uniform height while maintaining its aspect ratio and then input into a convolutional neural network encoder. After multiple convolutions, batch normalization, and non-linear activation operations, a two-dimensional local visual feature map of the text line is extracted. Global average pooling and global max pooling are calculated along the channel dimension of the two-dimensional local visual feature map. The two pooling results are concatenated and then passed through a convolutional layer and a sigmoid activation function to generate a spatial attention weight matrix. The spatial attention weight matrix is ​​multiplied element-wise with the original feature map to obtain an enhanced feature map that suppresses background noise and highlights the text region.

[0031] The enhanced feature map is extracted column by column along the width direction to form a sequence of visual feature vectors arranged in reading order. The sequence of visual feature vectors is input into a sequence text recognition decoder based on an attention mechanism. At each decoding time step, the decoder calculates attention weights on the sequence of visual feature vectors based on the current decoder hidden state and aggregates them to obtain a context vector. The context vector and the embedding vector of the output character in the previous step are input into the decoder unit to update the hidden state. The probability distribution of each candidate character in the current time step is predicted through a fully connected layer and a Softmax function. The character with the highest probability is taken as the recognition output of the current step. The process is repeated step by step until the output sequence terminates, thus obtaining the preliminary recognized text of the text line.

[0032] For each line of text, a recognition confidence score is calculated, which is the geometric mean of the probabilities of the largest candidate characters at each decoding time step. The recognition confidence score is compared with a preset confidence threshold. If it is higher than the threshold, the preliminary recognition text is directly output. If it is lower than the threshold, the visual feature column interval corresponding to the low-confidence character is located. The residual between the feature vector in the interval and the local interval mean is calculated and locally enhanced through a feedforward network before being re-input into the decoder for refined recognition. The refined recognition result replaces the corresponding low-confidence segment. Finally, the recognition results of all lines of text are concatenated in spatial order to obtain the complete text content of the block image.

[0033] As a preferred embodiment, step S4, which extracts semantic feature vectors from the text content, is specifically implemented as follows: the text content is input into the pre-trained model, and after word segmentation and tokenization, the hidden state vector obtained by average pooling the last layer of tokens is used as the semantic feature vector.

[0034] Preferably, in step S5, the semantic feature vector is matched with a preset template rule feature library; the matching score is calculated and compared with a preset compliance threshold, and the judgment result is output. The specific implementation is as follows:

[0035] Select the corresponding template rule feature library based on the document type, calculate the cosine similarity between the semantic feature vector and the feature vector of each rule in the rule library, and obtain the matching score of each rule.

[0036] The matching score of each rule is compared with its corresponding preset compliance threshold. If the matching score is greater than or equal to the compliance threshold, the rule is deemed qualified; otherwise, it is deemed unqualified.

[0037] When all rule items are qualified, the overall judgment result of the output document is qualified, and the text content is output as the document parsing result; when at least one rule is determined to be unqualified, the overall judgment result of the output document is unqualified, and the non-compliant rule information is output.

[0038] Compared with existing technologies, the advantages and positive effects of this invention are as follows: By designing an adaptive two-level tilt correction strategy, a global coarse correction is first performed using probabilistic Hough transform, followed by a sub-pixel-level fine secondary correction based on the document reference region. This effectively eliminates multi-dimensional geometric distortions introduced by factors such as scanner feed skew and paper curvature, providing high-quality standardized input images for subsequent feature extraction. By introducing a layout analysis network that integrates feature pyramids and a global multi-head self-attention mechanism, the multi-scale local feature extraction capability of convolutional neural networks and the global context modeling capability of Transformer encoders are fully combined, achieving high-precision automatic classification of various layout types in professional title application documents, such as personal information pages, certificate pages, and performance pages. By integrating a spatial attention enhancement module and a confidence-driven fine-grained re-recognition mechanism into the text recognition pipeline, significant robustness improvements are demonstrated for low-quality scanning scenarios with issues such as seal obstruction, ink stains, and table line interference. By using a pre-trained language model to extract deep semantic feature vectors and matching them with a preset template rule feature library in the vector space, the entire process from the original scanned image to the compliance judgment result is automated. This significantly reduces the workload of manual review, eliminates subjective judgment differences between different reviewers, and ensures the standardization and efficiency of the review of professional title application materials. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram illustrating the implementation process of an intelligent parsing method for scanned professional title application documents based on deep feature extraction. Detailed Implementation

[0041] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0042] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.

[0043] In this embodiment, within the professional and technical personnel title evaluation management system, personnel management departments at all levels need to accept and review a large number of title application documents submitted in the form of scanned images each year. Traditional manual review methods suffer from inefficiency, inconsistent subjective judgments, and poor adaptability to low-quality scanned documents. Although the industry has attempted to introduce traditional OCR technology to assist in text extraction, traditional methods have significant shortcomings in key areas such as scanned document tilt correction, complex layout analysis, recognition of text obscured by seals, and semantic compliance judgment. To address these technical bottlenecks, this embodiment provides an intelligent parsing method for scanned title application documents based on deep feature extraction. This method implements adaptive two-level tilt correction and quality perception enhancement strategies in the image preprocessing stage; constructs a layout analysis network integrating multi-scale convolutional features and global self-attention in the feature extraction stage; introduces spatial attention enhancement and a confidence-driven refined re-recognition mechanism in the text recognition stage; and utilizes a pre-trained language model to generate deep semantic representations and perform rule matching in the semantic understanding stage. This achieves end-to-end intelligent processing from the original scanned image input to the compliance judgment result output, comprehensively improving the automation level and judgment consistency of title application document parsing. The implementation process of this invention is as follows: Figure 1 As shown.

[0044] First, the uploaded scanned document image for professional title evaluation is received. Preprocessing operations are then performed on the image to output a standardized document image. The preprocessing includes tilt correction and image enhancement operations. Specifically, in the tilt correction operation, this embodiment considers that the existing single global Hough transform correction method can only eliminate the overall page tilt angle at the macro level. However, in the actual acquisition scenario of scanned documents for professional title evaluation, due to local warping of the paper on the scanner glass panel and uneven force applied by the paper feed rollers causing slight paper distortion, the image after global rotation correction still retains sub-pixel-level subtle tilt distortion in local areas. Although this residual distortion is not easily noticeable visually, even a local tilt of 0.1 to 0.5 degrees can cause the peak position of the horizontal projection histogram to shift, leading to inaccurate text line segmentation boundaries, for subsequent operations such as table line cropping and precise text block positioning. Therefore, this invention innovatively designs a two-stage correction strategy: global coarse correction + local reference area fine secondary correction. This eliminates both macro-tilt and micro-residual distortion, effectively ensuring the geometric regularity of the preprocessed image. Specifically, for the original scanned document image Adaptive binarization is performed to obtain a binary image. The Canny edge detection operator, based on connected component area weighting, is used to extract the main line segment feature set L in the document layout. Each line segment... It is characterized by its midpoint coordinates, length, and tilt angle; based on the detected line segment feature set L, using an improved probabilistic Hough transform and with the line segment length as the confidence weight, the globally dominant tilt angle of the document layout is calculated. , ,in, The length weight of the i-th line segment is... Given its corresponding tilt angle; construct the affine transformation matrix. According to the global dominant tilt angle Perform initial rotation correction on the entire image to obtain the initially corrected image. The affine transformation matrix is ​​defined as: ,in, This is the translation amount, used to ensure the image is centered after rotation; initial image correction. It is obtained from the following mapping: ,in The pixel coordinates of the output image are used in the initially calibrated image. The above describes a method for locating and cropping a set of reference regions with standard rectangular geometric features in a document using a pre-trained document reference region detection network. Subpixel-level edge refinement detection is then performed on the four boundaries of each reference region, and the local residual tilt angles of these four boundaries relative to the horizontal axis of the image are calculated. Finally, the global residual tilt angle is obtained by weighting and fusing the regions based on their areas. The calculation formula is as follows: ,in, Let j be the local residual tilt angles of the boundaries relative to the horizontal axis of the image. Let be the area of ​​each reference region. When the absolute value of the global residual tilt angle exceeds the preset fine correction threshold, construct a second-order fine affine transformation matrix. : For images A second fine rotation transformation is performed to obtain the final tilt-corrected image.

[0045] Furthermore, image enhancement operations are performed on the tilt-corrected document image. This embodiment considers that existing technologies typically employ fixed-parameter global histogram equalization or single denoising filtering for scanned document images, failing to account for the diverse types and degrees of image quality degradation introduced by different scanned documents due to variations in scanning equipment and lighting conditions. When using a fixed-parameter processing strategy, documents with good inherent contrast may produce artifacts due to over-enhancement, while documents severely contaminated with noise may experience a decrease in subsequent text recognition accuracy due to insufficient enhancement. Therefore, this invention designs an adaptive enhancement strategy selection mechanism based on automatic image quality assessment.

[0046] Specifically, after converting the tilt-corrected image to grayscale to obtain a grayscale image G, the image is divided into M×N non-overlapping local blocks. The global grayscale histogram and the local block grayscale distribution variance are calculated respectively. Based on the global grayscale histogram and the local variance, the illumination uniformity, overall contrast, and noise level of the image are evaluated, and a quality feature vector is generated. The illumination uniformity is: ,in, For local blocks The average gray level, The global grayscale average; the overall contrast ratio ,in, The global standard deviation of the grayscale image G; the noise level ,in, For local blocks Noise standard deviation estimated based on the Laplacian operator, To prevent extremely small positive numbers from being divided by zero, the indicators in the quality feature vector are normalized. Based on the comparison results between the normalized quality feature vector and each preset threshold, operators are adaptively selected and processed from a preset enhancement strategy library to generate an enhancement processing sequence. The selection rules are as follows: when the illumination uniformity indicator is lower than the corresponding illumination uniformity threshold, illumination normalization processing based on homomorphic filtering is selected; when the overall contrast indicator is lower than the overall contrast threshold, contrast-limited adaptive histogram equalization processing is selected; when the noise level indicator is higher than the noise level threshold, filtering and denoising processing based on non-local means is selected. Finally, the operators in the processing sequence are topologically sorted according to their dependencies. The processor first generates an initial set of enhancement operator activations based on the comparison results of the three quality indicators. Specifically, three Boolean status flags are set: when the normalized noise level index is greater than the noise level threshold, the denoising flag F_denoise is set to logical true; when the normalized illumination uniformity index is lower than the illumination uniformity threshold, the illumination normalization flag F_homomorphic is set to logical true; and when the normalized overall contrast index is lower than the overall contrast threshold, the contrast enhancement flag F_clahe is set to logical true. If a certain index does not trigger the threshold condition, the corresponding flag remains logically false, and the operator does not enter the processing sequence. After determining the set of activated operators, the processor constructs a directed acyclic graph (DAG) data structure in memory to represent the dependencies between operators. This DAG consists of three nodes (representing the denoising operator, illumination normalization operator, and contrast enhancement operator, respectively) and directed edges. The rules for constructing directed edges are as follows: denoising nodes point to illumination normalization nodes (indicating that denoising must be performed before illumination normalization), denoising nodes point to contrast enhancement nodes (indicating that denoising must be performed before contrast enhancement), and illumination normalization nodes point to contrast enhancement nodes (indicating that illumination normalization must be performed before contrast enhancement). Inactive operator nodes and their associated directed edges are marked as disabled and removed from the graph before topological sorting. The processor executes the classic Kahn topological sorting algorithm on the constructed directed acyclic graph. Specifically, the algorithm maintains an in-degree count array to count the number of in-edges for each active node. All nodes with an in-degree of zero are pushed into a working queue. Then, nodes are iteratively removed from the head of the queue, appended to the end of the sorted sequence, and the in-degree of all successor nodes pointed to by their outgoing edges is decremented by one. When the in-degree of a successor node reaches zero, it is pushed into the working queue. This process is repeated until the working queue is empty; at this point, the sorted sequence represents the execution order of the enhancement operators that satisfies all dependency constraints.Due to the inherent hierarchical nature of dependencies, when the number of activation operators is 0 to 3, the unique valid output of topological sorting is as follows: When only denoising is activated, the execution sequence is [Denoising]; when only illumination normalization is activated, the execution sequence is [Illumination Normalization]; when only contrast enhancement is activated, the execution sequence is [Contrast Enhancement]; when both denoising and illumination normalization are activated simultaneously, the execution sequence is [Denoising → Illumination Normalization]; when both denoising and contrast enhancement are activated simultaneously, the execution sequence is [Denoising → Contrast Enhancement]; when both illumination normalization and contrast enhancement are activated simultaneously, the execution sequence is [Illumination Normalization → Contrast Enhancement]; when all three operators are activated, the execution sequence is [Denoising → Illumination Normalization → Contrast Enhancement]. According to the sorting result sequence, the processor sequentially executes each enhancement operator on the grayscale image G: the output image after each operator is processed serves as the input for the next operator. After the previous operator is processed, the processor caches the intermediate result image in the working buffer of random access memory, waiting for the next operator to read and process it. After all operators are executed, the final output image is the standardized document image. This approach can dynamically select the most suitable combination of enhancement operators and their execution order from the strategy library based on the actual quality of each document image, avoiding the quality fluctuation problem caused by a one-size-fits-all fixed processing.

[0047] The next step is to extract global layout feature maps from standardized document images and output document type probability distribution vectors. Specifically, this embodiment considers that most traditional document layout analysis methods in the prior art rely on rule-based connected component analysis or shallow convolutional feature classifiers. Rule-based connected component analysis methods are prone to missegmentation or omission of blocks when dealing with complex layouts such as overlapping seal areas and text areas, and interwoven table lines and body text lines in professional title application documents. Although shallow convolutional feature classifiers can adaptively learn layout features to a certain extent, their receptive field is limited by the small convolutional kernel size, and they cannot capture long-range spatial dependencies in the document layout. For example, the format information of the title bar at the top of the document may be of important reference value for determining the layout type of the review comments area at the bottom, but shallow networks are difficult to model such long-range associations across the entire image. Therefore, this invention innovatively combines a multi-scale convolutional feature pyramid with a global multi-head self-attention mechanism, which not only retains the ability of convolutional neural networks to extract local texture and structural features at different scales, but also introduces the global information interaction capability of the Transformer encoder, achieving high-precision classification of the layout of professional title application documents.

[0048] Specifically, the standardized document image is first input into the backbone feature extraction network. The backbone network employs a multi-level residual convolutional structure (ResNet architecture), consisting of 4 to 6 residual stages stacked sequentially (preferably 5 residual stages in this embodiment). At different depth levels, the network progressively reduces the spatial resolution (downsampling is achieved through convolution or pooling operations with a stride of 2, reducing the spatial resolution to 1 / 2 to 1 / 4 of the previous level with each level reduction) and increases the channel dimension (in this embodiment, the number of channels at each level is 64, 128, 256, 512, and 1024, respectively), extracting multiple feature maps {C2, C3, C4, C5} at different scales to form a multi-scale feature map set. C2 has the highest spatial resolution and the shallowest semantic level, capturing fine-grained texture features in the document image (such as table lines, text stroke edges, etc.); C5 has the lowest spatial resolution and the deepest semantic level, encoding high-level semantic information in the document image (such as region categories, overall layout structure, etc.).

[0049] A top-down Feature Pyramid Network (FPN) operation is performed on the multi-scale feature map set. Specifically, starting from the deepest layer C5, the feature map of the previous layer is sequentially processed by a 1×1 horizontal convolution to unify the channel dimension to 256. This is then added element-wise with the higher-level fused feature map that has undergone bilinear upsampling (Scale Factor 2), gradually fusing feature information from different scales to obtain a fused feature pyramid set {P2, P3, P4, P5} with unified channel dimensions for each layer, where each layer's feature map has a channel dimension of 256. Each layer's feature map in the fused feature pyramid set is then flattened along the spatial dimensions into a one-dimensional feature sequence. Specifically, for a feature map Pi with spatial dimensions Hi×Wi and channel number C=256, it is flattened along the height and width dimensions into a one-dimensional sequence with length Hi×Wi and a feature dimension of 256. The feature sequences from each level are concatenated along their length to form a global layout feature sequence Sglobal, representing global layout information. The total sequence length of Sglobal is Ltotal = Σ(Hi × Wi), and the feature dimension of each position element is 256. Learnable hierarchical position encoding and spatial position encoding are added to each position element in the global layout feature sequence Sglobal. Specifically, for feature elements from level i and spatial position (row, col), the system maintains two sets of learnable embedding vectors: a hierarchical position encoding vector ei_level ∈ (There are 4 in total, corresponding to P2 to P5 respectively) and the spatial location encoding vector ei_spatial(row, col) ∈ (Constructed using a row-column separation encoding method, with each row and column encoding maintaining a set of learnable parameters, which are then added together to obtain a two-dimensional spatial position encoding). The hierarchical position encoding and spatial position encoding are added element-wise to the feature vectors at the corresponding positions of the original feature sequence to obtain the input sequence Spos with position-aware capabilities.

[0050] The position-encoded feature sequence Spos is input into a global multi-head self-attention encoder consisting of L stacked layers (in this embodiment, L is an integer between 3 and 8, preferably 6 layers). In each encoder layer, the input sequence is transformed by three independent linear projection transformations W_Q, W_K, and W_V ∈ […]. (In this embodiment, d_k is 64, and the number of attention heads h is 8, so the key / query dimension of each head is d_k / h = 8.) Generate a query matrix Q, a key matrix K, and a value matrix V. Calculate the dot product similarity between the query and the key and perform scaling: Attention(Q,K,V)=Softmax(QK^T / sqrt(d_k))·V. Normalize the attention weight matrix using the Softmax function, and multiply the attention weights with the value matrix to achieve global information interaction between any two positions in the sequence. After the multi-head self-attention operation, each encoder layer outputs through residual connections and layer normalization, then passes through a feedforward network consisting of two fully connected layers (intermediate dimension 1024, activation function GELU), residual connections, and layer normalization, outputting the encoding result of that layer as the input to the next layer. The output of the final layer (layer L) encoder is the global layout feature map Fglobal∈ .

[0051] Finally, the global layout feature map Fglobal output by the final encoder layer is taken, and a global average pooling operation is performed along the sequence length dimension to compress the sequence information into a global layout feature vector vlayout with a fixed dimension (256 dimensions). The global layout feature vector vlayout is then input into a document type classification head consisting of a fully connected layer and a Softmax activation function. Specifically, the classification head contains a fully connected layer that maps the 256-dimensional features to K dimensions (K equals the total number of predefined document types; in this embodiment, K is an integer between 5 and 15, preferably 8), followed by a Softmax activation function, mapping to a probability distribution vector p = [p1, p2, ..., pK] with dimension K, where p1 represents the predicted probability that the input document belongs to the i-th layout type. The category with the highest probability value is taken as the document type prediction result: type = argmax(p). The document type information is used in subsequent steps to select the corresponding template rule feature library for matching.

[0052] Next, in the subsequent step, the text block image is cropped from the standardized document image based on table lines, and visual content feature extraction is performed on the block image to obtain the text content of the block image. In this embodiment, this step takes into account the following technical challenges faced by the traditional document text recognition methods in the prior art when processing scanned documents for professional title applications: First, there are a large number of dense table lines in professional title documents, and traditional methods tend to misjudge the horizontal table lines as the baselines of text lines during text line positioning, resulting in offset of line segmentation positions; Second, traditional methods lack quantitative evaluation of the reliability of recognition results after text recognition, and cannot perform targeted refined reprocessing on low-quality recognition areas, leading to a high misrecognition rate of similar-shaped characters (such as "己", "已", "巳", etc.) and similar numbers (such as "1" and "l", "0" and "O", etc.). Therefore, on the basis of the traditional text recognition pipeline, the present invention introduces a spatial attention enhancement module to adaptively suppress background interference, and a refined re-recognition mechanism based on confidence evaluation to targeted improve the recognition quality of low-confidence characters, thereby integrally improving the text recognition accuracy for professional title documents with complex layouts.

[0053] Specifically, for each cropped block image B, grayscale processing is performed first, and a grayscale projection histogram is calculated along the horizontal direction: , , where, is the height of the block image, is the width of the block image, represents the grayscale value of the block image at the coordinate (x,y), is the cumulative sum of all pixel grayscale values in the y-th row, for performs mean smoothing filtering with a window size of to eliminate noise burrs, and sets an adaptive threshold: , where, is the mean value of all values in the projection histogram, is the standard deviation of all values in the projection histogram, is an adjustment coefficient, which is 0.3 in this embodiment. The row intervals that continuously satisfy are identified as the upper and lower boundaries of the i-th text line, where is the starting row coordinate of the i-th text line, is the ending row coordinate. Accordingly, the block image is segmented into text line images along the horizontal direction , where, is the height of the i-th text line image. For each text line image , a grayscale projection histogram is calculated along the vertical direction , that is, summing all pixel grayscale values in each column: , ​The segmentation points between characters are identified by comparing the gap width between adjacent non-zero pixel segments with a preset character spacing threshold. A character segmentation point is determined when dg > Tg, where dg is the width of consecutive zero-value pixels between two adjacent non-zero pixel segments, and Tg is a preset character spacing threshold based on the document font size. Connected components of adjacent characters with a spatial distance less than a preset merging threshold are clustered. A connected component refers to a connected region formed by adjacent non-zero pixels in a binary image. The preset merging threshold is used to determine whether adjacent connected components belong to the same character (e.g., in Chinese characters composed of multiple separate strokes). After the above processing, a complete text line image sequence is formed.

[0054] Image of each text line Scale to uniform height Pixels are preserved while maintaining aspect ratio to obtain a normalized text line image. ,in The scaled width depends on the aspect ratio of the original text line image. The normalized text line image is input into a convolutional neural network encoder. The encoder consists of seven convolutional blocks stacked sequentially, each containing a convolutional layer, a batch normalization layer, and a ReLU nonlinear activation function. The first three convolutional blocks have a 3×3 kernel size, a stride of 1, and padding of 1, used to extract low- and mid-level visual features while maintaining spatial resolution. The fourth and fifth convolutional blocks have a 3×3 kernel size and a stride of (2,1), indicating a stride of 2 in the height direction and 1 in the width direction, i.e., downsampling only in the height direction. The sixth and seventh convolutional blocks have a 3×3 kernel size and a stride of (2,1), continuing downsampling in the height direction. After processing by these seven convolutional blocks, the height dimension of the text line image is reduced from [previous value]. Gradually reduce the width to 1, preserving the sequence structure in the width direction. The encoder outputs a two-dimensional local visual feature map: ,in, This represents the length of the feature sequence in the width direction, i.e., the number of feature columns retained in the width direction after the original width is downsampled twice in the height direction. The number of feature channels. For the aforementioned two-dimensional local visual feature map. Spatial attention enhancement is performed. Since the height dimension is reduced to 1 after convolutional encoding, spatial attention degenerates into one-dimensional attention along the width direction. Specifically, global average pooling is calculated along the channel dimension. and global max pooling The two pooling results are concatenated into a two-dimensional vector, which is then passed through a one-dimensional convolutional layer with a kernel size of 7 and a sigmoid activation function to generate a spatial attention weight vector. The attention weight vector is multiplied element-wise with the original feature map to obtain the enhanced feature map: ,in, This represents the feature value of the c-th channel at position (1, w) after spatial attention enhancement. The spatial attention mechanism can adaptively allocate weights, assigning higher weights to feature columns containing character strokes and lower weights to feature columns containing table residual line segments or background noise, thereby enhancing the feature representation of the text region and suppressing interference from irrelevant regions.

[0055] Enhance feature maps Extracting column by column along the width direction to form a sequence of visual feature vectors. ,in, Let V represent the feature vector of column t. Input V into the attention-based sequence text recognition decoder. In this embodiment, the decoder uses a one-way gated recurrent unit (GRU), where GRU is a type of recurrent neural network unit that controls the retention and forgetting of information by updating and resetting gates. The character set includes commonly used Chinese characters, numbers, punctuation marks, and sequence start characters. <sos>and terminator <eos>The character set size is That is, the character set contains Each character is distinct. At each decoding time step t (t=1,2,…,Ti, where Ti is the total number of decoding steps for the i-th text line), the decoder's hidden state from the previous step is first determined. Calculate attention weights at each position in the visual feature vector sequence, where Let be the hidden state vector output by the decoder at step t-1. The dimension of the hidden state is 256 in this embodiment: , ,in, Let be the original attention score for the j-th visual feature position at the t-th decoding step. The weight vector calculated for the attention score; The weight matrix is ​​used to map the hidden states to the attention intermediate space. The weight matrix is ​​used to map visual features to the attention intermediate space. The bias vector calculated for attention; The attention weights, after Softmax normalization, represent the degree of attention paid to the j-th visual feature position in the t-th decoding step. Weighted aggregation yields the context vector for the current step. , The context vector is obtained by weighting and summing all visual features according to attention weights in the current decoding step, and it integrates the image region information that is most important in the current step. The context vector... The embedding vector of the output character in the previous step After splicing, input into the GRU unit: ,in The dense vector representation of the output character at step t-1 obtained by the embedding layer is updated with the hidden state. After predicting the probability distribution of each candidate character at the current time step using a fully connected layer and a softmax function, the character with the highest probability is selected as the recognition output for the current step. The character embedding vector corresponding to the character with the highest probability is then used. As input for the next step. The input for the first decoding step is the start symbol. <sos>The embedding vector. The process is executed iteratively step-by-step until a terminator is output. <eos>Or, once the maximum number of decoding steps is reached, the preliminary recognized text of that line of text is obtained.

[0056] For each line of text, a recognition confidence score is calculated. This score is the geometric mean of the probabilities of the largest candidate characters at each decoding time step, used to comprehensively measure the reliability of the entire line's text recognition result. To avoid underflow due to floating-point precision limitations during multiplication, a logarithmic form is used in the actual calculation. ,in, The output character at step t, Given the character sequence that has been output in the first t-1 steps, The confidence score is used to determine the quality of the text line's recognition. If the confidence score is greater than or equal to a set threshold, the text line is considered to have reliable recognition quality, and the preliminary recognition text is directly output as the final recognition result.

[0057] If the confidence score is less than a set threshold, the text line is considered to contain characters that are not accurately recognized, requiring refined attention window re-recognition. The specific process is as follows: First, based on the attention weight distribution recorded at each time step during decoding, the highest character prediction probability at each decoding time step is checked one by one. Time steps with a single-step recognition probability lower than a preset single-step confidence threshold are marked as low-confidence time steps, and all low-confidence time steps are collected into a set. Then, for each low-confidence time step in the set, the index of the position where the maximum attention weight is located during decoding is found, i.e., the position of the visual feature column most concerned in that decoding step. Using this position as the center, a preset half-window width is extended to the left and right to determine a local feature column interval, ensuring that this interval does not exceed the legal index boundaries of the feature sequence. Next, all visual feature vectors within this local interval are refined and enhanced. Specifically, first, the element-wise average of all feature vectors within the interval is calculated to obtain a mean vector representing the overall feature level of the local region. Then, for each feature vector within the interval, the difference between it and the mean vector is calculated, i.e., the residual vector. This residual reflects the degree to which the feature column deviates from the local average level. The residual vector is input into a feedforward network with one hidden layer for nonlinear transformation, outputting a feature correction value. This correction value is multiplied by a preset enhancement coefficient and added to the original feature vector to obtain the enhanced feature vector. In this way, the unique feature components that differ from adjacent characters in each feature vector are amplified, enhancing the differential expression between similar characters within the same window, thereby improving the ability to distinguish easily confused characters such as similar-looking characters. Finally, the enhanced feature vector replaces the original feature vector at the corresponding position, leaving the other positions unchanged, resulting in an updated visual feature sequence. The updated sequence is re-input into the decoder, and decoding and recognition are performed only for low-confidence time steps. The refined recognition result replaces the low-confidence characters at the corresponding positions, while the recognition results for the other high-confidence time steps remain unchanged. Finally, the recognition results of all text lines are concatenated in spatial order from top to bottom to obtain the complete text content of the block image.

[0058] Then feature extraction is performed on the text content respectively to obtain semantic feature vectors. Specifically, the text content is input into a pre-trained language model. The pre-trained language model adopts a Chinese pre-trained model based on the Transformer encoder architecture, with a hidden layer dimension of 768, which can learn lexical semantics and context association information on large-scale Chinese corpora, and has the ability of deep semantic understanding of professional domain texts. The built-in tokenizer of the model is called to perform word segmentation and tokenization processing on the text content, and the text is divided into an ordered sub-word token sequence. A special classification token is inserted at the head of the sequence, and padding or truncation is performed at the tail of the sequence to make the sequence length uniformly a preset maximum sequence length, and an attention mask sequence is generated at the same time to identify the positions of valid tokens and padding tokens. The processor inputs the token sequence and the attention mask sequence into the pre-trained model, maps them to dense vectors through the embedding layer and superimposes the position encoding, and then performs forward inference through multiple layers of Transformer encoders in sequence. Each layer of the encoder performs multi-head self-attention calculation and feed-forward network transformation, and outputs the encoding result layer by layer through residual connection and layer normalization. The final layer of the encoder outputs the hidden state vector matrix corresponding to the tokens at each position. The positions of valid tokens are determined according to the attention mask sequence, the hidden state vectors of the last layer corresponding to all valid tokens are extracted, these vectors are summed element-wise on the 768 dimensions and then divided by the total number of valid tokens to complete the average pooling operation, and the semantic feature vector is obtained.

[0059] Finally, the corresponding template rule feature library is selected according to the document type, the cosine similarity calculation is performed between the semantic feature vector and each rule feature vector in the rule library to obtain the matching score of each rule, then the matching score of each rule is compared with its corresponding preset compliance threshold respectively. If the matching score is greater than or equal to the compliance threshold, the rule item is determined to be qualified, otherwise it is determined to be unqualified. When all rule items are qualified, the overall judgment result of the document is output as qualified, and the text content is output as the document parsing result; when at least one rule is determined to be unqualified, the overall judgment result of the document is output as unqualified, and the non-compliant rule information is output.

[0060] The above description is only the preferred embodiments of the present invention, and is not intended to limit the present invention in other forms. Any person skilled in the art may use the technical content disclosed above to make alterations or modifications into equivalent embodiments with equivalent changes applied to other fields. However, any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the protection scope of the technical solution of the present invention.< / eos> < / sos> < / eos> < / sos>

Claims

1. A method for intelligent parsing of scanned professional title application documents based on deep feature extraction, characterized in that: Includes the following steps: S1. Receive the uploaded scanned image of the professional title document, perform preprocessing operations on the image, and output a standardized document image; the preprocessing includes tilt correction and image enhancement operations. S2. Extract the global layout feature map from the standardized document image and output the document type probability distribution vector; S3. Crop text blocks from standardized document images based on table lines, extract visual content features from the blocks, and obtain the text content of the blocks. S4. Perform feature extraction on the text content to obtain semantic feature vectors; S5. Match the semantic feature vector with the preset template rule feature library; calculate the matching score and compare it with the preset compliance threshold, output the judgment result, and if qualified, output the text content to obtain the document parsing result.

2. The intelligent parsing method for scanned professional title application documents based on deep feature extraction according to claim 1, characterized in that, The tilt correction operation in step S1 specifically includes: The original scanned document image is binarized, and the main line segment features in the image are detected. Based on the detected straight line segment features, the global dominant tilt angle of the document layout is calculated using the probabilistic Hough transform. The entire image is initially rotated and corrected using an affine transformation based on the global dominant tilt angle. On the image after initial rotation correction, locate and crop a reference area with standard rectangular features that is preset in the document. The reference area includes the title bar, the stamp area, or the fixed-format table lines. Perform local edge refinement detection on the cropped reference region image and calculate the local residual tilt angle of its boundary relative to the horizontal axis of the image; Based on the local residual tilt angle, a second fine rotation transformation is performed on the entire image to complete the tilt correction, and the corrected image is output.

3. The intelligent parsing method for scanned professional title application documents based on deep feature extraction according to claim 1, characterized in that, The image enhancement operation in step S1 specifically includes: The tilt-corrected image is converted to grayscale to obtain a grayscale image G, which is then divided into M×N non-overlapping local blocks. Calculate the variance of the global grayscale histogram and the local block grayscale distribution, respectively. Based on the global grayscale histogram and local variance, the image's illumination uniformity, overall contrast, and noise level are evaluated, and a quality feature vector is generated; the illumination uniformity is: ,in, For local blocks The average gray level, The global grayscale average; the overall contrast ratio ,in, The global standard deviation of the grayscale image G; the noise level ,in, For local blocks Noise standard deviation estimated based on the Laplacian operator, To prevent extremely small positive numbers from being divided by zero; The indicators in the quality feature vector are normalized. Based on the comparison results between the normalized quality feature vector and each preset threshold, operators are adaptively selected and processed from the preset enhancement strategy library to generate an enhancement processing sequence. The selection rules are as follows: when the illumination uniformity indicator is lower than the corresponding illumination uniformity threshold, illumination normalization processing based on homomorphic filtering is selected; when the overall contrast indicator is lower than the overall contrast threshold, contrast-limited adaptive histogram equalization processing is selected; when the noise level indicator is higher than the noise level threshold, filtering and denoising processing based on non-local means is selected. The operators in the processing sequence are topologically sorted according to their dependencies. The enhancement operators are then applied to the grayscale image G in sequence according to the sorting results, and the final output is a standardized document image. The dependencies are: denoising before illumination normalization, denoising before contrast enhancement, and illumination normalization before contrast enhancement.

4. The intelligent parsing method for scanned professional title application documents based on deep feature extraction according to claim 1, characterized in that, The specific implementation of step S2, which extracts the global layout feature map from the standardized document image and outputs the document type probability distribution vector, is as follows: Standardized document images are input into a backbone feature extraction network, which employs a multi-level residual convolutional structure. At different depth levels, the spatial resolution is progressively reduced while the channel dimension is increased, extracting multiple feature maps at different scales to form a multi-scale feature map set. A top-down feature pyramid fusion operation is then performed on this multi-scale feature map set. The feature map of the previous level is sequentially convolved laterally to unify the channel dimension, and then added element-wise with the higher-level fused feature map that has undergone bilinear upsampling. This process gradually fuses feature information from different scales, resulting in a fused feature pyramid set with unified channel dimensions across all layers. The feature maps of each layer in the fused feature pyramid set are flattened into one-dimensional feature sequences along the spatial dimension. The feature sequences of each layer are spliced ​​along the sequence length dimension to form a global layout feature sequence representing global layout information. Learnable hierarchical position codes and spatial position codes are added to each position element in the global layout feature sequence. The two sets of position codes are added to the original feature sequence element by element to obtain an input sequence with position awareness. The feature sequence with position encoding is input into a multi-layer stacked global multi-head self-attention encoder. In each layer of the encoder: the input sequence is transformed by three independent linear projection transformations to generate a query matrix, a key matrix, and a value matrix. The dot product similarity between the query and the key is calculated and scaled. The attention weight matrix is ​​obtained by normalization through the Softmax function. The attention weight is multiplied by the value matrix to realize global information interaction between any two positions in the sequence. Each layer encoder outputs its code after multi-head self-attention operation through residual connections and layer normalization. Then, it passes through a feedforward network, residual connections, and layer normalization to output the encoding result of that layer, which serves as the input for the next layer. The output of the final layer encoder is the global layout feature map. The global layout feature map output by the final encoder layer is taken, and a global average pooling operation is performed along the sequence length dimension to compress the sequence information into a global layout feature vector of fixed dimension. The global layout feature vector is input into a document type classification head composed of a fully connected layer and a Softmax activation function, and mapped to a probability distribution vector with a dimension equal to the total number of predefined document types. The category with the highest probability value is taken as the document type prediction result. The document type information is used for subsequent matching by selecting the corresponding template rule feature library.

5. The intelligent parsing method for scanned professional title application documents based on deep feature extraction according to claim 1, characterized in that, Step S3 involves extracting visual content features from the block image to obtain the text content of the block image, specifically including: For each cropped block image, a grayscale projection histogram is calculated along the horizontal direction. The upper and lower boundaries of each text line are determined based on the start and end positions of continuous non-zero pixel segments in the projection histogram, thus segmenting the block image into multiple text line images. For each text line image, a grayscale projection histogram is calculated along the vertical direction. The segmentation points between characters are identified based on the comparison between the gap between adjacent non-zero pixel segments and a preset character spacing threshold. Connected components of adjacent characters with a spatial distance less than a preset merging threshold are clustered to form a complete text line image sequence. Each text line image is scaled to a uniform height while maintaining its aspect ratio and then input into a convolutional neural network encoder. After multiple convolutions, batch normalization, and non-linear activation operations, a two-dimensional local visual feature map of the text line is extracted. Global average pooling and global max pooling are calculated along the channel dimension of the two-dimensional local visual feature map. The two pooling results are concatenated and then passed through a convolutional layer and a sigmoid activation function to generate a spatial attention weight matrix. The spatial attention weight matrix is ​​multiplied element-wise with the original feature map to obtain an enhanced feature map that suppresses background noise and highlights the text region. The enhanced feature map is extracted column by column along the width direction to form a sequence of visual feature vectors arranged in reading order. The sequence of visual feature vectors is input into a sequence text recognition decoder based on an attention mechanism. At each decoding time step, the decoder calculates attention weights on the sequence of visual feature vectors based on the current decoder hidden state and aggregates them to obtain a context vector. The context vector and the embedding vector of the output character in the previous step are input into the decoder unit to update the hidden state. The probability distribution of each candidate character in the current time step is predicted through a fully connected layer and a Softmax function. The character with the highest probability is taken as the recognition output of the current step. The process is repeated step by step until the output sequence terminates, thus obtaining the preliminary recognized text of the text line. For each line of text, a recognition confidence score is calculated, which is the geometric mean of the probabilities of the largest candidate characters at each decoding time step. The recognition confidence score is compared with a preset confidence threshold. If it is higher than the threshold, the preliminary recognition text is directly output. If it is lower than the threshold, the visual feature column interval corresponding to the low-confidence character is located. The residual between the feature vector in the interval and the local interval mean is calculated and locally enhanced through a feedforward network before being re-input into the decoder for refined recognition. The refined recognition result replaces the corresponding low-confidence segment. Finally, the recognition results of all lines of text are concatenated in spatial order to obtain the complete text content of the block image.

6. The intelligent parsing method for scanned professional title application documents based on deep feature extraction according to claim 1, characterized in that, Step S4, which extracts semantic feature vectors from the text content, is implemented as follows: The text content is input into the pre-trained model, and after word segmentation and tokenization, the hidden state vector obtained by average pooling the last layer of tokens is used as the semantic feature vector.

7. The intelligent parsing method for scanned professional title application documents based on deep feature extraction according to claim 1, characterized in that, In step S5, the semantic feature vector is matched with a preset template rule feature library; the matching score is calculated and compared with a preset compliance threshold, and the judgment result is output. The specific implementation is as follows: Select the corresponding template rule feature library based on the document type, calculate the cosine similarity between the semantic feature vector and the feature vector of each rule in the rule library, and obtain the matching score of each rule. The matching score of each rule is compared with its corresponding preset compliance threshold. If the matching score is greater than or equal to the compliance threshold, the rule is deemed qualified; otherwise, it is deemed unqualified. When all rule items are qualified, the overall judgment result of the output document is qualified, and the text content is output as the document parsing result; when at least one rule is determined to be unqualified, the overall judgment result of the output document is unqualified, and the non-compliant rule information is output.

Images