Financial document image quality grading method based on retinal spatio-temporal antagonism and predictive feedback

Abstract

The invention provides a financial document image quality grading method based on retina spatio-temporal antagonism and predictive feedback, and aims to solve the problems of reduction of subsequent OCR (optical character recognition) rate and difficulty in information extraction caused by financial document image quality defects. The combined sensitivity of retinal ganglion cells to time change and space contrast is simulated, and primary enhancement of low-contrast defects and edge blur is achieved; by establishing an explicit dependency relationship between the scale and the direction, the scale-direction selectivity characteristic of simple cells of the primary visual cortex is simulated, and refined direction coding is realized; a space prediction mask is generated through a dorsal path, a semantic prediction error is calculated through a ventral path, and dynamic alignment of defect space positioning and quality semantic recognition is realized by adopting an iterative optimization strategy. Finally, according to the method, the quality conformity judgment of the document image can be given, and the existing defect type can be prompted.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of machine learning and computer vision, and particularly relates to a financial document image quality grading method based on retinal spatiotemporal antagonism and predictive feedback. BACKGROUND

[0002] In the financial industry, the electronicization and automatic processing of paper documents are core links. However, financial document images are easily affected by various factors during acquisition (such as mobile phone shooting, scanner), transmission and storage, resulting in a decline in image quality. Common defects include blur, stains, glare, wrinkles, watermark interference, incomplete cropping, low contrast, noise, etc. These quality defects seriously affect the subsequent OCR recognition rate, the accuracy of automatic information extraction, and the efficiency and experience of manual review. The existing deep learning methods mainly have the following problems in financial document image quality detection: first, the feature extraction lacks hierarchy, directly using the ImageNet pre-training network does not design for document defect features (such as fine wrinkles and weak glare), resulting in insufficient response of weak defect features; second, the multi-task learning coupling is insufficient, quality judgment, defect classification and defect positioning are often trained independently or simply concatenated, lacking information complementation and collaborative optimization between tasks; third, the defect positioning ability is weak, and the defect region segmentation accuracy significantly decreases in the scene where the defect boundary is fuzzy and the document background is complex (such as seal and underprint); fourth, there is a lack of biological explainability, and the existing methods are mostly black box models, which are difficult to explain their working mechanism from the perspective of neuroscience.

[0003] The primate visual system has formed an efficient hierarchical information processing mechanism through millions of years of evolution. The retinal ganglion cells are not only sensitive to spatial contrast but also highly sensitive to temporal changes. The lateral geniculate body forms an opponent color channel. The simple cells of the primary visual cortex have selective response to specific scale-direction combinations. The high-level visual cortex is differentiated into dorsal "Where" pathway and ventral "What" pathway, and the two pathways are collaboratively modulated through a predictive feedback mechanism. This mechanism provides an important inspiration for constructing an efficient financial document image quality detection system. However, the traditional center-periphery contrast enhancement method (such as DOG filter) can detect edges, but only performs spatial domain processing at a single scale without introducing the "pseudo-temporal" semantics of multi-scale difference, so it lacks sensitivity to weak contrast defects. Using independent filters to process scale and direction does not model their coupling relationship. The dual-pathway architecture only performs simple feature concatenation or addition without introducing a predictive feedback mechanism. SUMMARY

[0004] The purpose of the present application is to provide a financial document image quality grading method based on retinal spatiotemporal antagonism and predictive feedback, which realizes the collaborative optimization of image quality judgment, defect classification and defect area positioning by systematically simulating the information encoding and transmission mechanism of visual neural pathway.

[0005] Step 1: Obtain the RGB image of the financial document, use the two-dimensional Gaussian kernel function with different standard deviations for three times of smoothing processing, generate fine scale, medium scale and coarse scale smooth images, which correspond to three different time points respectively, so as to construct a pseudo-time sequence dimension in a single static image;

[0006] Calculate the pixel-by-pixel difference between the fine scale smooth image and the medium scale smooth image as the fast temporal change response, which is used to capture the small edges or fuzzy areas of the document;

[0007] Calculate the pixel-by-pixel difference between the medium scale smooth image and the coarse scale smooth image as the slow temporal change response, which is used to capture the large area low contrast area of the document;

[0008] By calculating the deviation of each pixel from the mean and standard deviation of the brightness of the pixels in its local window, the normalized spatial contrast response is obtained;

[0009] Define the ON center cell response as the sum of the positive part of the fast temporal change response and the positive part of the spatial contrast response;

[0010] Define the OFF center cell response as the sum of the positive part of the negative value of the slow temporal change response and the positive part of the negative value of the spatial contrast response;

[0011] Subtract the OFF center cell response from the ON center cell response to obtain the retinal boundary response, realizing the spatiotemporal antagonistic information fusion;

[0012] Step 2: Define three scale levels according to the size characteristics of the defects in the document image, assign different direction weights to different scale layers, and obtain the weighted response of all scale-direction combinations based on the retinal boundary response; select the maximum weighted response value of each pixel position in the picture as the final response;

[0013] Step 3: Perform dorsal and ventral processing on the final response to obtain dorsal spatial prediction mask and ventral semantic mask, and reduce the spatial prediction error between the dorsal spatial prediction mask and the ventral semantic mask through iterative optimization, realizing the dynamic alignment of defect spatial positioning and quality semantic recognition;

[0014] Step 4: Based on the back-side spatial prediction mask and the ventral semantic mask, perform image quality grading, defect type classification, and defect region segmentation.

[0015] Preferably, in step 1, the calculation of the normalized spatial contrast response adopts a Gaussian weighted method, specifically: within a local window centered on each pixel, a two-dimensional Gaussian function is used to assign different weights to pixels at different positions within the window, with the center pixel having the highest weight and the edge pixels having progressively lower weights; the Gaussian weighted mean brightness and the Gaussian weighted standard deviation of brightness within the local window are calculated; each pixel value is subtracted from its corresponding Gaussian weighted mean brightness, and then divided by the sum of the Gaussian weighted standard deviation of brightness and a very small positive number to obtain the normalized spatial contrast; the standard deviation of the Gaussian function is set to 1 / 6 of the window size to ensure that the weight of the window edges is close to zero.

[0016] Preferably, step 2 specifically includes: defining three scale levels based on the size characteristics of defects in the document image: a coarse scale level corresponding to the following defects: blur, incomplete cropping, and low contrast; a fine scale level corresponding to the following defects: noise; and a mid-scale level based on the geometric mean of the coarse and fine scales, corresponding to the following transitional defects: stains, reflections, wrinkles, and watermark interference; for each scale level, constructing a selective filter kernel in six directions, with a 30° difference between adjacent directions; and assigning differentiated directional weights to different scale levels: for the coarse scale level, the horizontal direction (0° or 180°) and the vertical direction (90°) have higher weights than the diagonal directions (30°, 60°, 120°, and 150°); for the fine scale level, the diagonal direction has higher weights than the watermark. The scale is divided into two layers: a flat vertical layer and a medium-scale layer. The weights in each direction are equal. The weight ratios are determined based on the statistical characteristics of the training data. Based on the retinal boundary response, all scale-direction combinations are weighted to obtain a weighted response for each combination. A competitive selection mechanism is used to select the largest weighted response value at each pixel location in the image as the final response, and the corresponding scale and direction labels are recorded for fine-grained encoding of document defect direction features. Specifically, the scale layers are defined as follows: the coarse-scale layer corresponds to a Gabor filter wavelength of 16 pixels; the fine-scale layer corresponds to a Gabor filter wavelength of 4 pixels; and the medium-scale layer corresponds to a Gabor filter wavelength that is approximately the geometric mean of the coarse and fine-scale wavelengths, about 8 pixels.

[0017] Preferably, in step 2, the selective filter kernel is constructed using a Gabor filter, specifically including: constructing a Gabor filter kernel for each scale layer and each directional angle, wherein the Gabor filter kernel is composed of a Gaussian envelope and cosine wave modulation; assigning differentiated weights to different directions of different scale layers according to scale-direction weights; and convolving the retinal boundary response with the weighted Gabor filter kernel to obtain the weighted response of the scale-direction combination.

[0018] Preferably, in step 2, the calculation of the final response specifically includes:

[0019] For each pixel location, the maximum value among the weighted responses of 18 scale-direction combinations consisting of 3 scales and 6 directions is selected as the final response value for that location.

[0020] Preferably, step 3 specifically includes: performing multi-scale upsampling processing on the final response obtained in step 2 sequentially, generating backside feature maps of three scale layers through three transposed convolution operations with a stride of 2, halving the number of channels after each transposed convolution; generating a prediction mask for each scale layer's backside feature map through a 3×3 convolution and a Sigmoid activation function; upsampling all layer prediction masks to the original image size through bilinear interpolation; and performing a weighted summation of the upsampled prediction masks, with higher weights for deeper features, to obtain the final backside prediction mask, used to predict possible defect locations in the document image; the generation process of the ventral semantic mask: performing multi-scale upsampling processing on the final response obtained in step 2 sequentially... Feature transformation is performed through three convolutional blocks, each containing a convolutional layer, a batch normalization layer, and an activation function. The ventral semantic feature map is obtained after passing through these three blocks. The ventral semantic feature map is then converted from multi-channel to single-channel using a 1×1 convolution, and upsampled to the original image size via bilinear interpolation. Finally, it is activated by a Sigmoid function to obtain the ventral semantic mask, which is used to identify defect types in the image, including blur, stains, reflections, wrinkles, watermark interference, incomplete cropping, and low contrast. The spatial prediction error is calculated by comparing the backside spatial prediction mask with the ventral semantic mask. Based on this spatial prediction error, iterative optimization is performed to improve the accuracy of defect region identification in the image.

[0021] Preferably, in step 3, the specific process of the iterative optimization includes: calculating the pixel-by-pixel difference between the back-side spatial prediction mask and the front-side semantic mask to obtain the spatial prediction error;

[0022] Updating the mask: Multiply the spatial prediction error element-wise with the ventral semantic feature map to obtain the error-weighted features;

[0023] The error-weighted features are multiplied by the ventral learning rate and then added to the ventral semantic feature map to obtain the updated ventral semantic feature map. The updated ventral semantic feature map is then mapped to a single-channel mask through a 1×1 convolution. The single-channel mask is multiplied by the backside learning rate and then added to the backside spatial prediction mask to obtain the updated backside spatial prediction mask.

[0024] Update the backside feature map: Downsample the spatial prediction error to the size of the backside feature map; multiply the downsampled spatial prediction error element-wise with the backside feature map to obtain the error-weighted feature; multiply the error-weighted feature by the ventral learning rate and add it to the backside feature map; process the sum by nonlinear transformations of convolution, batch normalization, and ReLU activation function to obtain the updated backside feature map;

[0025] Recalculate the prediction error between the updated backside spatial prediction mask and the frontside semantic mask; repeat the above process several times, and terminate the iteration early when the prediction error is lower than a preset threshold.

[0026] Preferably, in step 4, for the image quality grading, global average pooling is performed on the backside spatial prediction mask and the ventral semantic mask to obtain two global feature vectors. The two vectors are concatenated along the feature dimension, and the probability distributions of the three quality levels are obtained through a two-layer fully connected network and a softmax activation function. For the defect type classification, the two vectors are concatenated along the feature dimension, and the probabilities of eight defect categories are output through an independent two-layer fully connected network and a sigmoid activation function. A weighted binary cross-entropy loss is used to address the differences between category samples. For the defect region segmentation, a foreground mask and a background mask are generated through a foreground and background dual-branch segmentation architecture. The foreground mask is subtracted from the background mask to obtain a residual difference map, which is then added to the refined fusion map to obtain the final segmentation mask.

[0027] Preferably, the specific implementation process of the dual-branch segmentation architecture includes: the foreground branch and the background branch respectively take the backside feature map optimized by predictive feedback iteration in step 3 as input; both the foreground branch and the background branch gradually restore the resolution to the original image size through three bilinear interpolation and 3×3 convolution operations, the foreground branch and the background branch have the same structure and their parameters are trained independently to generate foreground masks and background masks respectively; the foreground mask is subtracted from the background mask to obtain a residual difference map; the foreground mask and the background mask are concatenated in the channel dimension and then finely fused through a 3×3 convolutional layer to obtain a fused feature map; the residual difference map is added to the fused feature map to obtain the final defect region segmentation mask.

[0028] Preferably, step 4 further includes: for quality grading, using cross-entropy loss as the quality grading loss function; for defect region segmentation, using the sum of binary cross-entropy loss and Dice loss as the defect region segmentation loss function; constructing a total loss function by weighting the quality grading loss function, the weighted binary cross-entropy loss for defect type classification, and the defect region segmentation loss function according to a preset weight ratio; minimizing the total loss function through the backpropagation algorithm and optimizer to achieve end-to-end training, ensuring mutual promotion among tasks, and achieving information complementarity and collaborative decision-making.

[0029] Beneficial effects of the present invention

[0030] 1. A pseudo-temporal dimension and spatiotemporal antagonistic coding model was constructed to capture weak defects in documents. Existing bio-inspired methods generally neglect the temporal dimension. This invention constructs a pseudo-temporal dimension in a single static image, simulates the temporal change response through multi-scale smoothing and inter-scale differencing, and combines spatial contrast with ON / OFF antagonistic fusion. This model introduces temporal change sensitivity into financial document image quality detection, significantly enhancing the robustness of the initial representation of weak defects such as low-contrast text, slight blurring, or minor stains, and providing richer boundary information for subsequent processing.

[0031] 2. A scale-direction encoding method based on differentiated directional weight coupling is proposed to accurately identify diverse defects in financial documents. Given the limitations of traditional filters that process scale and direction independently, this invention explicitly models the dependency between scale and direction and designs differentiated directional weight allocation strategies for different scale layers based on the selective characteristics of the biological V1 cortex. This mechanism effectively avoids the problem of chaotic response in traditional methods when dealing with defects of multiple sizes (such as large-area reflections, fine scratches, and wrinkles) in financial documents, achieving refined and highly adaptable encoding of directional features of defects of different sizes.

[0032] 3. A prediction error-driven iterative optimization dual-path collaborative model was constructed to achieve accurate detection and semantic understanding of financial document defects. Addressing the difficulty in correcting prediction biases caused by the simple fusion of existing dual-path architectures, this invention introduces a prediction error-driven iterative optimization mechanism. By calculating the prediction error between back-side spatial prediction and ventral-side semantic perception, and performing multiple rounds of error-driven updates and feedback adjustments, dynamic adaptive alignment of spatial localization of financial document defects and quality semantic recognition is achieved. This iterative reasoning mechanism significantly improves the accuracy of document defect region identification in complex backgrounds (such as seals and patterns).

[0033] 4. A dual-branch defect region segmentation strategy based on unique ON / OFF antagonistic coding is proposed to improve the accuracy of defect boundary localization in financial documents. Addressing the challenge of balancing boundary contrast and detail preservation in traditional segmentation methods, this invention draws on the antagonistic coding mechanism of retinal ON / OFF bipolar cells to construct parameter-independent foreground and background dual branches. A unique residual differential fusion enhances the contrast of defect region boundaries, supplemented by refined fusion learning of optimal weights. The final defect region segmentation mask significantly improves the accuracy of locating blurred defect boundaries (such as stain edges and weakly reflective areas) in financial documents while maintaining contrast.

[0034] 5. An end-to-end neurobiological mapping framework from the retina to the higher visual cortex has been implemented, providing an interpretable solution for intelligent processing of financial documents. This invention not only draws on bioinspiration at the individual module level, but also systematically simulates the complete hierarchical processing mechanism of the primate visual cortex, from the spatiotemporal antagonistic encoding at the bottom layer, to the scale-direction coupled selective encoding in the middle layer, and then to the dorsal-ventral predictive feedback collaboration at the high layer. This comprehensive and biologically interpretable architecture provides an efficient and reliable bioinspired solution for automated quality grading and defect indication of financial document images, improving the transparency and theoretical value of the model, and contributing to the rationality and efficiency of financial business. Attached Figure Description

[0035] To more clearly illustrate the specific implementation results of the present invention, the accompanying drawings used in the specific embodiments will be briefly introduced below. The specific embodiments of the present invention will be described later in an exemplary and non-limiting manner with reference to the accompanying drawings. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0036] Figure 1 Comparison of evaluation indicators for quality grading tasks.

[0037] Figure 2 Comparison of evaluation indicators for defect classification tasks.

[0038] Figure 3 Comparison of evaluation indicators for segmentation and positioning tasks.

[0039] Figure 4 Comparative ablation experiments of the core modules. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] This embodiment provides a method for quality grading of financial document images based on retinal spatiotemporal antagonism and predictive feedback. This method systematically simulates the information encoding and transmission mechanism of primate visual neural pathways. Starting from the spatiotemporal antagonistic characteristics of retinal ganglion cells, information is relayed through the lateral geniculate body to reach scale-direction selective encoding in the primary visual cortex. Finally, through the predictive feedback synergy of the dorsal spatial localization pathway and the ventral semantic recognition pathway, the method achieves coordinated optimization of quality judgment, defect classification, and precise defect region localization in financial document images.

[0042] Step 1: Construct a document image quality detection dataset for financial business scenarios.

[0043] This dataset covers various financial document types, including ID cards, passports, driver's licenses, business licenses, bank cards, and various bills and vouchers, and contains more than 1,400 document images from real-world scenarios.

[0044] Based on actual business needs, document image quality is divided into three levels: excellent, acceptable, and unacceptable.

[0045] A high-quality level indicates that the image is clear and complete, with uniform lighting and no obvious defects, and can be directly used for subsequent recognition and processing.

[0046] A passable level indicates that the image is generally usable but has minor defects and can be used for recognition after simple processing;

[0047] A failing grade indicates that the image has serious quality problems and cannot meet the recognition requirements, requiring re-acquisition;

[0048] Of the collected images, 45% were high-quality, 30% were acceptable, and 25% were unacceptable.

[0049] Eight common defect categories were defined for image defect types: blur, stains, reflections, wrinkles, watermark interference, incomplete cropping, low contrast, and noise defects.

[0050] Each image may contain multiple defect types simultaneously, so a multi-label annotation method is adopted, and the annotation content includes three levels: quality level annotation, defect type annotation, and defect area annotation;

[0051] The dataset is divided into training, validation, and test sets in a ratio of 7:1.5:1.5.

[0052] Step 2: Image preprocessing stage.

[0053] Scale all input financial document RGB images to 512. 512 pixels, bilinear interpolation is used during scaling to maintain image smoothness;

[0054] The pixel values ​​are normalized from an integer range of 0-255 to a floating-point range of 0-1, and then standardized using statistics from the ImageNet dataset, i.e., the mean is subtracted and the standard deviation is divided.

[0055] During training, various data augmentation methods are employed, including geometric transformation augmentation such as random rotation, random cropping, and perspective transformation; lighting transformation such as brightness adjustment, contrast adjustment, saturation adjustment, and hue adjustment; and noise augmentation such as Gaussian noise, salt and pepper noise, and blurring.

[0056] During training, two to four enhancement methods are randomly selected and superimposed.

[0057] All enhancement operations are performed while ensuring image realism, avoiding over-enhancement that could lead to image distortion;

[0058] Geometric transformations and clipping operations can affect the position and shape of defective areas, so the segmentation mask of the annotations needs to be transformed simultaneously.

[0059] Step 3: Retinal spatiotemporal antagonistic coding stage.

[0060] Convert the preprocessed RGB image of financial documents to a grayscale image;

[0061] The grayscale image is smoothed three times using a two-dimensional Gaussian kernel function with different standard deviations to generate three smoothed images at different scales.

[0062] The first smoothing uses a Gaussian kernel with a standard deviation of 0.5 to generate a fine-scale smoothed image, which can preserve the image's detailed information.

[0063] The second smoothing process uses a Gaussian kernel with a standard deviation of 1.5 to generate a medium-scale smoothed image, which moderately blurs the image.

[0064] The third smoothing process uses a Gaussian kernel with a standard deviation of 3.0 to generate a coarse-scale smoothed image, which provides a stronger smoothing effect on the image.

[0065] Three smoothed images at different scales correspond to three different pseudo-time points, constructing a pseudo-temporal dimension in a single static image;

[0066] Calculate the pixel-by-pixel difference between the fine-scale smoothed image and the medium-scale smoothed image to obtain the fast time-varying response;

[0067] Calculate the pixel-by-pixel difference between the medium-scale smoothed image and the coarse-scale smoothed image to obtain the slow-time-varying response;

[0068] Within a local window centered on each pixel, the spatial contrast of that pixel is calculated using a Gaussian weighted method.

[0069] A two-dimensional Gaussian function is used to assign different weights to pixels at different positions within the window, with the center pixel having the highest weight and the edge pixels having gradually decreasing weights.

[0070] The standard deviation of the Gaussian function is set to one-sixth of the window size to ensure that the weights at the window edges are close to zero.

[0071] Calculate the Gaussian weighted mean brightness and Gaussian weighted standard deviation brightness within the local window. Then, subtract the corresponding Gaussian weighted mean brightness from each pixel value and divide by the sum of the Gaussian weighted standard deviation brightness and a very small positive number to obtain the normalized spatial contrast.

[0072] Based on the fast time-varying response, slow time-varying response, and normalized spatial contrast response, the ON and OFF central cell responses of retinal ganglion cells are defined.

[0073] The ON central cell response is defined as the sum of the positive part of the rapid time-varying response and the positive part of the spatial contrast response.

[0074] The OFF central cell response is defined as the sum of the positive part of the negative value of the slow time-varying response and the positive part of the negative value of the spatial contrast response;

[0075] By subtracting the OFF central cell response from the ON central cell response, the retinal boundary response is obtained, thus achieving spatiotemporal antagonistic information fusion.

[0076] Step 4: Scale-direction selective coding stage of the primary visual cortex.

[0077] Based on the size characteristics of defects in document images, three scale levels are defined to accommodate the detection needs of defects of different sizes;

[0078] The coarse-scale layer corresponds to a Gabor filter wavelength of 16 pixels, the fine-scale layer corresponds to a Gabor filter wavelength of 4 pixels, and the medium-scale layer corresponds to a Gabor filter wavelength that is the geometric mean of the coarse and fine-scale wavelengths, approximately 8 pixels.

[0079] For each scale layer, six selective filter kernels are constructed, with adjacent directions differing by 30°, corresponding to the directions of 0°, 30°, 60°, 90°, 120° and 150° respectively;

[0080] Using Gabor filters as selective filter kernels, each Gabor filter kernel consists of a Gaussian envelope and cosine modulation, local analysis of images is performed in the spatial and frequency domains, and selective responses are given to edges or textures at specific scales and directions.

[0081] Assign differentiated directional weights to layers of different scales;

[0082] For coarse-scale layers, the weights of the horizontal and vertical directions are higher than those of the diagonal direction; for fine-scale layers, the weights of the diagonal direction are higher than those of the horizontal and vertical directions; and for meso-scale layers, the weights of all directions are equal.

[0083] The weight ratio is determined based on the statistical characteristics of the training data. By analyzing the distribution frequency of different defect types in each direction in the training set, the directional weights of each scale layer are adaptively adjusted so that the model can better adapt to the feature distribution of defects in actual financial documents.

[0084] Differentiated weights are assigned to different directions of different scale layers based on scale-direction weights. Then, the retinal boundary response is convolved with the weighted Gabor filter kernel to obtain the weighted response of the scale-direction combination.

[0085] Through a competitive selection mechanism, the maximum value among the weighted responses of all eighteen scale-orientation combinations is selected as the final response value for each pixel location, and the corresponding scale label and orientation label are recorded.

[0086] The final response is input into a residual convolutional neural network (ResNet-18), which contains four residual blocks. Deep features are extracted step by step through multiple convolutions, batch normalization, and ReLU activation function. The first residual block outputs a 128×128×64 feature map, the second residual block outputs a 64×64×128 feature map, the third residual block outputs a 32×32×256 feature map, and the fourth residual block outputs a 16×16×512 feature map. The outputs of the third and fourth residual blocks are used as inputs to the dorsal spatial localization pathway and the ventral semantic recognition pathway, respectively.

[0087] Step 5: Parallel processing stage of dorsal spatial localization pathway and ventral semantic recognition pathway.

[0088] Multi-scale upsampling and spatial prediction are performed on the backside feature map output by the third residual block in step four.

[0089] The feature maps of three scale layers, 64×64, 128×128 and 256×256, are generated by three transposed convolution operations with a stride of 2. The number of channels is halved after each transposed convolution.

[0090] On the original 32×32 scale layer and the three upsampled scale layers, 3×3 convolution and sigmoid activation function are applied respectively to generate the prediction mask for each layer;

[0091] The predicted masks of all layers are uniformly upsampled to the original image size of 512×512 through bilinear interpolation, and then weighted and summed according to the following weights: deep layer weight 0.4 (32×32 layers), middle layer weight 0.3 (64×64 layers), shallow layer weight 0.2 (128×128 layers) and 0.1 (256×256 layers) to obtain the final predicted mask of the back side.

[0092] The ventral feature map output from the fourth residual block in step four is sequentially processed through three convolutional blocks for semantic feature extraction. Each convolutional block contains a 3×3 convolution, batch normalization, and ReLU activation function.

[0093] The first convolutional block reduces the number of channels from 512 to 256 while keeping the spatial size unchanged at 16×16.

[0094] The second and third convolutional blocks also maintain a spatial size of 16×16, with the number of channels remaining at 256;

[0095] After three convolutional blocks, the ventral semantic feature map is obtained. It is then mapped to a single channel through a 1×1 convolution, upsampled to 512×512 through bilinear interpolation, and finally obtained as a ventral semantic mask through a Sigmoid activation function.

[0096] Step Six: Introduce a predictive feedback mechanism.

[0097] Calculate the pixel-by-pixel difference between the back-side spatial prediction mask and the front-side semantic mask in step five, and use the mean square error or absolute error as a metric to obtain the spatial prediction error.

[0098] The spatial prediction error is multiplied element-wise with the ventral semantic feature map to obtain the error-weighted feature;

[0099] The error-weighted features are multiplied by the ventral learning rate and then added to the ventral semantic feature map to obtain the updated ventral semantic feature map.

[0100] The ventral and dorsal learning rates are initialized to 0.1 as learnable parameters and their values ​​are automatically optimized through backpropagation during training.

[0101] The updated ventral semantic feature map is then processed through 1 1. Convolutional mapping is used to create a single-channel mask, generating an updated ventral semantic mask;

[0102] Multiply the single-channel mask by the back-side learning rate and add it to the back-side spatial prediction mask to obtain the updated back-side spatial prediction mask;

[0103] The backside learning rate also controls the adjustment of the backside prediction by the ventral feedback information, and can be the same as or slightly different from the ventral learning rate.

[0104] The prediction error between the updated backside spatial prediction mask and the updated ventral semantic mask is recalculated to form a new round of error feedback.

[0105] Repeat the above process several times, setting it to three to five iterations. Monitor the change in prediction error in each iteration, and terminate the iteration early when the prediction error is lower than a preset threshold.

[0106] In each iteration, the backside feature map needs to be updated to achieve adaptive adjustment of deep features;

[0107] The spatial prediction error downsampled to 32×32 is multiplied element-wise with the backside feature map to obtain the error-weighted feature;

[0108] The error-weighted features are multiplied by the ventral learning rate and then added to the backside feature map. The updated backside feature map is then processed by a non-linear transformation module that includes 3×3 convolution, batch normalization, and ReLU activation function.

[0109] After iteration, the final backside spatial prediction mask, ventral semantic mask, and updated backside feature map are output.

[0110] Step 7: Multi-task output stage.

[0111] For the image quality classification task, global average pooling is performed on the back side spatial prediction mask and the ventral semantic mask respectively to aggregate the spatial dimension information into a global feature vector.

[0112] The back-side spatial prediction mask is obtained as a scalar value after global average pooling, and the ventral semantic mask is obtained as another scalar value after global average pooling.

[0113] Concatenate these two scalars along the feature dimension to form a two-dimensional feature vector;

[0114] Nonlinear transformation and feature fusion are performed through two fully connected networks. The first fully connected network maps two-dimensional features to sixteen-dimensional hidden features, and the second fully connected network maps sixteen-dimensional hidden features to three-dimensional output, corresponding to three quality levels: excellent, qualified, and unqualified.

[0115] The probability distributions of the three quality levels are obtained by using the softmax activation function, and the category with the highest probability is selected as the final quality classification result.

[0116] For the defect type classification task, the global feature vectors of the dorsal and ventral sides are concatenated along the feature dimension, and then the feature transformation is performed through two independent fully connected networks.

[0117] The first fully connected layer maps two-dimensional features to thirty-two hidden features, and the second fully connected layer maps the thirty-two hidden features to an eight-dimensional output, corresponding to eight defect categories: blur, stains, reflection, wrinkles, watermark interference, incomplete cropping, low contrast, and noise.

[0118] The Sigmoid activation function is used to output the probabilities of eight defect categories, and the existence of each defect category is determined independently.

[0119] We employ a weighted binary cross-entropy loss method to assign higher weights to categories with fewer samples.

[0120] For the defect region segmentation task, a foreground and background dual-branch segmentation architecture is adopted;

[0121] The foreground and background branches take the backside feature maps optimized through predictive feedback iteration in step six as inputs, respectively.

[0122] The two branches have the same network structure but their parameters are trained independently. Both are gradually restored to the original image size through cubic bilinear interpolation and 3×3 convolution operations.

[0123] The first upsampling restores the feature map size from 32×32 to 64×64, and halves the number of channels;

[0124] The second upsampling restores the size to 128×128, and the number of channels is halved again;

[0125] The third upsampling restores the size to the original image size of 512×512, and reduces the number of channels to 1;

[0126] The foreground branch ultimately generates the foreground mask, and the background branch ultimately generates the background mask;

[0127] Subtracting the background mask from the foreground mask yields a residual difference map, which enhances the boundary contrast between the defect region and the background region through an antagonistic mechanism.

[0128] The foreground and background masks are concatenated along the channel dimension and then refined through a 3x3 convolutional layer to obtain a fused feature map.

[0129] The residual difference map is added to the fused feature map to obtain the final defect region segmentation mask.

[0130] Step 8: Construct a multi-task loss function to achieve end-to-end joint optimization of the three tasks.

[0131] For quality grading tasks, cross-entropy loss is used as the quality grading loss function, which measures the difference between the predicted quality grade probability distribution and the true label.

[0132] For defect type classification tasks, weighted binary cross-entropy loss is used to assign different weights to different categories;

[0133] For the defect region segmentation task, the sum of the binary cross-entropy loss and the Dice loss is used as the defect region segmentation loss function;

[0134] The total loss function is constructed by weighting the quality grading loss function, the weighted binary cross-entropy loss for defect type classification, and the defect region segmentation loss function according to a preset weight ratio.

[0135] In this embodiment, the weight of the quality grading loss is set to 0.2, the weight of the defect type classification loss is set to 0.3, and the weight of the defect region segmentation loss is set to 0.5.

[0136] The gradient of the total loss function with respect to the network parameters is calculated by backpropagation algorithm, and the total loss function is minimized by optimizer such as Adam optimizer or stochastic gradient descent optimizer to achieve end-to-end training.

[0137] During training, the three tasks share the underlying retinal spatiotemporal antagonistic coding module and the primary visual cortex scale-direction selective coding module, and exchange information through dorsal and ventral predictive feedback mechanisms.

[0138] Step 9: Optimize network parameters.

[0139] The weights of convolutional layers are initialized using the He initialization method, which adaptively initializes the weights based on the number of neurons in the previous layer; the weights of fully connected layers are initialized using the Xavier initialization method.

[0140] The shared feature extractor part is initialized using weights pre-trained on the ImageNet dataset, and transfer learning is used to accelerate model convergence.

[0141] The optimizer chosen is the Adam adaptive learning rate optimizer, with an initial learning rate of 0.001, momentum parameter of 0.9, second-moment estimation parameter of 0.999, and numerical stability term set to... ;

[0142] The learning rate scheduling strategy uses cosine annealing. Each cycle contains ten training rounds. The learning rate gradually decays from the initial value to the minimum learning rate, which is set to one percent of the initial learning rate, and then increases again in the new cycle.

[0143] The batch size is set to sixteen, and the training is performed in a total of five hundred rounds, with each round iterating through the entire training set once.

[0144] After each round, the model performance is evaluated on the validation set, including quality grading accuracy, defect classification F1 score, and segmentation IoU metric. The optimal model parameters are selected based on the validation set performance.

[0145] Multiple regularization techniques are used during training to prevent overfitting, and the weight decay coefficient is set to... During the optimization process, L2 regularization constraints are applied to the weights, and Dropout randomly drops some neurons in the fully connected layer with a dropout probability of 0.5.

[0146] A class-weighted strategy is adopted, which assigns higher loss weights to classes with fewer samples, and a class-balanced sampling strategy is used during training.

[0147] Step 10: Evaluate model performance.

[0148] For quality grading tasks, the main evaluation metrics are classification accuracy, precision, recall, and F1 score. Training results are as follows: Figure 1 ;

[0149] For defect classification tasks, the main evaluation metrics are sample-level accuracy and label-level accuracy. The training results are as follows: Figure 2 ;

[0150] For segmentation and localization tasks, the main evaluation metrics are Intersection over Union (IoU), Dice coefficient, and pixel accuracy. Training results are as follows: Figure 3 ;

[0151] To verify the effectiveness of each core module in this example, ablation experiments were conducted. The performance of the complete model was as follows: quality classification accuracy 94.5%, defect classification F1 score 91.2%, segmentation IoU 87.3%, and average performance 91.0%. The experimental results are as follows: Figure 4 As shown in the table below, the ablation experiment results are as follows:

[0152]

[0153] The table above shows that the organic integration of the four modules of this invention results in a synergistic enhancement of system performance rather than a simple summation.

[0154] The above description is only a part of the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for image quality grading of financial documents based on retinal spatiotemporal antagonism and predictive feedback, characterized in that, Includes the following steps: Step 1: Obtain the RGB image of the financial document, and perform three smoothing processes using a two-dimensional Gaussian kernel function with different standard deviations to generate smoothed images at fine, medium, and coarse scales, corresponding to three different time points, thereby constructing a pseudo-temporal dimension in a single static image; Calculate the pixel-wise difference between a fine-scale smoothed image and a medium-scale smoothed image as a fast temporal change response to capture fine edges or blurred areas in a document; The pixel-wise difference between a medium-scale smoothed image and a coarse-scale smoothed image is calculated as a slow-temporal change response to capture large low-contrast areas in a document. The normalized spatial contrast response is obtained by calculating the degree of deviation of each pixel from the mean and standard deviation of the pixel brightness within its local window. The ON central cell response is defined as the sum of the positive parts of the rapid time-varying response and the positive parts of the spatial contrast response. The OFF central cell response is defined as the sum of the positive part of the negative value of the slow time-varying response and the positive part of the negative value of the spatial contrast response; The retinal boundary response is obtained by subtracting the OFF central cell response from the ON central cell response, thus achieving spatiotemporal antagonistic information fusion. Step 2: Define three scale levels based on the size characteristics of defects in the document image, assign differentiated directional weights to different scale levels, and obtain the weighted response of all scale-directional combinations based on the retinal boundary response; select the largest weighted response value at each pixel location in the image as the final response; Step 3: Perform back-side and ventral-side processing on the final response to obtain the back-side spatial prediction mask and the ventral-side semantic mask. Reduce the spatial prediction error between the back-side spatial prediction mask and the ventral-side semantic mask through predictive feedback iterative optimization to achieve dynamic alignment between defect spatial localization and quality semantic recognition. Step 4: Based on the back-side spatial prediction mask and the ventral semantic mask, perform image quality grading, defect type classification, and defect region segmentation.

2. The image quality grading method for financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 1, characterized in that, In step 1, the calculation of the normalized spatial contrast response adopts a Gaussian weighted method, specifically as follows: Within a local window centered on each pixel, a two-dimensional Gaussian function is used to assign different weights to pixels at different positions within the window, with the center pixel having the highest weight and the edge pixels having gradually decreasing weights. Calculate the Gaussian-weighted mean and standard deviation of brightness within the local window; Subtract the corresponding Gaussian weighted mean brightness value from each pixel value, and then divide by the sum of the Gaussian weighted standard deviation brightness value and a very small positive number to obtain the normalized spatial contrast. The standard deviation of the Gaussian function is set to 1 / 6 of the window size to ensure that the weights at the window edges are close to zero.

3. The image quality grading method for financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 1, characterized in that, Step 2 specifically includes: Based on the size characteristics of defects in the document image, three scale levels are defined: the coarse scale level corresponds to the following defects: blur, incomplete cropping, and low contrast; the fine scale level corresponds to the following defects: noise; and the meso scale level, based on the geometric mean of the coarse and fine scales, corresponds to the following defects: stains, reflections, wrinkles, and watermark interference. For each scale layer, a selective filter kernel with six directions is constructed, with a 30° difference between adjacent directions; Differentiated directional weights are assigned to layers of different scales: for coarse-scale layers, the horizontal direction (0° or 180°) and the vertical direction (90°) have higher weights than the diagonal direction (30°, 60°, 120°, and 150°). For fine-scale layers, the weights in the diagonal direction are higher than those in the horizontal and vertical directions; for mesoscale layers, the weights in each direction are equal; the proportions of these weights are determined based on the statistical characteristics of the training data. Based on the retinal boundary response, all scale-direction combinations are weighted to obtain the weighted response of all scale-direction combinations; The largest weighted response value at each pixel location in the image is selected as the final response through a competitive selection mechanism, and the corresponding scale and orientation labels are recorded for fine encoding of document defect orientation features. The specific definition of the scale hierarchy is as follows: The coarse-scale layer corresponds to a Gabor filter wavelength of 16 pixels. The fine-scale layer corresponds to a Gabor filter wavelength of 4 pixels; The mesoscale layer corresponds to a Gabor filter wavelength that is approximately 8 pixels, which is the geometric mean of the coarse and fine scale wavelengths.

4. The image quality grading method for financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 1, characterized in that, In step 2, the selective filter core is constructed using a Gabor filter, specifically including: A Gabor filter kernel is constructed for each scale layer and each directional angle. The Gabor filter kernel consists of a Gaussian envelope and cosine modulation. Different weights are assigned to different directions at different scales based on scale-direction weights; The retinal boundary response is convolved with the weighted Gabor filter kernel to obtain the weighted response of this scale-direction combination.

5. The method for image quality grading of financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 1, characterized in that, Step 2, the calculation of the final response, specifically includes: For each pixel location, the maximum value among the weighted responses of 18 scale-direction combinations consisting of 3 scales and 6 directions is selected as the final response value for that location.

6. The method for image quality grading of financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 1, characterized in that, Step 3 specifically includes: The process of generating the backside spatial prediction mask: The final response obtained in step 2 is subjected to multi-scale upsampling processing in sequence. Three transposed convolution operations with a stride of 2 are used to generate back-side feature maps of three scale layers respectively. The number of channels is halved after each transposed convolution. For the backside feature map of each scale layer, a prediction mask for that layer is generated by 3×3 convolution and sigmoid activation function; The predicted masks of all layers are upsampled to the original image size using bilinear interpolation; The upsampled prediction masks of each layer are weighted and summed, with higher weights for deeper features, to obtain the final backside prediction mask, which is used to predict the possible locations of defects in the document image. The process of generating the ventral semantic mask: The final response obtained in step 2 is sequentially transformed through three convolutional blocks. Each convolutional block contains a convolutional layer, a batch normalization layer, and an activation function. The ventral semantic feature map is obtained after three convolutional blocks; The ventral semantic feature map is converted from multi-channel features to single-channel features through 1×1 convolution, and then upsampled to the original image size through bilinear interpolation. After passing through the Sigmoid activation function, the ventral semantic mask is obtained, which is used to identify the types of defects in the image, including blur, stains, reflections, wrinkles, watermark interference, incomplete cropping, low contrast, and noise. The spatial prediction error is obtained by calculating the difference between the back-side spatial prediction mask and the front-side semantic mask. Based on the spatial prediction error, iterative optimization is performed to improve the accuracy of defect region identification in the image.

7. The method for image quality grading of financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 6, characterized in that, Step 3, the specific process of the iterative optimization includes: The spatial prediction error is obtained by calculating the pixel-by-pixel difference between the backside spatial prediction mask and the frontside semantic mask. Update the backside spatial prediction mask: The spatial prediction error is multiplied element-wise with the ventral semantic feature map to obtain the error-weighted feature; The error-weighted features are multiplied by the ventral learning rate and then added to the ventral semantic feature map to obtain the updated ventral semantic feature map. The updated ventral semantic feature map is mapped to a single-channel mask through a 1×1 convolution. The single-channel mask is multiplied by the back-side learning rate and then added to the back-side spatial prediction mask to obtain the updated back-side spatial prediction mask. Update the dorsal feature map: The spatial prediction error is downsampled to the size of the backside feature map. The downsampled spatial prediction error is multiplied element-wise with the backside feature map to obtain the error-weighted feature. The error-weighted features are multiplied by the ventral learning rate and then added to the dorsal feature map; The summation result is processed by nonlinear transformation of convolution, batch normalization and ReLU activation function to obtain the updated backside feature map; Recalculate the prediction error between the updated backside spatial prediction mask and the frontside semantic mask; Repeat the above process several times, and terminate the iteration early when the prediction error is lower than the preset threshold.

8. The image quality grading method for financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 6, characterized in that, In step 4, For the image quality classification, global average pooling is performed on the back-side spatial prediction mask and the ventral semantic mask to obtain two global feature vectors. The two vectors are concatenated along the feature dimension and then passed through a two-layer fully connected network and a softmax activation function to obtain the probability distribution of the three quality levels. For the defect type classification, the two vectors are concatenated along the feature dimension, and the probabilities of eight defect categories are output through an independent two-layer fully connected network and a Sigmoid activation function. A weighted binary cross-entropy loss is used to address the differences between category samples. For the segmentation of the defective region, a foreground mask and a background mask are generated by a dual-branch segmentation architecture of foreground and background, respectively. The background mask is subtracted from the foreground mask to obtain a residual difference map, which is then added to the refined fusion map to obtain the final segmentation mask.

9. The method for image quality grading of financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 8, characterized in that, The specific implementation process of the dual-branch partitioning architecture includes: The foreground branch and background branch take the backside feature map optimized by predictive feedback iteration in step 3 as input, respectively; Both the foreground and background branches gradually restore the resolution to the original image size through cubic bilinear interpolation and 3×3 convolution operations. The foreground and background branches have the same structure and their parameters are trained independently to generate foreground and background masks respectively. Subtracting the background mask from the foreground mask yields the residual difference map; The foreground and background masks are concatenated along the channel dimension and then finely fused using a 3×3 convolutional layer to obtain a fused feature map. The residual difference map is added to the fused feature map to obtain the final defect region segmentation mask.

10. The image quality grading method for financial documents based on retinal spatiotemporal antagonism and predictive feedback as described in claim 9, characterized in that, Step 4 also includes: For quality classification, cross-entropy loss is used as the quality classification loss function; For defect region segmentation, the sum of binary cross-entropy loss and Dice loss is used as the defect region segmentation loss function; The total loss function is constructed by weighting the quality grading loss function, the weighted binary cross-entropy loss for defect type classification, and the defect region segmentation loss function according to a preset weight ratio. By minimizing the total loss function through the backpropagation algorithm and optimizer, end-to-end training is achieved, ensuring mutual promotion among tasks and realizing information complementarity and collaborative decision-making.

Images