A paper currency recognition method based on deep learning

Abstract

This invention discloses a banknote recognition method based on deep learning, comprising: a data acquisition and standardized labeling step, which involves acquiring banknote sample images according to a unified acquisition standard and labeling them with composite tags to construct a training dataset; a deep neural network construction step, which involves constructing a banknote recognition neural network with a deep residual network as its backbone architecture; a model training and optimization step, which involves iteratively training the network using the training dataset to obtain a banknote recognition model; and an inference recognition and result output step, which involves inputting the banknote image to be recognized into the model and outputting the corresponding banknote inventory unit code, front and back identification, and confidence score. This invention, through a standardized multi-person, multi-scene, and multi-lighting data acquisition strategy combined with a ResNet deep residual network, effectively overcomes technical difficulties such as large intra-class differences in banknotes, blurred inter-class boundaries, and environmental interference, achieving efficient, accurate, and robust automatic banknote recognition. It can be widely applied in fields such as financial automation and collectible authentication.

Description

Technical Field

[0001] This invention relates to the field of computer vision and pattern recognition technology, and in particular to a deep learning-based banknote recognition method. Specifically, it relates to a deep learning-based banknote recognition method, system, electronic device, and readable storage medium. Background Technology

[0002] As legal tender, paper money is not only a medium of exchange but also an important carrier of history, culture, and art. In numerous business scenarios such as collecting, authentication, and automated financial processing, the ability to quickly and accurately identify the type (e.g., version, denomination, variety) of paper money is a core requirement. Taking Chinese paper money as an example, it covers the first to fifth sets of Renminbi, as well as local bank notes and border region currencies from various periods, exhibiting a wide variety of types, complex versions, and a vast historical span.

[0003] Currently, traditional banknote authentication and identification processes rely primarily on manual labor. Authenticators must consult banknote catalogs or professional guides, using the naked eye or a magnifying glass to observe paper characteristics, ink color, printing techniques, security features, and pattern details to comprehensively determine the authenticity and version of the banknote. However, this manual identification method has the following significant drawbacks: First, the identification process is inefficient and costly. Manual authentication relies heavily on the personal experience of experts, and processing a single banknote often takes several minutes or even longer. When dealing with large quantities of banknotes (such as the sorting of banknotes withdrawn from circulation or bulk purchases by collectors), the efficiency bottleneck becomes extremely prominent. Furthermore, training a qualified authentication expert is time-consuming and costly.

[0004] Second, the process is highly subjective, lacking consistency and objectivity. The results of authentication are easily influenced by factors such as the experience level of the authenticator, visual fatigue, and subjective judgment, which may lead to differences in the judgments of different authenticators on the same banknote. Even the judgments of the same authenticator at different times are difficult to guarantee to be completely consistent, lacking a unified objective standard.

[0005] Third, the ability to identify rare or damaged samples is limited. For rare varieties with limited surviving quantities, ordinary appraisers lack sufficient reference experience; and for banknotes with wear, stains, creases, fading, and other condition problems, manual identification is extremely difficult, often requiring the use of expensive professional instruments for auxiliary testing, which further increases costs and time.

[0006] To address these issues, the industry has attempted to introduce computer vision technology for assisted recognition. However, banknote recognition itself presents significant technical challenges, primarily in the following aspects: banknotes of the same version exhibit substantial intra-class differences under varying conditions (e.g., newness, wear); different versions, especially those from similar eras, show high inter-class similarity in patterns, denomination numbers, and main colors, leading to blurred classification boundaries. Furthermore, changes in lighting, shooting angle, and background interference in the acquisition environment introduce domain shifts, causing drastic changes in the apparent features of the same banknote under different imaging conditions. Traditional image recognition methods based on manually designed features (such as SIFT and HOG) struggle to achieve satisfactory accuracy and robustness in such complex and noisy environments due to limitations in feature representation capabilities.

[0007] In recent years, deep learning technology has made groundbreaking progress in the field of image classification, providing new solutions for complex visual recognition tasks. Some deep learning-based methods for banknote processing have emerged in the existing technology. For example, patent application CN121438449A discloses a deep learning-based method and system for detecting abnormal banknote shapes. Its focus is on detecting physical anomalies such as folding, dropping, and leaving behind banknotes, rather than performing detailed classification and recognition of the banknote's denomination or other identity attributes. Therefore, how to construct a solution that overcomes the aforementioned technical difficulties and achieves efficient, accurate, and robust automatic banknote recognition is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] In view of the shortcomings of existing technologies, the purpose of this invention is to provide a banknote recognition method based on deep learning, so as to solve the technical problems of low efficiency and strong subjectivity of manual recognition, as well as the difficulty of traditional visual algorithms in dealing with large intra-class differences, blurred inter-class boundaries, and strong environmental interference in banknotes.

[0009] The above-mentioned objective of this invention is achieved through the following technical solutions: This invention provides a deep learning-based banknote recognition method, comprising: Step S1, Data Acquisition and Standardized Labeling: Collect banknote sample images according to the preset unified acquisition specifications, and label each sample image with a composite label. The composite label uniquely indicates the corresponding banknote inventory unit code and front and back information to construct a training dataset. Step S2, Deep Neural Network Construction: Construct a banknote recognition neural network with a deep residual network as the backbone architecture. The dimension of the classification output layer of the network is equal to the total number of categories of all composite labels in the training dataset. Step S3, Model Training and Optimization: Using the training dataset, the cross-entropy loss function is minimized through the backpropagation algorithm and gradient descent optimizer to iteratively update the network parameters of the banknote recognition neural network until the model converges, thus obtaining the trained banknote recognition model. Step S4, Reasoning Recognition and Result Output: After performing the same preprocessing as in the training stage on the banknote image to be recognized, input it into the trained banknote recognition model to obtain the probability distribution output by the model, and parse out the corresponding banknote inventory unit code, front and back identification and confidence level based on the probability distribution as the final recognition result.

[0010] According to one embodiment of the present invention, the unified acquisition standard includes: acquiring multiple images of the front and back sides of the same banknote inventory unit code, and having multiple collectors take the photos in multiple different scenes and under multiple lighting conditions; the scenes include multiple different work or life backgrounds, and the lighting conditions include at least normal lighting, bright lighting and dark lighting.

[0011] According to one embodiment of the present invention, the deep residual network is a ResNet-50 network, which includes an input layer, an initial convolutional layer, multiple residual block stages, a pooling layer, a fully connected layer, and a classification output layer connected in sequence; the multiple residual block stages include res2 stage, res3 stage, res4 stage, and res5 stage.

[0012] According to one embodiment of the present invention, at the location of the pooling layer, a spatial pyramid pooling layer is used to replace the global average pooling layer to accommodate input images of different sizes and extract multi-scale features.

[0013] According to one embodiment of the present invention, in the model training and optimization step, a data augmentation strategy is used to expand the training samples online. The data augmentation strategy includes geometric transformation and color transformation. The geometric transformation includes at least random horizontal flipping, random rotation, and random cropping. The color transformation includes at least random scaling of brightness, contrast, and saturation.

[0014] According to one embodiment of the present invention, in the model training and optimization step, a transfer learning strategy is adopted, and the parameters of the banknote recognition neural network are initialized using weights pre-trained on a general large-scale image dataset.

[0015] According to one embodiment of the present invention, the reasoning recognition and result output step further includes a step of aggregating multiple recognition results: multiple images to be recognized or multiple batches of recognition results are grouped according to the banknote inventory unit code and the front and back identification, and the confidence scores of each recognition result in the group are summed or averaged to obtain an aggregation score, and the final sorting and output are performed based on the aggregation score.

[0016] The present invention also provides a system for a deep learning-based banknote recognition method according to the above embodiments, comprising: The data construction module is used to perform the data acquisition and standardized labeling steps to construct the training dataset; The network construction module is used to perform the deep neural network construction steps to construct a banknote recognition neural network; The model training module is used to perform the model training and optimization steps, train the banknote recognition neural network, and obtain a trained banknote recognition model. The recognition output module is used to perform the inference recognition and result output steps, inputting the banknote image to be recognized into the trained banknote recognition model and outputting the recognition result.

[0017] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a deep learning-based banknote recognition method according to the above embodiments.

[0018] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a deep learning-based banknote recognition method according to the above embodiments.

[0019] In summary, compared with the prior art, the present invention has at least one of the following beneficial technical effects: High recognition efficiency: Through an end-to-end deep learning model, the recognition time of a single banknote is reduced to milliseconds. Compared with the traditional manual method, the efficiency is improved by several orders of magnitude, which can meet the real-time recognition needs of large scale and high concurrency.

[0020] High accuracy and robustness: By designing refined data collection specifications (multiple people, multiple scenes, multiple lighting conditions, multiple angles), the diversity of training data is significantly improved, enabling the model to learn more generalizable features. This effectively overcomes the impact of differences in appearance and environmental interference, and demonstrates stronger robustness to wear, stains, and changes in lighting. Employing deep networks such as ResNet, it can automatically learn hierarchical features from texture to semantics, with representational capabilities far exceeding those of hand-designed features.

[0021] Objectivity and consistency: The model is trained on a large amount of standardized data, and the recognition process is completed automatically by the algorithm, eliminating the interference of human factors and ensuring the objectivity and repeatability of the recognition results.

[0022] Fine-grained recognition capability: By combining the "SKU code" with the "front and back" into a fine-grained composite label, the model can simultaneously complete the identification of banknote type and the determination of front and back, meeting the refined requirements of practical applications.

[0023] Highly adaptable and scalable: The solution of this invention does not rely on specific manual features. When it is necessary to add new banknote categories, it is only necessary to expand the corresponding training data and fine-tune the model, which has good scalability and adaptability. Attached Figure Description

[0024] Figure 1 This is a flowchart of the present invention.

[0025] Figure 2 This is an overall flowchart of the deep learning-based banknote recognition method in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of the architecture of the ResNet-50 deep neural network used in an embodiment of the present invention.

[0027] Figure 4 This is a structural block diagram of the deep learning-based banknote recognition system of the present invention.

[0028] Figure reference numerals: 100, System; 110, Data construction module; 120, Network construction module; 130, Model training module; 140, Recognition output module. Detailed Implementation

[0029] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0030] In the description of this application, it should be noted that the terms "upper," "lower," "inner," "outer," "top / bottom," etc., indicating the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0031] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installed," "equipped with," "sleeved / connected," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0032] On the one hand, this invention provides a banknote recognition method based on deep learning. This method models banknote recognition as a multi-class image classification task, including data acquisition and standardization annotation steps, deep neural network construction steps, model training and optimization steps, and inference recognition and result output steps.

[0033] Specifically, in the data acquisition and standardization labeling steps, banknote sample images are collected according to a preset unified acquisition standard, and each sample image is labeled with a fine-grained composite label. The composite label uniquely indicates its corresponding banknote stock unit (SKU) code and front and back information to construct training datasets, validation datasets and test datasets.

[0034] In the deep neural network construction step, a banknote recognition neural network with a deep residual network as its backbone is constructed. This network includes, in sequence, an input layer, a convolutional feature extraction layer, multiple residual block stages, a pooling layer, a fully connected layer, and a classification output layer. The dimension N of the classification output layer is equal to the total number of categories of all composite labels in the training dataset, and is used to output the probability that an image belongs to each composite label category.

[0035] In the model training and optimization steps, the training dataset is used to minimize the cross-entropy loss function through the backpropagation algorithm and gradient descent optimizer, and the network parameters of the banknote recognition neural network are iteratively updated until the model converges, thus obtaining the trained banknote recognition model.

[0036] In the inference recognition and result output steps, the banknote image to be recognized is preprocessed in the same way as in the training stage, and then input into the trained banknote recognition model to obtain the probability distribution output by the model. Based on the probability distribution, the corresponding SKU code, front and back markings and confidence level are parsed out as the final recognition result.

[0037] Furthermore, the unified acquisition standard includes: acquiring multiple images of the front and back sides of the same SKU, and having multiple acquirers take the photos in multiple different scenes and under multiple lighting conditions; the scenes include multiple different work or life backgrounds, and the lighting conditions include at least normal lighting, bright lighting and dark lighting.

[0038] Furthermore, the deep residual network is a ResNet-50 network, and its residual block stages include res2, res3, res4 and res5 stages connected in sequence; before the pooling layer, a spatial pyramid pooling (SPP) layer can also be introduced to be compatible with input images of different sizes and extract multi-scale features.

[0039] Furthermore, in the model training and optimization steps, data augmentation strategies are used to expand the training samples online. The data augmentation strategies include geometric transformation and color transformation. The geometric transformation includes at least random horizontal flipping, random rotation, and random cropping. The color transformation includes at least random scaling of brightness, contrast, and saturation.

[0040] Furthermore, in the model training and optimization steps, a transfer learning strategy is adopted, and the parameters of the banknote recognition neural network are initialized using weights pre-trained on a general large-scale image dataset.

[0041] Furthermore, the reasoning recognition and result output steps also include a step of aggregating multiple recognition results: multiple images to be recognized or multiple batches of recognition results are grouped according to SKU code and front and back labels, and the confidence scores of each recognition result in the group are summed or averaged to obtain an aggregate score, and the final sorting and output are performed based on the aggregate score.

[0042] On the other hand, the present invention also provides a deep learning-based banknote recognition system 100, comprising: Data construction module 110 is used to perform the above data acquisition and standardized annotation steps to construct training dataset, validation dataset and test dataset; Network building module 120 is used to perform the above deep neural network building steps to build a banknote recognition neural network; The model training module 130 is used to perform the above model training and optimization steps, train the banknote recognition neural network, and obtain a trained banknote recognition model. The recognition output module 140 is used to perform the above-mentioned reasoning recognition and result output steps, input the banknote image to be recognized into the trained banknote recognition model, and output the recognition result.

[0043] In another aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described deep learning-based banknote recognition method.

[0044] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described deep learning-based banknote recognition method. Example

[0045] This embodiment provides a banknote recognition method based on deep learning. For example... Figure 1 and Figure 2 As shown, the method includes steps S110 to S140.

[0046] Step S110: Data acquisition and standardized labeling.

[0047] This step aims to build a high-quality, highly diverse labeled dataset, which is fundamental to determining the upper limit of model performance. To ensure data quality and model generalization ability, this step follows a strict and unified set of data collection and labeling specifications.

[0048] The specific implementation method is as follows: S111: Directory and Label Organization. Establish a separate directory for each banknote stock unit (SKU), with each directory named using a unique SKUCode. Images of the front and back of the same banknote should be physically separated, for example, stored in separate "Front" and "Back" subdirectories, or distinguished by filename. Labels should use a composite label format, such as "{Front / Back Identifier}-{SKUCode}", to clearly distinguish the front and back of the same banknote. In a specific example, the front is labeled "CONS" and the back is labeled "PROS". File names are recommended to use the format "{SKUCode}{Front / Back Identifier}{Serial Number}.jpg" for batch processing, traceability, and quality auditing.

[0049] S112: Sample Size and Coverage Strategy. To cover the apparent changes of banknotes under different conditions, a predetermined number of images are collected for both the front and back of the same SKU. In a preferred embodiment, 18 images are collected for each SKU, one for each side. These 18 images are taken by three different collectors, with each collector taking six images. Each collector's six samples must cover at least three different background scenarios, such as being held, placed on a keyboard, placed in front of a screen, or placed next to everyday office supplies. The background selection should not be too bright or complex to avoid introducing strong discriminative interference features unrelated to the banknote itself. This "multi-person, multi-scenario" collection design aims to force the model to learn the inherent characteristics of the banknote itself, rather than artifacts related to specific collector habits or specific backgrounds, thereby greatly enhancing the model's generalization ability across devices and scenarios.

[0050] S113: Shooting Quality and Angle Constraints. Banknotes should be laid flat as much as possible during acquisition to reduce feature obstruction and deformation caused by creases. Samples of different conditions should be included, including brand new, circulated, and samples with minor creases or stains, to improve the model's robustness to common condition issues in real-world applications. Ensure the banknote occupies as much of the frame as possible, with the left and right margins not exceeding 1 / 4 of the overall image width. Shooting angles must be controlled, with horizontal angles limited to 10° and vertical angles limited to 20°, avoiding extreme side angles. The same collector should cover multiple angles to avoid repeatedly capturing images from the same angle. Each banknote should be captured under normal, slightly bright, and slightly dark lighting conditions to simulate complex lighting changes in real-world environments. If local overexposure occurs, it is only allowed in edge areas, and the exposed area of ​​a single image should not exceed 100×100 pixels, with the exposure positions distributed discretely to avoid concentration in key feature areas. The captured images must be clear and distinguishable. It is forbidden to capture blurry images caused by shaking or defocusing, or images that are severely damaged or copied from the screen.

[0051] S114: Image Technical Specifications. To ensure the distinguishability of details such as banknote texture, microtext, and anti-counterfeiting features, recommended requirements for image technical parameters are provided, as shown in Table 1 below: Table 1: Recommended parameters for image technology specifications S115: Data Validation and Quality Control. Before data is entered into the database, rigorous automated validation and manual sampling inspection are required. Automated validation includes checking whether the directory structure conforms to specifications, whether file naming is consistent, and whether there is a one-to-one correspondence between tags and files. Quality sampling inspection is handled by designated personnel, covering key items such as clarity, exposure, and occlusion. For scenarios with multiple annotators, statistical checks on annotation consistency should be conducted, such as calculating the Kappa coefficient. It is recommended that the Kappa coefficient be no less than 0.85 to ensure the reliability and consistency of the tags.

[0052] After the above steps S111-S115, a supervised learning sample pair (I_i, y_i) is finally obtained, consisting of a high-resolution banknote image and the corresponding composite classification label. Here, I_i ∈ R^{H×W×3} is the RGB image, y_i ∈ {1,2,...,N} is the category label, and N is the total number of categories, that is, the number of all possible combinations of "{front / back identification}-{SKUCode}".

[0053] Step S120: Network architecture design.

[0054] In this step, a deep neural network is constructed for joint recognition of banknote type and front / back. Given the fine texture and complex structure of banknote images, and the requirement for the model to have good training stability and generalization ability, this embodiment uses a ResNet-50 deep residual network as the backbone architecture.

[0055] ResNet effectively mitigates the vanishing / exploding gradients and network degradation problems in deep networks by introducing a residual learning mechanism. Its core idea is to learn the residual mapping F(x) = H(x) - x, rather than directly learning the target mapping H(x), which allows the network to maintain a low training error even as the number of layers increases.

[0056] Reference Figure 3 The specific structure of the banknote recognition network constructed in this embodiment is as follows: Input layer: Accepts a pre-processed RGB image of size 224×224×3.

[0057] The initial convolutional layer (conv1) contains a 7×7 convolutional kernel with a stride of 2, producing an output feature map of 112×112 pixels with 64 channels. This is followed by a BatchNorm (BN) layer for batch normalization, a ReLU activation function layer, and a 3×3 max-pooling layer with a stride of 2, further reducing the output feature map size to 56×56.

[0058] Residual block stage: There are four residual block stages in total, namely res2, res3, res4 and res5.

[0059] The res2 stage contains three residual units. Each residual unit employs a "bottleneck" structure, consisting of 1×1, 3×3, and 1×1 convolutional layers in sequence. The first residual unit's 1×1 convolution compresses the input's 256-dimensional channels; after a 3×3 convolution, the final 1×1 convolution restores the dimensions to 256. The output feature map of this stage has a size of 56×56 and 256 channels.

[0060] The res3 stage contains four residual units. The first residual unit in this stage downsamples the feature map to 28×28 by setting the convolution stride to 2, while increasing the number of channels to 512. Subsequent residual units extract features on a 28×28 scale.

[0061] The res4 stage contains 6 residual units. Similarly, the first residual unit is downsampled, reducing the feature map size to 14×14 and increasing the number of channels to 1024.

[0062] The res5 stage contains 3 residual units. The first residual unit performs downsampling, reducing the feature map size to 7×7 and increasing the number of channels to 2048.

[0063] Within each residual unit, the unit's input x is directly added to its output F(x, {W_i}) via a "shortcut," i.e., y = F(x, {W_i}) + x. When the number of input and output channels is inconsistent (as in downsampling), a 1×1 convolutional layer is added to the shortcut to adjust the dimension of x to match the dimension of F(x) for the addition operation. This structure significantly improves gradient backpropagation in deep networks.

[0064] Pooling layer: After the res5 stage, a global average pooling (GAP) layer is connected to compress the 7×7 feature map of each channel into a single value, thus obtaining a 2048-dimensional feature vector.

[0065] Fully connected layer: The 2048-dimensional feature vector output by the GAP layer is connected to a fully connected layer of dimension N. N is the total number of categories, for example, 878. The output of this layer is an N-dimensional logits vector, where each dimension corresponds to the raw score of a composite category (such as "CONS-0123456" or "PROS-0123456").

[0066] Classification output layer: Finally, a Softmax function is connected to transform the logits vector into a probability distribution p = [p_1, p_2, ..., p_N] that sums to 1, where p_k represents the probability that the input image belongs to the k-th class.

[0067] The evolution of feature map size and number of channels at each stage of the entire network can be summarized in Table 2 below: Table 2: ResNet-50 Network Structure Parameters The network architecture was further optimized. To enhance the model's robustness to input image scales and extract richer multi-scale features, a Spatial Pyramid Pooling (SPP) layer was introduced to replace the original global average pooling layer at the pooling layer location described in step S120 of the implementation example.

[0068] After the feature map output from the res5 stage (assuming a size of a×a×2048), an SPP layer is connected. The SPP layer contains multiple pooling operations of different scales, such as max pooling or average pooling on the input feature map using pooling windows of 1×1, 2×2, and 4×4 respectively. Thus, regardless of the original size 'a' of the input feature map, 1×1 pooling always produces one value, 2×2 pooling produces four values, and 4×4 pooling produces 16 values. Flattening and concatenating these pooling results yields a fixed-length feature vector. For example, for 2048 channels, using 1×1, 2×2, and 4×4 pooling, the final feature vector dimension is 2048 × (1×1 + 2×2 + 4×4) = 2048 × (1+4+16) = 2048×21.

[0069] Introducing the SPP layer offers several advantages: First, it allows the network to accept input images of arbitrary sizes, avoiding image distortion and feature loss caused by forced scaling. This is particularly important when processing banknote images containing numerous fine textures, where maintaining the original aspect ratio and details is crucial. Second, multi-scale pooling can simultaneously extract both overall image features and local detail features, providing richer discriminative information for subsequent classification. Experimental data shows that, under the same conditions, introducing the SPP layer can improve the Top-1 accuracy of banknote recognition by approximately 1% to 1.5% compared to single-scale global average pooling.

[0070] Step S130 is described in detail: Model training and optimization.

[0071] The goal of this step is to drive the network parameters to be optimized in a direction that can accurately map banknote images to category labels using a large amount of labeled data.

[0072] S131: Data Partitioning. The dataset containing all samples constructed in step S110 is divided into a training set, a validation set, and a test set according to a preset ratio. To ensure that the distribution ratio of samples of each category in the three sets is consistent with the original dataset, especially to ensure that long-tail categories with few samples also appear in the validation and test sets, this embodiment adopts a stratified sampling strategy. A recommended partitioning ratio is training set:validation set:test set = 8:1:1 or 7:1.5:1.5.

[0073] S132: Data Preprocessing and Augmentation. Before inputting images into the network, a uniform preprocessing operation is required. First, all input images are scaled to a fixed size required by the network, such as 224×224. This is mandatory for embodiments without SPP; for embodiments using SPP, although any size is acceptable, it is usually scaled to a fixed size for batch training convenience. Second, channel order conversion is performed, such as converting RGB to BGR (if the pre-trained model is based on this order). Finally, normalization is performed. This embodiment uses a mean subtraction method, that is, subtracting the mean of the corresponding channel from each of the B, G, and R channels for each pixel. In a specific example, the mean vector used is μ=[136,147, 156] (corresponding to the B, G, and R channels). This mean can be obtained statistically from the training dataset or the statistical mean from the ImageNet dataset can be used.

[0074] To improve the model's generalization ability and robustness, online data augmentation strategies are applied to the input images during training. Data augmentation simulates various changes that may exist in the real world, forcing the model to learn more essential invariant features. The data augmentation strategies used in this embodiment include: Geometric transformations: random horizontal flipping with a probability of 0.5; random rotation with the rotation angle controlled within ±15° to avoid excessive rotation that could lead to loss of banknote orientation information; random cropping, randomly cropping 80% to 100% of the area from the original image and then scaling it to the network input size.

[0075] Color transformation: The brightness, contrast and saturation of the image are randomly adjusted, with the adjustment range being randomly sampled within ±20% to simulate color differences between different lighting conditions and acquisition devices.

[0076] Advanced enhancements (optional): Further random erasing strategies can be employed, randomly selecting a rectangular region on the image and filling it with random values ​​or noise, forcing the model to focus on the features of the unoccluded region; or a mixup strategy can be used, mixing two training images of different categories in a certain proportion, and mixing the corresponding labels in the same proportion, so that the model learns a smoother decision boundary.

[0077] It is important to emphasize that the extent of data enhancement needs to be carefully controlled, and the key distinguishing features of banknotes should not be excessively damaged, such as the denomination numerals, portraits, national emblems, and other core patterns.

[0078] S133: Loss Function and Optimizer Settings. This embodiment uses the cross-entropy loss function as the objective function for optimization. For a single training sample (I, y), the loss is calculated as L(θ) = -log p_θ(y|I), where p_θ(y|I) is the probability that the model predicts the sample to be the true label y. In actual training, a mini-batch training method is used, and the average loss of all samples within the batch is taken as the final loss value.

[0079] The optimizer chosen is stochastic gradient descent (SGD) or a variant thereof. In a specific instance, a momentum-based SGD optimizer is used, with the momentum factor set to 0.9. The Adam optimizer can also be used, with hyperparameters set to β1=0.9 and β2=0.999. The initial learning rate depends on the training strategy: if training from scratch, the initial learning rate can be set to 0.01; if fine-tuning is based on a pre-trained model, the initial learning rate can be set to 0.001. To improve convergence later in training, a learning rate decay strategy is used, such as step decay, where the learning rate is reduced to 0.1 times its original value every 30 epochs; or a cosine annealing strategy is used, where the learning rate gradually decreases according to a cosine function throughout the training period.

[0080] Other hyperparameter settings: The batch size is recommended to be 32 or 64, which needs to be adjusted according to the GPU memory size. Too small a batch size will cause instability in the statistics of the batch normalization (BatchNorm) layer. The number of training epochs is recommended to be set to 50-100, and can be combined with an early stopping mechanism. That is, if the accuracy on the validation set no longer improves within several consecutive epochs (e.g., 10 epochs), training should be terminated early to prevent overfitting. The weight decay coefficient should be set to 1e-4 for L2 regularization.

[0081] S134: Transfer Learning. To fully utilize the knowledge learned from massive amounts of general image data, accelerate model convergence, and improve final performance, this embodiment employs a transfer learning strategy. Specifically, training begins with the weights of a ResNet-50 model pre-trained on large datasets such as ImageNet. Different fine-tuning strategies can be used for different amounts of data: Strategy 1 (Full Model Fine-tuning): Initialize all layers of the network using pre-trained weights, then perform a full forward and backward propagation on the banknote dataset to update the parameters of all layers. This is the most direct approach.

[0082] Strategy Two (Freeze Shallow Layers, Fine-tune Deep Layers): Since the shallow layers of the network (such as conv1, res2, res3) learn general low-level features such as edges, textures, and color patches, these features are universal across different tasks. Therefore, the parameters of these layers can be frozen and kept unchanged during training, training only the deeper layers of the network (such as res4, res5) and the newly initialized fully connected layers. This can significantly reduce the number of parameters that need to be trained, reduce the risk of overfitting, and accelerate the training process.

[0083] Strategy 3 (Layered Learning Rate): Set different learning rates for different layers. For example, set a smaller learning rate for shallow networks (such as conv1, res2), a medium learning rate for mid-level networks (res3, res4), and a larger learning rate for deep networks (res5, FC). The aim is to preserve the general knowledge of the pre-trained model while enabling the model to learn the high-level semantic features specific to banknotes more effectively.

[0084] S135: Model Evaluation and Selection. After each training epoch, the current model is evaluated on the validation set. Evaluation metrics include: Top-1 accuracy: The proportion of samples where the class predicted by the model has the highest probability matches the true label. This is the primary evaluation metric.

[0085] Top-5 accuracy: The proportion of samples where the true label appears in the top 5 categories with the highest probability predicted by the model. This metric is used to evaluate the model's fault tolerance and coverage of easily confused categories.

[0086] Confusion Matrix: By analyzing the confusion matrix, we can intuitively discover which categories are easily confused, and thus analyze the reasons in a targeted manner (for example, whether the front and back of two banknotes are similar, or whether different versions of banknotes of the same denomination are similar).

[0087] Precision and recall for each category: For long-tail categories with a small number of samples, precision and recall need to be calculated separately to evaluate the model's ability to identify rare varieties in a fine-grained manner and prevent the model from ignoring these categories due to sample imbalance.

[0088] Finally, the model that performs best on the validation set (e.g., with the highest Top-1 accuracy) is selected as the final banknote recognition model, and its performance is evaluated on the test set to objectively measure the model's generalization ability.

[0089] Step S140: Inference Recognition and Result Output. This step describes how to apply the trained model to a real-world banknote recognition scenario.

[0090] S141: Preparation before inference. Set the trained model to evaluation mode. In this mode, the Dropout layer in the network will be turned off, and the BatchNorm layer will use the global statistics (runningmean and running variance) learned during training instead of the statistics of the current batch data for normalization to ensure the determinism of the inference process and the stability of the results.

[0091] S142: Single image inference link.

[0092] 1. Image Input and Preprocessing: Receive the banknote image I to be recognized. First, perform the same preprocessing operations as in the training phase: if the input image is in RGB format, convert it to BGR channel order; adjust the image size to the input size required by the network (e.g., 224×224); subtract the mean value used during training (e.g., [136, 147, 156]) from the B, G, and R channels of each pixel to obtain the preprocessed tensor K.

[0093] 2. Forward propagation: Input Ĩ into the trained banknote recognition model, perform forward computation, and finally obtain the N-dimensional logits vector z = [z_1, z_2, ..., z_N] from the fully connected layer.

[0094] 3. Probability Calculation and Sorting: The logits vector is converted into a probability distribution using the Softmax function, yielding the probability p_k = exp(z_k) / Σ_{j=1}^N exp(z_j) of the image belonging to each composite category. The probability values ​​are then sorted in descending order, and the K results with the highest probabilities are selected, for example, K=5, to obtain the Top-K category indices and their corresponding probabilities.

[0095] 4. Result Parsing: Based on the pre-built label mapping file (e.g., "labels.txt", where each line corresponds to a label index in the format "CONS-0123456" or "PROS-0123456"), the Top-K category indices are parsed into specific SKU codes and front / back identifiers, and the corresponding probability values ​​are used as confidence scores. Finally, each result is encapsulated into a data structure containing id (SKU code), positive (front / back identifier), and confidenceRate (confidence score). For example: {"id": "0123456", "positive": "front", "confidenceRate": 0.9856}.

[0096] S143: Multi-result aggregation strategy. In practical applications, to improve the stability of recognition results, users may take multiple photos of the same banknote or perform multiple recognitions of the same banknote in a video stream. Therefore, this embodiment introduces a multi-result aggregation strategy, denoted as result_group.

[0097] Suppose we have obtained a list of M independent recognition results R = [r_1, r_2, ..., r_M]. The aggregation process is as follows: 1. Grouping: Iterate through the Top-K elements in all M results and group them according to the (id, positive) key-value pairs.

[0098] 2. Aggregated score calculation: For each unique (id, positive) group, sum (or average) the confidence scores of its occurrence in all identification results to obtain the aggregated score of the group.

[0099] 3. Sorting and Output: Sort all groups in descending order of their aggregate scores. The group with the highest score is the final comprehensive recognition result. For example, if four out of five recognitions identify the banknote as "front-0123456" with high confidence, the aggregate score of this result will be much higher than other occasional erroneous results, thus yielding a more robust final conclusion.

[0100] S144: Inference Performance Optimization. For applications with high real-time requirements, the following measures can be taken to optimize the inference process: Batch inference: Combining multiple images to be recognized into a batch and inputting them into the GPU for inference at once can significantly improve GPU utilization and overall throughput.

[0101] Model quantization: Converts the model's weights and activation values ​​from 32-bit floating-point numbers (FP32) to lower-bit integers, such as INT8. INT8 quantization can significantly reduce model size and memory footprint, and leverages the hardware's native support for low-precision computation to accelerate inference, while the loss of precision is usually kept within an acceptable range.

[0102] Inference engine optimization: The trained model (such as the Caffe model) is exported to the Open Neural Network Exchange Format (ONNX), and then loaded and optimized using a high-performance inference engine such as TensorRT. TensorRT reconstructs the network structure, performs inter-layer fusion, and automatically tunes the kernels, which can further reduce inference latency and improve throughput.

[0103] Example 2 This embodiment provides a deep learning-based banknote recognition system to implement the method described in Embodiment 1 above. Figure 4 As shown, the system 100 includes: The data construction module 110 is used to perform the data acquisition and standardized labeling steps described in step S110 of the first embodiment above. This module acquires banknote sample images according to a preset unified acquisition standard, and labels each sample image with fine-grained composite labels, ultimately constructing a training dataset, a validation dataset, and a test dataset.

[0104] The network construction module 120 is used to execute step S120 in Embodiment 1 above, or the deep neural network construction step described therein. This module constructs a banknote recognition neural network with a deep residual network (such as ResNet-50) as its backbone architecture. The dimension of the classification output layer of this network is equal to the total number of categories N of all composite labels in the training dataset. This module may also introduce spatial pyramid pooling layers into the network.

[0105] The model training module 130 is used to execute the model training and optimization steps described in step S130 of Embodiment 1 above. This module utilizes the training dataset provided by the data construction module 110, and iteratively updates the parameters of the network constructed by the network construction module 120 by minimizing the cross-entropy loss function through the backpropagation algorithm and a preset optimizer (such as SGD with momentum). During training, this module can also call data augmentation strategies to expand the training samples online and can load pre-trained weights to achieve transfer learning. After training is completed, the model training module 130 outputs the trained banknote recognition model.

[0106] The recognition output module 140 is used to execute the reasoning recognition and result output steps described in step S140 of Embodiment 1. This module loads the model trained by the model training module 130, performs preprocessing on the user-input banknote image to be recognized in the same manner as during the training phase, and then inputs it into the model for forward inference. Then, it analyzes the probability distribution output by the model to obtain the corresponding SKU code, front / back identification, and confidence level. This module can also, according to Embodiment 1, execute a multi-result aggregation strategy on multiple recognition results to output a more stable aggregated result.

[0107] Example 3 This embodiment provides an electronic device that can be used to perform the steps in the method embodiments described above. The electronic device includes a central processing unit (CPU), which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) or a program loaded from a storage portion into random access memory (RAM). The RAM also stores various programs and data required for system operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0108] The following components are connected to the I / O interface: input sections including keyboards, mice, touchscreens, and image acquisition devices (such as cameras and scanners); output sections including liquid crystal displays (LCDs) and speakers; storage sections including hard drives; and communication sections including network interface cards such as LAN cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0109] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit, it performs the functions defined above in the system of this application.

[0110] Example 4 This embodiment provides a computer-readable storage medium. This computer-readable storage medium may be included in the electronic device described in Embodiment 3 above, or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the deep learning-based banknote recognition method described in Embodiment 1 above.

[0111] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0112] Example 5 This embodiment's system consists of a network structure definition file, a model weight file, a label mapping file, an inference engine, and preprocessing parameters. The network structure definition uses ResNet_50_deploy_878.prototxt (Caffe protobuf format) to describe the configuration and connections of each layer. The model weights use resnet50_banknote_878_20181115.caffemodel to represent the parameters of the trained 878-class classifier. The label mapping uses labels_878_rotate_20181115.txt, with a label format of CONS / PROS-{SKUCode}, corresponding one-to-one with the network output dimensions. The inference engine uses Caffe, supporting switching between GPU (CUDA) and CPU modes according to the deployment environment. Preprocessing parameters include image_resize=224 and mean=[136,147,156], and are required to remain consistent with those used in the training phase to avoid input distribution drift, as shown in Table 3.

[0113] Table 3 2. Example of inference call Initialize the classifier classifier = BanknoteClassify( model_def_file='... / ResNet_50_deploy_878.prototxt', pretrained_model_file='... / resnet50_banknote_878_20181115.caffemodel', class_labels_file='... / labels_878_rotate_20181115.txt', image_resize=224, cpu_mode=False or True) Single image prediction result = classifier.predict(image) Returns: [{"id": "xxx", "positive": "front / back", "confidenceRate": "0.xxxx"}, ...] Multi-result aggregation aggregated = result_group(result) 3. Identification process diagram For practical applications, after users capture or upload images of banknotes to be recognized via mobile devices or scanning devices, subject segmentation or ROI localization (e.g., banknote region localization based on edge detection, contour extraction, or object detection) can be performed first when background noise or multi-target interference exists to obtain the cropped banknote ROI. Subsequently, the ROI is input into a ResNet-50 classification network according to preprocessing specifications to obtain an 878-dimensional probability distribution. The Top-5 results are then parsed into SKU Code, front / back information, and confidence scores, and fed back to the user. For higher robustness, multiple prediction results can be aggregated and fused to improve stability.

[0114] 4. Deployment and Maintenance Key Points On the deployment side, Caffe's CUDA (GPU) or CPU environment dependencies must be met, and the Python interface must be configured with pycaffe. The model files must be compatible with the Caffe version, and it is recommended to use the same version as during training. In terms of performance benchmarks, the inference time for a single 224×224 image on an NVIDIA T4 GPU is approximately 5–10 ms, which is sufficient for real-time recognition. At the same time, it can be connected to the TensorFlow / PyTorch ecosystem by exporting ONNX, or TensorRT can be used for inference acceleration and deployment optimization.

[0115] The implementation principle of this invention is as follows: This invention discloses a banknote recognition method, system, electronic device, and readable storage medium based on deep learning. The method includes: a data acquisition and standardized labeling step, which involves acquiring banknote sample images according to unified acquisition specifications and labeling them with composite tags to construct a training dataset; a deep neural network construction step, which involves constructing a banknote recognition neural network with a deep residual network as its backbone architecture; a model training and optimization step, which involves iteratively training the network using the training dataset to obtain a banknote recognition model; and an inference recognition and result output step, which involves inputting the banknote image to be recognized into the model and outputting the corresponding banknote inventory unit code, front and back identification, and confidence score. This invention, through a standardized multi-personnel, multi-scene, and multi-lighting data acquisition strategy combined with a ResNet deep residual network, effectively overcomes technical difficulties such as large intra-class differences in banknotes, blurred inter-class boundaries, and environmental interference, achieving efficient, accurate, and robust automatic banknote recognition. It can be widely applied in fields such as financial automation and collection authentication.

[0116] The embodiments described herein are preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape, and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A banknote recognition method based on deep learning, characterized in that, include: Step S1, Data Acquisition and Standardized Labeling: Collect banknote sample images according to the preset unified acquisition specifications, and label each sample image with a composite label. The composite label uniquely indicates the corresponding banknote inventory unit code and front and back information to construct a training dataset. Step S2, Deep Neural Network Construction: Construct a banknote recognition neural network with a deep residual network as the backbone architecture. The dimension of the classification output layer of the network is equal to the total number of categories of all composite labels in the training dataset. Step S3, Model Training and Optimization: Using the training dataset, the cross-entropy loss function is minimized through the backpropagation algorithm and gradient descent optimizer to iteratively update the network parameters of the banknote recognition neural network until the model converges, thus obtaining the trained banknote recognition model. Step S4, Reasoning Recognition and Result Output: After performing the same preprocessing as in the training stage on the banknote image to be recognized, input it into the trained banknote recognition model to obtain the probability distribution output by the model, and parse out the corresponding banknote inventory unit code, front and back identification and confidence level based on the probability distribution as the final recognition result.

2. The deep learning-based banknote recognition method according to claim 1, characterized in that, The unified data collection standard includes: collecting multiple images of the front and back of the same banknote inventory unit code, and having multiple collectors take the photos in multiple different scenes and under various lighting conditions; the scenes include a variety of different work or life backgrounds, and the lighting conditions include at least normal lighting, bright lighting, and dark lighting.

3. The deep learning-based banknote recognition method according to claim 1, characterized in that, The deep residual network is a ResNet-50 network, which includes an input layer, an initial convolutional layer, multiple residual block stages, a pooling layer, a fully connected layer, and a classification output layer connected in sequence; the multiple residual block stages include res2 stage, res3 stage, res4 stage, and res5 stage.

4. The deep learning-based banknote recognition method according to claim 3, characterized in that, At the location of the pooling layer, a spatial pyramid pooling layer is used to replace the global average pooling layer to accommodate input images of different sizes and extract multi-scale features.

5. The deep learning-based banknote recognition method according to claim 1, characterized in that, In the model training and optimization steps, data augmentation strategies are used to expand the training samples online. The data augmentation strategies include geometric transformation and color transformation. The geometric transformation includes at least random horizontal flipping, random rotation, and random cropping. The color transformation includes at least random scaling of brightness, contrast, and saturation.

6. The deep learning-based banknote recognition method according to claim 1, characterized in that, In the model training and optimization steps, a transfer learning strategy is adopted, and the parameters of the banknote recognition neural network are initialized using weights pre-trained on a general large-scale image dataset.

7. The deep learning-based banknote recognition method according to claim 1, characterized in that, The reasoning, recognition, and result output steps also include a step of aggregating multiple recognition results: multiple images to be recognized or multiple batches of recognition results are grouped according to the banknote inventory unit code and the front and back identification, and the confidence scores of each recognition result within the group are summed or averaged to obtain an aggregate score, and the final sorting and output are performed based on the aggregate score.

8. A system (100) for a banknote recognition method based on deep learning as described in any one of claims 1-7, characterized in that, include: The data construction module (110) is used to perform the data acquisition and standardization annotation steps to construct the training dataset; The network construction module (120) is used to perform the deep neural network construction steps to construct a banknote recognition neural network; The model training module (130) is used to perform the model training and optimization steps, train the banknote recognition neural network, and obtain a trained banknote recognition model. The recognition output module (140) is used to perform the steps of reasoning recognition and result output, inputting the banknote image to be recognized into the trained banknote recognition model and outputting the recognition result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of a deep learning-based banknote recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of a deep learning-based banknote recognition method as described in any one of claims 1 to 7.

Images