Banknote defect detection method and device, storage medium and electronic equipment

Abstract

The invention provides a banknote defect detection method and device, a storage medium and electronic equipment, and is applied to the technical field of image detection and recognition. According to the invention, the paper money defect picture training data set with multiple defect type labels is constructed, and the prior frame suitable for detection is generated through clustering by using the K-means algorithm, so that the basic capability of the model is improved. The number of convolution layers is increased and a CBAM-PANet structure is introduced in YOLOv4, so that the complex feature recognition and attention mechanism is enhanced, the detection accuracy and efficiency are remarkably improved, the generalization ability for different banknote defects is enhanced, and efficient and accurate defect type judgment is realized.

Description

Technical Field

[0001] The present invention relates to the field of image detection and recognition technology, and in particular to a banknote defect detection method, device, storage medium and electronic equipment. Background Art

[0002] In the financial industry, banknotes are inevitably damaged and soiled during circulation. Every day, financial institutions need to exchange a large number of damaged and soiled RMB. These banknotes need to be manually identified for defects and classified, and then handed over to the People's Bank of China for destruction. However, relying on manual experience to judge banknote defects is not only inefficient, but also long-term visual operation can easily lead to eye fatigue, which in turn causes misjudgment.

[0003] Traditional image detection methods rely on artificially designed feature extractors to identify target features in images, which requires R&D personnel to have deep knowledge of image processing and to design them independently for different types of defects. The defects on the surface of banknotes are diverse in form and complex in background, which makes traditional image processing technology face challenges in detecting various defect targets, and it is difficult to achieve ideal detection results through conventional feature extraction methods.

[0004] Therefore, how to improve the efficiency and accuracy of banknote defect detection has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of the above problems, the present invention provides a banknote defect detection method, device, storage medium and electronic device that overcome the above problems or at least partially solve the above problems. The technical solution is as follows:

[0006] A banknote defect detection method, comprising:

[0007] Obtain a banknote defect image training data set, wherein the banknote defect image training data set includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels;

[0008] Clustering the banknote defect image training data set using a K-means algorithm to regenerate a priori boxes suitable for banknote defect detection;

[0009] Based on the prior frame, the number of convolutional layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm is increased;

[0010] Adding the CBAM-PANet neck network structure to the detection network to obtain a banknote defect detection model;

[0011] The image of the banknote to be detected is input into the banknote defect detection model to obtain the banknote defect type of the banknote to be detected output by the banknote defect detection model.

[0012] Optionally, obtaining a banknote defect image training dataset includes:

[0013] Obtaining an original data set of banknote defect images, wherein the original data set of banknote defect images includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels;

[0014] Offline data enhancement is performed on the original data set of banknote defect images to obtain a training data set of banknote defect images.

[0015] Optionally, clustering the banknote defect image training data set using a K-means algorithm to regenerate a priori boxes suitable for banknote defect detection includes:

[0016] Obtaining the annotation data of each of the banknote defect images in the banknote defect image training data set, wherein the annotation data includes the center point coordinates and width and height information of the banknote defect image;

[0017] In the banknote defect image training data set, randomly selecting a preset number of the annotation data of the banknote defect images as current frame parameters of the prior frame;

[0018] Using the current frame parameters, calculating the distance between each of the prior frames and the defect marking frames of each banknote defect type, and classifying the defect marking frame as the center of the prior frame with the smallest distance therefrom;

[0019] Calculate the average width and height of the multiple prior frames classified by the center of each prior frame as the current frame parameter, return to execute the step of using the current frame parameters to calculate the distance between each prior frame and the defect marking frame of each banknote defect type, and classify the defect marking frame as the prior frame center with the smallest distance thereto, until the size change of the prior frame is lower than a preset threshold or no longer changes.

[0020] Optionally, based on the prior frame, increasing the number of convolutional layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm includes:

[0021] Based on the prior box, the number of convolutional layers between the output of the CSPDarket53 module and the PANet module is increased from 1 to 3, and the convolutional kernel sizes used are 1×1, 3×3, and 1×1 respectively;

[0022] Based on the prior frame, the number of convolutional layers on the left and right of the SSP module is increased to 5, and the convolutional kernel sizes used are 1×1, 3×3, 1×1, 3×3, and 1×1, respectively.

[0023] Optionally, the CBAM-PANet neck network structure includes three attention modules, each of which includes a channel attention module and a spatial attention module.

[0024] Optionally, the channel attention feature calculation formula of the channel attention module is:

[0025] M C (F)=σ(MLP(AvgPool(F)+MLP(MaxPool(F)))

[0026] Among them, M C (F) represents the channel attention feature map of the input feature map F after the channel attention mechanism, σ is the activation function; MLP represents multi-layer perceptron; AvgPool represents the global average pooling operation; MaxPool represents the global maximum pooling operation;

[0027] The spatial attention feature calculation formula of the spatial attention module is:

[0028] M S (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))

[0029] Among them, M S (F) represents the spatial attention feature map output after the input feature map F passes through the spatial attention mechanism; σ is the activation function; f 7×7 Indicates a convolution operation using a 7×7 convolution kernel; [AvgPool(F); MaxPool(F)] indicates channel concatenation of the pooling results after performing global average pooling operation AvgPool and global maximum pooling operation MaxPool on the input feature map F.

[0030] Optionally, the banknote defect types include surface defects, cracks and blurred patterns.

[0031] A banknote defect detection device comprises: a training data set acquisition unit, a priori frame generation unit, a convolution layer number increase unit, a banknote defect detection model acquisition unit and a banknote defect detection unit.

[0032] The training data set acquisition unit is used to acquire a banknote defect image training data set, wherein the banknote defect image training data set includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels;

[0033] The prior frame generating unit is used to cluster the banknote defect image training data set using a K-means algorithm to regenerate a prior frame suitable for banknote defect detection;

[0034] The convolution layer number increasing unit is used to increase the number of convolution layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm based on the priori frame;

[0035] The banknote defect detection model acquisition unit is used to add the CBAM-PANet neck network structure to the detection network to obtain the banknote defect detection model;

[0036] The banknote defect detection unit is used to input the image of the banknote to be detected into the banknote defect detection model to obtain the banknote defect type of the banknote to be detected output by the banknote defect detection model.

[0037] A computer-readable storage medium stores a program, wherein when the program is executed by a processor, the banknote defect detection method described in any one of the above items is implemented.

[0038] An electronic device, comprising at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other via the bus; and the processor is used to call program instructions in the memory to execute any of the above-mentioned banknote defect detection methods.

[0039] By means of the above technical scheme, the present invention provides a method, device, storage medium and electronic device for detecting banknote defects, obtains a training data set of banknote defect images, wherein the training data set of banknote defect images includes multiple banknote defect images, and the banknote defect images are annotated with banknote defect type labels; clusters the training data set of banknote defect images using the K-means algorithm, and regenerates a priori boxes suitable for banknote defect detection; based on the priori boxes, increases the number of convolutional layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm; adds the CBAM-PANet neck network structure to the detection network to obtain a banknote defect detection model; inputs the banknote image to be detected into the banknote defect detection model, and obtains the banknote defect type of the banknote to be detected output by the banknote defect detection model. The present invention can effectively generate a priori boxes suitable for banknote defect detection by constructing a training data set of banknote defect images containing multiple banknote defect type labels, and clustering the data set using the K-means algorithm. These priori boxes can more accurately match the characteristics of banknote defects and improve the basic capabilities of the detection model. On this basis, by increasing the number of convolutional layers between CSPDarknet53, PANet and SSP modules in the detection network of the YOLOv4 algorithm, and introducing the CBAM-PANet neck network structure, the model's recognition ability and attention mechanism for complex features are enhanced. This series of optimizations not only improves the accuracy and detection efficiency of the model, but also enhances the model's generalization ability for different types of banknote defects, so that after inputting the banknote image to be detected, it can efficiently output accurate defect type judgments.

[0040] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0042] Figure 1 A schematic diagram showing a flow chart of an implementation of a banknote defect detection method provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of a model framework of the YOLOv4 algorithm provided in an embodiment of the present invention is shown;

[0044] Figure 3 A schematic flow chart showing another implementation of a banknote defect detection method provided by an embodiment of the present invention;

[0045] Figure 4 A schematic diagram showing the structure of a banknote defect detection device provided by an embodiment of the present invention is shown;

[0046] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0047] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0048] In recent years, deep learning technology has developed rapidly in the fields of image processing and big data analysis. Compared with traditional methods, deep learning has the ability to automatically extract and learn the features of defective targets in banknote images, greatly reducing the cost of feature extraction, while having good detection accuracy and generalization ability. At present, the target detection model based on convolutional neural network (CNN) has been widely used in defect detection, video surveillance and image analysis, and has achieved remarkable success.

[0049] In the prior art, the method of comparing the grayscale value with the preset grayscale threshold is often used to obtain the defective area of ​​the banknote. This method belongs to the category of traditional image processing, has a single detection type, and can only identify the defect type of the banknote. In addition, the grayscale value is easily affected by the changes in the background environment of the banknote, and it is difficult to determine the appropriate grayscale threshold. Therefore, it has high requirements for the detection environment and lacks universality.

[0050] The present invention constructs a banknote defect dataset and trains a banknote defect detection model based on the improved YOLOv4 algorithm to perform defect detection. Combining the structural characteristics of the YOLOv4 algorithm and the characteristics of banknote defects, the prior frame size and network structure are optimized, and the attention mechanism is introduced to achieve efficient and intelligent detection of banknote defects. This method solves the problems of low efficiency and poor robustness in manual detection and traditional image processing methods, and is of great significance to the automation and intelligent development of the financial industry.

[0051] like Figure 1 As shown, a flow chart of an implementation of a banknote defect detection method provided by an embodiment of the present invention is provided, and the method may include:

[0052] S100. Obtain a banknote defect image training data set, wherein the banknote defect image training data set includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels.

[0053] The Banknote Defect Image Training Dataset is a data set used to train machine learning models specifically for identifying defects on banknotes. This dataset contains a large number of images, each showing a different type of banknote defect. This data is used to teach the model how to identify and classify various defects so that it can automatically detect and judge the quality of banknotes in practical applications.

[0054] Among them, banknote defect images refer to each image in the dataset that contains banknote defects. Each image shows one or more defects of the banknote. Banknote defect images are the basic component of the training dataset, providing visual information for model learning.

[0055] The banknote defect type label is a tag attached to each banknote defect image that describes the specific defect type shown in the image. For example, a label might indicate that the banknote in the image is stained or torn. The banknote defect type label provides the model with the correct information needed for supervised learning, helping it learn how to identify and classify different defect types.

[0056] Optionally, banknote defect types include surface defects, cracks and blurred patterns.

[0057] S110. Clustering the paper money defect image training data set using a K-means algorithm to regenerate a priori boxes suitable for paper money defect detection.

[0058] Among them, the K-means algorithm is an unsupervised learning clustering algorithm, which is mainly used to divide the samples in the data set into K clusters, each cluster is represented by a center point (centroid). The goal of the algorithm is to minimize the distance between each sample and the centroid of its corresponding cluster through iterative optimization. The basic steps of K-means include initializing K centroids, assigning each sample to the nearest centroid, and then updating the centroid position, and repeating the above steps until convergence.

[0059] Among them, the prior box is a set of rectangular boxes pre-defined in the target detection task to help the detection model identify the position and size of the target in the image. During the training process, the model adjusts the prediction according to the overlap between the actual target bounding box and these prior boxes to optimize the target detection results. By clustering the real boxes of the training data through the K-means algorithm, a set of prior boxes that are more suitable for banknote defect images can be generated. These boxes are closer to the distribution of actual data, thereby improving the performance and accuracy of the detection model.

[0060] S120. Based on the prior frame, increase the number of convolutional layers between the CSPDarket53 module, PANet module and SSP module in the detection network of the YOLOv4 algorithm.

[0061] Among them, the YOLOv4 (You Only Look Once version 4) algorithm is a real-time detection algorithm in the field of target detection. It is designed to provide high-precision and high-speed target detection capabilities. The detection network of YOLOv4 consists of multiple modules, which are responsible for image feature extraction, enhancement and target detection.

[0062] Among them, the CSPDarknet53 module is the backbone network of YOLOv4, which is used for feature extraction. CSPDarknet53 is an improved version of Darknet53, in which the CSP (Cross Stage Partial) structure is used to reduce the computational burden of the model and enhance the feature fusion capability. This module extracts multi-scale features of the image through cascaded convolutional layers to improve the accuracy and efficiency of detection.

[0063] The PANet (Path Aggregation Network) module is part of YOLOv4 and is used to enhance the feature pyramid network (FPN) structure. It improves the upstream and downstream information flow of features at different levels by introducing a path aggregation mechanism. PANet helps the detection network better handle target objects with different scales and complex backgrounds.

[0064] Among them, the SSP (Spatial Pyramid Pooling) module is used for feature enhancement in YOLOv4. It enriches feature expression by performing pooling operations at different scales and segmenting feature maps at different scales. This multi-scale processing method helps to improve the network's detection ability for objects of different sizes, so that the spatial information of the object can be better captured without increasing the amount of calculation.

[0065] The embodiment of the present invention can further enhance the capability of the YOLOv4 detection network by increasing the number of convolutional layers between these modules, thereby improving its recognition and detection performance of banknote defects in complex scenarios.

[0066] S130. Add the CBAM-PANet neck network structure to the detection network to obtain a banknote defect detection model.

[0067] Among them, CBAM (Convolutional Block Attention Module) is an attention mechanism module that aims to improve the ability of neural networks to pay attention to different features. It enhances feature representation capabilities through attention mechanisms in two dimensions: channel and space. CBAM first performs channel attention and then spatial attention, which allows the network to better focus on important features, thereby improving overall performance.

[0068] Among them, PANet (Path Aggregation Network) is a network structure for feature fusion, which is particularly suitable for use in target detection tasks. It introduces a path aggregation mechanism to enhance the information flow between different levels in the Feature Pyramid Network (FPN), helping the network to more effectively utilize multi-scale features to improve detection performance.

[0069] In the detection network, CBAM and PANet are combined to form the CBAM-PANet neck structure, which aims to further improve the feature fusion and feature enhancement capabilities. CBAM helps the model better focus on specific features, while PANet ensures that these features are effectively integrated and utilized at different scales. The combined structure can improve the model's detection capabilities for complex backgrounds and multi-scale targets.

[0070] Among them, the banknote defect detection model is a deep learning model specifically used to identify and detect surface defects on banknotes. Its goal is to automatically identify various defects on banknotes, such as cracks, stains, fading, and blurred patterns. The model extracts image features through a convolutional neural network (CNN) and processes these features using a detection network structure to identify potential defects. By introducing the CBAM-PANet neck structure, the model can significantly improve the ability to recognize complex textures and subtle defects on banknotes, providing higher accuracy and reliability.

[0071] In order to facilitate the understanding of the YOLOv4 algorithm provided in the embodiment of the present invention, Figure 2 To explain: Figure 2The model framework of the YOLOv4 algorithm is shown in the figure. In the YOLOv4 algorithm, the sample image first passes through the backbone network CSPDarknet53 to extract the feature data of banknote defects. Next, the SPP and PANet modules are used for feature fusion and enhancement to generate feature maps of three sizes: 19×19, 38×38 and 76×76, so as to detect and classify large, medium and small targets in the classification regression layer (YOLOv4 Head). Based on the original YOLOv4, the present invention first re-clustered through the K-means algorithm to obtain a priori boxes that are more suitable for banknote defect detection, thereby improving the matching effect between feature layers. Secondly, the number of convolutional layers between the CSPDarknet53, PANet and SSP modules is increased to enhance the model's ability to extract defect features. Finally, the attention module is introduced and the CBAM-PANet neck network structure is proposed, which further improves the accuracy of the model in detecting defects such as banknote surface defects, cracks and pattern blur.

[0072] S140, inputting the image of the banknote to be detected into the banknote defect detection model, and obtaining the banknote defect type of the banknote to be detected output by the banknote defect detection model.

[0073] The banknote image to be detected may be a banknote image obtained by a scanner, a camera or other image acquisition equipment.

[0074] After the image of the banknote to be inspected is input into the banknote defect detection model, the model will analyze the features in the image to detect whether there are defects on the banknote. The model output results include the banknote defect type of the banknote to be inspected.

[0075] The present invention provides a method for detecting banknote defects, the method comprising: obtaining a training data set of banknote defect images, wherein the training data set of banknote defect images includes a plurality of banknote defect images, and the banknote defect images are annotated with a banknote defect type label; clustering the training data set of banknote defect images using a K-means algorithm, and regenerating a priori frames suitable for banknote defect detection; based on the priori frames, increasing the number of convolutional layers between the CSPDarket53 module, the PANet module, and the SSP module in the detection network of the YOLOv4 algorithm; adding a CBAM-PANet neck network structure to the detection network to obtain a banknote defect detection model; inputting a banknote image to be detected into the banknote defect detection model, and obtaining the banknote defect type of the banknote to be detected output by the banknote defect detection model. The present invention can effectively generate a priori frames suitable for banknote defect detection by constructing a training data set of banknote defect images containing a plurality of banknote defect type labels, and clustering the data set using a K-means algorithm. These priori frames can more accurately match the characteristics of banknote defects and improve the basic capabilities of the detection model. On this basis, by increasing the number of convolutional layers between CSPDarknet53, PANet and SSP modules in the detection network of the YOLOv4 algorithm, and introducing the CBAM-PANet neck network structure, the model's recognition ability and attention mechanism for complex features are enhanced. This series of optimizations not only improves the accuracy and detection efficiency of the model, but also enhances the model's generalization ability for different types of banknote defects, so that after inputting the banknote image to be detected, it can efficiently output accurate defect type judgments.

[0076] Optional, based on Figure 1 The method shown, such as Figure 3 As shown, a flow chart of another implementation of the banknote defect detection method provided by an embodiment of the present invention is shown, and step S100 may include:

[0077] S300, obtaining an original data set of banknote defect images, wherein the original data set of banknote defect images includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels;

[0078] S310, performing offline data enhancement on the original data set of banknote defect images to obtain a training data set of banknote defect images.

[0079] Specifically, the embodiment of the present invention can obtain banknote defect pictures through an image acquisition device, and these pictures show various defects on banknotes. Use LabelImg software to annotate these banknote defect pictures, and affix a banknote defect type label to each picture to indicate its specific defect type. The annotated pictures constitute a banknote defect picture training data set. Subsequently, the data set is operated by an offline data enhancement method, including flipping, scaling, translation and adding noise, to further increase the data volume and diversity. The final data set contains banknote defect pictures annotated with banknote defect type labels, mainly covering three defect conditions: banknote surface defects, cracks and blurred patterns.

[0080] In the banknote defect detection, the embodiment of the present invention can provide reliable basic data for model training by obtaining the original data set of banknote defect images and ensuring that these images have been correctly labeled with defect type labels. Then, the original data set is enhanced offline to expand the diversity of the data set by methods such as rotation, flipping, scaling, and adding noise. By constructing a high-quality data set, it is not only possible to resist the overfitting problem, but also to improve the generalization ability of the model, making it more accurate in identifying banknote defects under different environments and conditions, thereby significantly improving the performance and robustness of the banknote defect detection model.

[0081] Optional, in the above Figure 1 On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, a K-means algorithm is used to cluster a banknote defect image training data set to regenerate a priori frames suitable for banknote defect detection, which may specifically include:

[0082] Obtain the annotation data of each banknote defect image in the banknote defect image training data set, wherein the annotation data includes the center point coordinates and width and height information of the banknote defect image. In the banknote defect image training data set, randomly select the annotation data of a preset number of banknote defect images as the current frame parameters of the prior frame. Using the current frame parameters, calculate the distance between each prior frame and the defect annotation frame of each banknote defect type, and classify the defect annotation frame as the prior frame center with the smallest distance therefrom. Calculate the average width and height of the multiple prior frames classified by each prior frame center as the current frame parameter, and return to execute the step of calculating the distance between each prior frame and the defect annotation frame of each banknote defect type using the current frame parameters, and classify the defect annotation frame as the prior frame center with the smallest distance therefrom, until the size change of the prior frame is lower than the preset threshold or no longer changes.

[0083] In banknote defect detection, it is necessary to use a self-made data set to re-cluster the prior frames to improve the detection accuracy and adaptability of the model. The clustering process based on the banknote surface defect set includes: extracting the annotation information of each defect target from the data set, including the center point coordinates and width and height information. In the acquired annotation data, 9 sets of values ​​are randomly selected as the initial prior frame parameters. These initial parameters will be used to start the iterative clustering process. For each prior frame, calculate its distance from all defect annotation frames. Classify each defect annotation frame to the nearest prior frame, and record the set of defect annotation frames corresponding to each prior frame. After all annotation frames are classified, calculate the average width and height of multiple annotation frames corresponding to the center of each prior frame, and update them as new prior frame parameters. Repeat the above steps of calculating distance, classifying and updating prior frame parameters, and continue to update the prior frame parameters until the change is lower than the preset threshold or no longer changes. Finally, output the result, that is, get a prior frame that is more suitable for banknote defect detection tasks.

[0084] The embodiment of the present invention uses the K-means algorithm to cluster the banknote defect image training data set, and can generate a priori boxes that are more suitable for the banknote defect characteristics, thereby improving the detection performance of the model on small-sized and long-strip defects.

[0085] In order to more effectively detect defects on the surface of banknotes and improve prediction accuracy under complex background patterns, the embodiments of the present invention improve the YOLOv4 network structure. These improvements are intended to allow the detection network to better learn detailed local information while combining global features to avoid false detection and missed detection.

[0086] Optional, in the above Figure 1 On the basis of one or more corresponding embodiments, in another optional embodiment provided by an embodiment of the present invention, based on the prior frame, increasing the number of convolutional layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm, specifically may include:

[0087] Based on the prior box, the number of convolutional layers between the output of the CSPDarket53 module and the PANet module is increased from 1 to 3, and the convolutional kernel sizes used are 1×1, 3×3, and 1×1, respectively.

[0088] Specifically, the embodiment of the present invention can increase the number of convolutional layers between the output of the CSPDarket53 module and the PANet module from 1 to 3. The convolution kernel sizes of these convolutional layers are 1×1, 3×3, and 1×1, respectively. This adjustment helps to refine feature processing and improve the ability to capture local information.

[0089] Based on the prior frame, the number of convolutional layers on the left and right of the SSP module is increased to 5, and the convolutional kernel sizes used are 1×1, 3×3, 1×1, 3×3, and 1×1, respectively.

[0090] Specifically, in the embodiment of the present invention, the number of convolutional layers on both sides of the SSP structure is increased to 5. The convolution kernel sizes on each side are 1×1, 3×3, 1×1, 3×3, and 1×1, respectively. By increasing the number of convolutional layers and adjusting the convolution kernel size, the model can better integrate local and global information and improve the depth and breadth of feature extraction.

[0091] The embodiments of the present invention not only expand the receptive field by increasing the number of convolutional layers and adjusting the size of the convolution kernel, so that the model can more accurately identify complex defects on banknotes, but also increase the ability to understand feature semantics, ultimately improving the detection accuracy and robustness of banknote surface defects.

[0092] Optional, in the above Figure 1 On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, the CBAM-PANet neck network structure includes three attention modules, each of which includes a channel attention module and a spatial attention module.

[0093] The embodiment of the present invention improves the PANet network of YOLOv4 and proposes a new CBAM-PANet neck network structure to solve the scale mismatch problem that may occur in the process of multi-layer semantic feature fusion. This structure introduces three attention modules between the backbone network and the neck network. Each attention module consists of a channel attention module and a spatial attention module.

[0094] These CBAM (Convolutional Block Attention Module) attention modules can learn the weights of different feature output layers and dynamically enhance the effect of feature fusion, thereby improving the accuracy of banknote defect detection. By combining channel and spatial attention mechanisms, CBAM enables the model to focus more on the more important parts of the input data, not just the entire image.

[0095] The Channel Attention Module (CAM) generates channel attention features by performing global average pooling and global maximum pooling on the input feature map, and then sigmoid activation. These features are multiplied with the input and passed to the Spatial Attention Module (SAM). SAM also performs global pooling on the input, generates spatial attention features through convolution and sigmoid layers, and then combines them with the input features. This mechanism enables the model to capture channel and spatial relationships more effectively, thereby improving detection performance.

[0096] The embodiment of the present invention introduces three attention modules, each of which includes a channel attention module and a spatial attention module, which effectively integrates the channel attention mechanism and the spatial attention mechanism, so that CBAM-PANet can more accurately capture the complex defect features on banknotes and improve the accuracy and robustness of detection.

[0097] Optionally, the channel attention feature calculation formula of the channel attention module is:

[0098] M C (F)=σ(MLP(AvgPool(F)+MLP(MaxPool(F)))

[0099] Among them, M C (F) represents the channel attention feature map of the input feature map F after the channel attention mechanism, σ is the activation function; MLP represents multi-layer perceptron; AvgPool represents the global average pooling operation; MaxPool represents the global maximum pooling operation.

[0100] The calculation formula of the spatial attention feature of the spatial attention module is:

[0101] M S (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))

[0102] Among them, M S (F) represents the spatial attention feature map output after the input feature map F passes through the spatial attention mechanism; σ is the activation function; f 7×7 Indicates a convolution operation using a 7×7 convolution kernel; [AvgPool(F); MaxPool(F)] indicates channel concatenation of the pooling results after performing global average pooling operation AvgPool and global maximum pooling operation MaxPool on the input feature map F.

[0103] Although operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or in a sequential order.Multitasking and parallel processing may be advantageous under certain circumstances.

[0104] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0105] Corresponding to the above method embodiment, the present invention also provides a banknote defect detection device, the structure of which is as follows: Figure 4 As shown, it includes: a training data set obtaining unit 10, a priori frame generating unit 20, a convolution layer number increasing unit 30, a banknote defect detection model obtaining unit 40 and a banknote defect detection unit 50.

[0106] The training data set obtaining unit 10 is used to obtain a banknote defect image training data set, wherein the banknote defect image training data set includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels.

[0107] The prior frame generating unit 20 is used to cluster the paper money defect image training data set using the K-means algorithm to regenerate a prior frame suitable for paper money defect detection.

[0108] A convolution layer number increasing unit 30 is used to increase the number of convolution layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm based on the prior frame;

[0109] The banknote defect detection model obtaining unit 40 is used to add the CBAM-PANet neck network structure into the detection network to obtain the banknote defect detection model.

[0110] The banknote defect detection unit 50 is used to input the image of the banknote to be detected into the banknote defect detection model, and obtain the banknote defect type of the banknote to be detected output by the banknote defect detection model.

[0111] Optionally, the training data set acquisition unit 10 is specifically used to obtain an original data set of banknote defect images, wherein the original data set of banknote defect images includes multiple banknote defect images, and the banknote defect images are annotated with banknote defect type labels; offline data enhancement is performed on the original data set of banknote defect images to obtain a banknote defect image training data set.

[0112] Optionally, the prior frame generation unit 20 is specifically used to obtain the annotation data of each banknote defect image in the banknote defect image training data set, wherein the annotation data includes the center point coordinates and width and height information of the banknote defect image; in the banknote defect image training data set, randomly select the annotation data of a preset number of banknote defect images as the current frame parameters of the prior frame; use the current frame parameters to calculate the distance between each prior frame and the defect annotation frame of each banknote defect type, and classify the defect annotation frame as the prior frame center with the smallest distance therefrom; calculate the average width and height of multiple prior frames classified by each prior frame center as the current frame parameter, return to execute the calculation of the distance between each prior frame and the defect annotation frame of each banknote defect type using the current frame parameters, and classify the defect annotation frame as the prior frame center with the smallest distance therefrom, until the size change of the prior frame is lower than a preset threshold or no longer changes.

[0113] Optionally, the convolution layer increasing unit 30 is specifically used to increase the number of convolution layers between the output of the CSPDarket53 module and the PANet module from 1 to 3 based on the prior frame, and the convolution kernel sizes used are 1×1, 3×3 and 1×1 respectively; based on the prior frame, the number of convolution layers on the left and right of the SSP module is increased to 5 layers, and the convolution kernel sizes used are 1×1, 3×3, 1×1, 3×3, and 1×1 respectively.

[0114] Optionally, the CBAM-PANet neck network structure includes three attention modules, each of which includes a channel attention module and a spatial attention module.

[0115] Optionally, the channel attention feature calculation formula of the channel attention module is:

[0116] M C (F)=σ(MLP(AvgPool(F)+MLP(MaxPool(F)))

[0117] Among them, M C (F) represents the channel attention feature map of the input feature map F after the channel attention mechanism, σ is the activation function; MLP represents multi-layer perceptron; AvgPool represents the global average pooling operation; MaxPool represents the global maximum pooling operation.

[0118] The calculation formula of the spatial attention feature of the spatial attention module is:

[0119] M S (F) = σ(f 7×7 ([AvgPool(F); MaxPool(F)]))

[0120] Among them, M S(F) represents the spatial attention feature map output after the input feature map F passes through the spatial attention mechanism; σ is the activation function; f 7×7 Indicates a convolution operation using a 7×7 convolution kernel; [AvgPool(F); MaxPool(F)] indicates channel concatenation of the pooling results after performing global average pooling operation AvgPool and global maximum pooling operation MaxPool on the input feature map F.

[0121] Optionally, banknote defect types include surface defects, cracks and blurred patterns.

[0122] The present invention provides a banknote defect detection device, which is used to: obtain a banknote defect image training data set, wherein the banknote defect image training data set includes multiple banknote defect images, and the banknote defect images are annotated with banknote defect type labels; cluster the banknote defect image training data set using the K-means algorithm to regenerate a priori boxes suitable for banknote defect detection; based on the priori boxes, increase the number of convolution layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm; add the CBAM-PANet neck network structure to the detection network to obtain a banknote defect detection model; input the banknote image to be detected into the banknote defect detection model to obtain the banknote defect type of the banknote to be detected output by the banknote defect detection model. The present invention can effectively generate a priori boxes suitable for banknote defect detection by constructing a banknote defect image training data set containing multiple banknote defect type labels and clustering the data set using the K-means algorithm. These priori boxes can more accurately match the characteristics of banknote defects and improve the basic capabilities of the detection model. On this basis, by increasing the number of convolutional layers between CSPDarknet53, PANet and SSP modules in the detection network of the YOLOv4 algorithm, and introducing the CBAM-PANet neck network structure, the model's recognition ability and attention mechanism for complex features are enhanced. This series of optimizations not only improves the accuracy and detection efficiency of the model, but also enhances the model's generalization ability for different types of banknote defects, so that after inputting the banknote image to be detected, it can efficiently output accurate defect type judgments.

[0123] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0124] The banknote defect detection device includes a processor and a memory. The above-mentioned training data set acquisition unit 10, prior frame generation unit 20, convolution layer number increase unit 30, banknote defect detection model acquisition unit 40 and banknote defect detection unit 50 are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0125] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set. By adjusting the kernel parameters, a training dataset of banknote defect images with multiple defect type labels is constructed, and the K-means algorithm is used to cluster and generate a priori boxes suitable for detection, which improves the basic capabilities of the model. Increasing the number of convolutional layers and introducing the CBAM-PANet structure in YOLOv4 enhances complex feature recognition and attention mechanisms, significantly improves detection accuracy and efficiency, and enhances the generalization ability of different banknote defects, achieving efficient and accurate defect type judgment.

[0126] An embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the banknote defect detection method is implemented.

[0127] An embodiment of the present invention provides a processor, which is used to run a program, wherein the banknote defect detection method is executed when the program is running.

[0128] like Figure 5 As shown, an embodiment of the present invention provides an electronic device 1000, which includes at least one processor 1001, at least one memory 1002 connected to the processor 1001, and a bus 1003; wherein the processor 1001 and the memory 1002 communicate with each other through the bus 1003; the processor 1001 is used to call the program instructions in the memory 1002 to execute the above-mentioned banknote defect detection method. The electronic device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0129] The present invention also provides a computer program product, which, when executed on an electronic device, is suitable for executing a program that initializes the steps of the banknote defect detection method.

[0130] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, electronic devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] In a typical configuration, an electronic device includes one or more processors (CPU), a memory, and a bus. The electronic device may also include an input / output interface, a network interface, and the like.

[0132] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0133] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0134] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0135] In the description of the present invention, it should be understood that the terms "up", "down", "front", "back", "left" and "right" etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the positions or elements referred to must have specific directions, be constructed and operate in specific directions. Therefore, they should not be understood as limitations of the present invention.

[0136] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0137] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0138] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A banknote defect detection method, characterized in that: include: Obtain a banknote defect image training data set, wherein the banknote defect image training data set includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels; Clustering the banknote defect image training data set using a K-means algorithm to regenerate a priori boxes suitable for banknote defect detection; Based on the prior frame, the number of convolutional layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm is increased; Adding the CBAM-PANet neck network structure to the detection network to obtain a banknote defect detection model; The image of the banknote to be detected is input into the banknote defect detection model to obtain the banknote defect type of the banknote to be detected output by the banknote defect detection model.

2. The method according to claim 1, characterized in that The step of obtaining a banknote defect image training data set comprises: Obtaining an original data set of banknote defect images, wherein the original data set of banknote defect images includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels; Offline data enhancement is performed on the original data set of banknote defect images to obtain a training data set of banknote defect images.

3. The method according to claim 1, characterized in that The method of clustering the banknote defect image training data set using the K-means algorithm to regenerate a priori frames suitable for banknote defect detection includes: Obtaining the annotation data of each of the banknote defect images in the banknote defect image training data set, wherein the annotation data includes the center point coordinates and width and height information of the banknote defect image; In the banknote defect image training data set, randomly selecting a preset number of the annotation data of the banknote defect images as current frame parameters of the prior frame; Using the current frame parameters, calculating the distance between each of the prior frames and the defect marking frames of each banknote defect type, and classifying the defect marking frame as the center of the prior frame with the smallest distance therefrom; Calculate the average width and height of the multiple prior frames classified by the center of each prior frame as the current frame parameter, return to execute the step of using the current frame parameters to calculate the distance between each prior frame and the defect marking frame of each banknote defect type, and classify the defect marking frame as the prior frame center with the smallest distance thereto, until the size change of the prior frame is lower than a preset threshold or no longer changes.

4. The method according to claim 1, characterized in that: The method of increasing the number of convolutional layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm based on the prior frame includes: Based on the prior box, the number of convolutional layers between the output of the CSPDarket53 module and the PANet module is increased from 1 to 3, and the convolutional kernel sizes used are 1×1, 3×3, and 1×1 respectively; Based on the prior frame, the number of convolutional layers on the left and right of the SSP module is increased to 5, and the convolutional kernel sizes used are 1×1, 3×3, 1×1, 3×3, and 1×1, respectively.

5. The method according to claim 1, characterized in that The CBAM-PANet neck network structure includes three attention modules, each of which includes a channel attention module and a spatial attention module.

6. The method according to claim 5, characterized in that The channel attention feature calculation formula of the channel attention module is: M C (F)=σ(MLP(AvgPool(F)+MLP(MaxPool(F))) Among them, M C (F) represents the channel attention feature map of the input feature map F after the channel attention mechanism, σ is the activation function; MLP represents multi-layer perceptron; AvgPool represents the global average pooling operation; MaxPool represents the global maximum pooling operation; The spatial attention feature calculation formula of the spatial attention module is: M S (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)])) Among them, M S (F) represents the spatial attention feature map output after the input feature map F passes through the spatial attention mechanism; σ is the activation function; f 7×7 Indicates a convolution operation using a 7×7 convolution kernel; [AvgPool(F); MaxPool(F)] indicates channel concatenation of the pooling results after performing global average pooling operation AvgPool and global maximum pooling operation MaxPool on the input feature map F.

7. The method according to any one of claims 1 to 6, characterized in that The types of banknote defects include surface defects, cracks and blurred patterns.

8. A banknote defect detection device, characterized in that: include: A training data set acquisition unit, a priori frame generation unit, a convolution layer number increase unit, a banknote defect detection model acquisition unit and a banknote defect detection unit. The training data set acquisition unit is used to acquire a banknote defect image training data set, wherein the banknote defect image training data set includes a plurality of banknote defect images, and the banknote defect images are annotated with banknote defect type labels; The prior frame generating unit is used to cluster the banknote defect image training data set using a K-means algorithm to regenerate a prior frame suitable for banknote defect detection; The convolution layer number increasing unit is used to increase the number of convolution layers between the CSPDarket53 module, the PANet module and the SSP module in the detection network of the YOLOv4 algorithm based on the prior frame; The banknote defect detection model acquisition unit is used to add the CBAM-PANet neck network structure to the detection network to obtain the banknote defect detection model; The banknote defect detection unit is used to input the image of the banknote to be detected into the banknote defect detection model to obtain the banknote defect type of the banknote to be detected output by the banknote defect detection model.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the banknote defect detection method according to any one of claims 1 to 7 is implemented.

10. An electronic device, comprising at least one processor, and at least one memory and a bus connected to the processor; wherein: The processor and the memory communicate with each other via the bus; The processor is used to call the program instructions in the memory to execute the banknote defect detection method according to any one of claims 1 to 7.

Images