Currency recognition method, device, equipment and storage medium

Abstract

The present specification relates to the technical field of artificial intelligence, and provides a currency recognition method, device, equipment and storage medium. The method comprises: acquiring a currency image of a currency to be recognized; preprocessing the currency image of the currency to be recognized; inputting the preprocessed currency image of the currency to be recognized into a preset image feature extraction model to extract a currency image feature vector of the currency to be recognized; recognizing a denomination in the currency image of the currency to be recognized; matching, from a plurality of currency image feature vector libraries indexed by denominations, a currency image feature vector library corresponding to the denomination of the currency to be recognized to serve as a target library; performing similarity comparison between the currency image feature vector of the currency to be recognized and each currency image feature vector in the target library; and determining a currency type of the currency to be recognized according to a similarity comparison result. The embodiments of the present specification can reduce currency recognition cost and improve currency recognition efficiency.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a currency recognition method, apparatus, device, and storage medium. Background Technology

[0002] Financial institutions encounter various currencies when exchanging cash for their clients. With the development of economic globalization, financial institutions are handling more and more currency exchange business. Usually, even experienced account managers (hereinafter referred to as users) can only recognize a dozen or so common and widely circulated currencies. It is very difficult to identify less commonly used currencies. Relying entirely on manual identification carries certain risks and has low efficiency.

[0003] Therefore, with the development of technology, a currency recognition technology based on a deep learning classification model has emerged. Its main steps include: first, constructing a currency database; then, using images from the currency database as training data to train a deep learning classification model (such as VGG-16) to obtain a classification model with currency recognition capabilities (i.e., a currency classification and recognition model); after training the currency classification and recognition model, using this model for inference calculations, inputting a currency image automatically calculates the classification and recognition result. Based on the classification and recognition result, the system retrieves other detailed information about the currency in the currency database, such as sample banknote front and back images, anti-counterfeiting identification methods, and detailed images, for users to use for anti-counterfeiting verification.

[0004] However, countries and regions issue new currencies periodically. Each time a currency changes, the classification and recognition model needs to be retrained to recognize the latest currency. This makes currency recognition based on deep learning classification models costly. Furthermore, there are numerous countries and regions with independent currencies globally, and a large number of issuance batches, currencies, and denominations. This results in a large number of categories that the classification model needs to classify, making it difficult to guarantee overall classification and recognition efficiency. Summary of the Invention

[0005] The purpose of the embodiments in this specification is to provide a currency recognition method, apparatus, device, and storage medium to reduce currency recognition costs and improve currency recognition efficiency.

[0006] To achieve the above objectives, in one aspect, embodiments of this specification provide a currency identification method, including:

[0007] Acquire an image of the currency to be identified;

[0008] The currency image of the currency to be identified is preprocessed;

[0009] The preprocessed currency image of the currency to be identified is input into a preset image feature extraction model to extract the currency image feature vector of the currency to be identified;

[0010] Identify the denomination in the currency image of the currency to be identified;

[0011] From multiple currency image feature vector libraries indexed by denomination, match the currency image feature vector library corresponding to the denomination of the currency to be identified, and use it as the target library;

[0012] The similarity of the feature vector of the currency image to be identified with the feature vector of each currency image in the target database is compared.

[0013] The currency to be identified is determined based on the similarity comparison results.

[0014] In the currency recognition method of this specification embodiment, the currency image of the currency to be recognized is preprocessed, including:

[0015] The currency image of the currency to be identified is scaled proportionally to a fixed size;

[0016] The color space of the scaled currency image is converted to RGB color space to obtain the pixel matrix of the currency to be identified in RGB color space.

[0017] In the currency recognition method of this specification, the image feature extraction model includes a pre-trained Siamese network model.

[0018] In the currency recognition method of this specification, the image feature extraction model is pre-trained in the following manner:

[0019] Get tagged currency images for multiple currencies;

[0020] The labeled currency images are preprocessed to form a labeled currency image dataset;

[0021] The labeled currency image dataset was added to the open-source dataset to form a labeled image dataset;

[0022] A Siamese network model is trained based on the labeled image dataset to obtain an image feature extraction model.

[0023] In the currency recognition method of this specification, the loss function used when training the Siamese network model based on the labeled image dataset can be:

[0024]

[0025] Where L represents the loss value of the loss function, N represents the number of samples, Y represents the label indicating whether two samples match, X1 represents the sample input to the first Siamese subnet of the Siamese network model, X2 represents the sample input to the second Siamese subnet of the Siamese network model, and D... w (X1, X2) represents the similarity between X1 and X2, max represents the maximum value function, m represents the set threshold, and P represents the feature dimension of the samples. This represents the j-th sample input to the first twin subnet of the twin network model. This represents the j-th sample input to the second twin subnet of the twin network model.

[0026] In the currency recognition method of the embodiments of this specification, the twin network model includes a twin network model based on ResNetN, where N represents the number of network layers contained in ResNet.

[0027] In the currency recognition method of this specification, the currency image feature vector library is obtained in advance through the following method:

[0028] Obtain images of all currencies and their denominations;

[0029] Each currency image in the currency image set is preprocessed to form a complete currency image set;

[0030] Each currency image in the full set of currency images is input into the image feature extraction model to extract the currency image feature vector of each currency image;

[0031] An index is established for the feature vectors of each currency image, with denomination as the object, thus forming multiple currency image feature vector libraries indexed by denomination.

[0032] In the currency recognition method of this specification, after forming multiple currency image feature vector libraries indexed by denomination, the method further includes:

[0033] When newly issued currency is received, the currency image of the newly issued currency is preprocessed;

[0034] The preprocessed currency image of the newly issued currency is input into the image feature extraction model to extract the currency image feature vector of the newly issued currency;

[0035] Identify the face value of the newly issued currency;

[0036] Based on the face value of the newly issued currency, the currency image feature vector of the newly issued currency is added to the currency image feature vector library with the corresponding index value.

[0037] In the currency recognition method of this specification embodiment, recognizing the denomination in the currency image of the currency to be recognized includes:

[0038] The OCR recognition module identifies the denomination in the currency image of the currency to be identified.

[0039] In the currency recognition method of this specification embodiment, the OCR-based recognition module identifies the denomination in the currency image of the currency to be recognized, including:

[0040] Using a selection box of a preset size, the four corners of the first side image of the currency to be identified and the four corners of the second side image of the currency to be identified are selected by edge-corner alignment, forming eight selection areas;

[0041] Based on OCR, the eight selected regions are identified respectively to obtain multiple character sequences;

[0042] Character sequences that match the denomination characteristics are identified from the plurality of character sequences to form a denomination set;

[0043] The denomination that appears most frequently in the set of denominations is determined, and the denomination that appears most frequently is used as the denomination of the currency image to be identified.

[0044] In the currency identification method of this specification embodiment, determining the currency type of the currency to be identified based on the similarity comparison result includes:

[0045] The multiple similarity values ​​obtained based on the similarity comparison are sorted from approximately smallest.

[0046] The currency type mapped by the currency image feature vector corresponding to the first similarity value in the target library is taken as the currency type to be identified.

[0047] In the currency identification method of this specification embodiment, determining the currency type of the currency to be identified based on the similarity comparison result includes:

[0048] The multiple similarity values ​​obtained based on the similarity comparison are sorted from approximately smallest.

[0049] The currencies mapped by the currency image feature vectors corresponding to the top K similarity values ​​in the target library in the sorted order are used as candidate currencies for users to select.

[0050] On the other hand, embodiments of this specification also provide a currency recognition device, including:

[0051] The image acquisition module is used to acquire images of the currency to be identified.

[0052] The preprocessing module is used to preprocess the currency image of the currency to be identified;

[0053] The feature extraction module is used to input the preprocessed currency image of the currency to be identified into a preset image feature extraction model in order to extract the currency image feature vector of the currency to be identified.

[0054] A denomination recognition module is used to identify the denomination in the currency image of the currency to be identified;

[0055] The feature matching module is used to match the currency image feature vector library corresponding to the denomination of the currency to be identified from multiple currency image feature vector libraries indexed by denomination, and use it as the target library;

[0056] The feature comparison module is used to compare the feature vector of the currency image to be identified with the feature vector of each currency image in the target database.

[0057] The currency determination module is used to determine the currency of the currency to be identified based on the similarity comparison results.

[0058] On the other hand, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when run by the processor, executes instructions for the above-described method.

[0059] On the other hand, embodiments of this specification also provide a computer storage medium storing a computer program thereon, which, when run by the processor of a computer device, executes instructions for the above-described method.

[0060] On the other hand, embodiments of this specification also provide a computer program product, which includes a computer program that, when run by the processor of a computer device, executes instructions for the above-described method.

[0061] As can be seen from the technical solutions provided in the embodiments of this specification above, in these embodiments, a pre-trained image feature extraction model extracts the currency image feature vector of the currency to be identified. Then, the currency image feature vector of the currency to be identified is compared with the currency image feature vectors in a known currency image feature vector library to determine the currency type. This transforms the currency identification problem into an image similarity retrieval problem. Thus, whenever there is a currency change, the currency image feature vector library can be maintained as needed without retraining the model, significantly reducing the cost of currency identification. Furthermore, since the currency image feature vector library is pre-built using denominations as indexes, after identifying the denomination of the currency to be identified, all currency image feature vectors corresponding to that denomination can be matched from the library to form a target library. Then, the currency image feature vector of the currency to be identified is compared with the similarity of each currency image feature vector in the target library. This significantly narrows the currency retrieval range, thereby improving the efficiency of currency identification. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0063] Figure 1 Schematic diagrams of currency recognition systems in some embodiments of this specification are shown;

[0064] Figure 2 Flowcharts of currency recognition methods in some embodiments of this specification are shown;

[0065] Figure 3 It shows Figure 2 The flowchart shown is a preprocessing step in the currency recognition method.

[0066] Figure 4a This specification shows schematic diagrams of the four corners of a currency front image framed in some embodiments of this specification;

[0067] Figure 4b This specification shows schematic diagrams of the four corners of a currency reverse image selected in some embodiments;

[0068] Figure 5 It shows Figure 2 The flowchart shown illustrates the currency identification method for recognizing currency denominations.

[0069] Figure 6 It shows Figure 2 The flowchart shown is a process for model training in the currency recognition method.

[0070] Figure 7 It shows Figure 2 The flowchart shown illustrates the process of generating a currency image feature vector library in the currency recognition method.

[0071] Figure 8 It shows Figure 2 The flowchart illustrates a currency identification method that determines the currency to be identified based on a comparison result.

[0072] Figure 9 It shows Figure 2 The flowchart illustrates another currency identification method that determines the currency to be identified based on comparison results.

[0073] Figure 10 This specification shows structural block diagrams of currency recognition devices in some embodiments;

[0074] Figure 11 A structural block diagram of a computer device in some embodiments of this specification is shown.

[0075] [Explanation of Labels in the Attached Image]

[0076] 10. Client-side;

[0077] 20. Server-side;

[0078] 101. Image acquisition module;

[0079] 102. Preprocessing module;

[0080] 103. Feature Extraction Module;

[0081] 104. Denomination Recognition Module;

[0082] 105. Feature matching module;

[0083] 106. Feature Comparison Module;

[0084] 107. Currency Determination Module;

[0085] 1102. Computer equipment;

[0086] 1104. Processor;

[0087] 1106. Memory;

[0088] 1108. Drive mechanism;

[0089] 1110. Input / output interface;

[0090] 1112. Input devices;

[0091] 1114. Output devices;

[0092] 1116. Presentation device;

[0093] 1118. Graphical User Interface;

[0094] 1120. Network interface;

[0095] 1122. Communication link;

[0096] 1124. Communication bus. Detailed Implementation

[0097] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0098] Figure 1 This specification illustrates a currency recognition system based on some embodiments. The system includes a client 10 and a server 20. The server 20 can acquire a currency image of the currency to be identified from the client 10 and, based on image search, query a database of currency image feature vectors of known currencies to find a matching currency image feature vector, thereby determining the currency type. In some embodiments, the client 10 can be a self-service terminal, a mobile terminal (i.e., a smartphone), a display, a desktop computer, a tablet computer, a laptop computer, a digital assistant, or a smart wearable device. Smart wearable devices may include smart bracelets, smartwatches, smart glasses, or smart helmets. Of course, the client 10 is not limited to the aforementioned physical electronic devices; it can also be software running on such electronic devices. In some embodiments, the server 20 can be an electronic device with computing and network interaction functions; it can also be software running on the electronic device that provides business logic for data processing and network interaction.

[0099] This specification provides a currency recognition method, which can be applied to the server-side described above. (See reference...) Figure 2 As shown, in some embodiments, the currency identification method may include the following steps:

[0100] Step 201: Obtain the currency image of the currency to be identified.

[0101] Step 202: Preprocess the currency image of the currency to be identified.

[0102] Step 203: Input the preprocessed currency image of the currency to be identified into a preset image feature extraction model to extract the currency image feature vector of the currency to be identified.

[0103] Step 204: Identify the denomination in the currency image of the currency to be identified.

[0104] Step 205: Match the currency image feature vector library corresponding to the denomination of the currency to be identified from multiple currency image feature vector libraries indexed by denomination, and use it as the target library.

[0105] Step 206: Compare the similarity of the currency image feature vector of the currency to be identified with the feature vector of each currency image in the target database.

[0106] Step 207: Determine the currency type of the currency to be identified based on the similarity comparison results.

[0107] In this embodiment, a pre-trained image feature extraction model extracts the currency image feature vector of the currency to be identified. Then, the similarity of this feature vector is compared with the currency image feature vectors in a known currency feature vector library to determine the currency type. This transforms the currency identification problem into an image similarity retrieval problem. Thus, whenever there is a currency change, the currency image feature vector library can be maintained as needed without retraining the model, significantly reducing the cost of currency identification. Furthermore, since the currency image feature vector library is pre-built using denominations as indexes, after identifying the denomination of the currency to be identified, all currency image feature vectors corresponding to that denomination can be matched from the library to form a target library. Then, the similarity comparison of the currency image feature vector of the currency to be identified with each currency image feature vector in the target library is performed. This significantly narrows the currency search range, thereby improving the efficiency of currency identification.

[0108] In some embodiments, the server can acquire an image of the currency to be identified through a client. The client may have an image acquisition module. Based on instructions from the server, the client can acquire an image of the currency to be identified (for example, if the client is a mobile app, the app can use the phone's camera to acquire the image of the currency to be identified), and provide the acquired image to the server for processing. It should be noted that the currency image in the embodiments of this specification generally includes both the front and back images of the currency.

[0109] To improve currency recognition efficiency, the acquired images of the currency to be recognized can be preprocessed. (Reference) Figure 3 As shown, in some embodiments, preprocessing of the currency image to be identified may include:

[0110] Step 301: Scale the currency image of the currency to be identified to a fixed size proportionally.

[0111] Due to factors such as acquisition equipment and environment, the currency images obtained by the server may vary in size. Image feature extraction models are generally suitable for currency images of a fixed size (e.g., image feature extraction models require a fixed image size when performing convolution operations). Therefore, the currency image can be scaled proportionally to the pre-required fixed size. In practice, it's advisable to first determine if the currency image is a fixed size. If not, the image can be scaled proportionally to that size (e.g., scaled to 128×128). If it is already a fixed size, no scaling is performed.

[0112] Step 302: Convert the color space of the scaled currency image to RGB color space to obtain the pixel matrix of the currency to be identified in RGB color space.

[0113] Images use various color spaces, such as RGB, BGR, and YUV, and different clients may support different color spaces. Therefore, for standardized processing, the color space of scaled currency images can be uniformly converted to the same color space. Since the RGB color space is closest to the colors seen by the human eye, the color space of scaled currency images can be uniformly converted to the RGB color space to obtain the pixel matrix of the currency to be identified in the RGB color space.

[0114] The preprocessing described above is merely illustrative. In other embodiments, any other suitable image preprocessing may be performed as needed. For example, due to the shooting angle, the acquired currency image may exhibit distortion or deformation. In such cases, distortion correction processing may be performed.

[0115] By inputting the preprocessed image of the currency to be identified into a preset image feature extraction model, the model can extract the currency image feature vector. This facilitates comparison with currency image feature vectors in a currency image feature vector library indexed by denomination, enabling currency identification based on the similarity of the currency image feature vectors. The currency image feature vector is a vectorized representation or feature encoding of the currency image features (e.g., pattern, color, shape).

[0116] In some embodiments, the image feature extraction model may employ a trained Siamese network (i.e., a Siamese neural network) model (the training process of the image feature extraction model will be described separately below). A Siamese network is a coupled neural network architecture based on two artificial neural networks with identical structures and shared weights. The idea behind a Siamese network is to take two samples as input, output a high-dimensional feature representation (i.e., extract high-dimensional features), and then compare the similarity between the two samples. The difference between genuine and counterfeit currency often lies in the differences in detailed features. Therefore, in the embodiments of this specification, the currency image feature vector extracted using the image feature extraction model can help uncover more detailed features of the currency image, thereby improving the accuracy of currency recognition.

[0117] Furthermore, given that the ResNet network in Siamese networks possesses very strong feature fitting capabilities and achieves a good balance between computational cost and accuracy, the ResNetN network can be considered a preferred Siamese network; where N represents the number of layers in the ResNet network. For example, in an exemplary embodiment, the Siamese network can be ResNet101, where 101 indicates that the ResNet network contains 101 layers (including 100 convolutional layers and 1 fully connected layer).

[0118] In some embodiments, the denomination of the currency image to be identified can be determined based on an Optical Character Recognition (OCR) module (here, the currency image to be identified can be the currency image before preprocessing or the currency image after preprocessing).

[0119] refer to Figure 5 As shown, in some embodiments, identifying the denomination of a currency image based on OCR may specifically include the following steps:

[0120] Step 501: Using a selection box of a preset size, perform corner alignment selection on the four corners of the first side image of the currency to be identified and the four corners of the second side image of the currency to be identified, forming eight selection areas.

[0121] Typically, the Arabic numerals for the denomination of currency are located in the four corners of the currency. For example, some currencies may only have the denomination marked in Arabic numerals in one or more corners on the obverse; others may only have it in one or more corners on the reverse; still others may have it in one or more corners on both sides. To facilitate narrowing the recognition area and avoid recognition failures caused by recognizing only one side of the image or only a portion of the corners of one side; for example, some currencies do not have the denomination designated in the corners on the obverse but have it in the corners on the reverse, and recognizing only one side of the image may lead to recognition failure; therefore, a selection box of a preset size can be used to select each of the four corners of the obverse and reverse images of the currency to be recognized, with the corners aligned (e.g., ...). Figure 4a and Figure 4b (As shown). In actual recognition, if the first image is a frontal image, then the second image is a backal image; if the first image is a backal image, then the second image is a frontal image.

[0122] The preset size of the selection box can be set as needed. In an exemplary embodiment, the preferred size of the selection box is the corresponding side length. In this way, we can effectively avoid missing the denomination information at the corners and edges, and also effectively reduce the recognition range.

[0123] Step 502: Based on OCR, the eight selected regions are identified to obtain multiple character sequences.

[0124] The OCR recognition principle mainly includes three processes: preprocessing, image segmentation, and character recognition. Preprocessing primarily includes steps such as image grayscale conversion, image binarization, image denoising, and image tilt correction. Image segmentation refers to the process of separating the image portion corresponding to each character within each selected region, thereby obtaining the corresponding character image and forming a character image sequence. Character recognition involves extracting the feature vector of each character image in the character image sequence and matching it with a feature template library to identify the character corresponding to each character image. The identified characters are arranged in the same order as the character image sequence, thus forming a character sequence. For example, in an exemplary implementation, the character sequence identified from a selected region might be "1 0 0".

[0125] Step 503: Identify character sequences that conform to the denomination characteristics from the multiple character sequences to form a denomination set.

[0126] Currency images vary widely, and some images may contain not only the denomination but also other information such as the currency unit, serial number, or Braille near the edges. This additional information can affect the accuracy of denomination recognition and therefore needs to be removed.

[0127] The inventors have discovered that denomination numerals typically conform to the following pattern: the first character is any one of the Arabic numerals 1 to 9, and the remaining characters (if any) are all Arabic numerals 0. For example, 1 yuan, 5 yuan, 10 yuan, 20 yuan, 50 yuan, and 100 yuan all conform to this pattern. Therefore, "the first character is any one of the Arabic numerals 1 to 9, and the remaining characters (if any) are all Arabic numerals 0" can be used as a denomination feature, and this feature can be used to identify whether individual character sequences conform to the denomination feature, thereby removing character sequences that do not represent denomination information.

[0128] For example, in an exemplary implementation, if a character sequence is 4 7 9 6 6 3 28B, although its first character "4" satisfies the condition that "the first character is any one of the Arabic numerals 1 to 9", the remaining digits (7 9 6 6 3 28B) do not meet the condition that "the remaining characters (if any) are all Arabic numerals 0". Therefore, the character sequence does not meet the denomination characteristics and can be removed.

[0129] For example, in another exemplary implementation, if a character sequence is 1 0 0, its first digit "1" satisfies the condition that "the first character is any one of the Arabic numerals 1 to 9", and the remaining digits (0 0) other than the first digit "2" also satisfy the condition that "the remaining characters (if any) are all Arabic numerals 0". Therefore, the character sequence meets the denomination characteristics and should be retained.

[0130] In this way, the character sequences that match the denomination characteristics of all the selected areas can eventually form a denomination set.

[0131] Step 504: Determine the denomination that appears most frequently in the denomination set, and use the denomination that appears most frequently as the denomination of the currency image to be identified.

[0132] When performing character recognition on a segmented single-character image using OCR, there is a possibility of misidentification, which may lead to distortion of individual or partial character sequences (i.e., discrepancies with reality). However, by using the most frequently occurring denomination as the denomination in the currency image to be identified, the accuracy of denomination recognition can be further improved, thus enhancing the accuracy of currency identification.

[0133] To improve the efficiency and accuracy of currency recognition, multiple currency image feature vector libraries indexed by denomination can be pre-constructed based on the entire currency set (the construction process of these libraries will be described separately below). In this way, during currency recognition, the currency image feature vector of the currency to be identified can be compared only with the feature vector of each currency image in the library corresponding to that specific denomination, without needing to compare it with feature vector libraries corresponding to other denominations. This avoids omissions and significantly reduces the search scope, thereby improving the efficiency and accuracy of currency recognition.

[0134] In the embodiments of this specification, when extracting the currency image feature vector of the currency to be identified, since the input preprocessed currency image contains both the front and back images of the currency, after inputting the preprocessed currency image of the currency to be identified into a preset image feature extraction model, the extracted currency image feature vector is actually: a vector combination (f) of the image feature vector of the front image and the image feature vector of the back image. f ,f r ), where f f f represents the image feature vector of a frontal image. r This represents the image feature vector of the reverse side of the image. Therefore, subsequent feature comparison is also based on vector combination. For example, the feature vector (f) of the currency image of a currency to be identified... 1f ,f 1r ), and a currency image feature vector (f) from the corresponding denomination currency image feature vector library. 2f ,f 2r When making a comparison, f can be used. 1f with f 2f A similarity value is obtained by comparison, and f is used to calculate the similarity. 1r with f 2r Another similarity value is obtained by comparison, and the average of the two similarity values ​​can be used as the similarity value of the feature vectors of the two currency images.

[0135] The total currency refers to all real currencies issued and still in use by countries or regions worldwide as of now, as collected by financial institutions (excluding their own national currencies). It should be noted that each currency image feature vector in the currency image feature vector library uniquely corresponds to one currency (and the currency type is known). Therefore, the currency image feature vector library indexed by denomination actually represents a set of currencies indexed by denomination.

[0136] For example, in an exemplary embodiment, when constructing the currency image feature vector library, it is assumed that the total currency denominations are divided into five types: 1 yuan, 5 yuan, 10 yuan, 50 yuan, and 100 yuan. On average, each denomination contains 2,000 currency images. Then, the currency image feature vectors of all currency images are divided into five categories according to the denomination, thereby forming five currency image feature vector libraries (i.e., establishing five currency image feature vector libraries with index values ​​of 1, 5, 10, 50, and 100 respectively). When performing currency recognition, the currency image feature vector library of a specific denomination can be quickly matched according to the denomination (index value) without having to match other denominations, thereby greatly narrowing the scope of currency recognition. This reduces the time required for currency recognition and greatly improves the accuracy of recognition.

[0137] Accordingly, when performing currency recognition, if the denomination of the currency to be recognized is 100, the currency image feature vector library with an index value of 100 (i.e., the matching target library) can be matched from the five currency image feature vector libraries corresponding to index values ​​of 1, 5, 10, 50, and 100, and used as the actual search scope for this operation.

[0138] In some embodiments, the currency image feature vector of the currency to be identified can be compared with the feature vector of each currency image in the target library based on any suitable similarity calculation method (e.g., Euclidean distance or cosine similarity) to facilitate implementation.

[0139] refer to Figure 8 As shown, in some embodiments, determining the currency to be identified based on the similarity comparison result may include the following steps:

[0140] Step 801: Sort the multiple similarity values ​​obtained based on the similarity comparison from the smallest to the largest.

[0141] Step 802: The currency mapped by the currency image feature vector corresponding to the first similarity value in the target database is taken as the currency to be identified. This allows for fully automated identification without human intervention.

[0142] For example, in an exemplary embodiment, the currency image feature vector of the currency to be identified, x, is λ, and the corresponding target library obtained by full matching from the currency image feature vector library is {λ1, λ2, λ3, ..., λ...}. 100},{λ1,λ2,λ3,…,λ 100 The corresponding currencies are {κ1,κ2,κ3…,κ}. 100}, λ and λ1, λ2, λ3,…,λ respectively 100 The similarity is s1, s2, s3, ..., s 100If s1, s2, s3, ..., s 100 The approximate order from smallest to largest is s2→s3→s1→…→s 100 Clearly, s2 is the similarity value at the top of the ranking, and s2 corresponds to the currency κ2. Therefore, x = κ2, meaning the currency to be identified is κ2.

[0143] refer to Figure 9 As shown, in some other embodiments, determining the currency of the currency to be identified based on the similarity comparison result may include the following steps:

[0144] Step 901: Sort the multiple similarity values ​​obtained based on the similarity comparison from the smallest to the largest.

[0145] Step 902: The currencies mapped by the currency image feature vectors corresponding to the top K (e.g., top 3, top 5, etc.) similarity values ​​in the target database are used as candidate currencies for user selection (i.e., user confirmation). This helps to further improve recognition accuracy.

[0146] For example, in an exemplary embodiment, the currency image feature vector of the currency to be identified, x, is λ, and the corresponding target library obtained by full matching from the currency image feature vector library is {λ1, λ2, λ3, ..., λ...}. 100},{λ1,λ2,λ3,…,λ 100 The corresponding currencies are {κ1,κ2,κ3…,κ}. 100}, λ and λ1, λ2, λ3,…,λ respectively 100 The similarity is s1, s2, s3, ..., s 100 If s1, s2, s3, ..., s 100 The approximate order from smallest to largest is s2→s3→s1→…→s 100 If the first three values ​​in the sorted list (i.e., s2, s3, and s1) are taken, then κ2, κ3, and κ2 are the range of selectable values ​​for x, which can be chosen by the user.

[0147] In the embodiments of this specification, after determining the currency to be identified based on the similarity comparison results, the server can output the currency and denomination as the currency identification result.

[0148] refer to Figure 6 As shown, in some embodiments, the image feature extraction model described above can be pre-trained in the following manner:

[0149] Step 601: Obtain labeled currency images of multiple currencies.

[0150] Labels, or classification labels, are used to indicate the currency type. Labeled currency images can be created manually or through other methods after the currency images are obtained. To improve the model's generalization ability, labeled currency images of multiple currencies can be acquired when constructing the dataset.

[0151] Step 602: Preprocess the labeled currency images to form a labeled currency image dataset.

[0152] The preprocessing here is the same as above. Figure 2 The preprocessing mentioned in the illustrated embodiments is the same and will not be repeated here.

[0153] Step 603: Add the labeled currency image dataset to the open-source dataset to form a labeled image dataset.

[0154] The model requires a large number of samples. A currency only has two images, the front and the back. However, the number of currencies in the world is limited, and the amount of currency image data is small, which is insufficient to train an image retrieval model with good generalization ability. Therefore, in order to improve the generalization ability of the model, the labeled currency image dataset can be added to open source datasets (such as Oxford5k and Paris6k) to form a larger-scale dataset.

[0155] Step 604: Train a Siamese network model based on the labeled image dataset to obtain an image feature extraction model.

[0156] The training of the image feature extraction model can be terminated by setting a training termination condition. In some embodiments, the training termination condition can be set when the loss value of the loss function is less than a set value. For example, the following loss function can be used when training a Siamese network model based on the labeled image dataset:

[0157]

[0158] Where L represents the loss value of the loss function, N represents the number of samples, Y represents the label indicating whether two samples match, X1 represents the sample input to the first Siamese subnet of the Siamese network model, X2 represents the sample input to the second Siamese subnet of the Siamese network model, and D... w (X1, X2) represents the similarity between X1 and X2, max represents the maximum value function, m represents the set threshold, and P represents the feature dimension of the samples. This represents the j-th sample input to the first twin subnet of the twin network model. This represents the j-th sample input to the second twin subnet of the twin network model.

[0159] From the above expression for the loss function, it can be seen that when Y = 1 (i.e., the two samples are similar), the loss function can be expressed as: When Y = 0 (i.e., the two samples are not similar), the loss function can be expressed as: That is, when two samples are not similar, if the distance between their feature spaces (such as Euclidean distance) is small, the loss value will be large, which is beneficial for training a model with higher recognition accuracy.

[0160] It should be noted that in the above expression of the loss function, setting the threshold m means that only dissimilar features with a distance between 0 and m are considered. For features with a distance greater than m, their corresponding loss value is regarded as 0 (that is, dissimilar features with a large distance should have a very low loss value). For similar features with a large distance, the loss value can be appropriately increased to update the matching degree of paired samples during training, so as to obtain a model with greater generalization ability.

[0161] refer to Figure 7 As shown, in some embodiments, based on the image feature extraction model trained above, the currency image feature vector library can be obtained in advance through the following methods:

[0162] Step 701: Obtain the currency images and denominations of the full amount of currency.

[0163] Step 702: Preprocess each currency image in the currency image set of the full currency set to form a full currency image set.

[0164] The preprocessing here is the same as above. Figure 2 The preprocessing mentioned in the illustrated embodiments is the same and will not be repeated here.

[0165] Step 703: Input each currency image in the full set of currency images into the image feature extraction model to extract the currency image feature vector of each currency image.

[0166] Step 704: Establish an index for the currency image feature vector of each currency image based on its face value, thereby forming a library of multiple currency image feature vectors indexed by face value.

[0167] In the embodiments of this specification, considering that new currencies may be issued in various countries and regions around the world at any time, the constructed multi-currency image feature vector library indexed by denomination needs to be updated in a timely manner to avoid the problem of being unable to identify newly issued currencies, thereby ensuring the stability of the currency recognition service.

[0168] For example, in some embodiments, the server can sign agreements with monetary authorities in various countries and regions around the world to subscribe to images of newly issued currencies. When a monetary authority in a country or region issues a new currency, the corresponding monetary authority can promptly push the image of the newly issued currency to the server. Of course, any other suitable method can also be used to obtain images of newly issued currencies from monetary authorities in various countries and regions.

[0169] When the server receives newly issued currency, it preprocesses the currency image (see above for details). Figure 2 (Preprocessing mentioned in the illustrated embodiment); inputting the preprocessed currency image of the newly issued currency into the image feature extraction model to extract the currency image feature vector of the newly issued currency; identifying the denomination of the newly issued currency; and adding the currency image feature vector of the newly issued currency to the currency image feature vector library with the corresponding index value according to the denomination of the newly issued currency.

[0170] For example, in an exemplary embodiment, based on the existing construction of five currency image feature vector libraries with index values ​​of 1, 5, 10, 50, and 100, if the central bank of country XX issues a new denomination of 50 yuan and pushes the currency image of the newly issued currency to the server, the server can extract the currency image feature vector of the newly issued currency and identify that the denomination of the newly issued currency is 50 yuan. Then, the server can add the currency image feature vector of the newly issued currency to the currency image feature vector library with index value 50.

[0171] In rare cases, the face value of newly issued currency may not match any of the existing currency image feature vector libraries indexed by face value. Given five existing currency image feature vector libraries with index values ​​of 1, 5, 10, 50, and 100, if the central bank of country XX issues a new 500 yuan denomination currency, a new currency image feature vector library with an index value of 500 can be created.

[0172] In the field of currency recognition, traditional classification models typically require pre-determining the category before training. However, new currencies (even new denominations) may be issued in various countries and regions around the world at any time, meaning the category is not fixed. When the category changes, the classification model needs to be retrained. In the embodiments of this specification, when a new currency is added, only the image feature extraction model of this specification needs to be used to extract features, and the extracted features are added to the currency image feature vector library for the corresponding denomination (index value), without retraining the image feature extraction model; thus further reducing the cost of currency recognition.

[0173] Although the process described above includes multiple operations that occur in a specific order, it should be clearly understood that these processes may include more or fewer operations, which may be executed sequentially or in parallel (e.g., using parallel processors or a multithreaded environment).

[0174] Corresponding to the currency recognition method described above, this specification also provides a currency recognition device, which can be configured on the aforementioned server. (Refer to...) Figure 10 As shown, in some embodiments, the currency recognition device may include:

[0175] Image acquisition module 101 is used to acquire a currency image of the currency to be identified;

[0176] The preprocessing module 102 is used to preprocess the currency image of the currency to be identified;

[0177] The feature extraction module 103 is used to input the preprocessed currency image of the currency to be identified into a preset image feature extraction model in order to extract the currency image feature vector of the currency to be identified.

[0178] The denomination recognition module 104 is used to recognize the denomination in the currency image of the currency to be recognized;

[0179] The feature matching module 105 is used to match the currency image feature vector library corresponding to the denomination of the currency to be identified from multiple currency image feature vector libraries indexed by denomination, and use it as the target library;

[0180] The feature comparison module 106 is used to compare the feature vector of the currency image of the currency to be identified with the feature vector of each currency image in the target library.

[0181] The currency determination module 107 is used to determine the currency of the currency to be identified based on the similarity comparison result.

[0182] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0183] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this specification are all information and data authorized and agreed upon by the user and fully authorized by all parties.

[0184] Embodiments of this specification also provide a computer device. For example... Figure 11As shown, in some embodiments of this specification, the computer device 1102 may include one or more processors 1104, such as one or more central processing units (CPUs) or graphics processing units (GPUs), each of which may implement one or more hardware threads. The computer device 1102 may also include any memory 1106 for storing information of any kind, such as code, settings, data, etc. In one specific embodiment, a computer program is stored on the memory 1106 and can run on the processor 1104. When the computer program is run by the processor 1104, it can execute instructions of the currency recognition method described in any of the above embodiments. Without limitation, for example, the memory 1106 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Furthermore, any memory may provide volatile or non-volatile retention of information. Furthermore, any memory may represent a fixed or removable component of the computer device 1102. In one scenario, when processor 1104 executes associated instructions stored in any memory or combination of memories, computer device 1102 can perform any operation of the associated instructions. Computer device 1102 also includes one or more drive mechanisms 1108 for interacting with any memory, such as hard disk drive mechanisms, optical disk drive mechanisms, etc.

[0185] Computer device 1102 may also include an input / output interface 1110 (I / O) for receiving various inputs (via input device 1112) and providing various outputs (via output device 1114). A specific output mechanism may include a presentation device 1116 and an associated graphical user interface 1118 (GUI). In other embodiments, the input / output interface 1110 (I / O), input device 1112, and output device 1114 may be omitted, and the device may function solely as a computer device within a network. Computer device 1102 may also include one or more network interfaces 1120 for exchanging data with other devices via one or more communication links 1122. One or more communication buses 1124 couple the components described above together.

[0186] Communication link 1122 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 1122 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0187] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), computer-readable storage media, and computer program products according to some embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processor to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processor, create a mechanism for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0188] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processor to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions may also be loaded onto a computer or other programmable data processor, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0190] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0191] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0192] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by computer equipment. As defined in this specification, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0193] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0194] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processors connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0195] It should also be understood that, in the embodiments of this specification, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0196] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0197] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0198] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A currency identification method, characterized in that, include: Acquire an image of the currency to be identified; The currency image of the currency to be identified is preprocessed; The preprocessed currency image of the currency to be identified is input into a preset image feature extraction model to extract the currency image feature vector of the currency to be identified; Identify the denomination in the currency image of the currency to be identified; Based on the identified denomination, the corresponding target feature subset is retrieved from a pre-built currency image feature vector library. This library is organized into multiple independently stored feature subsets, each corresponding to a specific denomination, and is decoupled from the image feature extraction model. The currency image feature vector library supports incremental updates. When a new currency type needs to be added, its denomination is identified and its currency image feature vector is extracted, then this feature vector is added to the feature subset corresponding to the denomination, without modifying or retraining the image feature extraction model. The feature vector of the currency image to be identified is compared with the feature vector of each currency image in the target database. The currency to be identified is determined based on the similarity comparison results.

2. The currency identification method as described in claim 1, characterized in that, Preprocessing the currency image of the currency to be identified includes: The currency image of the currency to be identified is scaled proportionally to a fixed size; The color space of the scaled currency image is converted to RGB color space to obtain the pixel matrix of the currency to be identified in RGB color space.

3. The currency identification method as described in claim 1, characterized in that, The image feature extraction model includes a pre-trained Siamese network model.

4. The currency identification method as described in claim 3, characterized in that, The image feature extraction model is pre-trained using the following method: Get tagged currency images for multiple currencies; The labeled currency images are preprocessed to form a labeled currency image dataset; The labeled currency image dataset was added to the open-source dataset to form a labeled image dataset; A Siamese network model is trained based on the labeled image dataset to obtain an image feature extraction model.

5. The currency identification method as described in claim 4, characterized in that, The loss function used when training the Siamese network model based on the labeled image dataset includes: in, This represents the loss value of the loss function. Indicates the number of samples. Labels indicating whether two samples match. This represents the sample input to the first twin subnet of the twin network model. Let represent the sample input to the second twin subnet of the twin network model. express and Similarity between them Represents the maximum value function. This indicates that a threshold value is set. The feature dimension of the sample. This represents the input to the first twin subnet of the twin network model. One sample, This represents the input to the second twin subnet of the twin network model. One sample.

6. The currency identification method as described in claim 3, characterized in that, The twin network model includes a twin network model based on ResNetN, where N represents the number of network layers contained in ResNet.

7. The currency identification method as described in claim 1, characterized in that, The currency image feature vector library was obtained in advance through the following methods: Get the currency image of the full currency set; Identify the denomination of each currency image; Each currency image is input into the image feature extraction model to extract the corresponding currency image feature vector; Based on the identified denominations, the currency image feature vectors are classified and stored into corresponding feature subsets, forming a currency image feature vector library organized by denomination.

8. The currency identification method as described in claim 1, characterized in that, Identifying the denomination in the currency image of the currency to be identified includes: The OCR recognition module identifies the denomination in the currency image of the currency to be identified.

9. The currency identification method as described in claim 8, characterized in that, The OCR-based recognition module identifies the denomination in the currency image of the currency to be identified, including: Using a selection box of a preset size, the four corners of the first side image of the currency to be identified and the four corners of the second side image of the currency to be identified are selected by edge-corner alignment, forming eight selection areas; Based on OCR, the eight selected regions are identified respectively to obtain multiple character sequences; Character sequences that match the denomination characteristics are identified from the plurality of character sequences to form a denomination set; The denomination that appears most frequently in the set of denominations is determined, and the denomination that appears most frequently is used as the denomination of the currency image to be identified.

10. The currency identification method as described in claim 1, characterized in that, The currency to be identified is determined based on the similarity comparison results, including: The multiple similarity values ​​obtained based on the similarity comparison are sorted from largest to smallest; The currency type mapped by the currency image feature vector corresponding to the first similarity value in the target library is taken as the currency type to be identified.

11. The currency identification method as described in claim 1, characterized in that, The currency to be identified is determined based on the similarity comparison results, including: The multiple similarity values ​​obtained based on the similarity comparison are sorted from largest to smallest; The currencies mapped by the currency image feature vectors corresponding to the top K similarity values ​​in the target library in the sorted order are used as candidate currencies for users to select.

12. A currency recognition device, characterized in that, include: The image acquisition module is used to acquire images of the currency to be identified. The preprocessing module is used to preprocess the currency image of the currency to be identified; The feature extraction module is used to input the preprocessed currency image of the currency to be identified into a preset image feature extraction model in order to extract the currency image feature vector of the currency to be identified. A denomination recognition module is used to identify the denomination in the currency image of the currency to be identified; The feature matching module is used to retrieve a corresponding target feature subset from a pre-built currency image feature vector library based on the identified denomination. The currency image feature vector library is organized into multiple independently stored feature subsets, each corresponding to a specific denomination, and is decoupled from the image feature extraction model. The currency image feature vector library supports incremental updates. When a new currency denomination needs to be added, its denomination is identified and its currency image feature vector is extracted, then the currency image feature vector is added to the feature subset corresponding to the denomination without modifying or retraining the image feature extraction model. The feature comparison module is used to compare the feature vector of the currency image to be identified with the feature vector of each currency image in the target database. The currency determination module is used to determine the currency of the currency to be identified based on the similarity comparison results.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is run by the processor, it executes the instructions of the method according to any one of claims 1-11.

14. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor of the computer device, it executes the instructions of the method according to any one of claims 1-11.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, performs instructions according to any one of claims 1-11.

Images