Banknote identification and inspection method and system based on photoelectric multi-mode fusion

Abstract

The invention discloses a photoelectric multi-mode fusion-based banknote identification and inspection method and system, and relates to the technical field of banknote identification. The method comprises the following steps: synchronously acquiring visible light, infrared, ultraviolet, magnetic and other multi-mode original signals of a banknote to be identified by using a multi-dimensional sensor group; pixel-level space registration of the multi-dimensional signals is achieved through space geometric correction and normalization processing; extracting shallow physical features of each modal to construct a feature set; introducing an attention mechanism, dynamically distributing fusion weights according to the mutual information amount and the signal-to-noise ratio, and generating a fusion feature vector; and finally, inputting a pre-training model to calculate a matching degree with a standard real currency template so as to output an identification result. According to the invention, through accurate registration and dynamic weighted fusion of the multi-dimensional signals, the problem of insufficient identification accuracy of complicated posture, dirty old and altered banknotes caused by signal space dislocation and modal isolation processing in the prior art is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of banknote authentication technology, specifically to a banknote authentication and inspection method and system based on photoelectric multimodal fusion. Background Technology

[0002] Due to the wide variety of currency types and their diverse materials, including cotton paper, polymers, and composite substrates, anti-counterfeiting technologies and printing processes exhibit significant heterogeneity. In actual high-frequency circulation scenarios, banknotes are highly susceptible to oil stains, mechanical wear, and aging fading, leading to blurred or missing key anti-counterfeiting features. Furthermore, to improve processing efficiency, users often insert banknotes into the equipment in any orientation and with any orientation (front or back). Most existing currency authentication technologies employ a parallel detection mode using independent sensors, comparing thresholds for optical, magnetic, and electrical signals separately, and finally summarizing the results using simple logic gates.

[0003] However, this loosely coupled technical architecture has two major flaws: First, it lacks a pixel-level spatial registration mechanism for cross-modal signals. When banknotes are transmitted at an angle or offset, the data collected by different sensors cannot be precisely aligned in spatial position, leading to misalignment in feature extraction. Second, the processing of data from each modality is isolated, failing to establish a dynamic complementary mechanism based on signal quality. For example, when the visible light texture of a banknote is damaged due to dirt, the system cannot automatically utilize magnetic or infrared penetration features with stronger anti-interference capabilities for weighted compensation. This "lone wolf" detection method results in a high false rejection rate when faced with dirty and worn banknotes from multiple countries and cannot effectively address the fraud risk of sophisticatedly altered banknotes. Therefore, it is urgent to improve the existing technology to solve the problem of insufficient accuracy in identifying banknotes with complex postures, dirt, and alterations due to the lack of precise spatial registration of multi-dimensional signals and deep weighted fusion of multi-modal features in current currency identification methods. Summary of the Invention

[0004] The purpose of this application is to provide a banknote identification and inspection method and system based on photoelectric multimodal fusion, which has the advantages of achieving accurate registration of multidimensional signal space and dynamic weighted fusion of multimodal features, effectively improving the reliability of identification of complex postures and dirty and old banknotes.

[0005] This application provides a banknote identification and inspection method based on photoelectric multimodal fusion, the technical solution of which is as follows: Includes the following steps: A multi-dimensional sensor array arranged in the banknote transmission channel is used to synchronously collect multi-modal raw signals of the banknote to be identified. The multi-dimensional sensor array includes at least sensors for acquiring visible light reflection images, infrared transmission images, ultraviolet fluorescence response data, and magnetic distribution sequences. Spatial geometric correction and signal normalization are performed on the original multimodal signals to map signals of different physical dimensions to a standard coordinate system, thereby obtaining spatially registered multimodal data. Shallow physical features of each mode are extracted from the spatially registered multimodal data, and the shallow physical features are mapped to an isomorphic high-dimensional feature space to construct a multimodal feature set containing texture details, spectral properties and magnetic features. An attention mechanism is used to perform weighted fusion of multimodal feature sets. The fusion weights are dynamically allocated by calculating the mutual information and signal-to-noise ratio of each feature component, thereby generating a fusion feature vector that enhances effective information and suppresses noise. The fused feature vector is input into a pre-trained classification and decision model, and the matching degree between it and the standard genuine currency feature template is calculated. Based on the matching result, the country, version, denomination and authenticity information of the banknote are output.

[0006] Furthermore, this application also proposes that, The specific steps for performing spatial geometric correction and signal normalization on multimodal raw signals include: The physical boundaries of banknotes in visible light reflection images are identified using edge detection algorithms, and the deflection angle and displacement deviation of banknotes relative to the transmission channel are calculated. Based on the deflection angle and displacement deviation, an affine transformation matrix is ​​constructed and geometric correction is performed on the visible light reflection image, infrared transmission image and ultraviolet fluorescence response data to align them to the standard pose and obtain the registered multimodal image data. Based on the banknote transmission speed and sensor sampling frequency, the magnetic distribution sequence is interpolated, resampled, and spatially mapped to generate a two-dimensional magnetic distribution map with the same spatial scale as the registered multimodal image data.

[0007] Furthermore, this application also proposes that, The specific steps for extracting shallow physical features of each mode from spatially registered multimodal data include: For visible light reflection images, texture roughness and directional features are extracted using the gray-level co-occurrence matrix; For infrared transmission images, extract the binarized skeleton and connected component features of the infrared absorbing ink region; Based on the ultraviolet fluorescence response data, statistical characteristics of the distribution density of fluorescent fibers and the brightness of the fluorescence pattern were extracted; For the two-dimensional magnetic distribution map, the peak position of magnetic flux along the safety line and the waveform slope characteristics of the magnetic ink region are extracted.

[0008] Furthermore, this application also proposes that, The weighted fusion of multimodal feature sets using an attention mechanism specifically includes: Construct a dual attention network that includes a channel attention module and a spatial attention module; The importance of multimodal feature sets in the channel dimension is analyzed using the channel attention module, and channel weight coefficients are generated. The spatial attention module is used to analyze the response intensity of the feature map in the spatial dimension and generate spatial location weights. By combining channel weight coefficients and spatial location weights, a weighted fusion of multimodal feature sets is achieved.

[0009] Furthermore, this application also proposes that, The dynamic allocation of fusion weights specifically includes: Real-time monitoring of local contrast and data loss rate of each modal signal; When the local contrast of a certain modal signal is lower than a preset threshold or there is regional data loss, the weight of the corresponding feature of that modality is automatically reduced during the fusion process. Simultaneously, the weights of other undisturbed modal features are increased to maintain the overall energy stability of the fused feature vector and achieve cross-modal feature compensation.

[0010] Furthermore, this application also proposes that, Specifically, inputting the fused feature vectors into the pre-trained classification decision model includes: A multilayer perceptron is used as the main structure of the classification decision model; Calculate the Euclidean distance or cosine similarity between the fused feature vector of the banknote to be identified and the feature template of the standard genuine banknote for each currency type; Based on the matching degree, the country and version of the banknote are first determined, and then the authenticity of the banknote is determined based on the matching degree of the features of specific anti-counterfeiting areas.

[0011] Furthermore, this application also proposes that, After outputting the country of origin, version, denomination, and authenticity information of the banknotes, it also includes: If the matching degree is within the preset ambiguity range for true and false judgment, then the identification is marked as a suspicious event, and the corresponding multimodal original signal and fused feature vector are stored in the abnormal sample library. The classification and decision model is incrementally learned periodically using an abnormal sample library to update the standard genuine currency feature template and adaptively optimize the identification performance.

[0012] Furthermore, this application also proposes that, The multi-dimensional sensor group includes a visible light image sensor and a near-infrared image sensor. The visible light image sensor and the near-infrared image sensor share the same optical path through a beam splitter to ensure that the acquired visible light reflected image and infrared transmitted image are naturally registered in physical space.

[0013] Furthermore, this application also proposes that, The output formats for banknotes include: country of origin, version, denomination, and authenticity information. The interface displays the current banknote's attributes and total value in real time. The mechanical sorting device is driven by a control interface to physically separate genuine banknotes from counterfeit or unidentifiable banknotes. The identification records and feature data are uploaded to the upper management system through the data communication interface.

[0014] Furthermore, this application also proposes that, A banknote authentication and verification system based on photoelectric multimodal fusion, used to implement the above method, includes: The signal acquisition module, arranged along the banknote transmission path, integrates a multi-dimensional sensor group to acquire the multi-modal raw signals of the banknote to be identified. The central processing module, electrically connected to the signal acquisition module, is used to perform spatial geometric correction, feature extraction, weighted fusion, and classification decision of the multimodal raw signals; The storage module, connected to the central processing module, is used to store standard genuine currency feature templates and anomaly sample libraries; The human-computer interaction and execution module is connected to the central processing module and is used to display the identification results and perform banknote sorting operations.

[0015] As can be seen from the above, the banknote identification and inspection method and system based on photoelectric multimodal fusion provided in this application achieves pixel-level registration by synchronously acquiring multimodal signals through a multidimensional sensor group and performing spatial geometric correction. Combined with the attention mechanism, the fusion weights are dynamically allocated to compensate for signal quality fluctuations. This effectively solves the problem of insufficient identification accuracy for banknotes with complex postures and dirty or old banknotes. It has the advantages of achieving accurate spatial registration of multidimensional signals and dynamic weighted fusion of multimodal features, effectively improving the reliability of identification for banknotes with complex postures and dirty or old banknotes. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the banknote identification and inspection method based on photoelectric multimodal fusion provided by the present invention; Figure 2 This is a schematic diagram of the structure of the banknote identification and inspection system based on photoelectric multimodal fusion provided by the present invention.

[0018] The following are the markings in the attached diagram: 100. Signal acquisition module; 200. Central processing module; 300. Storage module; 400. Human-computer interaction and execution module. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0021] like Figure 1 As shown in the figure, this application proposes a banknote identification and inspection method based on photoelectric multimodal fusion, including the following steps: S100. Using a multi-dimensional sensor group arranged in the banknote transmission channel, the multi-modal raw signals of the banknote to be identified are collected synchronously. The multi-dimensional sensor group includes at least sensors for acquiring visible light reflection images, infrared transmission images, ultraviolet fluorescence response data, and magnetic distribution sequences. S200: Perform spatial geometric correction and signal normalization processing on the original multimodal signals to uniformly map signals of different physical dimensions to the standard coordinate system and obtain spatially registered multimodal data. S300. Extract shallow physical features of each mode from the spatially registered multimodal data, and map the shallow physical features to an isomorphic high-dimensional feature space to construct a multimodal feature set containing texture details, spectral properties and magnetic features. S400: An attention mechanism is used to perform weighted fusion of multimodal feature sets. The fusion weights are dynamically allocated by calculating the mutual information and signal-to-noise ratio of each feature component, thereby generating a fusion feature vector that enhances effective information and suppresses noise. S500: Input the fused feature vector into the pre-trained classification and decision model, calculate the matching degree between it and the standard genuine currency feature template, and output the country, version, denomination and authenticity information of the banknote based on the matching result.

[0022] The multi-dimensional sensor array is a device integrating multiple sensing functions, primarily responsible for simultaneously capturing feature information of banknotes in different physical dimensions. For example, it can be implemented by configuring multiple independent sensor modules, each responsible for acquiring a specific type of signal. Visible light reflectance images can be acquired using CCD or CMOS image sensors, infrared transmission images using near-infrared sensors, ultraviolet fluorescence response data using ultraviolet detectors, and magnetic distribution sequences using magnetoresistive sensor arrays. This design ensures synchronous acquisition of multi-modal signals, avoiding data inconsistencies caused by time differences.

[0023] Furthermore, spatial geometric correction and signal normalization refer to the process of aligning the positions and standardizing the values ​​of the acquired multimodal raw signals. Specifically, spatial geometric correction can be achieved by establishing a mathematical transformation model, such as using polynomial fitting to correct image distortion, or using point cloud registration algorithms to align 3D data. Signal normalization, on the other hand, can be achieved by linear scaling or nonlinear mapping to adjust signals of different modes to the same numerical range for subsequent processing.

[0024] Furthermore, the extraction of shallow physical features can be understood as the process of separating physically meaningful feature parameters from the original signal. For example, for visible light reflectance images, contour features can be extracted using edge detection algorithms, or color distribution features can be extracted using color histogram analysis; for infrared transmission images, grayscale features of specific regions can be extracted using threshold segmentation; for ultraviolet fluorescence response data, periodic variation features of fluorescence intensity can be extracted using frequency domain analysis; and for magnetic distribution sequences, local fluctuation features can be extracted using wavelet transform. These features are further mapped to a homogeneous high-dimensional feature space to achieve cross-modal feature comparison and fusion.

[0025] When using an attention mechanism to perform weighted fusion of multimodal feature sets, different weight allocation strategies can be designed. For example, a static weight allocation method based on feature importance can assign fixed weights to features of different modalities based on prior knowledge; a dynamic weight allocation method based on signal quality can adjust the weights according to real-time monitored signal integrity indicators, for example, automatically reducing the weight of a certain modality signal and increasing the weights of other modalities when the signal-to-noise ratio of that modality is low. This mechanism can effectively cope with signal interference or data loss, improving the robustness of the fused features.

[0026] The innovation of this application lies in its systematic design, which achieves precise spatial registration of multi-dimensional signals and deep weighted fusion of multi-modal features. Specifically, the synchronous acquisition of multi-dimensional sensor arrays solves the signal misalignment problem caused by changes in banknote posture in existing technologies; spatial geometric correction and signal normalization ensure the spatial and numerical alignment of different modal data; the extraction and mapping of shallow physical features provides a unified foundation for cross-modal feature fusion; and the introduction of an attention mechanism enables dynamic weight allocation based on signal quality, overcoming the problem of insufficient identification accuracy when a single modal signal is damaged. Therefore, this application demonstrates higher reliability and adaptability in identification scenarios involving complex postures, dirty or altered banknotes.

[0027] The working principle of this application embodiment is as follows: A banknote authentication method based on photoelectric multimodal fusion synchronously acquires multimodal raw signals of the banknote to be authenticated through a multidimensional sensor group arranged in the banknote transmission channel. The multidimensional sensor group includes at least sensors for acquiring visible light reflectance images, infrared transmission images, ultraviolet fluorescence response data, and magnetic distribution sequences. This design ensures that multimodal data of the banknote's key anti-counterfeiting features can be comprehensively captured when the banknote is transmitted in any orientation, providing fundamental support for subsequent processing. Furthermore, spatial geometric correction and signal normalization processing are performed on the multimodal raw signals, uniformly mapping signals from different physical dimensions to a standard coordinate system to obtain spatially registered multimodal data. Specifically, this step solves the problem of inaccurate alignment of sensor data when the banknote is tilted or offset, thereby avoiding misalignment in feature extraction.

[0028] Shallow physical features of each modality are extracted from the spatially registered multimodal data, and these shallow physical features are mapped to a homogeneous high-dimensional feature space to construct a multimodal feature set containing texture details, spectral characteristics, and magnetic features. This process, based on the registered data, ensures the accuracy of feature extraction, while mapping to a unified space makes the features of different modalities comparable and fusionable. An attention mechanism is used to perform weighted fusion of the multimodal feature set. The fusion weights are dynamically allocated by calculating the mutual information and signal-to-noise ratio of each feature component, thereby generating a fused feature vector that enhances effective information and suppresses noise. Thus, when the visible light texture of a banknote is damaged due to dirt, the system can automatically reduce the weight of that modality and increase the weight of magnetic or infrared features with stronger anti-interference capabilities, achieving cross-modal feature compensation.

[0029] Finally, the fused feature vector is input into a pre-trained classification and decision model to calculate its matching degree with the standard genuine currency feature template. Based on the matching result, the model outputs the country of origin, version, denomination, and authenticity information of the banknote. This matching degree-based decision logic ensures comprehensive and accurate identification of banknotes with complex postures, dirt, and alterations, significantly improving the system's robustness in high-frequency scenarios such as cross-border circulation. The overall technical solution effectively solves the problem of insufficient identification accuracy in existing currency identification methods through precise spatial registration of multi-dimensional signals and deep weighted fusion of multi-modal features.

[0030] Furthermore, this application proposes specific steps for performing spatial geometric correction and signal normalization processing on the multimodal raw signal, including: The physical boundaries of banknotes in visible light reflection images are identified using edge detection algorithms, and the deflection angle and displacement deviation of banknotes relative to the transmission channel are calculated. Based on the deflection angle and displacement deviation, an affine transformation matrix is ​​constructed and geometric correction is performed on the visible light reflection image, infrared transmission image and ultraviolet fluorescence response data to align them to the standard pose and obtain the registered multimodal image data. Based on the banknote transmission speed and sensor sampling frequency, the magnetic distribution sequence is interpolated, resampled, and spatially mapped to generate a two-dimensional magnetic distribution map with the same spatial scale as the registered multimodal image data.

[0031] In the implementation process, the application of edge detection algorithms is crucial for achieving pose quantization. Its core lies in using mathematical tools such as the Canny operator, Sobel operator, or Laplacian operator to keenly capture areas in the image where brightness changes drastically, thereby accurately pinpointing the physical boundaries of the banknote. Subsequently, the system uses a constructed affine transformation matrix to perform geometric correction. This process essentially involves rotating, scaling, and translating the image data in a two-dimensional plane through linear algebraic operations, aiming to forcibly align images of different modalities to a unified standard coordinate system. For one-dimensional time-series magnetic data, interpolation resampling techniques are introduced. Whether linear interpolation or cubic spline interpolation is used, the essence is to transform discrete points into a continuous distribution through numerical estimation, thereby generating a two-dimensional distribution map that is fully compatible with the image data in terms of spatial scale.

[0032] As a specific implementation method, in order to build a unified data foundation, the system first establishes a standard coordinate system. For the spatial geometric correction step, the system uses an affine transformation model to map the acquired multimodal images to this standard coordinate system. Assume the horizontal displacement of the banknote's physical center relative to the image center, calculated by the edge detection algorithm, is... The vertical displacement is The rotation angle is For the original image coordinates of any pixel in The corrected target coordinates in the standard coordinate system The following matrix operations yielded the following: in, These are the original pixel coordinates. These are the corrected pixel coordinates. The system calculates them using bilinear interpolation. The pixel values ​​at each location are used to obtain the corrected multimodal image set. These correspond to the visible light, infrared, and ultraviolet modes, respectively.

[0033] Meanwhile, for one-dimensional magnetic distribution sequences In order to make it compatible with the corrected image set Alignment is performed on a spatial scale, and the system performs interpolation resampling. Assume the banknote transmission speed is... (Unit: mm / s), sensor sampling frequency is (Unit: Hz), the spatial resolution of the image is (Unit: dpi). Constructing resampling coefficients. as follows: Using cubic spline interpolation functions, the original magnetic sequence Mapped to a length of Feature vector of (number of columns of image pixels) ， This achieves physical dimension normalization between magnetic data and image data. At this point, the system obtains a spatially registered multimodal dataset. This serves as a unified input for subsequent processing.

[0034] Specifically, the aforementioned technical solution achieves pixel-level spatial registration of multimodal signals through the close integration of each stage. The entire process begins with precise analysis of the visible light reflection image, utilizing its clear boundary features to provide a high-confidence benchmark for attitude measurement. Once the deflection angle and displacement deviation are quantified, the affine transformation matrix acts as a correction engine, driving the visible light, infrared, and ultraviolet images to achieve strict spatial synchronization, completely eliminating the risk of texture or spectral feature misalignment caused by incorrect banknote insertion posture. Simultaneously, the spatial mapping of magnetic data and image correction proceed in parallel, bridging the dimensional gap between magnetic signals and image signals. This progressive correction strategy effectively solves the feature extraction offset problem caused by banknote posture deviation, laying a solid data foundation for subsequent deep fusion of multimodal features.

[0035] Furthermore, this application proposes specific steps for extracting shallow physical features of each mode from spatially registered multimodal data, including: For visible light reflection images, texture roughness and directional features are extracted using the gray-level co-occurrence matrix; For infrared transmission images, extract the binarized skeleton and connected component features of the infrared absorbing ink region; Based on the ultraviolet fluorescence response data, statistical characteristics of the distribution density of fluorescent fibers and the brightness of the fluorescence pattern were extracted; For the two-dimensional magnetic distribution map, the peak position of magnetic flux along the safety line and the waveform slope characteristics of the magnetic ink region are extracted.

[0036] At the feature extraction level, customized strategies were adopted for the physical characteristics of different modalities. For visible light reflectance images, the Gray-Level Co-occurrence Matrix (GLCM) was used to deeply mine the spatial correlation between pixels. This statistical method can effectively separate the roughness and directionality of the texture from the background noise, and remains robust even when there are stains on the banknote surface. For infrared transmission images, the technical solution introduced a binarized skeleton extraction technique, which transforms the infrared absorption region into a simplified representation with a clear topological structure through adaptive thresholding, thereby ensuring that the continuity of the anti-counterfeiting pattern can still be accurately judged when the ink boundaries are blurred due to banknote aging and fading. In the ultraviolet fluorescence dimension, the system quantifies the distribution density and brightness characteristics of fluorescent fibers per unit area through gridded statistics or region sampling. For the two-dimensional magnetic distribution map, differential positioning or peak detection algorithms are used to accurately locate the peak coordinates and waveform change rate of magnetic flux along the security line to capture subtle magnetic anti-counterfeiting features.

[0037] As a specific implementation method, based on the aforementioned registered dataset The system extracts shallow physical features for each mode. .

[0038] For visible light images ， System constructs gray-level co-occurrence matrix To extract texture roughness features .in For pixel grayscale levels, Pixel distance, This is the direction angle. The formula for calculating (i.e., contrast ratio) is as follows: For the resampled magnetic vector The system extracts waveform change rate features. .set up For the first in the sequence The magnetic flux values ​​at each point are used to calculate the local slope using a first-order difference operator: in To smooth the window radius. Finally, the system will extract features from each modality (including...) (And other modal features) are concatenated and mapped to construct an initial multimodal feature set. ,in This represents the total number of modal channels.

[0039] In detail, the aforementioned feature extraction scheme fully considers the complex interferences that banknotes may encounter in actual circulation. By replacing traditional threshold segmentation with a gray-level co-occurrence matrix, the damage to texture extraction caused by dirty areas is avoided; the combination of binarized skeleton and connected component features gives the system the ability to resist ink aging and blurring; and the introduction of dual statistical features quantifies the degree of fluorescence response decay. In particular, the joint analysis of magnetic features can keenly capture anomalies in magnetic waveforms when sophisticated alterations such as security thread displacement or ink tampering occur. These highly targeted feature extraction methods, together with the aforementioned spatial correction steps, form a powerful complement to construct a highly robust multimodal feature set.

[0040] The above technical solution effectively solves the problem that general feature extraction methods cannot accurately capture key anti-counterfeiting information when banknotes are subjected to complex conditions such as oil stains, mechanical wear, or aging and fading. It significantly improves the reliability of multimodal fusion and the accuracy of authenticity determination.

[0041] Furthermore, this application proposes a specific scheme for weighted fusion of multimodal feature sets using an attention mechanism, including the following steps: constructing a dual attention network containing a channel attention module and a spatial attention module; using the channel attention module to analyze the importance of the multimodal feature set in the channel dimension and generating channel weight coefficients; using the spatial attention module to analyze the response intensity of the feature map in the spatial dimension and generating spatial position weights; and combining the channel weight coefficients and spatial position weights to achieve weighted fusion of the multimodal feature set.

[0042] The dual attention network constructed in this embodiment is a deep neural network architecture capable of processing channel and spatial dimension information in parallel. The channel attention module acts as a "feature filter," dynamically evaluating the statistical reliability of each modality signal based on metrics such as signal-to-noise ratio and mutual information. The spatial attention module, on the other hand, focuses on "region focusing," intelligently identifying key anti-counterfeiting areas of the banknote by analyzing the local contrast and edge sharpness of the feature map. This design overcomes the limitations of traditional single attention mechanisms, achieving a comprehensive evaluation of features.

[0043] As a specific implementation method, in order to initialize the feature set... To optimize and integrate the system, a channel attention weight vector was introduced. Assume a set of features. ,in This represents the spatial dimensions of the feature map. First, channel descriptors are obtained through global average pooling. , No. Descriptors of each channel The calculation is as follows: Subsequently, normalized channel weights are generated using a multilayer perceptron. : in The learnable weight matrix activation function for the fully connected layer. Use the Sigmoid function to ensure that the output weights are within the range of... Between. The final fused feature vector. It is by using the original features With weight The result after flattening by weighting each channel is: in This represents the Hadamard product of the channel dimension. This represents flattening a multidimensional tensor into a one-dimensional vector. This is a fused feature vector that includes weighted information, used for subsequent decision-making.

[0044] A deeper analysis of this fusion strategy reveals its core advantage lies in its dynamic weight allocation mechanism. On one hand, channel weight coefficients can be adjusted in real time based on the quality of the modal signal. For example, when a significant attenuation of the ultraviolet fluorescence signal due to banknote aging is detected, the system automatically reduces the weight of that channel, relying instead on information from the infrared or magnetic channels. On the other hand, spatial location weights are based on feature distribution characteristics, providing targeted enhancement to key areas. For instance, when the texture of the security thread area is unclear due to oil stains, the algorithm automatically strengthens the waveform slope characteristics at the corresponding location in the magnetic distribution map. Through this two-dimensional weight superposition, the system can retain high-quality signals while maximally suppressing noise interference in damaged areas, generating a highly discriminative fusion feature vector.

[0045] Furthermore, this application proposes a specific implementation method for dynamically allocating fusion weights, including the following steps: real-time monitoring of the local contrast and data missing rate of each modal signal; when the local contrast of a certain modal signal is lower than a preset threshold or there is regional data missing, the weight of the corresponding feature of that modality is automatically reduced during the fusion process; and the weight of other undisturbed modal features is increased simultaneously to maintain the overall energy stability of the fused feature vector and achieve cross-modal feature compensation.

[0046] To achieve refined weight control, the system introduces local contrast and data missing rate as core monitoring indicators. Local contrast, by calculating the gray-level difference or gradient magnitude between adjacent pixels (e.g., using the Sobel operator), accurately reflects the loss of regional details on the banknote surface caused by oil stains or wear. Data missing rate, on the other hand, quantifies the information loss caused by physical obstruction or sensor malfunction by statistically analyzing the proportion of invalid pixels or outliers. The key to this monitoring mechanism is that its focus shifts from global averages to local details, thus providing a precise basis for fine-tuning the weights.

[0047] As a specific implementation method, in order to enhance the system's adaptability to dirty and worn banknotes, the aforementioned channel weights... It will be further affected by the signal quality factor. The system defines a local contrast ratio. To measure the first Signal quality of each mode. If Below the preset threshold Then a dynamic attenuation coefficient is introduced. Adjust the weights: in Hyperparameters used to control the steepness of decay. The final weights after correction. The calculation is as follows: The denominator term in the formula It plays a normalization role, ensuring that when a certain mode (such as visible light) is affected by... Lower levels lead to During descent, the weights of other modes (such as magnetic) It will automatically increase proportionally, thereby maintaining the total energy conservation of the feature vector and achieving cross-modal compensation.

[0048] In practical operation, this signal quality-driven adaptive mechanism plays a crucial role. Once the local contrast of a certain mode is detected to fall below a preset threshold, or a regional data black hole appears, the system immediately triggers weight reduction logic to actively suppress these noise-dominated feature components from participating in the fusion. Simultaneously, to maintain the overall energy conservation of the fused feature vector, the algorithm synchronously increases the weights of undisturbed modes. For example, in extreme cases where the visible light signal is rendered ineffective due to severe surface contamination, the weights of infrared transmission or magnetic features are significantly increased, utilizing these highly penetrating and interference-resistant modal information to complete feature compensation. This dynamic ebb and flow ensures that the system consistently outputs stable and reliable identification results when dealing with dirty or damaged banknotes.

[0049] Furthermore, this application proposes specific steps for inputting the fused feature vector into the pre-trained classification decision model, including: using a multilayer perceptron as the main structure of the classification decision model; calculating the Euclidean distance or cosine similarity between the fused feature vector of the banknote to be identified and the standard genuine banknote feature templates of each currency type; and, based on the matching degree, first determining the country and version of the banknote, and then determining the authenticity of the banknote based on the feature matching degree of a specific anti-counterfeiting area.

[0050] The classification and decision model primarily employs a multilayer perceptron (MLP). This deep structure, composed of fully connected layers, activation functions, and regularization techniques, can efficiently deconstruct the complex relationships within high-dimensional feature vectors through nonlinear transformations. Regarding similarity measurement, the system does not limit itself to a single metric but flexibly utilizes Euclidean distance and cosine similarity: the former excels at quantifying numerical differences in intact banknotes, while the latter demonstrates stronger adaptability in handling brightness fluctuations or localized degradation in soiled banknotes. The final matching degree calculation is based on this dynamically selected measurement method, achieving compatibility with banknotes of varying degrees of degradation.

[0051] As a specific implementation method, in obtaining the final fused feature vector Then, the classification and decision model calculates its comparison with the standard genuine currency feature template. The matching degree between them. The system uses Euclidean distance. Similarity to cosine Combined indicators: Final judgment score Defined as: in The preset maximum distance normalization factor, This is the balance coefficient. If... Greater than the authenticity threshold If it is true, it is considered genuine; otherwise, it is considered counterfeit or abnormal.

[0052] From a logical perspective, the phased judgment strategy significantly improves the system's robustness. The multilayer perceptron first identifies the country of origin and version of the banknote at the global feature level, effectively avoiding a chain of misjudgments caused by using incorrect templates. Subsequently, the judgment focus shifts to the matching of local features in specific anti-counterfeiting areas to confirm the banknote's authenticity. This surface-to-point judgment process fully utilizes the prior constraints of version information on anti-counterfeiting features and greatly enhances the targeted nature of authenticity verification. Especially for sophisticated counterfeit banknotes, this in-depth feature mining and multi-dimensional quantitative mechanism can effectively expose their disguise.

[0053] Furthermore, after presenting the country of origin, version, denomination, and authenticity verification results of the exported banknotes, this application also includes the following steps: If the matching degree is within the preset ambiguity range for true and false judgment, then the identification is marked as a suspicious event, and the corresponding multimodal original signal and fused feature vector are stored in the abnormal sample library. The classification and decision model is incrementally learned periodically using an abnormal sample library to update the standard genuine currency feature template and adaptively optimize the identification performance.

[0054] Specifically, the matching degree refers to the similarity between the fused feature vector of the banknote to be authenticated and the feature template of a standard genuine banknote, which can be achieved by calculating Euclidean distance or cosine similarity. In practical applications, the fuzzy range of the matching degree can be set as a dynamic threshold range based on historical data statistics. This is to avoid misjudgment caused by a single fixed threshold. Suspicious events can be understood as boundary cases where the system cannot clearly determine authenticity. Their processing methods include, but are not limited to, marking, storing, and subsequent analysis, with the aim of providing a basis for system improvement.

[0055] The anomaly sample library refers to a dataset specifically used to store multimodal raw signals and fused feature vectors with matching degrees in the ambiguity range. It can be implemented using a distributed database or a local storage structure. The classification and decision model is periodically incrementally trained using the anomaly sample library. Specifically, this can be achieved by fine-tuning the model parameters based on the mini-batch gradient descent algorithm. The goal is to achieve lightweight iterative updates of the model while reducing the resource consumption of full retraining.

[0056] As one specific implementation method, when the decision score In the fuzzy range Time (of which) The system performs incremental learning to update the standard template. The updated formula is as follows: in As the current standard feature template, The feature vector of the sample to be identified that falls into the fuzzy region. The learning rate is set to 0.001 (e.g., 0.001). This formula allows the standard template to slowly drift towards the true feature distribution of banknotes in the current circulation environment, thus adaptively accommodating the feature variations of dirty and worn banknotes.

[0057] In detail, when the matching degree falls within the preset ambiguity range for authenticity determination, the system marks the identification as a suspicious event, avoiding rigid judgments based on a single threshold and thus adopting a prudent strategy under uncertainty. This mechanism effectively prevents misclassification problems caused by banknotes being stained, worn, or having altered features close to the threshold. Simultaneously, the corresponding multimodal raw signals and fused feature vectors are stored in an anomaly sample library, ensuring the complete preservation of key data and providing a foundation of challenging samples in real-world scenarios for subsequent model optimization. Based on this, incremental learning dynamically adjusts model parameters, not only achieving continuous updates to the standard genuine banknote feature template but also enabling the system to adaptively optimize identification performance, thereby improving robustness against banknotes with complex postures, dirt, and sophisticated alterations.

[0058] Through the above technical solutions, the system has formed a closed-loop self-evolutionary capability of "detection-storage-learning-optimization," fundamentally solving the identification reliability problem caused by boundary cases. This mechanism, combined with the aforementioned multimodal signal acquisition, spatial registration, and weighted fusion steps, jointly improves the overall identification performance of the system, especially demonstrating higher accuracy and adaptability when dealing with dirty and old banknotes from multiple countries and sophisticatedly altered banknotes.

[0059] Furthermore, this application proposes a multi-dimensional sensor group including a visible light image sensor and a near-infrared image sensor. The visible light image sensor and the near-infrared image sensor share the same optical path through a beam splitter to ensure that the acquired visible light reflection image and infrared transmission image are naturally registered in physical space.

[0060] A multi-dimensional sensor array refers to a hardware module that integrates multiple types of sensors to simultaneously acquire signals from different modalities. It can be implemented using a combination of visible light and near-infrared image sensors, aiming to provide unified hardware support for multi-modal signal acquisition. The visible light and near-infrared image sensors share the same optical path via a beam-splitting prism. Specifically, the principle of optical beam splitting is used to decompose the incident light into different wavelengths and transmit them separately to the two sensors. This aims to eliminate spatial deviations caused by independent sensor settings, thereby improving the spatial consistency of multi-modal data.

[0061] Specifically, this solution fundamentally solves the spatial registration barrier of multimodal images by constructing a hardware architecture that shares the same optical path between a visible light image sensor and a near-infrared image sensor. The visible light and near-infrared image sensors integrated into the multi-dimensional sensor array can simultaneously acquire optical signals from different wavelengths, providing a high-precision raw data foundation for subsequent feature extraction and fusion. After sharing the optical path through a beam-splitting prism, the incident light is split and received by the two sensors. Since the sensors acquire banknote information from the same optical path and viewing angle, the visible light reflection image and the infrared transmission image are naturally aligned in physical space. This natural registration mechanism avoids image misalignment caused by the physical separation of the sensors and maintains spatial consistency of the image without relying on subsequent software correction steps. Furthermore, this design significantly improves the reliability of feature matching when dealing with banknotes in complex poses, effectively addressing spatial deviations caused by banknote tilt or offset transmission.

[0062] In summary, the above technical solutions not only solve the problem of accurate alignment of multimodal images in physical space, but also provide high-quality data support for subsequent multimodal feature extraction and fusion, thereby improving the accuracy and robustness of the banknote identification process.

[0063] In some of the solutions described above in this application, an output format for the identification results is proposed to provide multi-dimensional output options. However, in this process, the output mechanism fails to effectively support real-time information visualization, automated physical sorting, and remote data integration, resulting in low operational efficiency, high sorting error rates, and a lack of data traceability in cross-border high-frequency circulation scenarios, especially failing to meet actual business needs when dealing with dirty and worn banknotes from multiple countries.

[0064] Furthermore, this application proposes the following technical solution: The output formats of the identification results include: The interface displays the current banknote's attributes and total value in real time. The mechanical sorting device is driven by a control interface to physically separate genuine banknotes from counterfeit or unidentifiable banknotes. The identification records and feature data are uploaded to the upper management system through the data communication interface.

[0065] Specifically, the display interface refers to a visual device that can dynamically present key attribute information of banknotes. It can be implemented using an LCD screen, LED dot matrix screen or touch screen. The purpose is to provide operators with intuitive result feedback, reduce manual verification steps and improve processing efficiency.

[0066] The control interface can be understood as a hardware or software module used to transmit sorting instructions. It can be implemented through relay circuits, serial communication protocols or industrial buses. Its purpose is to ensure that the identification results can be directly converted into accurate physical action instructions, thereby avoiding sorting misjudgments caused by posture deviation or dirt.

[0067] In practical applications, a data communication interface refers to a functional module that supports data transmission and remote management. It can be implemented using an Ethernet interface, a USB interface, or a wireless communication module. Its purpose is to centrally store and analyze raw signals and feature vectors, providing a data foundation for subsequent model optimization and anomaly event tracing.

[0068] Specifically, this solution addresses the issue of limited application of authentication results in existing technologies by defining multi-dimensional output formats, achieving closed-loop management from result generation to actual operation. First, the display interface shows the current banknote's attribute information and accumulated amount in real time, enabling operators to instantly obtain key attributes such as the banknote's country of origin, version, denomination, and authenticity status, while simultaneously tracking the accumulated processing amount, thus quickly confirming results in high-frequency cross-border transaction scenarios. Second, the control interface drives a mechanical sorting device to physically separate genuine banknotes from counterfeit or unidentifiable banknotes, significantly improving automated processing efficiency. Finally, the data communication interface uploads authentication records and feature data to the upper-level management system, enhancing the system's adaptability and management capabilities in complex circulation environments.

[0069] Furthermore, this solution, combined with the aforementioned multimodal signal acquisition, spatial registration, and feature fusion technologies, forms a complete banknote identification and inspection process. Through the coordinated design of the display interface, control interface, and data communication interface, not only is the overall system operating efficiency improved, but reliable technical support is also provided for handling complex scenarios involving dirty and worn banknotes from multiple countries.

[0070] In another implementation, such as Figure 2 As shown in the embodiments, this application also discloses a banknote identification and inspection system based on photoelectric multimodal fusion, used to implement the aforementioned method, including: The signal acquisition module 100 is arranged along the banknote transmission path and integrates a multi-dimensional sensor group to acquire the multi-modal raw signals of the banknote to be identified. The central processing module 200 is electrically connected to the signal acquisition module 100 and is used to perform spatial geometric correction, feature extraction, weighted fusion and classification decision of the multimodal raw signal; The storage module 300 is connected to the central processing module 200 and is used to store standard genuine currency feature templates and an anomaly sample library. The human-computer interaction and execution module 400 is connected to the central processing module 200 and is used to display the identification results and perform banknote sorting operations.

[0071] Specifically, this system integrates multimodal signal acquisition, precise spatial registration, deep feature fusion, and intelligent decision-making through a modular architecture design, effectively overcoming the identification failure problems caused by signal spatial misalignment and isolated modal processing in existing technologies. The signal acquisition module 100 is arranged along the banknote transmission path. This layout ensures that the sensor captures data in real time during the dynamic transmission of the banknote, avoiding signal timing deviations caused by transmission speed fluctuations, and providing a time-synchronized multimodal raw signal foundation for subsequent processing. The integrated multidimensional sensor group synchronously acquires visible light reflection images, infrared transmission images, ultraviolet fluorescence response data, and magnetic distribution sequences, enabling synchronous acquisition of signals from different physical dimensions at the source, creating a prerequisite for cross-modal pixel-level registration.

[0072] The central processing module 200 is electrically connected to the signal acquisition module 100. This connection ensures low-latency transmission of the original signal. During spatial geometric correction, an affine transformation matrix is ​​generated based on the edge detection results of the visible light reflectance image. This matrix is ​​then used to perform geometric correction on the infrared transmission image and ultraviolet fluorescence response data, ensuring accurate alignment of the multimodal images in the standard coordinate system. This solves the core problem of feature extraction misalignment under banknote tilt or offset postures. In the feature extraction stage, shallow physical features for each modality are generated based on the spatially registered multimodal data. For example, texture roughness features are extracted from the visible light reflectance image, and features are extracted from the infrared transmission image. Binarized skeleton features are used to capture different anti-counterfeiting characteristics, avoiding misjudgments caused by the lack of information in a single modality. The weighted fusion process dynamically allocates weights based on the mutual information and signal-to-noise ratio of each feature component. When the ultraviolet fluorescence response data is regionally missing due to banknote dirt, the weight of that modality is automatically reduced and the weight of the magnetic distribution feature is increased, achieving cross-modal feature compensation, strengthening effective information and suppressing noise interference. The classification decision is based on the matching degree calculation between the fused feature vector and the standard genuine banknote feature template. The country of origin, version and authenticity are determined step by step according to Euclidean distance or cosine similarity, ensuring the reliability of identification in complex scenarios.

[0073] The storage module 300 is connected to the central processing module 200, storing standard genuine banknote feature templates for rapid comparison. It also maintains an abnormal sample library; when the matching degree is in an ambiguous range, the multimodal original signals and fused feature vectors of suspicious samples are stored in this library. These samples are periodically used to incrementally learn the classification decision model, enabling the standard feature templates to be updated adaptively, significantly improving the system's long-term adaptability to aging, faded, or altered banknotes. The human-machine interaction and execution module 400 is connected to the central processing module 200, displaying real-time banknote attribute information and cumulative amounts, facilitating operator monitoring of high-frequency circulation scenarios; driving the mechanical sorting device to perform physical sorting, ensuring accurate separation of genuine and counterfeit banknotes; and uploading identification records through a data communication interface, supporting risk analysis by the upper-level management system. This transforms the technical solution into practical application effectiveness, comprehensively solving the industry problem of insufficient accuracy in identifying dirty and worn banknotes under complex conditions.

[0074] Through the above technical solutions, this application can demonstrate higher reliability and adaptability in the identification scenarios of banknotes with complex postures, dirt, and alteration, providing strong technical support for currency identification in cross-border cash circulation.

[0075] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A banknote authentication and inspection method based on photoelectric multimodal fusion, characterized in that, Includes the following steps: A multi-dimensional sensor array arranged in the banknote transmission channel is used to synchronously collect multi-modal raw signals of the banknote to be identified. The multi-dimensional sensor array includes at least sensors for acquiring visible light reflection images, infrared transmission images, ultraviolet fluorescence response data, and magnetic distribution sequences. Spatial geometric correction and signal normalization processing are performed on the original multimodal signals to uniformly map signals of different physical dimensions to a standard coordinate system, thereby obtaining spatially registered multimodal data; Shallow physical features of each mode are extracted from the spatially registered multimodal data, and the shallow physical features are mapped to an isomorphic high-dimensional feature space to construct a multimodal feature set containing texture details, spectral characteristics and magnetic features. The multimodal feature set is weighted and fused using an attention mechanism. The fusion weights are dynamically allocated by calculating the mutual information and signal-to-noise ratio of each feature component, thereby generating a fused feature vector that enhances effective information and suppresses noise. The fused feature vector is input into a pre-trained classification and decision model to calculate its matching degree with the standard genuine currency feature template, and the country, version, denomination and authenticity information of the banknote are output according to the matching result.

2. The banknote identification and inspection method based on photoelectric multimodal fusion according to claim 1, characterized in that, The specific steps for performing spatial geometric correction and signal normalization on the multimodal raw signal include: The physical boundaries of the banknote in the visible light reflection image are identified using an edge detection algorithm, and the deflection angle and displacement deviation of the banknote relative to the transmission channel are calculated. Based on the deflection angle and displacement deviation, an affine transformation matrix is ​​constructed, and the visible light reflection image, the infrared transmission image, and the ultraviolet fluorescence response data are geometrically corrected to align them to a standard pose, thereby obtaining registered multimodal image data. Based on the banknote transmission speed and sensor sampling frequency, the magnetic distribution sequence is interpolated, resampled, and spatially mapped to generate a two-dimensional magnetic distribution map with the same spatial scale as the registered multimodal image data.

3. The banknote identification and inspection method based on photoelectric multimodal fusion according to claim 2, characterized in that, The specific steps for extracting shallow physical features of each mode from the spatially registered multimodal data include: For the visible light reflection image, texture roughness and directional features are extracted using the gray-level co-occurrence matrix; For the infrared transmission image, extract the binarized skeleton and connected component features of the infrared absorbing ink region; Based on the aforementioned ultraviolet fluorescence response data, statistical characteristics of the distribution density of fluorescent fibers and the brightness of the fluorescence pattern were extracted; For the two-dimensional magnetic distribution map, the peak position of magnetic flux along the safety line and the waveform slope characteristics of the magnetic ink region are extracted.

4. The banknote identification and inspection method based on photoelectric multimodal fusion according to claim 1, characterized in that, The weighted fusion of the multimodal feature set using an attention mechanism specifically includes: Construct a dual attention network that includes a channel attention module and a spatial attention module; The channel attention module is used to analyze the importance of the multimodal feature set in the channel dimension and generate channel weight coefficients; The spatial attention module is used to analyze the response intensity of the feature map in the spatial dimension to generate spatial position weights; By combining the channel weight coefficients and the spatial location weights, a weighted fusion of the multimodal feature set is achieved.

5. The banknote identification and inspection method based on photoelectric multimodal fusion according to claim 4, characterized in that, The dynamic allocation of fusion weights specifically includes: Real-time monitoring of local contrast and data loss rate of each modal signal; When the local contrast of a certain modal signal is lower than a preset threshold or there is regional data loss, the weight of the corresponding feature of that modality is automatically reduced during the fusion process. Simultaneously, the weights of other undisturbed modal features are increased to maintain the overall energy stability of the fused feature vector and achieve cross-modal feature compensation.

6. The banknote identification and inspection method based on photoelectric multimodal fusion according to claim 1, characterized in that, The specific steps of inputting the fused feature vector into the pre-trained classification decision model include: A multilayer perceptron is used as the main structure of the classification decision model. Calculate the Euclidean distance or cosine similarity between the fused feature vector of the banknote to be identified and the feature template of the standard genuine banknote for each currency type; Based on the matching degree, the country and version of the banknote are first determined, and then the authenticity of the banknote is determined based on the feature matching degree of a specific anti-counterfeiting area.

7. The banknote identification and inspection method based on photoelectric multimodal fusion according to claim 6, characterized in that, The information regarding the country of origin, version, denomination, and authenticity of the exported banknotes also includes: If the matching degree is within the preset fuzzy range of true and false judgment, then the identification is marked as a suspicious event, and the corresponding multimodal original signal and the fused feature vector are stored in the abnormal sample library. The classification and decision model is incrementally learned periodically using the abnormal sample library to update the standard genuine currency feature template and adaptively optimize the identification performance.

8. The banknote identification and inspection method based on photoelectric multimodal fusion according to claim 1, characterized in that, The multi-dimensional sensor group includes a visible light image sensor and a near-infrared image sensor. The visible light image sensor and the near-infrared image sensor share the same optical path through a beam splitter to ensure that the acquired visible light reflection image and the infrared transmission image are naturally registered in physical space.

9. The banknote identification and inspection method based on photoelectric multimodal fusion according to claim 1, characterized in that, The output formats for the country of origin, version, denomination, and authenticity information of the banknotes include: The interface displays the current banknote's attributes and total value in real time. The mechanical sorting device is driven by a control interface to physically separate genuine banknotes from counterfeit or unidentifiable banknotes. The identification records and feature data are uploaded to the upper management system through the data communication interface.

10. A banknote authentication and verification system based on photoelectric multimodal fusion, used to implement the method according to any one of claims 1 to 9, characterized in that, include: The signal acquisition module, arranged along the banknote transmission path, integrates the multi-dimensional sensor group to acquire the multi-modal raw signals of the banknote to be identified. The central processing module is electrically connected to the signal acquisition module and is used to perform spatial geometric correction, feature extraction, weighted fusion and classification decision of the multimodal raw signal; A storage module, connected to the central processing module, is used to store the standard genuine currency feature templates and the abnormal sample library; The human-computer interaction and execution module is connected to the central processing module and is used to display the identification results and perform banknote sorting operations.

Images