Decentralized authentication of anti-counterfeit qr codes using federated learning based on visual transformers

Abstract

A privacy-preserving authentication method of anti-counterfeit QR codes using federated learning (FL) based on visual transformers (ViTs) is provided. In the method, individual clients authenticate anti-counterfeit QR codes captured in images presented to the individual clients. Local machine learning (ML) models of the individual clients determine authenticity of the anti-counterfeit QR codes captured in the images presented to the individual clients. The local ML models are initialized as pre-trained models based on ViTs that are pre-trained on a large-scale ImageNet dataset to process input images to determine authenticity of anti-counterfeit QR codes captured in the input images. The plurality of clients perform a cyclic weight transmission FL process to update respective local ML models of the plurality of clients according to respective instant training data owned by different clients of the plurality of clients, while protecting training data privacy among the different clients.

Description

[0001] abbreviation

[0002] CDP Replication Detection Mode

[0003] CLTP Circumferential Local Ternary Mode

[0004] CNN (Convolutional Neural Network)

[0005] CWT Cyclic Weight Transfer

[0006] DMFNet dual-branch multi-scale feature fusion network

[0007] FedAVG Federal Average

[0008] FG-DPANet Feature-Guided Dual-Pool Attention Network

[0009] FL Federal Learning

[0010] ML Machine Learning

[0011] MLP (Multilayer Perceptron)

[0012] PUF (Physically Unclonable Function)

[0013] QR Quick Response

[0014] SA Self-Attention

[0015] SGD (Stochastic Gradient Descent)

[0016] TACA Triple Anti-counterfeiting Authentication

[0017] ViT Visual Converter Technical Field

[0018] This disclosure relates to an ML technique using a ViT-based model and FL for authenticating anti-counterfeiting QR codes captured in an image presented to an individual client from among multiple clients. Background Technology

[0019] In recent years, as the core sensing technology of the Internet of Things and an important information entry point of the Internet, QR codes have been widely used for product information traceability and anti-counterfeiting[1]. The principle of QR code anti-counterfeiting traceability is to generate a unique QR label for each product and establish a reliable anti-counterfeiting mark. Existing QR code product authentication systems rely on serial numbers. That is, users scan the QR code with their smartphones and decode it to obtain the serial number information, and then initiate an authentication request. The authentication system returns the authentication result based on the serial number[2]. However, the above authentication scheme is vulnerable to illegal copying attacks, which are usually achieved by scanning and printing real QR codes[3].

[0020] To enhance the security and anti-counterfeiting capabilities of QR codes, researchers have developed various types of copy-resistant QR codes by combining them with additional anti-counterfeiting measures. These measures include digital watermarking [4-6], halftone encryption [7-9], PUF [10-12], CDP [13,14], and anti-counterfeiting patterns [15-19]. These copy-resistant QR codes are designed in such a way that any attempt to copy them results in distortion or alteration of the pattern's details, thus enabling the development of corresponding authentication methods to achieve anti-counterfeiting. Among these methods, anti-counterfeiting patterns stand out due to the random distribution of their fine textures. These patterns can be directly embedded into the QR code during printing, offering advantages such as low cost, high copy sensitivity, and extreme difficulty in counterfeiting. Given their increasing attention, this disclosure focuses on anti-counterfeiting QR codes with embedded anti-counterfeiting patterns. Figure 1 An example of an anti-counterfeiting QR code is shown, depicting an anti-counterfeiting QR code 100 having a normal data-carrying area 110 and an anti-copying pattern 120 located in the center of the QR code 100.

[0021] Various detection methods have been proposed to effectively capture forgery features in illegal copying, including spectral and spatial channel models

[20] , Gaussian models based on channel noise characteristics, CLTP

[16] , TACA

[21] , DMFNet

[22] , and FG-DPANet

[17] . These detection methods can be classified into two main strategies: manual feature extraction and deep learning methods.

[0022] Manual feature extraction based on expert experience involves theoretical analysis and rigorous modeling, providing high interpretability. However, this approach is often complex and time-consuming due to the need for extensive experimentation and validation

[23] . Deep learning methods, represented by CNNs, can automatically learn representations of forged features in a data-driven manner and have become one of the most popular methods for most computer vision tasks due to their highly competitive performance compared to manual feature extraction

[24] . These methods typically involve centralized training and testing by aggregating QR code data from multiple mobile clients on a central server [2,3,15], making them the dominant approach for forgery detection. However, increasingly stringent privacy legislation and growing concern for data privacy pose unprecedented challenges to the traditional centralized training paradigm [25,26]. Consumers are reluctant to transfer and share their local private data due to concerns about data breaches, making centralized models of data collection, storage, and processing increasingly unsustainable.

[0023] In summary, all the methods described above [2, 3, 15-17, 20-26] operate in a centralized authentication mode. That is, data needs to be uploaded to a central server for processing and authentication. There is a need in this field for decentralized authentication technologies used to authenticate anti-counterfeiting QR codes. Summary of the Invention

[0024] One aspect of this disclosure is to provide a method for authenticating an anti-counterfeiting QR code captured in an image presented to an individual client from among multiple clients. This method achieves decentralized and privacy-preserving authentication of the anti-counterfeiting QR code.

[0025] In this method, when an image is presented to an individual client, the individual client uses its local ML model to determine the authenticity of the anti-counterfeiting QR code captured in the image. The individual client's local ML model is initialized as a local copy of an ML model shared by multiple clients. The ML model is a ViT-based model that has been pre-trained to process the input image to determine the authenticity of the captured anti-counterfeiting QR code. Furthermore, the multiple clients perform a CWT FL process to update their respective local ML models based on real-time training data available to each client, while protecting training data privacy across different clients.

[0026] In some embodiments, the CWT FL process includes: sorting a plurality of clients to generate an ordered list of clients; and forming an extended ordered list of clients by repeating the ordered list of clients a predetermined number of times. The CWT FL process also includes a sub-process of repeatedly fine-tuning the local ML model of the currently selected client until predetermined convergence conditions for the respective local ML models of the plurality of clients are met, or until the respective clients, ordered sequentially according to the extended ordered list of clients, are used as the currently selected client in the running sub-process. This sub-process includes: fine-tuning the local ML model of the currently selected client using the corresponding immediate multiple training data possessed by the currently selected client; identifying the next client from the extended ordered list of clients, such that the next client becomes the currently selected client in the next execution of the sub-process; and if the next client is identifiable, replacing the local ML model of the next client with the local ML model of the currently selected client after fine-tuning the local ML model of the currently selected client, such that the corresponding immediate multiple training data possessed by the currently selected client is used to update the local ML model of the next client, but not disclosed to the next client. In addition, the CWT FL process also includes updating the corresponding local ML models of the multiple clients using the local ML model of the currently selected client used in the last execution of the subprocess.

[0027] In some embodiments, the ML model includes first and second plurality of model parameters for configuring the ML model, such that the local ML model of an individual client is configured by the corresponding first and second plurality of model parameters of that individual client. During the execution of the CWT FL process, the corresponding first plurality of model parameters of the individual client are fixed, while the corresponding second plurality of model parameters of the individual client are adjustable to fine-tune the local ML model of the individual client during the CWT FL process. Advantageously, replacing the local ML model of the next client with the local ML model of the currently selected client during the execution of a subprocess includes: overwriting the corresponding second plurality of model parameters in the local ML model of the next client with the corresponding second plurality of model parameters in the local ML model of the currently selected client, thereby replacing the local ML model of the next client with the local ML model of the currently selected client.

[0028] In some embodiments, a first plurality of model parameters configure the ML model to identify the edges and shape of the anti-counterfeiting QR code.

[0029] In some embodiments, multiple training data sets corresponding to the individual client are generated based on the authentication results obtained by authenticating the QR code image received by the individual client using the individual client's local ML model.

[0030] In some embodiments, the CWT FL process is repeated from time to time to periodically update the corresponding local ML model.

[0031] In some embodiments, the ViT-based model is selected as the ViT(S) model.

[0032] In some embodiments, the ML model is stored on a server serving multiple clients. The CWTFL procedure also includes updating the ML model using the local ML model of the currently selected client used in the previous execution of the subprocess, thereby allowing the server to initialize a new local ML model for a new client using the updated ML model when the server adds a new client to the multiple clients.

[0033] Other aspects of this disclosure are disclosed as shown in the following embodiments. Attached Figure Description

[0034] Figure 1 An example of an anti-counterfeiting QR code with an embedded anti-copying pattern is depicted.

[0035] Figure 2 ViT is described for use in authenticating anti-counterfeiting QR codes.

[0036] Figure 3 A pre-trained ViT model for anti-counterfeiting QR code authentication is described according to an exemplary embodiment of the present disclosure.

[0037] Figure 4 A schematic diagram of a ViT-based FL for anti-counterfeiting QR code authentication is depicted.

[0038] Figure 5 The impact of communication rounds on the performance of various models under the FedAVG algorithm is shown.

[0039] Figure 6 The impact of communication rounds on the performance of various models under the CWT FL algorithm is shown.

[0040] Figure 7 A flowchart depicts exemplary steps of the method disclosed herein, which is used for authenticating an anti-counterfeiting QR code captured in an image presented to an individual client among a plurality of clients.

[0041] Figure 8 Flowcharts depicting certain embodiments of the CWT FL process used in implementing the disclosed method are provided.

[0042] Those skilled in the art will understand that the elements in the figures are shown for simplicity and clarity only and are not necessarily depicted to scale. Detailed Implementation

[0043] As used herein, “anti-counterfeiting QR code” or “anti-copying QR code” refers to a QR code that includes one or more anti-counterfeiting features, wherein the individual anti-counterfeiting features are configured to produce a distortion or alteration of at least one pattern on the QR code in an attempt to copy the QR code.

[0044] As used in this article, "client" refers to a computing device that plays a client role according to the client-server architecture well-known in computer science. Examples of such computing devices include general-purpose computers, desktop computers, laptops, mobile computing devices, smartphones, tablets, etc. These computers may be equipped with cameras or imaging devices for capturing images.

[0045] This document discloses a decentralized method for authenticating anti-counterfeiting QR codes. After proposing and describing the decentralized method in detail, embodiments of this disclosure will be elaborated based on the disclosed details, examples, applications, etc.

[0046] I. The proposed decentralized approach

[0047] As an emerging research paradigm, FL [27,28] can train models on local data distributed across multiple mobile devices. Specifically, each client independently trains the model using its local dataset and then shares the model parameters with other participants. The actual data remains local, thus ensuring personal data privacy and reducing the risk of data leakage, making it suitable for anti-counterfeiting QR code authentication using mobile devices. However, due to the different environmental interferences and noise faced by multiple smart devices, CNN models struggle to achieve satisfactory authentication performance on multiple devices simultaneously. How to achieve decentralized high-precision authentication while protecting user privacy is a challenge that urgently needs to be addressed.

[0048] Recent research has shown that ViT exhibits better generalization ability and robustness than CNN in image classification [29-32], thanks to its self-attention-like architecture. Inspired by the above research, the inventors have innovatively introduced ViT into anti-counterfeiting QR code authentication to improve the performance of authentication models deployed on different smart devices.

[0049] Therefore, this disclosure proposes a privacy-preserving authentication scheme for anti-counterfeiting QR codes based on ViT and FL. Privacy protection refers to the practice of ensuring that ML models do not disclose any confidential information about the data owner during training or inference. It is worth noting that the proposed scheme belongs to the block-based counterfeiting detection method

[24] . That is, the anti-counterfeiting QR code image is first divided into non-overlapping blocks, which can effectively highlight the subtle differences in pattern distortion between real and counterfeit QR codes and avoid interference caused by image content. QR code data is distributed on each user's mobile device and the amount of data per client is small. Therefore, pre-trained models on the large dataset ImageNet-1k [33, 34] for transfer learning are introduced. These pre-trained models have shown good generalization ability. The initial and intermediate layers for identifying edges and shapes can be used without modification, and only the final layer is adjusted to adapt to the authentication task of anti-counterfeiting QR codes. In the FL framework, individual smartphones fine-tune the corresponding pre-trained models using local data and then share the model parameters with other participants. However, different smartphones face different lighting environments, camera fingerprint noise and blur jitter. To establish an authentication model that better adapts to individual mobile devices, the inventors innovatively introduced the CWT FL algorithm, in which weights are transferred and updated cyclically among clients to better capture and integrate the unique data distribution characteristics on each client, thereby improving the generalization ability of the authentication model on data from different clients. To the inventors' knowledge, there are no publicly available datasets in the field of QR code authentication research. Therefore, the inventors constructed a self-built dataset using nine printers and eight smartphones for experimentation. Experimental results show that the proposed method exhibits competitive performance while protecting data privacy compared to traditional centralized authentication schemes based on CNNs.

[0050] The following sections will elaborate on the details of the proposed method. First, a pre-trained visual converter model for anti-counterfeiting QR code authentication will be introduced. Then, the serial FL framework and CWT FL, which effectively protect personal privacy data, will be described. Finally, the proposed decentralized authentication scheme combining the above strategies will be detailed.

[0051] A. Pre-trained ViT for anti-counterfeiting QR code authentication

[0052] The Transformer architecture was first applied to the field of machine translation and then performed well in natural language processing tasks

[39] . As research continued, Transformers were also found to be applicable to image and video tasks and showed good results. Dosovitskiy et al.

[40] attempted to apply Transformers with global attention directly to computer vision tasks and proposed ViT, which outperformed CNNs on various computer vision tasks. Figure 2 The block diagram structure of ViT is described.

[0053] ViT first divides each image into n evenly distributed blocks, and then divides each block into tokens, embedding... Then, all these tokens are fed into a set of transformer blocks. Each transformer block uses SA to mix the tokens and MLP to perform channel-mode feature transformation. SA is used to aggregate global information, and the input token embedding tensor can be represented as... And for parameter W K W Q and W V Apply linear transformations respectively, where

[0054]

[0055]

[0056] Then, the SA module calculates the attention matrix and aggregates the token features, as follows:

[0057]

[0058] in It is a linear transformation, X = [z1, z2, ..., z n ] represents the aggregated token features, and d is the scaling factor. The output of SA is then normalized and fed into the MLP to generate the input for the next block. The MLP consists of two linear layers and a GELU layer, which transforms the input tokens into features Z:

[0059] Z′=MLP(Z). (5)

[0060] In authentication tasks using smartphones, the quality of the captured QR code image directly impacts the accuracy and reliability of the authentication results. Due to camera shake and environmental noise, captured QR code images can become degraded and distorted, affecting the overall quality of the QR code. Therefore, a robust authentication model is needed to address these issues.

[0061] Recent studies have found that ViT is highly robust to severe occlusion, perturbation and domain shift

[41] . Inspired by the above findings, this disclosure introduces ViT into QR code authentication. To the best of the inventors’ knowledge, this disclosure is the first attempt to use ViT in QR code authenticity recognition research. Compared with CNN, ViT can capture global relationships between elements well and has stronger representation capabilities. However, due to the lack of inductive bias from convolution, ViT needs to rely on a large number of training samples to fully learn local features, which requires a large amount of data. In order to reduce the need for data, this disclosure introduces a ViT model pre-trained on a large ImageNet-1K dataset and transfers the ViT model to the task of anti-counterfeiting QR code authenticity recognition. Figure 3 A schematic diagram depicts the entire example process for verifying the authenticity of an anti-counterfeiting QR code.

[0062] B. Overseen Federal Fine-Tuning

[0063] FL [42,43] is a promising distributed machine learning paradigm. Its key feature is that each client independently trains the model using local data and then shares the model parameters with other participants, thus protecting data privacy. Compared to traditional centralized learning methods, FL reduces the risk of data transmission and privacy breaches, while improving the scalability and adaptability of the model. Widely used FL algorithms include FedAVG and CWT.

[0064] FedAVG

[44] is a fundamental algorithm in FL that aggregates model updates from local computations on multiple client devices to form a global model. Each client trains its model on local data, and the server averages these model updates to iteratively improve the global model. This process protects data privacy by keeping data decentralized while enabling collaborative model training.

[0065] CWT

[45] is a typical serial FL method, in which local clients train in a serial and cyclic manner. In each round of training, CWT uses local data to train a global model on a local client, performs one or more loops, and then transfers this global model to the next client for training, until all local clients have completed one training cycle. This training process is repeated between clients until the model converges or a predetermined number of communication rounds are reached.

[0066] Due to CWT's multi-round cyclic transmission mechanism, individual participant models incorporate data features from different participants. Therefore, it can capture the global data distribution better than FedAVG. Furthermore, each participant's model is updated using parameters from multiple other participants, which reduces the impact of a single participant on the global model and improves the overall model's robustness. This disclosure innovatively deploys pre-trained visual transducer models in the two typical FL algorithms mentioned above, and then applies them to QR code authenticity recognition through fine-tuning, thereby realizing a robust QR code authentication method with privacy protection features.

[0067] C. ViT-based CWTFL for anti-counterfeiting QR code authentication

[0068] Most existing authentication methods involve scanning anti-counterfeiting QR codes and uploading them to global cloud servers. The servers then use stored databases and authentication models for comparison and analysis to arrive at the authentication result, which is then fed back to the user. This centralized authentication method poses a risk of privacy breaches. In fact, users are reluctant to share their private anti-counterfeiting QR code data due to concerns about data leakage, and this data often exists in silos across multiple users, making it difficult to aggregate into large datasets. Unlike previous centralized authentication methods, this disclosure defines QR code authentication as distributed authentication via smartphones. This approach better reflects the objective fact that QR codes are authenticated by each user's smartphone. Figure 4 As shown.

[0069] Due to the differences between the counterfeit QR code data from different printing devices and the different brands and models of smartphones used by different users, directly using FL to aggregate model parameters from multiple users may lead to poor authentication results. Furthermore, the direct model trained by users using local data cannot establish an effective recognition model due to the limited amount of known label data. To address the above issues, this disclosure proposes a federated transfer learning framework for QR code authentication. This framework uses the parameter transfer strategy of CWT

[28] to reduce the number of local model parameters and improve the security of parameter transfer. Finally, multiple users collaborate to build a shared model for transfer learning, and users fine-tune the shared model using local data to achieve QR code authentication.

[0070] The detailed process of the proposed method is shown in Algorithm 1. There are N local clients, and each client's local dataset is represented as D. i For each client, the local model is represented as i∈{1,2,…,N}.

[0071] Algorithm 1: ViT-CWT method.

[0072]

[0073] II. Experiment

[0074] This section introduces the experiments used to validate the proposed methods. Experiments were conducted on a newly created dataset on which all proposed methods were tested and evaluated. The effectiveness of the proposed methods was verified through a series of comparative experiments, including comparisons with previous centralized training models, comparisons with different FL algorithms, ablation studies, and anti-fuzzing experiments.

[0075] A. Dataset

[0076] At the time of our experiment, there were no publicly available datasets for studying decentralized authentication of anti-counterfeiting QR codes collected from multiple smartphones. Therefore, we built our own anti-counterfeiting QR code dataset for authentication.

[0077] 1) Creation and Anti-counterfeiting of Anti-counterfeiting QR Codes: First, eight types of anti-counterfeiting QR codes, 100 of each, were used, including C5-D1-Ft2, C5-D1-Ft3, C5-D1.2-Ft2, C5-D1.3-Ft3, C6-D1-Ft2, C6-D1-Ft3, C6-D1.2-Ft2, and C6-D1.2-Ft3, where C represents the QR code type, D represents the texture density of the anti-counterfeiting pattern, and Ft represents the fault tolerance level. This setup was intended to verify that our proposed method can be applied to various types of anti-counterfeiting codes. Next, an authorized printer, named Xerox DocuCentre color-C7500, was used to print genuine anti-counterfeiting QR codes, resulting in 800 genuine anti-counterfeiting QR codes. Then, the anti-counterfeiting QR codes were illegally copied using a scanner and an unauthorized printer. Considering the diversity of scanners and printers that might be used in the counterfeiting process, three scanners were used: an Epson-WF-M21000a for scanning C5-D1-Ft2, C5-D1-Ft3, and C5-D1.2-Ft2; a SHARP MX-5608N PCL6 for scanning C5-D1.3-Ft3, C6-D1-Ft2, and C6-D1-Ft3; and a Cannon 9000F mark2 for scanning C6-D1.2-Ft2 and C6-D1.2-Ft3. Eight illegal printers were used for counterfeiting, each counterfeiting only one type of anti-counterfeiting QR code. The correspondence between them is shown in Table I. As a result, 800 counterfeit anti-counterfeiting QR codes were obtained. Finally, using eight smartphones of different brands and models, which were also considered as clients, all genuine and counterfeit anti-counterfeiting QR codes were captured. The brand and model information of these smartphones is shown in Table I. Each smartphone collects 100 genuine anti-counterfeiting QR codes and 100 counterfeit anti-counterfeiting QR codes, resulting in a total of 1600 anti-counterfeiting codes. All anti-counterfeiting QR codes have a uniform image size of 512×512.

[0078] Table I: Brand and Model Information for Printers, Scanners, and Smartphones

[0079]

[0080]

[0081] After obtaining data on genuine and counterfeit anti-counterfeiting QR codes, we created an anti-counterfeiting QR code dataset for decentralized authentication using smartphones.

[0082] 2) Decentralized Anti-counterfeiting QR Code Dataset Collected Using Smartphones: We randomly sampled 10 genuine anti-counterfeiting codes and 10 counterfeit anti-counterfeiting codes from the data collected from each smartphone. A total of 80 genuine and 80 counterfeit anti-counterfeiting codes were obtained from 8 smartphones, which were used as the verification set. Similarly, we performed the same sampling procedure to obtain the test set. Finally, each smartphone retained 80 genuine and 80 counterfeit anti-counterfeiting codes as local training data for each client. In deep learning-based anti-counterfeiting QR code authentication, the differences between genuine and counterfeit anti-counterfeiting codes are very subtle. A block-based preprocessing method can effectively avoid interference from image content and amplify the differences between categories. Therefore, each anti-counterfeiting QR code was divided into 64 pieces, each piece being 64×64 pixels. Therefore, each client's local training set, validation set, and test set contain 5120 genuine anti-counterfeiting code chips and 5120 counterfeit anti-counterfeiting code chips, resulting in a total of 102,400 chips.

[0083] B. Implementation Details

[0084] The proposed method is implemented on the PyTorch deep learning framework, and our model is trained on an NVIDIA Geforce RTX4070 GPU platform with 12GB of memory. The local training batch size is set to 32, optimized using SGD, and the initial learning rate is 0.003. Since data is collected using eight smartphones, we set up eight different data centers as local clients. The number of local training epochs on each client is set to 1, and the total number of communication rounds is set to 100. For fair comparison, all pre-trained models are trained on ImageNet-1K

[46] . Meanwhile, since this disclosure focuses on decentralized authentication on smartphones, models with too many parameters and high complexity are not conducive to deployment on smartphones. Therefore, the size of all models used for comparison is less than 30MB.

[0085] C. Comparison with previous centralized training models

[0086] In this section, we compare our proposed method with existing methods on a self-constructed anti-counterfeiting QR code dataset, and we use classification accuracy as a quantitative indicator. As shown in Table II, we reimplement several cutting-edge deep learning algorithms widely used in the field of image forensics, including ResNet18

[47] , EfficientNet

[48] and ViT

[40] . For these algorithms, we adopt two training strategies: fine-tuning the pre-trained model and training from scratch. We also compare our proposed method with representative CNNs specifically for QR code authentication, namely DMFNet

[22] and FG-DPANet

[17] . DMFNet is a dual-branch multi-feature fusion network composed of residual modules, while FG-DPANet is a feature-guided dual-pool attention network with embedded attention modules. The above methods achieve good authentication results using a centralized training mode, but the disadvantage is that they cannot protect the privacy of training data in the QR code authentication task. It should be noted that the method proposed in this paper belongs to the distributed training method, in which personal data is stored on the local client, which is different from the previous centralized training methods that do not consider privacy issues.

[0087] Table II: Comparison with the previous centralized training model.

[0088]

[0089]

[0090] As shown in Table II, without considering user privacy, the accuracies of ResNet18, EfficientNetb0, DMFNet, FG-DPNet, ViT(T), and ViT(S) trained from scratch are 99.90%, 99.98%, 99.45%, 99.64%, 99.79%, and 99.81%, respectively. On the other hand, the accuracies of ResNet18, EfficientNetb0, ViT(T), and ViT(S) fine-tuned using pre-trained models are 99.98%, 100%, 99.98%, and 100%, respectively. The accuracy of the centralized models was improved after using the fine-tuning method with pre-trained models.

[0091] When considering user privacy protection, we adopted two FL algorithms: FedAVG and CWT. FedAVG trains each local client in parallel, either synchronously or asynchronously, while CWT trains each client serially and cyclically. The accuracy data of the pre-trained ViT(S) model in the FedAVG and CWT FL frameworks are 99.95% and 100%, respectively. We found that the CWT training method in CWT performs better on QR code authentication because CWT uses its local data to train a global model on local clients, and then transfers this global model to the next client for training after each cycle, and so on, until all local clients have been trained. The training process is then repeated on the clients until the model converges or reaches a predefined number of communication rounds, which makes the training more thorough.

[0092] D. Comparison of different FL algorithms

[0093] This study primarily employs two mainstream FL algorithms for privacy protection: parallel FedAVG and serial CWT. This section compares these two algorithms. We implemented six mainstream deep learning models—ResNet18, ResNet34, EfficientNet-b0, EfficientNetb5, ViT(S), and ViT(T)—within both FL frameworks. For these models, we adopted two training strategies: training from scratch; and fine-tuning pre-trained models.

[0094] Table III: Experiments with different federated learning frameworks.

[0095]

[0096] As shown in Table III, firstly, by comparing different training strategies, the method of fine-tuning using a pre-trained model achieves better results than training from scratch on both CNN-based and ViT-based models. Secondly, comparing different types of models, regardless of the training strategy used, the ViT-based model consistently outperforms the CNN-based model. Finally, comparing different FL algorithms, we find that for CNN-based models, we cannot definitively say which FL algorithm is better, but for ViT-based models, CWT outperforms FedAVG, with the pre-trained ViT(S) model performing best and achieving 100% accuracy under the CWT FL algorithm. Therefore, the pre-trained ViT(S) model within the CWT FL framework is used as the proposed method.

[0097] E. Ablation Research

[0098] In this section, we conduct ablation comparison experiments on our proposed method, including different ViT model settings (ViT(S) and ViT(T)), different training mode settings (with and without pre-training), and different FL framework settings (FedAVG and CWT).

[0099] Table IV: Ablation experiments of our proposed method.

[0100] set up Accuracy (%) ViT(T)+FedAVG 96.35 ViT(S)+FedAVG 97.74 ViT(T)+CWT 98.59 ViT(S)+CWT 99.19 Pre-trained ViT(T) + FedAVG 99.87 Pre-trained ViT(S) + FedAVG 99.95 Pre-trained ViT(T)+CWT 99.98 Pre-trained ViT(S) + CWT (our) 100.00

[0101] As shown in Table IV, we explored various combinations of settings for the three variables—ViT model selection, FL framework selection, and whether pre-training was performed—to determine the most suitable scheme for authenticating anti-counterfeiting QR codes. We found that, when considering the ViT model selection, keeping the other two variables constant, ViT(S) exhibited better performance than ViT(T). Similarly, CWT showed better performance than FedAVG, and the pre-trained mode demonstrated better performance than the de novo trained mode. The best performance, achieving 100% accuracy, was obtained when using the pre-trained ViT(S) model within the CWT FL framework; therefore, we selected this as the proposed scheme.

[0102] F. The impact of the number of communication rounds on the performance of the authentication model

[0103] In this section, we primarily discuss the impact of the number of communication rounds on the performance of the authentication model. In privacy-preserving QR code authentication, the setting of the number of communication rounds in FL (Focus Function) is crucial. Faster convergence of the authentication model can minimize communication requirements and effectively reduce training time.

[0104] We set the number of communication rounds to 100. At the end of each training round, the current model was tested. After the overall training process was completed, we identified which models converged faster and more stably while maintaining better authentication performance. Experiments were conducted using both FedAVG and CWT FL algorithms, considering models such as EfficientNet-b0, ResNet18, ViT(T), and ViT(S), employing both fine-tuning of pre-trained models and de novo training strategies.

[0105] Figure 5 The relationship between the number of communication rounds and test accuracy using the FedAVG FL algorithm is shown. Figure 5In the diagram, curve 511 shows the results of training from scratch using EfficientNet-b0; curve 512 shows the results of training from scratch using ViT(T); curve 513 shows the results of training from scratch using pre-trained EfficientNet-b0; curve 514 shows the results of training from scratch using pre-trained ViT(T); curve 515 shows the results of training from scratch using ResNet18; curve 516 shows the results of training from scratch using ViT(S); curve 517 shows the results of training from scratch using ResNet18; and curve 518 shows the results of training from scratch using ViT(S). The FedAVG FL algorithm was used in curves 511-518.

[0106] From a training methodology perspective, fine-tuning a pre-trained model outperforms training from scratch. Comparing the ViT model with CNN models, the ViT model, under the same training method, exhibits not only more stable and faster convergence but also better authentication performance. Specifically, the pre-trained ViT(S) model converges faster and more stably, achieving the best performance. This section's experiments primarily utilize pre-trained models within the CWT FL framework.

[0107] Figure 6 The relationship between the number of communication rounds and test accuracy using the CWT FL algorithm is shown. Figure 6 In the diagram, curve 611 shows the results of training from scratch using EfficientNet-b0; curve 612 shows the results of training from scratch using ViT(T); curve 613 shows the results using pre-trained EfficientNet-b0; curve 614 shows the results using pre-trained ViT(T); curve 615 shows the results of training from scratch using ResNet18; curve 616 shows the results of training from scratch using ViT(S); curve 617 shows the results using pre-trained ResNet18; and curve 618 shows the results using pre-trained ViT(S). The CWT algorithm was used in curves 611-618.

[0108] From a training methodology perspective, regardless of whether it's a ViT or CNN model, fine-tuning using a pre-trained model achieves faster and more stable convergence than training from scratch. Comparing ViT and CNN, regardless of whether it's trained from scratch or pre-trained, when the number of communication rounds reaches 100, the ViT model outperforms the CNN model. Specifically, ViT(S), fine-tuned using a pre-trained model, achieves the fastest and best convergence and exhibits the best performance in authentication.

[0109] In summary, comparison Figure 5 and Figure 6 We found that fine-tuning the pre-trained ViT(S) model on the CWT FL algorithm can achieve convergence faster and better than the FedAVG FL algorithm. Furthermore, combined with the ablation experiments analyzed in Table IV, the pre-trained ViT(S) model has the highest accuracy in the CWT algorithm, reaching 100%.

[0110] G. Conduct anti-fuzzing experiments using a pre-trained model under CWT.

[0111] In real-world scenarios, the relative motion between the smartphone and the anti-counterfeiting QR code during handheld shooting can easily cause blurring in the captured image, affecting the performance of the authentication method. Therefore, it is necessary to test the anti-blurring capability of the proposed method. We used the OpenCV computer vision library to add different degrees of motion blur to all QR code blocks in the test set in batches, setting the length of the motion blur kernel to 0, 2, 4, and 6 respectively, and the angle of the blur kernel to 45 degrees. When the length of the blur kernel is 0, it indicates that the anti-counterfeiting QR code image is clear and unblurred, and this is used as the baseline value. Note that the experiments in the previous two sections have demonstrated that pre-trained models perform better than those trained from scratch in anti-counterfeiting QR code authentication. Therefore, in this section, we consider pre-trained models, and the models used for comparison include ResNet18, EfficientNet-b0, ViT(S), and ViT(T). The FL framework used is CWT. The results of the anti-blurring experiments are shown in Table V.

[0112] Table V: Anti-fuzzing experiments using pre-trained models within the CWT federated framework.

[0113]

[0114] As shown in Table V, under the CWT FL framework, the authentication performance of the four pre-trained models decreases with increasing blur kernel length. Among these models, EfficientNetb0 experiences the most significant performance drop, with its accuracy plummeting from 92.03% to 67.96% as the blur kernel length increases from 0 to 2. In contrast, the ViT model demonstrates better robustness against blurring compared to CNN-based models. The pre-trained ViT(S) consistently performs best in most cases, maintaining an authentication accuracy of 98.33% even with a blur kernel length of 4, thus confirming its advantage in authentication tasks under blurred conditions.

[0115] III. Embodiments of this Disclosure

[0116] Based on the details, examples, and applications of the decentralized method for authenticating anti-counterfeiting QR codes disclosed above, the embodiments of this disclosure are developed as follows and may be universal.

[0117] One aspect of this disclosure is to provide a method for authenticating an anti-counterfeiting QR code captured in an image presented to an individual client among a plurality of clients.

[0118] This method uses Figure 7 To explain, Figure 7 A flowchart 700 is depicted, illustrating exemplary steps of the disclosed method. Exemplarily, the method includes steps 710, 720, and 730.

[0119] In step 720, the individual client uses its local ML model to determine the authenticity of the anti-counterfeiting QR code captured in the image when the image is presented to the individual client. The individual client's local ML model is initialized in step 710 as a local copy of the ML model shared by multiple clients. Advantageously, the ML model is a ViT-based model that is pre-trained to process the input image to determine the authenticity of the anti-counterfeiting QR code captured in the input image. ViT-based models (e.g., the ViT proposed in

[40] ) are transformer models suitable for processing images. As mentioned above, ViT-based models can be pre-trained on the large ImageNet-1K dataset

[46] .

[0120] The ViT-based model can be selected from the ViT(T) model defined in

[49] , the ViT(S) model defined in

[50] , etc. Preferably, the ViT-based model is selected as the ViT(S) model.

[0121] In step 730, advantageously, the multiple clients execute the CWT FL procedure 800 to update their respective local ML models based on real-time multiple training data available to each of the multiple clients, while protecting the privacy of the training data between the different clients. As used herein, "real-time multiple training data" refers to multiple training data available when actually processing the multiple training data in the CWT FL procedure 800. That is, the aforementioned multiple training data can vary over time and can contain different training datasets at different times. By using self-built anti-counterfeiting QR code datasets generated by the multiple clients to fine-tune these local ML models, the update of the respective local ML models is achieved, thereby allowing the model parameters of these local ML models to adapt to the specific characteristics of the anti-counterfeiting QR codes encountered by the multiple clients.

[0122] In practice, multiple training data sets 720 owned by the client are typically generated based on the information obtained by the client when executing step 720 multiple times. In some embodiments, multiple training data sets owned by the individual client are generated based on the authentication result obtained by authenticating the QR code image received by the individual client using the individual client's local ML model.

[0123] Since the authentication result generated by the individual client in step 720 produces the corresponding real-time multiple training data owned by that individual client, step 720 is generally considered to precede step 730. Furthermore, this real-time multiple training data is private to that client and is not shared with other clients among those multiple clients.

[0124] In actual implementations of the disclosed method, the local ML model of an individual client is typically updated continuously over time using newly emerging training data. In some embodiments, step 730 is repeated periodically to periodically update the corresponding local ML models of the multiple clients. The local ML model is periodically updated to signify that the local ML model is being repeatedly updated from time to time.

[0125] Figure 8 Flowcharts depicting some embodiments of implementing the CWT FL process 800 are provided.

[0126] The CWT FL process 800 begins with initialization step 810. In initialization step 810, the multiple clients are sorted to produce an ordered list of clients, and an extended ordered list of clients is formed by repeating the ordered list of clients a predetermined number of times. Refer to Algorithm 1. The ordered list of clients specifies the order in which the clients are used in a single communication round to sequentially fine-tune the corresponding local ML model so that the corresponding local ML model is updated incrementally. The predetermined number of times is R, which is the number of communication rounds used to cyclically update the corresponding local ML model. The extended ordered list of clients specifies the order in which the clients are used when sequentially fine-tuning the corresponding local ML model over R communication rounds.

[0127] After performing initialization step 810, subprocess 815, which fine-tunes the local ML model of the currently selected client, is repeated until stopping condition 850 is met. In some embodiments, stopping condition 850 is that all clients listed sequentially in the extended ordered list of clients have been processed by subprocess 815. In some other embodiments, stopping condition 850 is that at least one of the following conditions is met: predetermined convergence conditions of the corresponding local ML models of multiple clients are met; and the corresponding clients, arranged in the extended ordered list of clients, have been used as the currently selected clients when running subprocess 815.

[0128] Sub-procedure 815 includes steps 820, 830, and 840. The currently selected client in the current execution of the sub-procedure can be identified from the extended ordered list of clients.

[0129] In step 820, the local ML model of the currently selected client is fine-tuned using the corresponding real-time multiple training data available to the currently selected client.

[0130] In step 830, the next client is identified from the extended ordered list of clients, such that the next client becomes the currently selected client in the next execution of the subprocess. The next execution immediately follows the current execution. Note that in the special case where the currently selected client is already the last client in the extended ordered list of clients, the next client cannot be identified.

[0131] If the next client is identifiable, step 840 is executed. After fine-tuning the local ML model of the currently selected client in step 820, in step 840, the local ML model of the next client is replaced by the local ML model of the currently selected client. As a result, the corresponding immediate training data possessed by the currently selected client is used to update the local ML model of the next client, but is not disclosed to the next client.

[0132] After subprocess 815 is completed, the corresponding local ML models of the multiple clients are updated in step 860 using the local ML model of the currently selected client used in the previous execution of subprocess 815.

[0133] Typically, the ML model copied to individual clients in initialization step 710 to form a local ML model is stored on a server serving multiple clients. In some practical situations, it is preferable to also update the ML model using the local ML model of the currently selected client used in the previous execution of subprocess 815, so that when the server adds a new client to the plurality of clients, it allows the server to initialize a new local ML model for the new client using the updated ML model. In some embodiments of CWTFL process 800, step 870 is used to update the ML model using the local ML model of the currently selected client from the previous execution of subprocess 815.

[0134] Further implementation details of the disclosed method are described below.

[0135] Consider a scenario where the ML model includes first and second multiple model parameters for configuring the ML model, such that an individual client's local ML model is configured by the corresponding first and second multiple model parameters of that individual client. In this case, the corresponding first multiple model parameters of the individual client are fixed during the execution of CWT FL procedure 800, while the corresponding second multiple model parameters of the individual client are adjustable to fine-tune the individual client's local ML model during CWT FL procedure 800. Using the first and second multiple model parameters to operate the ML model simplifies the procedure of replacing the local ML model of the next client with the local ML model of the currently selected client when performing step 840.

[0136] In some embodiments of step 840, if a next client is identifiable, the corresponding second plurality of model parameters in the local ML model of the next client are overwritten by the corresponding second plurality of model parameters in the local ML model of the currently selected client, thereby replacing the local ML model of the next client with the local ML model of the currently selected client.

[0137] In one setting of the first and second plurality of model parameters as described above, the first plurality of model parameters configure the ML model to recognize the edges and shape of the anti-counterfeiting QR code. In this setting, the second plurality of model parameters are adjustable and trainable to adapt to the task of determining the authenticity of the anti-counterfeiting QR code based on the recognized edges and shape.

[0138] This disclosure may be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, this embodiment should be considered illustrative rather than restrictive in all respects. The scope of the invention is indicated by the appended claims rather than the foregoing description, and therefore, it is intended to cover all variations falling within the meaning and scope of equivalents of the claims.

[0139] References

[0140] The following is a list of references cited from time to time in this specification. Each publication of these references is incorporated herein by reference in its entirety.

[0141] [1] J.Zhang, Z.Wang, X.Huo,

[0142] [2]Y.Yan,Z.Zou,H.Xie,Y.Gao,and L.Zheng,“An IoT based anti-counterfeiting system using visual features on qr code,”IEEE Internet ofThings Journal,vol.8,no.8,pp.6789–6799,2020.

[0143] [3]J.Picard,P.Landry,and M.Bolay,“Counterfeit detection with QRcodes,”in Proceedings of the 21st ACM Symposium on Document Engineering,pp.1-4,2021.

[0144] [4]J.-S.Pan,X.-X.Sun,S.-C.Chu,A.Abraham,and B.Yan,“Digitalwatermarking with improved SMS applied for QR code,”Engineering Applicationsof Artificial Intelligence,vol.97,p.104049,2021.

[0145] [5]Y.-M.Wang,C.-T.Sun,P.-C.Kuan,C.-S.Lu,and H.-C.Wang,“Securedgraphic QR code with infrared watermark,”in Proceeding of 2018IEEEInternational Conference on Applied System Invention(ICASI),pp.690–693,IEEE,2018.

[0146] [6]M.K.Harahap and N.Khairina,“Copyright protection of scientificworks using digital watermarking by embedding DOIQR code,”Journal of ComputerNetworks,Architecture and High Performance Computing,vol.3,no.2,pp.234-240,2021.

[0147] [7]J.Liu,J.Han,K.Fu,J.Jia,D.Zhu,and G.Zhai,“Application of QR codewatermarking and encryption in the protection of data privacy of intelligentmouth-opening trainer,”IEEE Internet of Things Journal,vol.10,no.12,pp.10510-10518,2023.

[0148] [8]C.Shaik,“Preventing counterfeit products using cryptography,QRcode and webservice,”Computer Science&Engineering:An International Journal(CSEIJ),vol.11,no.1,2021.

[0149] [9]H.R.H.Al Dallal and W.N.M.Al Mukhtar,“A QR code used for personalinformation based on multilayer encryption system.,”Int.J.Interact.Mob.Technol.,vol.17,no.9,pp.44-56,2023.

[0150]

[10] A.Fernández-Benito,M.Hoyos,M.A.López-Manchado,and T.J.Sorensen,“Aphysical unclonable function based on recyclable polymer nanoparticles toenable the circular economy,”ACS Applied Nano Materials,vol.5,no.10,pp.13752-13760,2022.

[0151]

[11] H.Im,J.Yoon,J.Choi,J.Kim,S.Baek,D.H.Park,W.Park,and S.Kim,“Chaotic organic crystal phosphorescent patterns for physical unclonablefunctions,”Advanced Materials,vol.33,no.44,p.2102542,2021.

[0152]

[12] V.Lapidas,A.Zhizhchenko,E.Pustovalov,D.Storozhenko,andA.Kuchmizhak,“Direct laser printing of high-resolution physically unclonablefunction anti-counterfeit labels,”Applied Physics Letters,vol.120,no.26,2022.

[0153]

[13] R.Chaban,O.Taran,J.Tutt,Y.Belousov,B.Pulfer,T.Holotyak,andS.Voloshynovskiy,“Printing variability of copy detection patterns,”inProceedings of 2022 IEEE International Workshop on Information Forensics andSecurity (WIFS),pp.1-6,IEEE,2022.

[0154]

[14] O.Taran,J.Tutt,T.Holotyak,R.Chaban,S.Bonev,and S.Voloshynovskiy,“Mobile authentication of copy detection patterns,”EURASIP Journal onInformation Security,vol.2023,no.1,p.4,2023.

[0155]

[15] Z.Zheng,H.Zheng,J.Ju,D.Chen,X.Li,Z.Guo,C.You,and M.Lin,“A systemfor identifying an anticounterfeiting pattern based on the statisticaldifference in key image regions,”Expert Systems with Applications,vol.183,p.115410,2021.

[0156]

[16] Z.Zheng,B.Xu,J.Ju,Z.Guo,C.You,Q.Lei,and Q.Zhang,“Circumferentiallocal ternary pattern:New and efficient feature descriptors for anti-counterfeiting pattern identification,”IEEE Transactions on InformationForensics and Security,vol.17,pp.970–981,2022.

[0157]

[17] C.You,H.Zheng,Z.Guo,T.Wang,J.Ju,and X.Li,“Identification of aprinted anti-counterfeiting code based on feature guidance double poolattention networks,”Computers,Materials&Continua,vol.75,no.2,2023.

[0158]

[18] H.Zheng,C.Zhou,X.Li,T.Wang,and C.You,“Forgery detection for anti-counterfeiting patterns using deep single classifier,”Applied Sciences,vol.13,no.14,p.8101,2023.

[0159]

[19] T.Wang,H.Zheng,C.You,and J.Ju,“A texture-hidden anti-counterfeiting qrcode and authentication method,”Sensors,vol.23,no.2,p.795,2023.

[0160]

[20] C.Chen,M.Li,A.Ferreira,J.Huang,and R.Cai,“A copy-proof schemebased on the spectral and spatial barcoding channel models,”IEEE Transactionson Information Forensics and Security,vol.15,pp.1056–1071,2019.

[0161]

[21] T.Wang,H.Zheng,Z.Guo,C.You,and J.Ju,“Anti-counterfeiting texturedpattern,”Visual Computer,vol.40,no.3,p.2139–2160,2023.

[0162]

[22] Z.Guo,H.Zheng,C.You,T.Wang,and C.Liu,“DMF-Net:Dual-branch multi-scale feature fusion network for copy forgery identification of anti-counterfeiting QR code,”arXiv preprint arXiv:2201.07583,2022.

[0163]

[23] Z.Guo,S.Wang,Z.Zheng,and K.Sun,“Printer source identification ofquick response codes using residual attention network and smartphones,”Engineering Applications of Artificial Intelligence,vol.131,p.107822,2024.

[0164]

[24] F.Z.Mehrjardi,A.M.Latif,M.S.Zarchi,and R.Sheikhpour,“A survey ondeep learning-based image forgery detection,”Pattern Recognition,p.109778,2023.

[0165]

[25] S.Lu,Z.Gao,Q.Xu,C.Jiang,A.Zhang,and X.Wang,“Class-imbalanceprivacy-preserving federated learning for decentralized fault diagnosis withbiometric authentication,”IEEE Transactions on industrial informatics,vol.18,no.12,pp.9101–9111,2022.

[0166]

[26] D.Liu,Z.Dang,C.Peng,Y.Zheng,S.Li,N.Wang,and X.Gao,“Fedforgery:generalized face forgery detection with residual federated learning,”IEEETransactions on Information Forensics and Security,2023.

[0167]

[27] K.Wei,J.Li,M.Ding,C.Ma,H.H.Yang,F.Farokhi,S.Jin,T.Q.Quek,andH.V.Poor,“Federated learning with differential privacy:Algorithms andperformance analysis,”IEEE transactions on information forensics andsecurity,vol.15,pp.3454–3469,2020.

[0168]

[28] L.Qu,Y.Zhou,P.P.Liang,Y.Xia,F.Wang,E.Adeli,L.Fei-Fei,and D.Rubin,“Rethinking architecture design for tackling data heterogeneity in federatedlearning,”in Proceedings of the IEEE / CVF conference on computer vision andpattern recognition,pp.10061–10071,2022.

[0169]

[29] Y.Bai,J.Mei,A.L.Yuille,and C.Xie,“Are transformers more robustthan CNNs?”Advances in neural information processing systems,vol.34,pp.26831-26843,2021.

[0170]

[30] X.Mao,G.Qi,Y.Chen,X.Li,R.Duan,S.Ye,Y.He,and H.Xue,“Towards robustvision transformer,”in Proceedings of the IEEE / CVF conference on ComputerVision and Pattern Recognition,pp.12042-12051,2022.

[0171]

[31] S.Paul and P.-Y.Chen,“Vision transformers are robust learners,”inProceedings of the AAAI conference on Artificial Intelligence,vol.36,pp.2071-2081,2022.

[0172]

[32] D.Zhou,Z.Yu,E.Xie,C.Xiao,A.Anandkumar,J.Feng,and J.M.Alvarez,“Understanding the robustness in vision transformers,”in Proceedings ofInternational Conference on Machine Learning,pp.27378-27394,PMLR,2022.

[0173]

[33] Z.Yin,E.Xing,and Z.Shen,“Squeeze,recover and relabel:Datasetcondensation at imagenet scale from a new perspective,”Advances in NeuralInformation Processing Systems,vol.36,2024.

[0174]

[34] Y.You,Z.Zhang,C.-J.Hsieh,J.Demmel,and K.Keutzer,“Imagenettraining in minutes,”in Proceedings of the 47th international conference onparallel processing,pp.1-10,2018.

[0175]

[35] H.P.Nguyen,F.Retraint,F.Morain-Nicolier,and A.Delahaies,“Awatermarking technique to secure printed matrix barcode—application foranti-counterfeit packaging,”IEEE Access,vol.7,pp.131839-131850,2019.

[0176]

[36] N.Xie,J.Chen,Y.Chen,J.Hu,Q.Zhang,C.Chen,and L.Huang,“Detection ofinformation hiding at anticopying 2d barcodes,”IEEE Transactions on Circuitsand Systems for Video Technology,vol.32,no.1,pp.437-450,2021.

[0177]

[37] J.Chen,L.Dong,R.Wang,D.Yan,and C.Peng,“Mixed-bit samplinggraphic:When watermarking meets copy detection pattern,”IEEE SignalProcessing Letters,2023.

[0178]

[38] C.Vinh Loc,T.Xuan Viet,T.Hoang Viet,L.Hoang Thao,and N.HoangViet,“Deep learning based-approach for quick response code verification,”Applied Intelligence,vol.53,no.19,pp.22700-22714,2023.

[0179]

[39] A.Vaswani,“Attention is all you need,”arXiv preprint arXiv:1706.03762,2017.

[0180]

[40] A.Dosovitskiy et al.,“An image is worth 16x16 words:Transformersfor image recognition at scale,”arXiv preprint arXiv:2010.11929,2020.

[0181]

[41] M.M.Naseer,K.Ranasinghe,S.H.Khan,M.Hayat,F.Shahbaz Khan,and M.-H.Yang,“Intriguing properties of vision transformers,”Advances in NeuralInformation Processing Systems,vol.34,pp.23296-23308,2021.

[0182]

[42] J. H.B.McMahan,F.X.Yu,P.Richtárik,A.T.Suresh,and D.Bacon,“Federated learning:Strategies for improving communication efficiency,”arXivpreprint arXiv:1610.05492,2016.

[0183]

[43] Y.Zhao,M.Li,L.Lai,N.Suda,D.Civin,and V.Chandra,“Federatedlearning with non-IID data,”arXiv preprint arXiv:1806.00582,2018.

[0184]

[44] B.McMahan,E.Moore,D.Ramage,S.Hampson,and B.A.y Arcas,“Communication-efficient learning of deep networks from decentralized data,”in Artificial intelligence and statistics,pp.1273-1282,PMLR,2017.

[0185]

[45] K.Chang,N.Balachandar,C.Lam,D.Yi,J.Brown,A.Beers,B.Rosen,D.L.Rubin,and J.Kalpathy-Cramer,“Distributed deep learning networks amonginstitutions for medical imaging,”Journal of the American Medical InformaticsAssociation,vol.25,no.8,pp.945-954,2018.

[0186]

[46] J.Deng,W.Dong,R.Socher,L.-J.Li,K.Li,and L.Fei-Fei,“ImageNet:Alarge-scale hierarchical image database,”in Proceedings of 2009 IEEEconference on computer vision and pattern recognition,pp.248-255,IEEE,2009.

[0187]

[47] K.He,X.Zhang,S.Ren,and J.Sun,“Deep residual learning for imagerecognition,”in Proceedings of the IEEE conference on computer vision andpattern recognition,pp.770-778,2016.

[0188]

[48] M.Tan and Q.Le,“EfficientNet:Rethinking model scaling forconvolutional neural networks,”in Proceedings of International conference onmachine learning,pp.6105-6114,PMLR,2019.

[0189]

[49] A.Steiner,A.Kolesnikov,X.Zhai,R.Wightman,J.Uszkoreit,and L.Beyer,“How to train your ViT?data,augmentation,and regularization in visiontransformers,”arXiv preprint arXiv:2106.10270,2021.

[0190]

[50] X.Chen,C.-J.Hsieh,and B.Gong,“When vision transformers outperformResNets without pre-training or strong data augmentations,”arXiv preprintarXiv:2106.01548,2021.

Claims

1. A method for authenticating a counterfeit-proof quick response (QR) code captured in an image presented to an individual client from among multiple clients, the method comprising: When the image is presented to the individual client, the individual client uses its local machine learning (ML) model to determine the authenticity of the anti-counterfeiting QR code captured in the image. The individual client's local ML model is initialized as a local copy of an ML model shared by the multiple clients. This ML model is a vision-transformer-based model pre-trained to process the input image and determine the authenticity of the anti-counterfeiting QR code captured in the input image. The multiple clients perform a recurrent weight transfer (CWT) federated learning (FL) process to update the corresponding local ML models of the multiple clients based on the real-time multiple training data held by each of the multiple clients, while protecting the privacy of the training data among the different clients.

2. The method of claim 1, wherein the CWT FL process comprises: The multiple clients are sorted to generate an ordered list of clients; The client's extended ordered list is formed by repeating the client's ordered list a predetermined number of times; The sub-process of fine-tuning the local ML model of the currently selected client is repeated until a predetermined convergence condition for the corresponding local ML model of the plurality of clients is met, or until the corresponding clients, arranged sequentially according to the extended ordered list of clients, are used as the currently selected client during the execution of the sub-process, wherein the sub-process includes: The local ML model of the currently selected client is fine-tuned using the corresponding real-time multiple training data available to the currently selected client. Identify the next client from the extended ordered list of clients, such that the next client becomes the currently selected client in the next execution of the sub-process; and If the next client is identifiable, after fine-tuning the local ML model of the currently selected client, the local ML model of the currently selected client is used to replace the local ML model of the next client, so that the corresponding real-time multiple training data possessed by the currently selected client are used to update the local ML model of the next client, but not disclosed to the next client. as well as The corresponding local ML models of the plurality of clients are updated using the local ML model of the currently selected client used in the previous execution of the subprocess.

3. The method according to claim 2, wherein: The ML model includes first and second plurality of model parameters for configuring the ML model, such that the local ML model of the individual client is configured by the corresponding first and second plurality of model parameters of the individual client; During the execution of the CWT FL process, the corresponding first plurality of model parameters of the individual client are fixed, while the corresponding second plurality of model parameters of the individual client are adjustable in order to fine-tune the local ML model of the individual client during the CWT FL process; and Replacing the local ML model of the next client with the local ML model of the currently selected client during the execution of the sub-process includes: overwriting the corresponding second plurality of model parameters in the local ML model of the next client with the corresponding second plurality of model parameters in the local ML model of the currently selected client, thereby replacing the local ML model of the next client with the local ML model of the currently selected client.

4. The method of claim 3, wherein the first plurality of model parameters configure the ML model to identify the edges and shape of the anti-counterfeiting QR code.

5. The method of claim 1, wherein multiple real-time training data belonging to the individual client are generated based on the authentication result obtained by authenticating the QR code image received by the individual client using the local ML model of the individual client.

6. The method of claim 1, wherein the CWT FL process is repeated from time to time to periodically update the corresponding local ML model.

7. The method of claim 1, wherein the visual converter-based model is selected as the ViT(S) model.

8. The method according to claim 1, wherein: The ML model is stored in a server that provides services to the multiple clients; and The CWT FL process also includes updating the ML model using the local ML model of the currently selected client used in the previous execution of the subprocess, thereby allowing the server to initialize a new local ML model for the new client using the updated ML model when the server adds a new client to the plurality of clients.

Images