Geographic landmark vinegar product identification method and device and storage medium

Abstract

The invention relates to a geographical landmark vinegar product identification method and device and a storage medium, and belongs to the technical field of machine learning. The method comprises the following steps of: constructing a federal learning framework of the geographical landmark vinegar product, wherein the framework comprises a central node and a plurality of local nodes; the central node performs feature extraction on the hyperspectral detection data of the vinegar product by adopting the constructed encoder model, and issues the constructed encoder model to each local node; the local node performs feature extraction on the hyperspectral detection data of the vinegar sample by adopting the encoder model issued by the central node; the central node constructs a global diffusion generation model by cyclically distilling the local diffusion generation model constructed by each local node; and training a classifier of the geographically symbolic vinegar product by adopting the simulated hyperspectral representation data and label data of the vinegar sample. According to the method, the classifier algorithm is continuously optimized based on federated learning, and the quality of vinegar products of different categories and brands can be quickly and accurately identified.

Description

Technical Field

[0001] The present application belongs to the field of machine learning technology, and specifically relates to a method, device and storage medium for identifying geographically indicated vinegar products. Background Art

[0002] Vinegar is a condiment for all major traditional Chinese cuisines and is deeply loved by the general public. Vinegar geographical indication products refer to the quality characteristics and types of vinegar produced based on the geographical characteristics of the region within the studied or identified geographical scope. Through data analysis and authenticity identification, the vinegar products can meet the quality requirements of traceability, rating, and characteristics, effectively improving consumers' cognition of vinegar products and the credibility of brands and quality. However, looking back at the consumer market for vinegar products in my country, it can be found that there is no reasonable connection between the quality and pricing of vinegar products. Many high-end vinegar products rely on the absolute advantages of the production area or the influence of the brand to charge exorbitant prices, which seriously affects consumers' ability to discern when purchasing vinegar products, and at the same time makes it impossible for truly high-quality vinegar products to gain an advantage in the chaotic market competition.

[0003] In the past, when authenticating vinegar products, manual and molecular information technology was mostly used, and there was no rigorous grading system. Many products that claimed to be high-quality vinegar did not have relevant certificates. When each testing center identified vinegar products, there were many problems such as insufficient data volume, urgent need for data confidentiality, low efficiency in identifying vinegar products, and lack of reliability and validity. Although existing machine learning and deep learning are constantly developing, they will incur expensive costs both in the calculation process and in the optimization of algorithms. There are many loopholes in data privacy and training data volume, and the final accuracy cannot be guaranteed.

[0004] Therefore, creating a stable intelligent technology for authenticity identification of vinegar products has great practical significance. Summary of the invention

[0005] In view of the above analysis, the embodiments of the present invention aim to provide a geographical indication vinegar product identification method, device and storage medium to quickly and accurately identify geographical indication vinegar products.

[0006] In a first aspect of the present application, a method for identifying a geographically indicated vinegar product is provided, comprising:

[0007] Constructing a federated learning framework for geographical indication vinegar products, including a central node and multiple local nodes; wherein the central node is used to coordinate the joint model training, and the local nodes independently manage their respective data and local models;

[0008] The central node collects the hyperspectral detection data of a large number of vinegar products, uses the constructed encoder model to extract features from the hyperspectral detection data of vinegar products, and sends the constructed encoder model to each local node;

[0009] The local nodes collect the hyperspectral detection data and label data of their respective local vinegar samples, use the encoder model sent by the central node to extract features from the hyperspectral detection data of the vinegar samples, and use the constructed local diffusion generation model to generate data for training;

[0010] The central node constructs a global diffusion generation model by cyclically distilling the local diffusion generation models constructed by each local node; the global diffusion generation model is used to generate simulated vinegar sample hyperspectral characterization data and label data;

[0011] The simulated hyperspectral characterization data of vinegar samples and label data are used to train the classifier of geographical indication vinegar products, and the trained classifier outputs the identification and classification results of geographical indication vinegar products.

[0012] Optionally, the step of using the constructed encoder model to extract features from the hyperspectral detection data of the vinegar product and sending the constructed encoder model to each local node includes:

[0013] The constructed sequence feature encoder model is used to extract the sequence modal features to form sequence feature data; the sequence feature encoder model is sent to each local node;

[0014] The constructed image feature encoder model is used to extract image features to form image feature data; the image feature encoder model is sent to each local node.

[0015] Optionally, the construction of the sequence feature encoder model includes:

[0016] Construct a pre-trained model based on multi-scale CNN, long short-term memory network and factor-aware attention network as the sequence feature encoder model;

[0017] The time series modal data of the hyperspectral detection data are collected at a preset sampling rate, and parallel CNN models of different scales are used to extract multi-scale features from the time series modal data of each sampling point; each input sequence data is converted from one data to multiple data, and features of different degrees are fused;

[0018] The factor-aware attention network adopts the following attention mechanism:

[0019]

[0020] in, is a learnable parameter, (·)T represents the transpose of a matrix, is the hidden state of LSTM at time step t, m is the size of LSTM unit; exogenous sequence is extracted by adaptively:

[0021]

[0022] Then the hidden state at time t is updated as:

[0023]

[0024] Where LSTM(·) is an LSTM unit, and the sequence feature input x of the central node SCt Newly calculated Replacement; symbol is the output of factor-aware attention at time step t;

[0025] The data output after module training is multi-dimensional data, and average pooling is used to extract average and overall features:

[0026]

[0027] Among them, x S F (t) represents the rectangular area related to the kth feature map The average pooling output value, x kpq Represents a rectangular area The element at (p,q) in Represents a rectangular area The number of elements in

[0028] A multi-way LSTM is used to learn from x SCt To the hidden state h t The mapping:

[0029] i t =σ(W i x SCt +U i h t-1 +b i )

[0030] f t =σ(W f x SCt +U f h t-1 +b f )

[0031] o t =σ(W o x SCt +U o ht-1 +b o )

[0032] c t =f t ⊙c t-1 +σ(W i x SCt +U i h t-1 +b i )⊙tanh(W c x SCt +U c h t-1 +b c )

[0033] h t =o t ⊙tanh(f t ⊙c t-1 +i t ⊙tanh(W c x SCt +U c h t-1 +b c ))

[0034] Where the symbol ⊙ is the element-wise product, σ = 1 / (1+e -x ) is the sigmoid function; is the hidden state of LSTM at time t; the symbol is learned during training

[0035] Optionally, the extracting of image features by using the constructed image feature encoder model of the hyperspectral data to form image feature data comprises:

[0036] After the input information is determined by the patching module and the fully connected layer, the Transformer model is constructed to form an image feature encoder model for image modality processing;

[0037] Among them, after extracting with the baseline Transformer model, the model parameters of the downstream task are directly fine-tuned by attaching a task layer on the pre-trained encoder; the input of the subsequent Transformer model architecture is a sequence of image patches The patch is then linearly projected to obtain the patch embedding Add a special token [S] before the input sequence; embed the standard learnable 1D position Added to the patch embedding, the input vector is fed into the Transformer architecture;

[0038] The Transformer image feature extraction network is trained in part by an effective attention mechanism, which includes: each token is scored separately externally to determine the importance of the class token for classification based on the input token; some tokens are retained based on the score sampling and pruned from the token dimension; from an internal perspective, a combination function is used instead of the Softmax operator to calculate the self-attention matrix, resulting in a new linear attention mechanism.

[0039] Optionally, the step of using the constructed local diffusion generation model to generate data for training includes:

[0040] Learn a unique feature tokenizer for each column of modal data, and then feed the tokens into a Transformer to capture the complex relationships between columns;

[0041] The feature tokenizer converts each column into a d-dimensional vector; categorical features are preprocessed using one-hot encoding, i.e. Each record is represented by

[0042] A linear transformation is applied to the numerical columns and an embedding lookup table is created for the categorical columns, where each category is assigned a learnable d-dimensional vector, i.e.:

[0043]

[0044] in, are the learnable parameters of the tagger, Each record is represented as a stack of all column embeddings;

[0045]

[0046] Each step of the diffusion process generates a noisy data, and the entire diffusion process is represented as a Markov chain:

[0047]

[0048] x0 is the original data, and x1, x2, ..., x are generated step by step. t .

[0049] Optionally, the step of using the constructed local diffusion generation model to generate data for training includes:

[0050] Use residual networks and multiple attention mechanisms to predict noise and train local diffusion generation models;

[0051] Among them, the residual layer is divided into a direct mapping layer and an indirect mapping layer; in the indirect mapping layer, a fully connected layer with a ReLU activation function is established to extract and normalize features:

[0052] FC(x F 0) = ReLU(BN(Linear(x F 0)))

[0053] In the formula, the function Linear represents the linear fully connected layer, the function BN represents the BorderlineNorm normalization layer, and the function ReLU is the ReLU activation function;

[0054] A soft threshold is set at the end of the multiple self-attention mechanism, and the Sigmoid activation function is used to ensure that the threshold of the soft threshold function is positive and within the set range.

[0055] Optionally, the central node constructs a global diffusion generation model by cyclically distilling the local diffusion generation models constructed by each local node, including:

[0056] The local diffusion generation model is distilled, and a global diffusion generation model is constructed based on the iterative cycle training mechanism of each local diffusion generation model:

[0057] Given a data distribution p data (x) sample x, noise scheduling function α (t) and σ (t) , by minimizing the weighted mean square error i Multiple diffusion models Diff of [I∈(0~R)] S (x s F (t)) and Diff I (x GI F (t)) Diffusion generation model Diff with distillation as the central node C :

[0058]

[0059] in, is the signal-to-noise ratio, and ω(λ t ) is a pre-specified weight function.

[0060] Optionally, the step of training a classifier of geographically indicated vinegar products using simulated vinegar sample hyperspectral characterization data and label data includes:

[0061] Using the LSTM long short-term memory network, the two modal data with time node t are used as input to train the classifier:

[0062] The data y output by the diffusion generation model F tAs the input data of LSTM,

[0063] Input Gate - I t =σ(W i y F t +U i h t-1 +b i )

[0064] Forget Gate - F t =σ(W f y t F +U f h t-1 +b f )

[0065] Output Gate——O t =σ(W o y f F +U o h t-1 +b o )

[0066] The Sigmoid activation function is used as the gating mechanism:

[0067] The data output through the output gate is scaled using the tanh function:

[0068]

[0069] Among them, the loss function model is:

[0070]

[0071] Among them, T represents the real data, Y represents the data generated by the geographical indication vinegar product identification method, and y i represents the i-th data, N represents the number of training samples, N>0 and N∈N + ; Use the stochastic gradient descent algorithm for training optimization, and optimize the model performance and training speed by calculating the gradient of the model loss value.

[0072] In a second aspect of the present application, a geographically-indicated vinegar product identification device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the geographically-indicated vinegar product identification method according to any one of the above-described methods is implemented.

[0073] According to a third aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for identifying a geographically-indicated vinegar product according to any one of the above-mentioned methods is implemented.

[0074] The geographical indication vinegar product identification method provided in the present application constructs a federated learning framework for geographical indication vinegar products, including a central node and multiple local nodes; wherein the central node is used to coordinate joint model training, and the local nodes independently manage their own data and local models. The central node uses the constructed encoder model to extract features from the hyperspectral detection data of the vinegar product, and sends the constructed encoder model to each local node; the local node uses the encoder model sent by the central node to extract features from the hyperspectral detection data of the vinegar sample; the central node constructs a global diffusion generation model by cyclically distilling the local diffusion generation model constructed by each local node; the classifier of the geographical indication vinegar product is trained using simulated hyperspectral characterization data of the vinegar sample and label data. The present application uses a hyperspectral data extraction model to analyze and extract features, combines the quality characteristics of the vinegar product itself, and continuously optimizes the classifier algorithm based on federated learning, thereby realizing the rapid and accurate identification of the quality of vinegar products of different categories and brands, so that consumers can see the grade of the vinegar product at a glance.

[0075] In addition, the present application also provides a geographically-indicative vinegar product identification device and storage medium having the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0077] Figure 1 A flowchart of a specific implementation method of the geographical indication vinegar product identification method provided in this application;

[0078] Figure 2 A flowchart of another specific implementation of the method for identifying geographically indicated vinegar products provided in this application;

[0079] Figure 3 A schematic diagram of the federated learning framework in the geographical indication vinegar product identification method provided in this application;

[0080] Figure 4 Schematic diagram of the product sequence feature extraction process for geographical indication vinegar products;

[0081] Figure 5 Schematic diagram of image feature extraction for geographical indication vinegar products;

[0082] Figure 6 A schematic diagram of the structure of the generation model in the method for identifying geographically indicated vinegar products provided in this application;

[0083] Figure 7 This is a structural block diagram of the geographical indication vinegar product identification device provided in this application. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be noted that, in the absence of conflict, the embodiments in the present disclosure and the features in the embodiments can be combined, separated, interchanged and / or rearranged with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0085] The terms used here are for the purpose of describing specific embodiments, and are not intended to be restrictive. As used here, unless the context clearly indicates otherwise, the singular forms "one (kind, person)" and "said (the)" are also intended to include plural forms. In addition, when the terms "comprise" and / or "include" and their variations are used in this specification, it is explained that there are stated features, integral bodies, steps, operations, parts, assemblies and / or their groups, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, parts, assemblies and / or their groups. It should also be noted that, as used here, the terms "substantially", "approximately" and other similar terms are used as approximate terms and not as degree terms, so that they are used to explain the inherent deviations of the measured values, calculated values ​​and / or the values ​​provided that will be recognized by those of ordinary skill in the art.

[0086] A flowchart of a specific implementation method of the method for identifying geographically indicated vinegar products provided in this application is as follows Figure 1 As shown, the method includes:

[0087] S101: Construct a federated learning framework for geographical indication vinegar products, including a central node and multiple local nodes; wherein the central node is used to coordinate joint model training, and the local nodes independently manage their respective data and local models.

[0088] Specifically, the central node (central server) is used to coordinate joint model training; each brand service provider can act as a local node to independently manage its own data and local models.

[0089] S102: The central node collects the hyperspectral detection data of a large amount of vinegar products, uses the constructed encoder model to extract features from the hyperspectral detection data of the vinegar products, and sends the constructed encoder model to each local node.

[0090] The constructed sequence feature encoder model is used to extract sequence modal features to form sequence feature data; the sequence feature encoder model is sent to each local node; the constructed image feature encoder model is used to extract image features to form image feature data; the image feature encoder model is sent to each local node.

[0091] Specifically, the central node collects hyperspectral detection data of large-scale vinegar products. When processing sequence modal data, multi-scale feature extraction is performed through parallel CNN models of different scales to process the input data. The pre-training model is constructed using LSTM and factor-aware attention network to construct a decoder-encoder. Further, a mask training mechanism can be used to realize sequence feature extraction in hyperspectral detection data. After training is completed, the sequence feature encoder model is sent to each local node.

[0092] The central node collects large-scale hyperspectral detection data of vinegar products, uses the image pre-training model integrated with E-Attention for pre-training, and constructs a decoder-encoder. It can also use a mask training mechanism to realize the construction of an image feature pre-training model in the hyperspectral detection data. After the training is completed, the image feature encoder model is sent to each local node.

[0093] S103: The local nodes collect the hyperspectral detection data and label data of their respective local vinegar samples, use the encoder model sent by the central node to extract features from the hyperspectral detection data of the vinegar samples, and use the constructed local diffusion generation model to generate data for training.

[0094] The local nodes collect the hyperspectral data of vinegar samples from their respective brands, as well as the label data of categories and levels. The distributed sequence feature encoder model and image feature encoder model are used to generate compressed representations and concatenate them. Furthermore, the TABSYN diffusion model can be used to form tabular data, and the residual network and multiple attention mechanisms can be used to predict noise and train the local diffusion generation model.

[0095] S104: The central node constructs a global diffusion generation model by cyclically distilling the local diffusion generation models constructed by each local node; and uses the global diffusion generation model to generate simulated vinegar sample hyperspectral characterization data and label data.

[0096] The local diffusion generation model constructed by the local node is distilled, and the diffusion generation model of the central node is constructed based on the iterative cycle training mechanism of the local diffusion generation model in each local node, and the generated data is decoded by the decoder.

[0097] S105: Using the simulated vinegar sample hyperspectral characterization data and label data to train a classifier of geographical indication vinegar products, the trained classifier outputs identification and classification results of the geographical indication vinegar products.

[0098] Specifically, the diffusion generation model of the central node is used to generate simulated hyperspectral characterization data of vinegar samples and their label data. The LSTM long short-term memory network is used to take the two modal data with time node t as input to build, train and update the classifier of geographical indication vinegar products to realize the identification of geographical indication vinegar products.

[0099] This application uses a hyperspectral data extraction model to analyze and extract features, combines the quality characteristics of the vinegar product itself, and continuously optimizes the classifier algorithm based on federated learning, so as to achieve rapid and accurate quality identification of vinegar products of different categories and brands, so that consumers can clearly see the grade of the vinegar product.

[0100] A flowchart of another specific implementation of the method for identifying geographically indicated vinegar products provided in this application is as follows Figure 2 As shown, combined Figure 2 , the geographical indication vinegar product identification method provided in this application is further elaborated in detail below.

[0101] S201: Construct a federated learning framework for geographical indication vinegar products, including a central node and multiple local nodes.

[0102] The central node is used to coordinate joint model training, and the local nodes manage their own data and local models independently. Figure 3 As shown in the schematic diagram of the federated learning framework in the geographical indication vinegar product identification method provided in this application, this application constructs a federated learning framework, including a central node and C local nodes, each local node corresponds to an independent client node. The central node maintains a parameter θ ξ Global diffusion generative model Each client i Maintaining a decentralized local diffusion generation model

[0103] The construction process of federated learning is as follows: First, the central node builds the decoder E C —Encoder D C Pre-trained model. Large-scale data x(t) = {x S (t),x I (t)} is input into the central node encoder for encoding, where t means that the data was sampled in the tth month of sample vinegar brewing. S (t) is the large-scale sequence modal data at time t, x I (t) is the large-scale image modality data at time t. After encoding, it forms x S F (t) The sequence modal data and x I F (t) The image data enters the local node for data generation. After the generators of each local node are trained, a central node generator is formed through cyclic distillation. After decoding at the central node, it is put into the classifier for recognition and finally the result is output.

[0104] By using the generative model to integrate the federated learning framework, the use of the generative model can enhance the privacy of the data and provide effective privacy protection for the data of the manufacturer as the node.

[0105] S202: The central node collects the hyperspectral detection data of a large number of vinegar products, extracts the sequence modal features using the constructed sequence feature encoder model, and forms sequence feature data; and sends the sequence feature encoder model to each local node.

[0106] The central node collects the time series modal data of the hyperspectral detection data of large-scale vinegar products at a sampling rate of once a month. The multi-scale parallel CNN model is used to extract multi-scale features from the hyperspectral sequence modal data of each sampling point. The pre-training model is constructed using LSTM and factor-aware attention network. The encoder E is fused. S -Decoder D S , and adopts a mask training mechanism to realize the sequential modal feature extraction of hyperspectral detection data.

[0107] like Figure 4 The schematic diagram of the feature extraction process of geographical indication vinegar products is shown in the figure. The hyperspectral detection data of large-scale vinegar products is trained as the input of the pre-trained model, and the encoder E is composed of factor-aware attention network, LSTM and filter. S The hyperspectral detection data is first processed by a CNN model with convolutional layers and pooling layers of different sizes in parallel on the sequence modality input data xS (t) Perform multi-scale feature extraction and convert each sequence data into From one data set to N data sets, features of different scales are finally fused to improve model performance and reduce resource usage:

[0108]

[0109] During the feature extraction pre-training process, the problem of different feature lengths may arise, which will have a certain impact on the training results. This application integrates the time series modal data in months to form multi-scale features of the sequence modal data at each time point, and then uses three convolutional layers to integrate the sequence modal multi-scale feature time series according to a fixed time length to avoid affecting the final training due to inconsistent data length.

[0110] The fermentation of geographical indication vinegar products is a long process, and monthly sampling can form time series data. Therefore, this application uses long short-term memory network (LSTM) and factor-aware attention network as the pre-training model of sequence modal data. The gating mechanism of LSTM can remember the long-term information in the sequence and realize the sequence feature extraction of hyperspectral detection data. However, in order to reduce the interference of negative information on prediction, this application proposes a factor-aware attention network to further optimize the time series data.

[0111] Module 1: Factor-Aware Attention Network

[0112] This application proposes a factor-aware attention network that adjusts weights according to the importance of factors, consisting of two layers of feedforward neural networks. In the first layer, appropriate weights are assigned to each factor in the exogenous sequence at time t. In the second layer, these hidden features are aggregated together to generate a high-level semantic And corresponds to the sequence feature input x of the corresponding central node SCt In general, given any exogenous sequence at time t (i.e. x k ), the following attention mechanism can be used:

[0113]

[0114] in, is a learnable parameter, and (·) T Represents the transpose of a matrix. is the hidden state of LSTM at time step t, and m is the size of LSTM unit. The attention weights are determined by the historical hidden state and the current input, indicating the impact of each exogenous factor on the prediction. Through these attention weights, important exogenous sequences can be extracted by adaptive means:

[0115]

[0116] Then the hidden state at time t can be updated as:

[0117]

[0118] where LSTM(·) is a LSTM unit, x SCt Newly calculated Instead of. Symbol is the output of the factor-aware attention at time step t.

[0119] The data output after module training is multi-dimensional data, and average pooling is used to extract its average and overall features:

[0120]

[0121] Among them, x S F (t) represents the rectangular area related to the kth feature map The average pooling output value, x kpq Represents a rectangular area The element at (p,q) in Represents a rectangular area The number of elements in .

[0122] Module 2: Long Short-Term Memory Network (LSTM)

[0123] LSTM has become a scalable and useful model for solving the problem of learning sequential modal data. The multi-channel data processed by parallel CNN is passed through multiple LSTM networks to improve the encoding efficiency of sequential modal data. Given an input in is a vector with n exogenous factors at time t. The present invention uses a multi-channel LSTM to learn SCt To the hidden state h t 's mapping.

[0124] i t =σ(W i x SCt +U i h t-1 +b i )

[0125] f t =σ(W f x SCt +U f h t-1 +b f )

[0126] ot =σ(W o x SCt +U o h t-1 +b o )

[0127] c t =f t ⊙c t-1 +σ(W i x SCt +U i h t-1 +b i )⊙tanh(W c x SCt +U c h t-1 +b c )

[0128] h t =o t ⊙tanh(f t ⊙c t-1 +i t ⊙tanh(W c x SCt +U c h t-1 +b c ))

[0129] The symbol ⊙ is the element-wise product, σ = 1 / (1+e -x ) is the sigmoid function. is the hidden state of LSTM at time t. During the training process, the symbol Obviously, after the sigmoid function, the outputs of the three gates are between 0 and 1. Therefore, if f t is close to 1 and tends to 0. The previous storage unit c t-1 is saved and passed to the current time step. Similarly, if o t Approximately 1, c t The information is passed to h t .

[0130] After training is completed, the sequence feature encoder model E is sent to each local node S .

[0131] The large-scale vinegar product sample data is passed through the central node encoder E SCAfter the pre-training of sequence feature extraction is completed, the validation set is used to evaluate the model performance, such as accuracy and loss value. According to the evaluation results, hyperparameter adjustment and model optimization are performed. Then it is converted into a format that can be distributed, and the encoder model is sent to each local node through a secure channel.

[0132] S203: extracting image features using the constructed image feature encoder model to form image feature data; and sending the image feature encoder model to each local node.

[0133] Aiming at the large-scale hyperspectral detection data of vinegar products, the image pre-training model fused with E-Attention is used for pre-training, and the fusion encoder E I -Decoder D I , and adopts a mask training mechanism to realize the construction of image feature pre-training model in hyperspectral detection data. After the training is completed, the image feature encoder model is sent to each local node.

[0134] like Figure 5 The image feature extraction diagram of geographical indication vinegar products is shown in the figure. The image pre-training model fused with E-Attention is used for pre-training. The fusion encoder E I -Decoder D I , and adopts a mask training mechanism to realize image feature extraction in hyperspectral detection data.

[0135] In the process of image modality data processing, the data input first passes through the patching module and the fully connected layer to determine the input information, and then the Transformer model is constructed to form the encoder E of this mechanism. I Perform image modality processing:

[0136] Module 1: Transformer Model

[0137] After first extracting from the baseline Transformer model, we directly fine-tune the model parameters of the downstream tasks by attaching a task layer to the pre-trained encoder. Pre-training uses the Mask Image Model (MIM) task in a self-supervised learning manner. The purpose of MIM is to recover occluded image patches based on the encoding vector. For downstream tasks, we attach a task layer to the pre-trained image representation construction model and fine-tune the parameters of the specific dataset to randomly mask a certain proportion of image patches and replace them with special mask embeddings [*]. At the same time, the purpose of the pre-training task is to predict the visual markers of the original image based on the encoding vector of the damaged image, so that even hyperspectral detection data that is damaged due to detection errors can participate in training.

[0138] The input to the subsequent Transformer architecture is a sequence of image patches These patches are then linearly projected to obtain the patch embeddings In addition, a special token [S] is added before the input sequence. The standard learnable 1D position embedding Added to the patch embedding, the input vector is fed into the Transformer architecture.

[0139] At the same time, each image is labeled with a 14×14 grid of visual labels. It should be emphasized that the number of visual labels for an image is the same as the number of image patches.

[0140] Module 2: E-Attention Mechanism

[0141] The Transformer image feature extraction network of the present invention is partially trained by the effective attention (E-Attention) mechanism, which is a Transformer image feature extraction network based on linear attention and pruning. First, each tag is scored separately externally to determine the importance of the class tag for classification based on the input tag. Then, some tokens are retained based on the score sampling and pruned from the token dimension. Then, from an internal perspective, a combination function is used instead of the Softmax operator to calculate the self-attention matrix, resulting in a new linear attention mechanism. The specific implementation is:

[0142] First, we describe the general form of the Transformer mathematically, where a given input is represented in the embedding space as T represents the transpose of the input into the Transformer module, thus defining the transpose T as:

[0143] T(X)=F(Att(X)+X)

[0144] Where F is a feedforward neural network with residual connections; Att is a function used to calculate the self-attention matrix.

[0145] The self-attention mechanism in the effective attention model is scaled dot product attention. For the convenience of expression, the scaling factor of ATT is omitted:

[0146] Att(Q,K,V)=Softmax(QK T )V

[0147] in can be calculated separately from the input.

[0148] In-depth analysis shows that the Softmax operator limits the performance of the scaled dot product attention mechanism, making it quadratic, so it is necessary to calculate QK first.T , we get an n×n matrix, making the complexity O(n 2 ) level. If there is no Softmax operator, then these three matrices QK T V will be multiplied, and using the combination law of matrix multiplication, the last two matrices K can be calculated first T V gets a matrix d k ×d v , and then let the matrix Q do the multiplication on the left. Due to the actual situation d k and d v It is much smaller than n, so the overall complexity can be considered as O(n). The definition of the Softmax operator is:

[0149]

[0150] Among them, z i represents the output value of the i-th node, and C represents the total number of output nodes.

[0151] The Softmax operation can convert the output values ​​of multiple clusters into a probability distribution between 0 and 1, the sum of which is 1. Then:

[0152]

[0153] Only positive values ​​are retained in the attention matrix, and negatively correlated features are ignored, effectively avoiding the aggregation of irrelevant contextual information. Secondly, the nonlinear weighting mechanism can focus the distribution of attention weights, thereby stabilizing the training process. It can also help the model expand locality, which means that a large part of the contextual dependencies come from neighboring tags.

[0154] This module uses a combination function to replace Softmax. This combination function satisfies the above two properties and consists of two sub-functions. They are used to achieve non-negativity and non-linear weighting respectively:

[0155] Att(Q,K,V)=g(f(Q,K))V

[0156] Among them, f(·) and g(·) are expressed as:

[0157]

[0158] Decouple g(·) and express the transformed Q as Q cos and Q sin Similarly, the transformed K is K cos He He K sin . Thus, the final expression after linear decomposition is obtained:

[0159] Att(Q,K,V)=g(f(Q,K))V=Q cos(K cos V)+Q sin (K sin V)

[0160] The Transformer-based image feature extraction network is pruned from the token dimension to reduce computational cost. The tokens converted from different inputs are first estimated with the scores of the image, and then some tokens are retained by sampling. All tokens input to the self-attention layer inside the model are divided into two parts: the first part contains only the first-class tokens, and the first-class tokens are additionally introduced in the final stage to determine the category of the input image and placed at the first position of the token sequence. The second part is all the remaining tokens, which are obtained by dividing and transforming the input image.

[0161] Pruning from the token dimension means reducing the second part of the token converted from the image as much as possible, and the number of this part of tokens is recorded as m. In other words, there are M+1 tokens in the input and r′+1 tokens in the output. Among them, r′ is the number of tokens retained in the second part after pruning, and the corresponding data range is r′≤r≤m. Here r is a pre-given parameter, and its function is to control the maximum number of tokens retained after sampling.

[0162] When designing a scoring formula to calculate the importance of each token except the class token, the specific implementation is:

[0163]

[0164] Among them, a 1,j and a 1,i denote the elements of the 1st row, jth column, and ith column of the self-attention matrix A, respectively. j is the fraction of the jth token. Therefore, based on these scores, some tokens can be randomly selected for sampling and retention. The lower the score, the lower the probability of being selected in the current stage and cannot be retained, and vice versa.

[0165] First, the cumulative distribution function (CDF) can characterize the probability distribution of a random variable x, denoted as X. For all real numbers x, the cumulative distribution function is defined as: F X (x) = P (X ≤ x):

[0166] If the cumulative distribution function F is a continuous and strictly increasing function, then there exists a corresponding inverse function F -1 (y), and the range of y is [0,1). The inverse function can be used to generate random variables that follow this random distribution. That is, if we take F X (x) is the CDF of x, then there exists an inverse function If a is a random variable with a value range of [0,1) and uniform distribution, then It follows the X distribution.

[0167] Therefore, CDF can be calculated as:

[0168]

[0169] The accumulation here starts from the second token because the first classification token must be retained. After obtaining the CDF, the inverse form of the sampling function can be obtained: η(v) = CDF -1 (v), v is between 0 and 1.

[0170] After sampling the tokens to be retained, the token pruning method is to replace QK T The calculated attention matrix A is changed from the original m+1 row to the current r′+1 row, and then added to the next calculation process. The attention calculation method after the fusion of the two methods is:

[0171] Att(Q′,K,V)=g(f(Q′,K))V=Q′ cos (K cos V)+Q′ sin (K sin V).

[0172] After fusion, a more effective attention mechanism (E-Attention) is obtained

[0173] The data Pre-training the encoder E through image modality IC , and obtain the output result x of the pre-training of image modality feature extraction I F (t):

[0174] x I F (t) = Att(x I (t))

[0175] After the training is completed, the hyperspectral detection data encoder model E is sent to each local node. I .

[0176] The large-scale vinegar product sample data is passed through the central node encoder E IC After the image feature extraction pre-training is completed, the model performance, such as accuracy and loss value, is evaluated using the validation set. Based on the evaluation results, hyperparameter adjustment and model optimization are performed. The encoder part is extracted from the pre-trained model, converted into a format that can be distributed, and the encoder model is sent to each local node through a secure channel.

[0177] S204: The local nodes collect the hyperspectral detection data and label data of their respective local vinegar samples, use the encoder model sent by the central node to extract features from the hyperspectral detection data of the vinegar samples, generate compressed representations and concatenate them, and use the TABSYN diffusion model to form tabular data. At the same time, the residual layer and multiple attention mechanisms are used to predict noise and train the local diffusion generation model.

[0178] like Figure 6 As shown in the structural diagram of the generation model in the geographical indication vinegar product identification method provided in the present application, the local nodes collect the hyperspectral data of vinegar samples of their respective brands, as well as category and grade labels, collect the hyperspectral detection data of the samples and perform preprocessing.

[0179] When selecting geographical indication vinegar samples, this application selects multiple designated vinegar brands for research or identification, and extracts sample data of different vinegar products of various brands based on the quality characteristics and types of vinegar produced according to the geographical characteristics of different regions. At the same time, brand category and grade labels are collected as the corresponding ones after identification and classification.

[0180] When collecting sample data, this application involves the management and monitoring of geographical indication vinegar products, with a particular focus on the continuous tracking and recording of the brewing conditions of vinegar products from different origins and varieties such as Shanxi, Fujian, and Jiangsu through intelligent monitoring technology, and the collection of vinegar brand category and grade labels at the same time. Specifically, this application randomly selects sample areas from each selected geographical indication vinegar product brewing site. In order to monitor the brewing conditions of vinegar products more carefully and collect the fermentation process of samples at different times, a total of t sample data are collected. After the collection is completed, the samples are taken back to the laboratory and properly stored.

[0181] After the vinegar sample collection is completed, this application uses a hyperspectral imaging system to collect hyperspectral detection data for geographical indication vinegar samples. In order to reduce the impact of the environment and instruments on the image data, the accuracy and reliability of the hyperspectral detection data collection of geographical indication vinegar samples are ensured. In order to reduce the impact of the environment and instruments on the image data, the original spectral data is RAD corrected using software after the image collection is completed. Subsequently, black and white plate correction is performed to eliminate the impact of external factors such as air on the image and obtain the relative reflectivity. After the correction is completed, in order to ensure the accuracy of the results, ENVI 5.3 software is used to manually select sample images as regions of interest (ROIs) in the image according to different shooting directions, and the average reflectivity of these ROIs is used to represent the spectral value of the sample. The spectral data from different lenses are manually integrated to construct a multi-band reflectivity data set.

[0182] In the actual test process, signal interference often comes from detector anomalies and current instability. These interference factors will cause spikes in the spectrum signal. In order to optimize the accuracy of the detection data, this application adopts four different pre-processing methods, including standard normal variable transformation (SNV), multivariate scatter correction (MSC), first-order derivative (Der1) and second-order derivative (Der2), and performs fine pre-processing on the spectral curve. Through a comprehensive comparison of these pre-processing results, the interference effect is effectively reduced and the accuracy and reliability of the data are improved.

[0183] The distributed hyperspectral sequence encoder and image encoder are used to generate compressed representations and concatenate them. The TABSYN diffusion model is used to form tabular data. The residual layer and multiple attention mechanisms are used to predict noise and train the local data generator.

[0184] The sequence feature data x will be formed S F (t) and image feature data x I F (t) respectively through the sequence modal data diffusion model Diff S and image data generator Diff I , and the dataset x F The test set, validation set, and training set are divided into 8:1:1 ratio.

[0185] Training a generative model with input from representation data is challenging because the modal representation data is very complex. Here, the original representation data is converted into a continuous embedding space, in which a good diffusion model with Gaussian noise becomes feasible. At the same time, multiple self-attention mechanisms are used to handle the correlation between data and establish connections between data, which can better reflect the data distribution and facilitate denoising.

[0186] Assume M num and M cat are the number of numerical columns and categorical columns, respectively.

[0187] Each row is represented as a numerical feature and a categorical feature vector x = [x num ,x cat ],in Specifically, the i-th category attribute has C i There are finite candidate values, so we have This application mainly studies unconditional generation tasks. The goal of this application is to learn a parameterized generative model This model can be used to generate realistic and diverse synthetic tabular data

[0188] This application effectively utilizes Transformer to assist in the automatic encoding of tables by first learning a unique tagger for each column of modal data and then feeding the tag (column) wise representation into Transformer to capture the complex relationships between columns.

[0189] Module 1: Feature Marker

[0190] The feature tokenizer converts each column (both numerical and categorical) into a d-dimensional vector. First, we preprocess the categorical features using one-hot encoding, i.e. Each record is represented by Then, a linear transformation is applied to the numerical columns and an embedding lookup table is created for the categorical columns, where each category is assigned a learnable d-dimensional vector, i.e.:

[0191]

[0192] in, are the learnable parameters of the tagger, Now, each record is represented as a stack of all column embeddings.

[0193]

[0194] Module 2: Forward Process of Latent Generative Diffusion Model

[0195] The diffusion model is a probabilistic model that aims to learn the data distribution p(x) by gradually denoising the normally distributed variables. The forward process makes the data progressively more blurred by adding Gaussian noise at each time step (T). This is equivalent to learning a fixed Markov chain of length T, adding Gaussian noise as follows:

[0196]

[0197] Where β is a predefined noise ratio parameter, which is usually gradually increased throughout the process. The process starts from x0 (original data) and gradually generates x1, x2, …, x t (pure noise).

[0198] Each step of the diffusion process generates noisy data, and the entire diffusion process is a Markov chain, as shown below:

[0199]

[0200] The model can be interpreted as an equally weighted denoising autoencoder sequence ∈ θ (x t ,t); t = 1…T, they are trained to predict their input x tA denoising variant of t is a noisy version of the input x. The corresponding objective can be simplified to:

[0201]

[0202] Module 3: Denoising process of generative model

[0203] After the above model has learned well, the encoder is used to extract the latent embedding and the output of the encoder is flattened to The embedding recorded in this way is a vector rather than a matrix. To learn the distribution p(z) of the latent embedding, consider the following forward diffusion process and backward sampling process:

[0204] ——Diffusion process

[0205] ——Sampling process

[0206] where x F 0 is the initial embedding from the encoder, x F t is the diffusion embedding at time t, and σ(t) is the noise level. For x t The score function, w t is a standard Wiener process. The training of the diffusion model is achieved by denoising score matching:

[0207]

[0208] where x F t =x F 0+σ(t)ε,∈ θ is a neural network (named as the denoising function) that uses the perturbed data x t and time t to approximate Gaussian noise. Then After the model training is completed, the synthetic data is obtained through the inverse process. The noise level σ(t) defines the noise scale of the perturbed data at different time steps and significantly affects the final solution trajectory of the differential equation. The noise level σ(t) = t is set, which is the linear WRT time.

[0209] The representation data is trained through TABSYN to generate high-quality and realistic fake data, where the sequence modal feature data is x GS (t) = Diff S (x GS F (t)), the image modality feature data is x GI (t) = Diff I (xGI F (t)), which facilitates the construction of the central node generator model after the subsequent random cycle distillation.

[0210] In the diffusion generation process, several basic network submodules are used to predict the noise. The first step is the residual layer, which is divided into a direct mapping layer and an indirect mapping layer (the indirect mapping layer is mainly composed of a new multi-attention mechanism with a soft threshold). In the indirect mapping module, the first step is to establish a fully connected layer with a ReLU activation function to extract and normalize the features:

[0211] FC(x F 0) = ReLU(BN(Linear(x F 0)))

[0212] In the formula, the function Linear represents the linear fully connected layer, the function BN represents the BorderlineNorm normalization layer, and the function ReLU is the ReLU activation function.

[0213] The second step is to set up multiple self-attentions to handle the correlation between data and establish connections between data, which can better reflect the data distribution and facilitate denoising. At the same time, a soft threshold is set at the end of the multiple self-attentions to enhance the denoising effect. The threshold can be automatically learned and set by the multi-layer perceptron. The Sigmoid activation function is used to ensure that the threshold of the soft threshold function is positive and within an appropriate range, avoiding the situation where the output is all zero. At the same time, the setting here allows each sample to have its own unique set of thresholds, making the residual network suitable for situations where the noise content of a single sample is different.

[0214] Since the generated data has sequence characteristics, the t vector is added to the vector data corresponding to time t, so the dimension of the sequence modal data becomes x GS (t) The image modality data dimension becomes x GI (t)

[0215] S205: The central node constructs a global diffusion generation model by cyclically distilling the local diffusion generation models constructed by each local node; and uses the global diffusion generation model to generate simulated vinegar sample hyperspectral characterization data and label data.

[0216] By performing model distillation on multiple diffusion generation models of different modes constructed by each local node, multiple rounds of cyclic distillation are performed on each local node for continuous iterative cyclic training to form a global diffusion generation model of the central node. The specific implementation is as follows:

[0217] Given a data distribution p data(x) sample x, noise scheduling function α (t) and α (t) , by minimizing the weighted mean square error i Multiple diffusion models Diff of [I∈(0~R)] S (x GS F (t)) and Diff I (x GI F (t)) Diffusion generator Diff with distillation as the central node C .

[0218]

[0219] in is the signal-to-noise ratio, and ω(λ t ) is a pre-specified weight function.

[0220] Once the diffusion model Diff is trained C , we can use the discrete-time DDIM sampler to sample from the model.

[0221] Specifically, the DDIM sampler is start:

[0222]

[0223] N is the total number of sampling steps. The last sample will use Diff C (z0) is generated.

[0224] It should be noted that when performing model distillation, in order to ensure the generalization of model learning, this application adopts a random method within the batch, that is, the distillation order adopted by each node is random.

[0225] Generate model Diff through central node diffusion C The generated data passes through the decoder D C Perform decoding operation. The global feature c output by the encoder encodes the entire input sequence x F t Information. Given the output sequence y in the training sample F t , for each time step t′ (the symbol is different from the input sequence or the encoder’s time step t), the decoder outputs y′ t The conditional probability will be based on the previous output sequence y F t and the global feature c, namely P(y t′ |y1,…,y t′-1 ,c).

[0226] To this end, this application can use another recurrent neural network as the decoder D C At time step t′ of the output sequence, the decoder D C The output y of the previous time step t′-1 and global features c as input and combine them with the hidden state s of the previous time step t′-1 Transformed to the hidden state s of the current time step t′ .

[0227] With decoder D C After the hidden state is obtained, we can use the custom output layer and softmax operation to calculate P(y t′ |y1,…,y t′-1 ,c).

[0228] S206: Use the global diffusion generation model of the central node to generate simulated vinegar sample hyperspectral characterization data and its label data, use the LSTM long short-term memory network, take the two modal data with t as the time node as input, train and update the classifier of geographical indication vinegar products, and realize the identification of geographical indication vinegar products.

[0229] The data collection, feature extraction, data generation and data distillation processes of the geographical indication vinegar product identification method have been completed. The generated false data is input into the classifier for the final identification process of the geographical indication vinegar product. Since the data of the two modes are both generated by hyperspectral detection data collected on a monthly basis, they both have obvious time series characteristics. When performing classifier identification, the LSTM long short-term memory neural network is used to complete it.

[0230] Will generate the model output data y F t As the input data of LSTM, the specific implementation process is as follows:

[0231] Input Gate - I t =σ(W i y F t +U i h t-1 +b i )

[0232] Forget Gate - F t =σ(W f y F t +U f h t-1 +b f )

[0233] Output Gate——O t =σ(Wo y F t +U o h t-1 +b o )

[0234] The Sigmoid activation function is used as the gating mechanism:

[0235] The data output through the output gate is scaled using the tanh function:

[0236] Among them, the loss function model of the geographical indication vinegar product identification method is:

[0237]

[0238] Among them, T represents the real data, Y represents the data generated by the geographical indication vinegar product identification method, and y i represents the i-th data, N represents the number of training samples, N>0 and N∈N + The stochastic gradient descent algorithm is used to continuously perform training optimization. By continuously calculating the gradient of the model loss value, the model performance and training speed are improved, thereby achieving the purpose of optimizing the recognition method.

[0239] During the testing phase of the geographical indication vinegar product identification method, for each input sample, the trained recognition model is first used to enter the central node for pre-training operations such as encoding. After obtaining the feature data of the sample, it is input into the classifier for identification to obtain the classification result.

[0240] When studying geographically indicated vinegar products, this application uses a pre-training method to extract sequence and image features from the hyperspectral molecular detection data of vinegar products, which can effectively compress data in subsequent federated learning and improve the efficiency of data transmission. When performing pre-training for sequence feature extraction, the gating mechanism of LSTM is used to memorize the sequence modal data, and the factor-aware attention network is used to reduce the interference with the prediction; when performing pre-training for image feature extraction, the effective attention (E-Attention) model is used to improve the efficiency of feature extraction, and feature extraction is performed based on linear attention and pruning methods to reduce the various costs incurred during model training. Feature extraction pre-training is performed according to different data types to speed up the speed of data processing and transmission, thereby effectively improving the efficiency of federated learning.

[0241] In order to effectively protect customer privacy and increase the amount of data, this application proposes a diffusion generation model, which adds a noise prediction module to the traditional diffusion generation model, and uses the residual layer and multiple attention mechanisms to effectively denoise. At the same time, the setting here allows each sample to have its own unique set of thresholds, making the residual network suitable for situations where the noise content of a single sample is different. By training on real sample data, the data volume is increased while effectively optimizing the central node and local node server.

[0242] The present application also proposes a federated learning framework that integrates generative models. Based on the federated learning framework, a diffusion generative model is used to generate data for uploaded large-scale data. By uploading hyperspectral detection data, the central node encoder is trained using large-scale data at the central node of the federated learning framework. The encoder is then sent to the local node of the federated learning to form a local node generator. Finally, the local generators of each node are aggregated by federated learning through a cyclic distillation method to form a central node generator, which is finally input into the central node classifier to effectively and accurately identify geographically indicated vinegar products.

[0243] In addition, the present application also provides a geographical indication vinegar product identification device, such as Figure 7 As shown in the structural block diagram of the geographically-indicated vinegar product identification device provided in the present application, the device specifically includes a memory 71 and a processor 72, and the memory 71 stores a computer program. When the computer program is executed by the processor 72, the geographically-indicated vinegar product identification method described above is implemented.

[0244] In addition, the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned methods for identifying geographically-indicative vinegar products is implemented.

[0245] Computer readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0246] The professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0247] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0248] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A method for identifying a geographically indicated vinegar product, characterized in that: include: Constructing a federated learning framework for geographical indication vinegar products, including a central node and multiple local nodes; wherein the central node is used to coordinate the joint model training, and the local nodes independently manage their respective data and local models; The central node collects the hyperspectral detection data of a large number of vinegar products, uses the constructed encoder model to extract features from the hyperspectral detection data of vinegar products, and sends the constructed encoder model to each local node; The local nodes collect the hyperspectral detection data and label data of their respective local vinegar samples, use the encoder model sent by the central node to extract features from the hyperspectral detection data of the vinegar samples, and use the constructed local diffusion generation model to generate data for training; The central node constructs a global diffusion generation model by cyclically distilling the local diffusion generation models constructed by each local node; the global diffusion generation model is used to generate simulated vinegar sample hyperspectral characterization data and label data; The simulated vinegar sample hyperspectral characterization data and label data are used to train the classifier of geographical indication vinegar products, and the trained classifier outputs the identification and classification results of geographical indication vinegar products.

2. The method for identifying geographically indicated vinegar products according to claim 1, characterized in that: The method of using the constructed encoder model to extract features from the hyperspectral detection data of vinegar products and sending the constructed encoder model to each local node includes: The constructed sequence feature encoder model is used to extract the sequence modal features to form sequence feature data; the sequence feature encoder model is sent to each local node; The constructed image feature encoder model is used to extract image features to form image feature data; the image feature encoder model is sent to each local node.

3. The method for identifying geographically indicated vinegar products according to claim 2, characterized in that: The construction of the sequence feature encoder model includes: Construct a pre-trained model based on multi-scale CNN, long short-term memory network and factor-aware attention network as the sequence feature encoder model; The time series modal data of the hyperspectral detection data are collected at a preset sampling rate, and parallel CNN models of different scales are used to extract multi-scale features from the time series modal data of each sampling point; each input sequence data is converted from one data to multiple data, and features of different degrees are fused; The factor-aware attention network adopts the following attention mechanism: in, is a learnable parameter, (·) T represents the transpose of a matrix, is the hidden state of LSTM at time step t, m is the size of LSTM unit; exogenous sequence is extracted by adaptively: Then the hidden state at time t is updated as: Where LSTM(·) is an LSTM unit, and the sequence feature input x of the central node SCt Newly calculated Replacement; symbol is the output of factor-aware attention at time step t; The data output after module training is multi-dimensional data, and average pooling is used to extract average and overall features: Among them, x S F (t) represents the rectangular area related to the kth feature map The average pooling output value, x kpq Represents a rectangular area The element at (p, q) in Represents a rectangular area The number of elements in A multi-way LSTM is used to learn from x SCt To the hidden state h t The mapping: i t =σ(W i x SCt +U i h t-1 +b i ) f t =σ(W f x SCt +U f h t-1 +b f ) the t =σ(W o x SCt +U o h t-1 +b o ) c t =f t ⊙c t-1 +σ(W i x SCt +U i h t-1 +b i )⊙tanh(W c x SCt +U c h t-1 +b c ) h t =o t ⊙tanh(f t ⊙c t-1 +i t ⊙tanh(W c x SCt +U c h t-1 +b c )) Where the symbol ⊙ is the element-wise product, σ = 1 / (1+e -x ) is the sigmoid function; is the hidden state of LSTM at time t; the symbol is learned during training 4. The method for identifying geographically indicated vinegar products according to claim 2, characterized in that: The image feature encoder model constructed for the hyperspectral data is used to extract the image features to form the image feature data, including: After the input information is determined by the patching module and the fully connected layer, the Transformer model is constructed to form an image feature encoder model for image modality processing; Among them, after extracting with the baseline Transformer model, the model parameters of the downstream task are directly fine-tuned by attaching a task layer on the pre-trained encoder; the input of the subsequent Transformer model architecture is a sequence of image patches The patch is then linearly projected to obtain the patch embedding Add a special token [S] before the input sequence; embed the standard learnable ID position Added to the patch embedding, the input vector is fed into the Transformer architecture; The Transformer image feature extraction network is trained in part by an effective attention mechanism, which includes: each token is scored separately externally to determine the importance of the class token for classification based on the input token; some tokens are retained based on the score sampling and pruned from the token dimension; from an internal perspective, a combination function is used instead of the Softmax operator to calculate the self-attention matrix, resulting in a new linear attention mechanism.

5. The method for identifying geographically indicated vinegar products according to any one of claims 1 to 4, characterized in that: The method of using the constructed local diffusion generation model to generate data for training includes: Learn a unique feature tokenizer for each column of modal data, and then feed the tokens into a Transformer to capture the complex relationships between columns; The feature tokenizer converts each column into a d-dimensional vector; categorical features are preprocessed using one-hot encoding, i.e. Each record is represented by A linear transformation is applied to the numerical columns and an embedding lookup table is created for the categorical columns, where each category is assigned a learnable d-dimensional vector, i.e.: in, is a learnable parameter of the tagger Each record is represented as a stack of all column embeddings; Each step of the diffusion process generates a noisy data, and the entire diffusion process is represented as a Markov chain: x0 is the original data, and x1, x2, ..., x are generated step by step. t .

6. The method for identifying geographically indicated vinegar products according to claim 5, characterized in that: The method of using the constructed local diffusion generation model to generate data for training includes: Use residual networks and multiple attention mechanisms to predict noise and train local diffusion generation models; Among them, the residual layer is divided into a direct mapping layer and an indirect mapping layer; in the indirect mapping layer, a fully connected layer with a ReLU activation function is established to extract and normalize features: FC(x F 0)=ReLU(BN(Linear(xF0))) In the formula, the function Linear represents the linear fully connected layer, the function BN represents the BorderlineNorm normalization layer, and the function ReLU is the ReLU activation function; A soft threshold is set at the end of the multiple self-attention mechanism, and the Sigmoid activation function is used to ensure that the threshold of the soft threshold function is positive and within the set range.

7. The method for identifying geographically indicated vinegar products according to claim 5, characterized in that: The central node constructs a global diffusion generation model by cyclically distilling the local diffusion generation model constructed by each local node, including: The local diffusion generation model is distilled, and a global diffusion generation model is constructed based on the iterative cycle training mechanism of each local diffusion generation model: Given a data distribution p data (x) sample x, noise scheduling function α (t) and σ (t) , by minimizing the weighted mean square error i Multiple diffusion models Diff of [I∈(0~R)] S (x s F (t)) and Diff I (x GI F (t)) Diffusion generation model Diff with distillation as the central node C : in, is the signal-to-noise ratio, and ω(λ t ) is a pre-specified weight function.

8. The method for identifying geographically indicated vinegar products according to claim 5, characterized in that: The method of training a classifier of geographical indication vinegar products using simulated vinegar sample hyperspectral characterization data and label data includes: Using the LSTM long short-term memory network, the two modal data with time node t are used as input to train the classifier: The data y output by the diffusion generation model F t As the input data of LSTM, Input Gate - I t =σ(W i y F t +U i h t-1 +b i ) Forget Gate - F t =σ(W f y F t +U f h t-1 +b f ) Output Gate——O t =σ(W o y F t +U o h t-1 +b o ) The Sigmoid activation function is used as the gating mechanism: The data output through the output gate is scaled using the tanh function: Among them, the loss function model is: Among them, T represents the real data, Y represents the data generated by the geographical indication vinegar product identification method, and y i represents the i-th data, N represents the number of training samples, N>0 and N∈N + ; Use the stochastic gradient descent algorithm for training optimization, and optimize the model performance and training speed by calculating the gradient of the model loss value.

9. A geographical indication vinegar product identification device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method for identifying a geographically-indicated vinegar product according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method for identifying a geographically-indicated vinegar product according to any one of claims 1 to 8 is implemented.

Images