A luxury goods counterfeit identification model training method, luxury goods counterfeit identification method and device

Abstract

The present invention provides a luxury goods counterfeit detection model training method, luxury goods counterfeit detection method and device, wherein a training sample set is used to train an initial luxury goods counterfeit detection model comprising a micro-feature extraction network, a macro-feature extraction network, a multimodal fusion network and a classification network; the micro-feature extraction network comprises a convolutional neural network layer and an overlapping patch embedding layer; the macro-feature extraction network comprises an overall nested edge detection layer and a residual network; the image patch embedding layer in the multimodal fusion network divides the micro-features and performs vector conversion, and then inputs the micro-features into the Transformer model for attention weight adjustment to obtain a first target micro-feature, which is input together with the macro-feature into a cross-attention feature fusion module to obtain a second target micro-feature, the two target micro-features are weightedly summed and fused into a classification vector, and then input into a multimodal fusion network for global feature capture to obtain a target classification vector, which is then input into the classification network to obtain a sample authenticity prediction result.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a luxury goods counterfeit identification model training method, a luxury goods counterfeit identification method and a luxury goods counterfeit identification device. Background Art

[0002] In the field of luxury goods authentication, existing methods primarily focus on capturing microscopic features of objects, such as the density and distribution of ink dots, enhancing feature learning by identifying these key details or applying attention mechanisms. While these methods excel at identifying subtle differences, they often rely heavily on microscopic features and tend to overlook macroscopic features and overall contextual clues. Furthermore, existing methods are complex and require precise microfeature extraction. In practical applications, inaccurate positioning or wear and tear on luxury goods due to environmental and transportation issues can make microfeatures less distinct, making it difficult to effectively distinguish genuine goods from counterfeits. Summary of the Invention

[0003] In view of this, the embodiments of the present invention provide a luxury goods counterfeit identification model training method, luxury goods counterfeit identification method and device to eliminate or improve one or more defects in the existing technology, and solve the problem that the existing technology cannot effectively identify the authenticity of luxury goods.

[0004] One aspect of the present invention provides a method for training a luxury goods counterfeit detection model, the method comprising the following steps:

[0005] Obtaining a training sample set, wherein the training sample set includes multiple samples, each sample including an image of a luxury print; the multiple samples in the training sample set have labels including true values ​​for authenticity classification;

[0006] An initial luxury goods authentication model is obtained, wherein the initial luxury goods authentication model includes a micro-feature extraction network, a macro-feature extraction network, a multimodal fusion network and a classification network; the macro-feature extraction network includes a global nested edge detection layer and a residual network; the multimodal fusion network includes an image patch embedding layer, a Transformer model and a cross-attention feature fusion module; the micro-feature network extraction network takes samples in the training sample set as input and outputs micro-features; the macro-feature network extraction network takes samples in the training sample set as input, obtains a target edge detection map through the global nested edge detection layer, and then extracts and outputs macro-features through the residual network; the image patch embedding layer in the multimodal fusion network divides the micro-features into the target edge detection map and the target edge detection map is obtained through the global nested edge detection layer. The features are converted into vectors and then input into the Transformer model for attention weight adjustment to obtain the first target micro-feature, which is input into the cross-attention feature fusion module together with the macro-feature to obtain the second target micro-feature. The first target micro-feature and the second target micro-feature are weightedly summed and fused into a classification vector, and then input into the multimodal fusion network for global feature capture to obtain a target classification vector; the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample; a loss function is constructed according to the deviation between the authenticity prediction result corresponding to each sample and the true value of the authenticity classification, and the parameters of the initial luxury goods authentication model are iteratively updated with the goal of minimizing the loss function to obtain a luxury goods authentication model.

[0007] In some embodiments, the micro-feature network extraction network takes samples in the training sample set as input and outputs micro-features, including:

[0008] Inputting samples in the training sample set into a first convolutional layer for downsampling to obtain a first feature map, and gradually extracting image features from the first feature map through three consecutive convolutional blocks; the convolutional block includes three consecutive convolutional layers;

[0009] The first feature map is downsampled using a maximum pooling layer to obtain a second feature map, and the second feature map is subjected to depthwise separable convolution through two shifted reverse bottleneck convolutions with a squeeze excitation mechanism to further enhance the extracted image features;

[0010] The second feature map is divided into multiple fixed-size image blocks with overlapping areas through an overlapping patch embedding layer, and the feature vectors of the multiple image blocks are obtained through an embedding operation and encoded to obtain the microscopic features; the microscopic features include the density and distribution of ink dots.

[0011] In some embodiments, the macro-feature network extraction network takes samples in the training sample set as input, obtains a target edge detection image through the overall nested edge detection layer, and then extracts and outputs macro-features through the residual network, including the following steps:

[0012] Extracting macroscopic feature information from the preprocessed samples layer by layer through multiple convolutional layers;

[0013] Constructing a plurality of edge detection branches including branch convolution layers, wherein each of the edge detection branches generates a plurality of branch feature maps with different resolutions through the corresponding branch convolution layer;

[0014] Inputting the plurality of branch feature maps into a bilinear upsampling layer for upsampling to obtain a plurality of edge detection maps of the same size as the luxury print image, and performing weighted fusion on the plurality of edge detection maps to generate a target edge detection map;

[0015] The residual network is used to perform coarse-grained feature extraction on the target edge detection image and obtain the macro features; the macro features include the pattern and material of the luxury goods.

[0016] In some embodiments, the Transformer model further includes a multi-head self-attention module, a multi-layer perceptron module, and a layer norm module that are arranged in series.

[0017] In some embodiments, the image patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, and then inputs the micro features into the Transformer model for attention weight adjustment to obtain a first target micro feature, which is input into the cross-attention feature fusion module together with the macro feature to obtain a second target micro feature, and the first target micro feature and the second target micro feature are weighted and summed, and the classification vector is fused and input into the multimodal fusion network for global feature capture to obtain a target classification vector. The classification network classifies the target classification vector to obtain the authenticity prediction result of the sample, and the steps include:

[0018] Divide the microscopic features into multiple fixed-size image patches, flatten each patch into a vector, concatenate the image patch vector and the classification vector initialized to all 0s into a feature matrix, and input the matrix into the multi-head self-attention module for feature processing and global feature capture;

[0019] The feature matrix is ​​nonlinearly mapped by the multilayer perceptron module and then input into the layer norm module for normalization to obtain the first target micro-feature after attention weight adjustment;

[0020] The cross-attention fusion module uses the macro features F macroCalculate the query vector Q using the first target micro-feature F micro Calculate the key vector K and value vector V as follows:

[0021] Q=W Q ·F macro ;

[0022] K=W K ·F micro ;

[0023] V=W V ·F micro ;

[0024] The cross attention score is calculated based on the query vector and the key vector and normalized by the softmax function to obtain the attention weight matrix, which is expressed as:

[0025]

[0026] The second target micro-feature F is obtained by weighting the value vector with the attention weight enhanced , the expression is:

[0027] F enhanced =A·V;

[0028] The second target micro-feature is expanded to the same size as the luxury print image, and a learnable dynamic weight parameter α is introduced to the first target micro-feature F self and the second target microfeature F expanded After weighted summation and fusion of the classification vectors, the vectors are input into the multimodal fusion network for global feature capture to obtain the target classification vector, which is expressed as:

[0029] F final =α·F self +(1-α)·F expanded ;

[0030] The classification network classifies the target classification vector to obtain a true or false prediction result of the sample.

[0031] In some embodiments, the method further comprises:

[0032] A softmax function layer is connected to the output end of the classification network to output the probability of each classification in the luxury goods authenticity prediction result.

[0033] In some embodiments, the loss function is expressed as:

[0034] Loss=-[ylog(p)+(1-y)·log(1-p)];

[0035] Wherein, y represents the value of the label, and p represents the probability that the luxury print image is authentic.

[0036] On the other hand, the present invention also provides a method for identifying counterfeit luxury goods, the method comprising the following steps:

[0037] Obtain the luxury goods print image to be detected;

[0038] The luxury goods print image to be detected is input into the luxury goods counterfeit detection model trained by the above-mentioned luxury goods counterfeit detection model training method, and the authenticity classification and identification result of the luxury goods print image to be detected is output.

[0039] On the other hand, the present invention also provides a luxury goods counterfeit detection model training device, including a processor, a memory and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0040] On the other hand, the present invention also provides a luxury goods authentication device, comprising a processor, a memory, and a computer program / instruction stored in the memory, wherein the processor is configured to execute the computer program / instruction, and when the computer program / instruction is executed, the device implements the steps of the above method.

[0041] The beneficial effects of the present invention are at least:

[0042] In the luxury goods authentication model training method, luxury goods authentication method and device of the present invention, the initial luxury goods authentication model includes a micro feature extraction network, a macro feature extraction network, a multimodal fusion network and a classification network; the micro feature extraction network includes a convolutional neural network layer and an overlapping patch embedding layer; the convolutional neural network first obtains a first feature map by downsampling and then inputs it into a maximum pooling layer for downsampling to obtain a second feature map, reducing the spatial resolution while retaining feature information; the overlapping patch embedding layer divides the second feature map into multiple image blocks, which can reduce the resolution of the feature map while retaining local information, providing more effective input for subsequent feature extraction and processing; the macro feature extraction network includes a global nested edge detection layer and a residual network; the global nested edge detection layer performs feature extraction layer by layer from low-level features to high-level features to obtain information of the luxury goods print image, The branch feature maps of multiple edge detection branches are upsampled and weightedly fused to obtain a target edge detection map, so that the detection obtains edge contours while retaining details; the multimodal fusion network includes an image patch embedding layer, a Transformer model and a cross-attention feature fusion module, the multi-head self-attention module guides the model to focus on key areas, so that the classification vector captures key features and obtains micro features after weight adjustment by the attention mechanism, the multi-layer perceptron module enhances the feature expression ability, the layer norm module ensures the stability of model training, the cross-attention module uses macro features to query micro features, enhances the model's context perception ability, and realizes multi-scale feature fusion; the loss function uses a cross-entropy loss function, and updates parameters with the goal of minimizing the loss function to obtain a luxury goods authentication model, and the authenticity classification results of luxury goods are obtained through the luxury goods authentication model.

[0043] Additional advantages, objects, and features of the present invention will be set forth in part in the following description and will become apparent to those skilled in the art upon examination of the following or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained by the structures particularly pointed out in the description and drawings.

[0044] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. In the drawings:

[0046] Figure 1 2 is a flow chart of a luxury goods counterfeit identification model training method according to an embodiment of the present invention.

[0047] Figure 2 Schematic diagram of the structure of a luxury goods authentication model according to one embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0049] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0050] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0051] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0052] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0053] In the prior art, luxury goods authentication mainly focuses on capturing the microscopic features of objects, such as the density and distribution of ink dots, and enhances feature learning by identifying these key details or applying attention mechanisms. This method is highly dependent on microscopic features and tends to ignore macroscopic features and overall context clues. These methods are highly complex and require high precision in the extraction of microscopic features. When luxury goods are worn due to inaccurate positioning or environmental and transportation problems, the microscopic features will become less obvious, making it difficult to effectively distinguish between genuine and counterfeit goods. The present invention proposes a training method for a luxury goods authentication model, a luxury goods authentication method and a device, which trains an initial luxury goods authentication model comprising a microscopic feature extraction network, a macroscopic feature extraction network, a multimodal fusion network and a classification network through a training sample set. The macroscopic feature extraction network comprises a globally nested edge detection layer and a residual network. The multimodal fusion network comprises an image patch embedding layer, a Transformer model and a cross-attention feature fusion module. The visual feature network extraction network takes the samples in the training sample set as input and outputs micro features; the macro feature network extraction network takes the samples in the training sample set as input, obtains the target edge detection map through the overall nested edge detection layer, and then extracts and outputs macro features through the residual network; the image patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, and then inputs the micro features into the Transformer model for attention weight adjustment to obtain the first target micro features, and inputs the micro features together with the macro features into the cross attention feature fusion module to obtain the second target micro features, performs weighted summation on the first target micro features and the second target micro features, fuses the classification vector, and then inputs the multimodal fusion network for global feature capture to obtain the target classification vector; the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample; the luxury print image to be detected is input into the luxury goods authentication model, and the authenticity classification recognition result is output.

[0054] Figure 1 2 is a flow chart of a luxury goods counterfeit identification model training method according to an embodiment of the present invention. Figure 2 Schematic diagram of the structure of a luxury goods authentication model according to an embodiment of the present invention. Specifically, the present application provides a method for training a luxury goods authentication model, a luxury goods authentication method and apparatus, and the method includes the following steps S101 to S103:

[0055] Step S101: Obtain a training sample set, where the training sample set includes multiple samples, each sample includes an image of a luxury print; and multiple samples in the training sample set have labels including true values ​​of authenticity classification.

[0056] Step S102: Obtain an initial luxury goods authentication model, which includes a micro-feature extraction network, a macro-feature extraction network, a multimodal fusion network, and a classification network; the macro-feature extraction network includes a global nested edge detection layer and a residual network; the multimodal fusion network includes an image patch embedding layer, a Transformer model, and a cross-attention feature fusion module; the micro-feature network extraction network takes samples from the training sample set as input and outputs micro-features; the macro-feature network extraction network takes samples from the training sample set as input, obtains a target edge detection map through the global nested edge detection layer, and then extracts and outputs macro-features through the residual network; the image patch embedding layer in the multimodal fusion network divides the micro-features and performs vector conversion, which is then input into the Transformer model for attention weight adjustment to obtain a first target micro-feature, which is then input together with the macro-feature into the cross-attention feature fusion module to obtain a second target micro-feature, the first target micro-feature and the second target micro-feature are weightedly summed and fused into a classification vector, which is then input into the multimodal fusion network for global feature capture to obtain a target classification vector; the classification network classifies the target classification vector to obtain a sample authenticity prediction result.

[0057] Step S103: construct a loss function based on the deviation between the authenticity prediction result of each sample and the true authenticity classification value, and iteratively update the parameters of the initial luxury goods authentication model with the goal of minimizing the loss function to obtain the luxury goods authentication model.

[0058] In step S101, the samples in the training sample set are partial printed images of luxury goods with ink-printed patterns, and the luxury goods include but are not limited to clothing, luggage, decorations and other products with ink-printed patterns; the samples are labeled with the true value of the authenticity classification.

[0059] In step S102, the initial luxury goods authentication model is a two-path model of micro and macro. The micro feature extraction network of the micro path inputs the luxury goods print image, which is sampled by a convolutional neural network and the feature map obtained by sampling is divided into multiple image blocks with overlapping areas through an overlapping patch embedding layer. The feature vectors extracted from the multiple image blocks are encoded to obtain micro features.

[0060] In some embodiments, the micro-feature network extraction network takes samples in the training sample set as input, and the steps of outputting micro-features include S11 to S13:

[0061] Step S11: Input the samples in the training sample set into the first convolution layer for downsampling to obtain a first feature map, and the first feature map is gradually extracted from the image features through three consecutive convolution blocks; the convolution block includes three consecutive convolution layers.

[0062] Step S12: Use the maximum pooling layer to downsample the first feature map to obtain a second feature map, perform depth-wise separable convolution on the second feature map through two moving inverted bottleneck convolutions with a squeeze excitation mechanism, and further enhance the extracted image features.

[0063] Step S13: Divide the second feature map into multiple fixed-size image blocks with overlapping areas through the overlapping patch embedding layer, obtain feature vectors of the multiple image blocks through the embedding operation and encode them to obtain micro features; the micro features include the density and distribution of ink dots.

[0064] Specifically, the convolutional neural network consists of three stages: a first convolutional layer, a maximum pooling layer, and a moving inverted bottleneck convolution layer. The first convolutional layer is a convolutional layer with a stride of 2. Downsampling reduces the spatial dimension of the luxury print image and produces a low-resolution first feature map. The first feature map passes through the first three-layer convolution block, then inputs the second three-layer convolution block, and finally passes through the third three-layer convolution block, reducing computational complexity while preserving important image information. The first feature map is further downsampled using a maximum pooling layer with a stride of 2 and a convolution kernel of 3x3 to produce a second feature map, further reducing spatial resolution while preserving important information. The size of the second feature map is smaller than that of the first feature map, which is smaller than the original luxury print image. Furthermore, the moving inverted bottleneck convolution with a squeeze excitation mechanism performs depthwise separable convolution on each input channel and then combines the outputs, reducing the computational complexity of data processing. The number of channels is expanded or contracted before and after the depthwise separable convolution.

[0065] In some embodiments, the macro-feature network extraction network takes samples in the training sample set as input, obtains a target edge detection map after the luxury print image passes through the overall nested edge detection layer, and extracts and outputs macro-features through the residual network after obtaining the target edge detection map through the overall nested edge detection layer, including steps S21 to S24:

[0066] Step S21: extracting macroscopic feature information from the preprocessed samples layer by layer through multiple convolutional layers.

[0067] Step S22: Construct multiple edge detection branches including branch convolution layers, and each edge detection branch generates multiple branch feature maps with different resolutions through the corresponding branch convolution layer.

[0068] Step S23: Input the multiple branch feature maps into the bilinear upsampling layer for upsampling to obtain multiple edge detection maps with the same size as the luxury print image, and perform weighted fusion on the multiple edge detection maps to generate a target edge detection map.

[0069] Step S24: performing coarse-grained feature extraction on the target edge detection image through a residual network and obtaining macro features of the luxury print image; the macro features include the pattern and material of the luxury product.

[0070] Specifically, after the luxury print image is input into the macro feature extraction network, the target edge detection map of the luxury print image is obtained through the overall nested edge detection layer (HED). During the edge detection process, the edge detection branches constructed by each branch convolution layer can extract useful edge information. The edge information extracted at the low level contains many details, and the edge information extracted at the high level contains more global semantic information, which is close to the pattern contour of the luxury print image. The low-level edge information and the high-level global semantic information are combined to obtain the required contour information and details; in the present invention, the edge detection maps of the first layer and the fourth layer of the branch convolution layer are weightedly fused to obtain the final target edge detection map; the edge detection map is obtained by inputting the branch feature maps generated by each edge detection branch into the bilinear upsampling layer, and the bilinear upsampling layer upsamples the branch feature maps to the same size as the luxury print image and performs weighted fusion according to the preset corresponding weights.

[0071] Furthermore, a residual network (ResNet) extracts coarse-grained features from the target edge detection map. The residual network internally contains multiple residual blocks with residual connections. These residual blocks include convolutional layers and batch normalization layers. The ReLU activation function is set at the output of the residual network. This helps prevent gradient vanishing and gradient explosion during feature extraction. Coarse-grained features help identify edge information in the printed image, and the residual network structure ensures stable training. Macro-level features reduce the computational complexity of detailed information and improve feature processing efficiency. The weights in the residual network can be adjusted during training to improve the model's adaptability.

[0072] In some embodiments, the Transformer model further includes a multi-head self-attention module, a multi-layer perceptron module, and a layer norm module that are consecutively arranged.

[0073] In some embodiments, the image patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, and then inputs the micro features into the Transformer model for attention weight adjustment to obtain the first target micro features, which are input together with the macro features into the cross-attention feature fusion module to obtain the second target micro features, and the first target micro features and the second target micro features are weighted summed and fused with the classification vector, and then input into the multimodal fusion network for global feature capture to obtain the target classification vector. The classification network classifies the target classification vector to obtain the authenticity prediction result of the sample, and the steps include S31 to S37:

[0074] Step S31: Divide the micro features into multiple fixed-size image patches, flatten each patch into a vector, concatenate the image patch vector and the classification vector initialized to all 0s into a feature matrix, input it into the multi-head self-attention module for feature processing and global feature capture.

[0075] Step S32: After performing nonlinear mapping on the feature matrix through the multi-layer perceptron module, the input layer norm module performs normalization processing to obtain the first target micro-feature after attention weight adjustment.

[0076] Step S33: Cross attention fusion module uses macro features F macro Calculate the query vector Q using the first target micro-feature F micro Calculate the key vector K and value vector V as follows:

[0077] Q=W Q ·F macro ;

[0078] K=W K ·F micro ;

[0079] V=W V ·F micro .

[0080] Step S34: Calculate the cross attention score based on the query vector and the key vector and normalize it through the softmax function to obtain the attention weight matrix, which is expressed as:

[0081]

[0082] Step S35: Weight the value vector by the attention weight to obtain the second target micro-feature F enhanced , the expression is:

[0083] F enhanced =A·V.

[0084] Step S36: Expand the second target micro-feature to the same size as the luxury print image, and introduce a learnable dynamic weight parameter α to the first target micro-feature F self and the second target microscopic feature F expanded After weighted summation and fusion of classification vectors, the vectors are input into the multimodal fusion network for global feature capture to obtain the target classification vector, which is expressed as:

[0085] F final =α·F self +(1-α)·F expanded .

[0086] Step S37: The classification network classifies the target classification vector to obtain the authenticity prediction result of the sample.

[0087] Specifically, the multimodal fusion network uses a Transformer model and a cross-attention feature fusion module. The micro-features of multiple image blocks are embedded in the image patch to generate multiple image patch vectors. The classification vector and the image patch vector are then concatenated into a two-dimensional feature matrix and input into the Transformer model. The self-attention mechanism is used to model global relationships, capture key information, and integrate all information into the classification vector. In the cross-attention fusion module, macro-features are used to query the micro-features output by the Transformer model after being optimized by adjusting the attention weights. The micro-features enhanced by the macro-features are obtained. The output of the cross-attention fusion module is then used as input back into the Transformer model. The multimodal fusion network is iterated multiple times. The classification primitives in the output of the last multimodal fusion network are used as the target classification vector and input into a fully connected layer to obtain the classification output. The fully connected layer is a classification head. Figure 2 A1, A2...A n Represents microscopic features, A o represents the first target microscopic feature, A t represents the second target micro-feature, B represents the macro-feature, and C represents the classification vector, which is initialized to a full 0 vector.

[0088] Furthermore, the multi-head self-attention module can simultaneously process features at different positions and capture the global context and the relationship between different features. Through the multi-head self-attention module, it can more accurately identify important areas in luxury print images to improve classification accuracy; the multi-layer perceptron module enhances the expressiveness of input features through nonlinear mapping, helping the Transformer model to interact with macro features and micro features; the layer norm module normalizes the input features to achieve normalization between different feature channels and ensure the stability of luxury goods authentication model training.

[0089] After the fully connected layer outputs the classification recognition results, in some embodiments, the method further includes: connecting a softmax function layer to the output end of the classification network to output the probability of each category in the luxury goods authenticity prediction result.

[0090] In step S103, in some embodiments, the loss function is expressed as:

[0091] Loss=-[ylog(p)+(1-y)·log(1-p)];

[0092] Where y represents the value of the label and p represents the probability that the luxury print image is authentic.

[0093] Specifically, the adaptive moment estimation optimizer (Adam optimizer) updates the parameters of the loss function and adjusts the loss function through regularization, improving the generalization ability of the luxury goods authentication model and preventing overfitting, thereby improving the training effect of the luxury goods authentication model. The initial luxury goods authentication model is trained and its parameters updated using the loss function. The loss function uses the cross-entropy loss function. The adaptive moment estimation optimizer uses the first-order and second-order moment estimates of the gradient to dynamically adjust the parameters, accelerating convergence and improving training results. Regularization of the loss function effectively prevents overfitting and ensures the generalization ability of the luxury goods authentication model.

[0094] On the other hand, the present invention also provides a method for identifying counterfeit luxury goods, which includes the following steps S201-S202:

[0095] Step S201: Acquire an image of a luxury goods print to be detected.

[0096] Step S202: inputting the luxury goods print image to be detected into the luxury goods counterfeit detection model trained by the luxury goods counterfeit detection model training method, and outputting the authenticity classification and identification result of the luxury goods print image to be detected.

[0097] On the other hand, the present invention also provides a luxury goods counterfeit detection model training device, including a processor, a memory and a computer program / instructions stored in the memory, the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0098] On the other hand, the present invention also provides a luxury goods authentication device, comprising a processor, a memory, and a computer program / instruction stored in the memory, wherein the processor is configured to execute the computer program / instruction, and when the computer program / instruction is executed, the device implements the steps of the above method.

[0099] The present invention will be described below in conjunction with a specific embodiment:

[0100] This invention provides a training method, method, and device for luxury goods authentication. The method employs a dual-path model consisting of a micro-feature extraction network and a macro-feature extraction network. The micro-feature extraction network extracts the microfeatures of luxury goods, primarily capturing microscopic details within the image; the macro-feature extraction network extracts macroscopic features from the image, acquiring information about the overall structure of the luxury goods. The features of these two paths are then fused through a multimodal fusion network. The self-attention mechanism enables the luxury goods authentication model to capture microscopic details, while the cross-attention mechanism enables the luxury goods authentication model to simultaneously focus on both microscopic details and macroscopic structure, thereby improving classification accuracy.

[0101] The micro-feature extraction network is responsible for extracting the micro-features of luxury goods. The macro-feature extraction network first extracts macroscopic information from the image and then feeds these macro-features into the multimodal fusion network. Finally, the multimodal fusion network fuses the micro- and macro-features to achieve a comprehensive judgment of the luxury goods' authenticity. The micro-feature extraction network encodes visual information using a convolutional neural network to capture the image's micro-features. The macro-feature extraction network first extracts edges using a globally nested edge detection layer to obtain macroscopic information from the image. It then uses a residual network to extract macro-features, providing a holistic structural perspective. The self-attention mechanism of the multimodal fusion network helps capture microscopic details, and the cross-attention mechanism of the multimodal fusion network fuses the macro- and micro-features. Finally, the classification network classifies the target classification vector.

[0102] 1. The micro-feature extraction network consists of a convolutional neural network layer and an overlapping patch embedding layer.

[0103] 1.1 The convolutional neural network layer is divided into three stages:

[0104] (1) The first stage: First, a convolutional layer with a stride of 2 is used to downsample the input features to obtain the first feature map, and then the features are gradually extracted through three simple structures containing 3 layers of convolution.

[0105] (2) The second stage: The first feature map is further downsampled using a maximum pooling layer with a stride of 2 and a convolution kernel of 3x 3 to obtain the second feature map, reducing the spatial resolution while retaining important feature information. The features are then further extracted through a moving inverted bottleneck convolution (MBConv) block with a squeeze excitation mechanism.

[0106] (3) The third stage: The feature extraction capability is further enhanced through the mobile inversion bottleneck convolution block of the squeeze incentive mechanism.

[0107] 1.2 Overlapping Patch Embedding Layer The second feature map of the input is divided into image patches with heavy regions by using a convolutional layer with a stride of 2, thereby extracting embedded features. This can reduce the resolution of the feature map while retaining more local information, providing a more efficient high-dimensional representation for subsequent feature extraction and processing.

[0108] 2. The macro feature extraction network includes a holistic nested edge detection layer and a residual network.

[0109] 2.1 Overall nested edge detection layer (HED) edge extraction: This part of the workflow includes input preprocessing, feature extraction, construction of edge detection branches, and the final fusion of edge detection results to obtain the edge detection map.

[0110] First, the input luxury print image is preprocessed. Next, the image is passed through a series of convolutional layers for feature extraction. These convolutional layers consist of multiple layers, gradually extracting rich image information from low-level features to high-level features.

[0111] After feature extraction, the model constructs multiple edge detection branches. Each edge detection branch generates a branch feature map of different resolutions through a branched convolutional layer. A bilinear upsampling layer then upsamples the branch feature map to the same size as the input luxury print image. The edge detection maps output by these branches are fused to generate the final target edge detection map. Specifically, the edge detection maps from layers 1 and 4 are weighted and fused to form the final target edge detection map.

[0112] The loss function uses a class-balanced cross-entropy loss. This introduces a class balancing factor to address class imbalance, allowing the model to better focus on the minority class during training. During model training, an adaptive moment estimation optimizer is used, along with regularization, to prevent overfitting.

[0113] 2.2 Residual Network Macro-Feature Extraction: This module uses the target edge detection map as input and performs coarse-grained feature extraction through the residual network. To improve the model's adaptability, the residual network weights remain trainable during the feature extraction phase. This allows the model to fine-tune the pre-trained weights during task-specific training, thereby better adapting to data characteristics and improving classification performance.

[0114] 3. Multimodal fusion of micro-features and macro-features.

[0115] The micro-feature extraction network encodes visual information through convolutional neural network layers to generate micro-features. Simultaneously, the macro-path uses a residual network to process the target edge detection map obtained from the global nested edge detection layer to extract macro-features. After being processed by the image patch embedding layer, the micro-visual features are fed into the Transformer model along with the classification vector for further processing.

[0116] In the Transformer model, the Multi-Head Self-Attention (MSA) module is first used. This module processes features at different locations in parallel, capturing global context and the connections between different features. This allows the luxury goods authentication model to better identify important areas in an image, combining information from both micro and macro features to improve classification accuracy.

[0117] Next, the Multilayer Perceptron (MLP) module performs nonlinear mapping of features, further enhancing their expressiveness. The MLP module combines the rich features extracted from the attention mechanism in a more complex manner, helping the model understand the interaction between micro- and macro-level features.

[0118] Finally, the Layer Normalization (LN) module is used to normalize the input features to ensure the stability of model training. It can normalize between different feature channels, helping to alleviate training instability caused by different data distributions.

[0119] The output of the Transformer model is then fed into the cross-attention feature fusion module. Q is calculated using macro features, and K and V are calculated using micro features adjusted for attention weights. This yields micro features enhanced by the macro features. These micro features are then upsampled to the size of the feature map output by the Transformer model and weightedly added to the output of the Transformer model to yield the final output of the multimodal fusion network. This output is then fed back into the multimodal fusion network, and the final target classification vector is obtained through multiple iterations of the multimodal fusion network.

[0120] This fusion process uses macro features to query micro features, effectively enhancing the representation of micro features with macro features. This allows the model to not only capture the object's microscopic details but also leverage macro information for more comprehensive classification, improving its robustness and accuracy in luxury goods authentication. 4. Luxury goods authentication model training method.

[0121] The luxury print image is input into the global nested edge detection layer to generate the corresponding target edge detection map. The target edge detection map is then input into the residual network extraction. The luxury print image is then input into the micro-feature extraction network to obtain micro-features. The macro-features and micro-features are input into the multimodal fusion network. The micro-visual features are processed by the image patch embedding layer and input into the Transformer model together with the classification vector to obtain micro-features optimized by the attention mechanism. The macro-features are used to query the micro-features, and the scaled click attention is calculated to obtain micro-features enhanced by the macro-features. The micro-features are upsampled to the feature map size output by the Transformer model and weightedly added with the output of the Transformer model to obtain the final output of the multimodal fusion network. This output is re-input into the multimodal fusion network. After multiple iterations of the multimodal fusion network, the final target classification vector is obtained. The classification network classifies the target classification network to produce the classification output.

[0122] First, the original image is input into the overall nested edge detection layer to generate the corresponding target edge detection map.

[0123] Next, the target edge detection image and the luxury print image are input into the initial luxury goods authentication model. The luxury print image is processed through a micro-feature extraction network to extract micro-features. The target edge detection image, on the other hand, is processed through a macro-feature extraction network to extract macro-features. The macro-features and micro-features are then input into a multimodal fusion network. The micro-visual features are processed by the image patch embedding layer and, along with the classification vector, are input into the Transformer model to obtain micro-features optimized by the attention mechanism. The macro-features are then used to query the micro-features, and the scaled click attention is calculated to obtain micro-features enhanced by the macro-features. These micro-features are then upsampled to the feature map size output by the Transformer model and weightedly added to the output of the Transformer model to obtain the final output of the multimodal fusion network. This output is then re-input into the multimodal fusion network. After multiple iterations of the multimodal fusion network, the final target classification vector is obtained. The classification network then classifies the target classification network to produce the classification output.

[0124] The present invention identifies luxury print images after processing them in three ways: blocking them with a grid mask, replacing the ink background of the luxury print images with background replacement, and sharpening the counterfeits. Table 1 is a comparison table of the recognition accuracy of samples processed by different methods.

[0125] Table 1 Comparison of recognition accuracy of samples processed by different methods

[0126] Use only micro information Combining macro and micro information Performance comparison No sample processing 99.73％ 99.88％ Increase by 0.15% Grid Mask 96.02％ 96.46％ Increased by 0.44% Background Replacement 90.95％ 93.67％ 2.72% increase Introducing difficult samples 98.23％ 98.67％ Increased by 0.44%

[0127] In summary, the present invention provides a luxury goods counterfeit identification model training method, luxury goods counterfeit identification method and device, which trains an initial luxury goods counterfeit identification model including a micro feature extraction network, a macro feature extraction network, a multimodal fusion network and a classification network through a training sample set; the micro feature extraction network includes a convolutional neural network layer and an overlapping patch embedding layer; the macro feature extraction network includes a global nested edge detection layer and a residual network; the multimodal fusion network includes a Transformer model and a cross-attention feature fusion module; the multi-head self-attention module guides the model to focus on key areas, so that the classification vector captures key features and obtains micro features after attention weight adjustment; the multi-layer perceptron module enhances feature expression capabilities; the layer norm module ensures the stability of model training; the cross-attention module uses macro features to query micro features, enhances the model's context perception capabilities, and realizes multi-scale feature fusion; the loss function uses a cross-entropy loss function, and the parameters are updated with the goal of minimizing the loss function to obtain a luxury goods counterfeit identification model, and the luxury goods authenticity classification results are obtained through the luxury goods counterfeit identification model.

[0128] Corresponding to the above method, the present invention also provides a luxury goods counterfeit detection model training device, including a processor, a memory and a computer program / instructions stored in the memory, the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0129] An embodiment of the present invention further provides a luxury goods authentication device, comprising a processor, a memory, and a computer program / instruction stored in the memory. The processor is configured to execute the computer program / instruction. When the computer program / instruction is executed, the device implements the steps of the above method.

[0130] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0131] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0132] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0133] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A luxury goods counterfeit detection model training method, characterized by: The method comprises the following steps: Obtaining a training sample set, wherein the training sample set includes multiple samples, each sample including an image of a luxury print; the multiple samples in the training sample set have labels including true values ​​for authenticity classification; Obtain an initial luxury goods authentication model, which includes a micro-feature extraction network, a macro-feature extraction network, a multimodal fusion network and a classification network; the macro-feature extraction network includes a global nested edge detection layer and a residual network; the multimodal fusion network includes a patch embedding layer, a Transformer model and a cross-attention feature fusion module; the micro-feature extraction network takes samples in the training sample set as input and outputs micro-features; the macro-feature extraction network takes samples in the training sample set as input, obtains a target edge detection map through the global nested edge detection layer, and then passes it through the residual network. The patch embedding layer in the multimodal fusion network extracts and outputs macro features; the patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, and then inputs the micro features into the Transformer model for attention weight adjustment to obtain a first target micro feature, which is input into the cross attention feature fusion module together with the macro feature to obtain a second target micro feature, and the first target micro feature and the second target micro feature are weighted and summed, and the classification vector is fused and then input into the multimodal fusion network for global feature capture to obtain a target classification vector; the classification network classifies the target classification vector to obtain a true or false prediction result of the sample; A loss function is constructed based on the deviation between the authenticity prediction result corresponding to each sample and the true authenticity classification value. The parameters of the initial luxury goods authentication model are iteratively updated with the goal of minimizing the loss function to obtain a luxury goods authentication model.

2. The luxury goods counterfeit detection model training method according to claim 1, characterized in that: The micro-feature extraction network takes samples in the training sample set as input, and the step of outputting micro-features includes: Inputting samples in the training sample set into a first convolutional layer for downsampling to obtain a first feature map, and gradually extracting image features from the first feature map through three consecutive convolutional blocks; the convolutional block includes three consecutive convolutional layers; The first feature map is downsampled using a maximum pooling layer to obtain a second feature map, and the second feature map is subjected to depthwise separable convolution through two shifted reverse bottleneck convolutions with a squeeze excitation mechanism to further enhance the extracted image features; The second feature map is divided into multiple fixed-size image blocks with overlapping areas through an overlapping patch embedding layer, and the feature vectors of the multiple image blocks are obtained through an embedding operation and encoded to obtain the microscopic features; the microscopic features include the density and distribution of ink dots.

3. The luxury goods counterfeit detection model training method according to claim 1, characterized in that: The macro feature extraction network uses samples in the training sample set as input, obtains a target edge detection image through the overall nested edge detection layer, and then extracts and outputs macro features through the residual network, including the following steps: Extracting macroscopic feature information from the preprocessed samples layer by layer through multiple convolutional layers; Constructing a plurality of edge detection branches including branch convolution layers, wherein each of the edge detection branches generates a plurality of branch feature maps with different resolutions through the corresponding branch convolution layer; Inputting the plurality of branch feature maps into a bilinear upsampling layer for upsampling to obtain a plurality of edge detection maps of the same size as the luxury print image, and performing weighted fusion on the plurality of edge detection maps to generate a target edge detection map; The residual network is used to perform coarse-grained feature extraction on the target edge detection image and obtain the macro features; the macro features include the pattern and material of the luxury goods.

4. The luxury goods counterfeit detection model training method according to claim 1, characterized in that: The Transformer model also includes a multi-head self-attention module, a multi-layer perceptron module and a layer norm module that are arranged in succession.

5. The luxury goods counterfeit detection model training method according to claim 4, characterized in that: The patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, and then inputs the micro features into the Transformer model for attention weight adjustment to obtain a first target micro feature, and inputs the micro features together with the macro features into the cross attention feature fusion module to obtain a second target micro feature, performs weighted summation on the first target micro feature and the second target micro feature, fuses the classification vector, and then inputs the multimodal fusion network for global feature capture to obtain a target classification vector, and the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample, including the following steps: Divide the microscopic features into multiple fixed-size image patches, flatten each patch into a vector, concatenate the image patch vector and the classification vector initialized to all 0s into a feature matrix, and input the matrix into the multi-head self-attention module for feature processing and global feature capture; The feature matrix is ​​nonlinearly mapped by the multilayer perceptron module and then input into the layer norm module for normalization to obtain the first target micro-feature after attention weight adjustment; The cross-attention feature fusion module uses the macro features F macro Calculate the query vector Q using the first target micro-feature F micro Calculate the key vector K and value vector V as follows: Q=W Q ·F macro ; K=W K ·F micro ; V=W V ·F micro ; The cross attention score is calculated based on the query vector and the key vector and normalized by the softmax function to obtain the attention weight matrix, which is expressed as: The second target micro-feature F is obtained by weighting the value vector with the attention weight enhanced , the expression is: F enhanced =A·V; The second target micro-feature is expanded to the same size as the luxury print image, and a learnable dynamic weight parameter α is introduced to the first target micro-feature F self and the second target microfeature F expanded After weighted summation and fusion of the classification vectors, the vectors are input into the multimodal fusion network for global feature capture to obtain the target classification vector, which is expressed as: F final =α·F self +(1-α)·F expanded ; The classification network classifies the target classification vector to obtain a true or false prediction result of the sample.

6. The luxury goods counterfeit detection model training method according to claim 5, characterized in that: The method further comprises: A softmax function layer is connected to the output end of the classification network to output the probability of each classification in the luxury goods authenticity prediction result.

7. The luxury goods counterfeit detection model training method according to claim 1, characterized in that: The expression of the loss function is: Loss=-[ylog(p)+(1-y)·log(1-p)]; Wherein, y represents the value of the label, and p represents the probability that the luxury print image is authentic.

8. A method for identifying counterfeit luxury goods, characterized in that: The method comprises the following steps: Obtain the luxury goods print image to be detected; The luxury goods print image to be detected is input into the luxury goods counterfeit identification model trained by the luxury goods counterfeit identification model training method according to any one of claims 1 to 7, and the authenticity classification and identification result of the luxury goods print image to be detected is output.

9. A luxury goods counterfeit detection model training device, comprising a processor, a memory, and a computer program / instruction stored in the memory, characterized in that: The processor is configured to execute the computer program / instructions. When the computer program / instructions are executed, the device implements the steps of the method according to any one of claims 1 to 7.

10. A luxury goods authentication device comprising a processor, a memory, and a computer program / instruction stored in the memory, characterized in that: The processor is configured to execute the computer program / instructions. When the computer program / instructions are executed, the device implements the steps of the method according to claim 8.

Images