Luxury authentic identification model training method, luxury authentic identification method and luxury authentic identification device

Abstract

The invention provides a luxury authentic identification model training method, a luxury authentic identification method and a luxury authentic identification device. A training sample set trains an initial luxury authentic identification model comprising a microscopic feature extraction network, a macroscopic feature extraction network, a multi-modal fusion network and a classification network; the microscopic feature extraction network comprises a convolutional neural network layer and an overlapped patch embedding layer; the macroscopic feature extraction network comprises an overall nested edge detection layer and a residual network; the image patch embedding layer in the multi-modal fusion network divides the microscopic features, carries out vector conversion, then inputs the microscopic features into the Transform model to carry out attention weight adjustment so as to obtain first target microscopic features, and inputs the first target microscopic features and the macroscopic features together into a cross attention feature fusion module so as to obtain second target microscopic features; carrying out weighted summation on the two target microscopic features, fusing classification vectors, inputting into a multi-modal fusion network, carrying out global feature capture to obtain target classification vectors, and inputting into the classification network to obtain an authenticity prediction result of the sample.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a luxury goods counterfeit identification model training method, a luxury goods counterfeit identification method and a luxury goods counterfeit identification device. Background Art

[0002] In the field of luxury goods authentication, existing methods mainly focus on capturing the microscopic features of objects, such as the density and distribution of ink dots, and enhance feature learning by identifying these key details or applying attention mechanisms. Although these methods perform well in identifying subtle differences, they are often highly dependent on microscopic features and tend to ignore macroscopic features and overall context clues; in addition, existing methods are highly complex and require high precision in the extraction of microscopic features. In practical applications, if the positioning is inaccurate or the luxury goods are worn due to environmental and transportation problems, the microscopic features will become less obvious, making it difficult to effectively distinguish between genuine and counterfeit goods. Summary of the invention

[0003] In view of this, the embodiments of the present invention provide a luxury goods counterfeit identification model training method, luxury goods counterfeit identification method and device to eliminate or improve one or more defects existing in the prior art, and solve the problem that the prior art cannot effectively identify the authenticity of luxury goods.

[0004] One aspect of the present invention provides a luxury goods counterfeit detection model training method, the method comprising the following steps:

[0005] Obtain a training sample set, wherein the training sample set includes multiple samples, each sample includes a luxury print image; the multiple samples in the training sample set have labels including true values ​​for authenticity classification;

[0006] An initial luxury goods counterfeit detection model is obtained, wherein the initial luxury goods counterfeit detection model includes a micro-feature extraction network, a macro-feature extraction network, a multimodal fusion network and a classification network; the macro-feature extraction network includes an overall nested edge detection layer and a residual network; the multimodal fusion network includes an image patch embedding layer, a Transformer model and a cross-attention feature fusion module; the micro-feature network extraction network takes samples in the training sample set as input and outputs micro-features; the macro-feature network extraction network takes samples in the training sample set as input, obtains a target edge detection map through the overall nested edge detection layer, and then extracts and outputs macro-features through the residual network; the image patch embedding layer in the multimodal fusion network divides the micro-features into The features are converted into vectors and then input into the Transformer model for attention weight adjustment to obtain the first target micro-feature, which is input into the cross-attention feature fusion module together with the macro-feature to obtain the second target micro-feature, and the first target micro-feature and the second target micro-feature are weighted summed and fused into the classification vector, and then input into the multimodal fusion network for global feature capture to obtain the target classification vector; the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample; a loss function is constructed according to the deviation between the authenticity prediction result corresponding to each sample and the true value of the authenticity classification, and the parameters of the initial luxury goods authentication model are iteratively updated as the goal of minimizing the loss function to obtain the luxury goods authentication model.

[0007] In some embodiments, the micro-feature network extraction network takes samples in the training sample set as input, and the step of outputting micro-features includes:

[0008] Inputting samples in the training sample set into the first convolution layer for downsampling to obtain a first feature map, and the first feature map is gradually extracted from image features through three consecutive convolution blocks; the convolution block includes three consecutive convolution layers;

[0009] The first feature map is downsampled using a maximum pooling layer to obtain a second feature map, and the second feature map is subjected to depthwise separable convolution through two moving inverted bottleneck convolutions with a squeeze excitation mechanism to further enhance the extracted image features;

[0010] The second feature map is divided into a plurality of fixed-size image blocks with overlapping areas through an overlapping patch embedding layer, and feature vectors of the plurality of image blocks are obtained through an embedding operation and encoded to obtain the microscopic features; the microscopic features include the density and distribution of ink dots.

[0011] In some embodiments, the macro-feature network extraction network takes samples in the training sample set as input, obtains a target edge detection map through the overall nested edge detection layer, and then extracts and outputs macro-features through the residual network, including:

[0012] Extracting macroscopic feature information of the preprocessed samples layer by layer through multiple convolutional layers;

[0013] Constructing a plurality of edge detection branches including branch convolution layers, wherein each of the edge detection branches generates a plurality of branch feature maps with different resolutions through the corresponding branch convolution layer;

[0014] Inputting the plurality of branch feature maps into a bilinear upsampling layer for upsampling to obtain a plurality of edge detection maps of the same size as the luxury print image, and performing weighted fusion on the plurality of edge detection maps to generate a target edge detection map;

[0015] The target edge detection image is subjected to coarse-grained feature extraction through the residual network to obtain the macro features; the macro features include the patterns and materials of the luxury goods.

[0016] In some embodiments, the Transformer model also includes a multi-head self-attention module, a multi-layer perceptron module and a layer norm module that are arranged in succession.

[0017] In some embodiments, the image patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, then inputs the micro features into the Transformer model for attention weight adjustment to obtain a first target micro feature, and inputs the micro features together with the macro features into the cross attention feature fusion module to obtain a second target micro feature, performs weighted summation of the first target micro feature and the second target micro feature, fuses the classification vector, and then inputs the multimodal fusion network for global feature capture to obtain a target classification vector, and the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample, including:

[0018] Divide the microscopic features into a plurality of image patches of fixed size, flatten each patch into a vector, concatenate the image patch vector and the classification vector initialized to all 0s into a feature matrix, and input the matrix into the multi-head self-attention module for feature processing and global feature capture;

[0019] The feature matrix is ​​nonlinearly mapped by the multilayer perceptron module and then input into the layer norm module for normalization processing to obtain the first target micro-feature after attention weight adjustment;

[0020] The cross-attention fusion module uses the macro features F macroCalculate the query vector Q using the first target micro-feature F micro Calculate the key vector K and value vector V, the expression is:

[0021] Q=W Q ·F macro ;

[0022] K=W K ·F micro ;

[0023] V=W V ·F micro ;

[0024] The cross attention score is calculated based on the query vector and the key vector and normalized by the softmax function to obtain the attention weight matrix, which is expressed as:

[0025]

[0026] The second target micro-feature F is obtained by weighting the value vector by the attention weight enhanced , the expression is:

[0027] F enhanced =A·V;

[0028] The second target micro-feature is expanded to the same size as the luxury print image, and a learnable dynamic weight parameter α is introduced to the first target micro-feature F self and the second target microscopic feature F expanded After weighted summation and fusion of the classification vectors, the multimodal fusion network is input to perform global feature capture to obtain the target classification vector, which is expressed as:

[0029] F final =α·F self +(1-α)·F expanded ;

[0030] The classification network classifies the target classification vector to obtain a true or false prediction result of the sample.

[0031] In some embodiments, the method further comprises:

[0032] A softmax function layer is connected to the output end of the classification network to output the probability of each classification in the luxury goods authenticity prediction result.

[0033] In some embodiments, the loss function is expressed as:

[0034] Loss=-[ylog(p)+(1-y)·log(1-p)];

[0035] Wherein, y represents the value of the label, and p represents the probability that the luxury print image is authentic.

[0036] On the other hand, the present invention also provides a method for identifying counterfeit luxury goods, the method comprising the following steps:

[0037] Obtain the luxury goods print image to be detected;

[0038] The luxury goods print image to be detected is input into the luxury goods counterfeit detection model trained by the above-mentioned luxury goods counterfeit detection model training method, and the authenticity classification and identification result of the luxury goods print image to be detected is output.

[0039] On the other hand, the present invention also provides a luxury goods counterfeit detection model training device, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0040] On the other hand, the present invention also provides a luxury goods counterfeit detection device, comprising a processor, a memory and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0041] The beneficial effects of the present invention are at least:

[0042] In the luxury goods authentication model training method, luxury goods authentication method and device of the present invention, the initial luxury goods authentication model includes a micro feature extraction network, a macro feature extraction network, a multimodal fusion network and a classification network; the micro feature extraction network includes a convolutional neural network layer and an overlapping patch embedding layer; the convolutional neural network first obtains a first feature map by downsampling and then inputs it into a maximum pooling layer for downsampling to obtain a second feature map, thereby reducing the spatial resolution while retaining feature information; the overlapping patch embedding layer divides the second feature map into multiple image blocks, thereby reducing the resolution of the feature map while retaining local information, and providing more effective input for subsequent feature extraction and processing; the macro feature extraction network includes an overall nested edge detection layer and a residual network; the overall nested edge detection layer performs feature extraction layer by layer from low-level features to high-level features to obtain information of luxury goods print images, and The branch feature maps of multiple edge detection branches are upsampled and weightedly fused to obtain a target edge detection map, so that the detection obtains the edge contour while retaining the details; the multimodal fusion network includes an image patch embedding layer, a Transformer model and a cross-attention feature fusion module, the multi-head self-attention module guides the model to focus on key areas, so that the classification vector captures key features and obtains micro features after weight adjustment by the attention mechanism, the multi-layer perceptron module enhances the feature expression capability, the layer norm module ensures the stability of model training, the cross-attention module uses macro features to query micro features, enhances the model's context perception capability, and realizes multi-scale feature fusion; the loss function uses a cross-entropy loss function, and updates parameters with the goal of minimizing the loss function to obtain a luxury goods authentication model, and the authenticity classification results of luxury goods are obtained through the luxury goods authentication model.

[0043] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0044] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0046] Figure 1 Schematic diagram of the flow of a luxury goods counterfeit identification model training method according to an embodiment of the present invention.

[0047] Figure 2 Schematic diagram of the structure of a luxury goods counterfeit identification model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0049] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0050] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0051] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection with an intermediate.

[0052] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0053] In the prior art, luxury goods authentication mainly focuses on capturing the microscopic features of objects, such as the density and distribution of ink dots, and enhances feature learning by identifying these key details or applying an attention mechanism. This method is highly dependent on microscopic features and tends to ignore macroscopic features and overall context clues. These methods are highly complex and require high precision in the extraction of microscopic features. When luxury goods are worn due to inaccurate positioning or environmental and transportation problems, the microscopic features will become less obvious, making it difficult to effectively distinguish between genuine and counterfeit goods. The present invention proposes a luxury goods authentication model training method, luxury goods authentication method and device, and trains an initial luxury goods authentication model including a microscopic feature extraction network, a macroscopic feature extraction network, a multimodal fusion network and a classification network through a training sample set. The macroscopic feature extraction network includes an overall nested edge detection layer and a residual network. The multimodal fusion network includes an image patch embedding layer, a Transformer model and a cross-attention feature fusion module. The micro-feature network extraction network takes the samples in the training sample set as input and outputs micro-features; the macro-feature network extraction network takes the samples in the training sample set as input, obtains the target edge detection map through the overall nested edge detection layer, and then extracts and outputs macro-features through the residual network; the image patch embedding layer in the multimodal fusion network divides the micro-features and performs vector conversion, and then inputs them into the Transformer model for attention weight adjustment to obtain the first target micro-features, and inputs them together with the macro-features into the cross-attention feature fusion module to obtain the second target micro-features, and performs weighted summation and fusion of the classification vector on the first target micro-features and then inputs them into the multimodal fusion network for global feature capture to obtain the target classification vector; the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample; the luxury print image to be detected is input into the luxury goods authentication model, and the authenticity classification recognition result is output.

[0054] Figure 1 Schematic diagram of the flow of a luxury goods counterfeit identification model training method according to an embodiment of the present invention. Figure 2 Schematic diagram of the structure of the luxury goods counterfeit detection model in one embodiment of the present invention. Specifically, the present application provides a luxury goods counterfeit detection model training method, luxury goods counterfeit detection method and device, the method comprising the following steps S101 to S103:

[0055] Step S101: obtaining a training sample set, where the training sample set includes multiple samples, each sample includes a luxury print image; the multiple samples in the training sample set have labels including true values ​​of authenticity classification.

[0056] Step S102: obtaining an initial luxury goods authentication model, the initial luxury goods authentication model includes a micro-feature extraction network, a macro-feature extraction network, a multimodal fusion network and a classification network; the macro-feature extraction network includes an overall nested edge detection layer and a residual network; the multimodal fusion network includes an image patch embedding layer, a Transformer model and a cross-attention feature fusion module; the micro-feature network extraction network takes samples in the training sample set as input and outputs micro-features; the macro-feature network extraction network takes samples in the training sample set as input, obtains a target edge detection map through the overall nested edge detection layer, and then extracts and outputs macro-features through the residual network; the image patch embedding layer in the multimodal fusion network divides the micro-features and performs vector conversion, and then inputs the micro-features into the Transformer model for attention weight adjustment to obtain the first target micro-features, and inputs the micro-features and the macro-features into the cross-attention feature fusion module to obtain the second target micro-features, and performs weighted summation on the first target micro-features and the second target micro-features and fuses the classification vector, and then inputs the multimodal fusion network for global feature capture to obtain the target classification vector; the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample.

[0057] Step S103: construct a loss function according to the deviation between the authenticity prediction result corresponding to each sample and the true value of the authenticity classification, and iteratively update the parameters of the initial luxury goods authentication model with the goal of minimizing the loss function to obtain the luxury goods authentication model.

[0058] In step S101, the samples in the training sample set are partial printed images of luxury goods with ink printed patterns, and the luxury goods include but are not limited to clothing, bags, decorations and other products with ink printed patterns; the samples are labeled with the true value of the authenticity classification.

[0059] In step S102, the initial luxury goods counterfeit identification model is a micro and macro dual-path model. The micro feature extraction network of the micro path inputs the luxury goods print image, which is sampled by a convolutional neural network and the feature map obtained by sampling is divided into multiple image blocks with overlapping areas through an overlapping patch embedding layer. The feature vectors extracted from the multiple image blocks are encoded to obtain micro features.

[0060] In some embodiments, the micro-feature network extraction network takes samples in the training sample set as input, and the steps of outputting micro-features include S11 to S13:

[0061] Step S11: Input the samples in the training sample set into the first convolution layer for downsampling to obtain a first feature map, and the first feature map is gradually extracted from the image features through three consecutive convolution blocks; the convolution block includes three consecutive convolution layers.

[0062] Step S12: Use the maximum pooling layer to downsample the first feature map to obtain a second feature map, perform depth-separable convolution on the second feature map through two moving inverted bottleneck convolutions with a squeeze excitation mechanism, and further enhance the extracted image features.

[0063] Step S13: Divide the second feature map into multiple fixed-size image blocks with overlapping areas through an overlapping patch embedding layer, obtain feature vectors of the multiple image blocks through an embedding operation and encode them to obtain microscopic features; the microscopic features include the density and distribution of ink dots.

[0064] Specifically, the convolutional neural network includes three stages: the first convolutional layer, the maximum pooling layer, and the moving inverted bottleneck convolution layer. The first convolutional layer is a convolutional layer with a stride of 2. Downsampling can reduce the data spatial dimension of the luxury print image and obtain a low-resolution first feature map. The first feature map passes through the first three-layer convolution block and then enters the second three-layer convolution block. Finally, it passes through the third three-layer convolution block, reducing the complexity of computational processing while retaining important information in the image; the first feature map is downsampled again using a maximum pooling layer with a stride of 2 and a convolution kernel of 3x 3 to obtain a second feature map, further reducing the spatial resolution and retaining important information; the size of the second feature map is smaller than that of the first feature map, and the size of the first feature map is smaller than that of the original luxury print image. Furthermore, when performing depth-separable convolution, the moving inverted bottleneck convolution with a squeeze excitation mechanism combines the outputs after convolving each input channel separately, reducing the amount of computational processing for data processing, and the number of channels is expanded or reduced before and after the depth-separable convolution.

[0065] In some embodiments, the macro-feature network extraction network takes samples in the training sample set as input, obtains a target edge detection map after the luxury print image passes through the overall nested edge detection layer, and extracts and outputs macro-features through the residual network after obtaining the target edge detection map through the overall nested edge detection layer, including S21-S24:

[0066] Step S21: extracting macroscopic feature information of the preprocessed samples layer by layer through multiple convolutional layers.

[0067] Step S22: construct multiple edge detection branches including branch convolution layers, and each edge detection branch generates multiple branch feature maps with different resolutions through the corresponding branch convolution layer.

[0068] Step S23: Input the multiple branch feature maps into the bilinear upsampling layer for upsampling to obtain multiple edge detection maps with the same size as the luxury print image, and perform weighted fusion on the multiple edge detection maps to generate a target edge detection map.

[0069] Step S24: Perform coarse-grained feature extraction on the target edge detection image through a residual network and obtain macro features of the luxury print image; the macro features include the patterns and materials of the luxury goods.

[0070] Specifically, after the luxury print image is input into the macro feature extraction network, the target edge detection map of the luxury print image is obtained through the overall nested edge detection layer (HED). In the edge detection process, the constructed edge detection branches including the branch convolution layer can extract useful edge information. The edge information extracted at the low level contains many details, and the edge information extracted at the high level contains more global semantic information, which is close to the pattern contour of the luxury print image. The low-level edge information and the high-level global semantic information are combined to obtain the required contour information and details; in the present invention, the edge detection maps of the first layer and the fourth layer of the branch convolution layer are weightedly fused to obtain the final target edge detection map; the edge detection map is obtained by inputting the branch feature map generated by each edge detection branch into the bilinear upsampling layer, and the bilinear upsampling layer upsamples the branch feature map to the same size as the luxury print image and performs weighted fusion according to the preset corresponding weights.

[0071] Furthermore, the residual network (ResNet) extracts coarse-grained features from the target edge detection map. The residual network has multiple residual blocks with residual connections. The residual blocks contain convolutional layers and batch normalization layers. The output of the residual network is set with a ReLU activation function, which is helpful in preventing gradient vanishing and gradient explosion during feature extraction. Coarse-grained features help identify edge information of printed images, and the structure of the residual network keeps the training process stable. Macro features reduce the amount of calculation of detailed information and improve the efficiency of feature processing. The weights in the residual network can be adjusted during training to improve the model's adaptability.

[0072] In some embodiments, the Transformer model also includes a multi-head self-attention module, a multi-layer perceptron module, and a layer norm module that are arranged in succession.

[0073] In some embodiments, the image patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, and then inputs the micro features into the Transformer model for attention weight adjustment to obtain the first target micro features, and inputs the micro features into the cross attention feature fusion module together with the macro features to obtain the second target micro features, and the first target micro features and the second target micro features are weighted summed and fused with the classification vector, and then input into the multimodal fusion network for global feature capture to obtain the target classification vector, and the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample, and the steps include S31 to S37:

[0074] Step S31: Divide the micro features into multiple fixed-size image patches, flatten each patch into a vector, concatenate the image patch vector and the classification vector initialized to all 0s into a feature matrix, input it into the multi-head self-attention module for feature processing and global feature capture.

[0075] Step S32: After performing nonlinear mapping on the feature matrix through the multi-layer perceptron module, the input layer norm module performs normalization processing to obtain the first target micro-feature after the attention weight is adjusted.

[0076] Step S33: Cross-attention fusion module uses macro features F macro Calculate the query vector Q using the first target micro-feature F micro Calculate the key vector K and value vector V, the expression is:

[0077] Q=W Q ·F macro ;

[0078] K=W K ·F micro ;

[0079] V=W V ·F micro .

[0080] Step S34: Calculate the cross attention score based on the query vector and the key vector and obtain the attention weight matrix by normalizing it through the softmax function. The expression is:

[0081]

[0082] Step S35: Weight the value vector by the attention weight to obtain the second target micro-feature F enhanced , the expression is:

[0083] F enhanced =A·V.

[0084] Step S36: Expand the second target micro-feature to the same size as the luxury print image, and introduce a learnable dynamic weight parameter α to the first target micro-feature F self and the second target microscopic feature F expanded After weighted summation and fusion of classification vectors, the vectors are input into the multimodal fusion network for global feature capture to obtain the target classification vector, which is expressed as:

[0085] F final =α·F self +(1-α)·F expanded .

[0086] Step S37: The classification network classifies the target classification vector to obtain the authenticity prediction result of the sample.

[0087] Specifically, the multimodal fusion network uses the Transformer model and the cross-attention feature fusion module. The micro features of multiple image blocks are embedded in the image patch to generate multiple image patch vectors. The classification vector and the image patch vector are spliced ​​into a two-dimensional feature matrix and then input into the Transformer model. The global relationship is modeled by the self-attention mechanism, key information is captured, and all information is integrated into the classification vector. In the cross-attention fusion module, the macro features are used to query the micro features output by the Transformer model that are optimized by adjusting the attention weights, and the micro features enhanced by the macro features are obtained. The output of the cross-attention fusion module is used as input again and input into the Transformer model. The multimodal fusion network is iterated multiple times, and the classification primitives in the output of the last multimodal fusion network are used as the target classification vector. After input into a fully connected layer, the classification output is obtained. The fully connected layer is a classification head. Figure 2 A1, A2, ... A n Represents microscopic features, A o represents the first target microscopic feature, A t represents the second target micro-feature, B represents the macro-feature, C represents the classification vector, and the classification vector is initialized to a full 0 vector.

[0088] Furthermore, the multi-head self-attention module can simultaneously process features at different positions and capture the global context and the association between different features. The multi-head self-attention module can more accurately identify important areas in luxury print images to improve classification accuracy; the multi-layer perceptron module enhances the expressiveness of input features through nonlinear mapping, helping the Transformer model to interact with macro features and micro features; the layer norm module normalizes the input features to normalize different feature channels and ensure the stability of luxury goods counterfeit detection model training.

[0089] After the fully connected layer outputs the classification recognition results, in some embodiments, the method further includes: connecting a softmax function layer at the output end of the classification network to output the probability of each classification in the luxury goods authenticity prediction result.

[0090] In step S103, in some embodiments, the loss function is expressed as:

[0091] Loss=-[ylog(p)+(1-y)·log(1-p)];

[0092] Among them, y represents the value of the label, and p represents the probability that the luxury print image is authentic.

[0093] Specifically, the parameters of the loss function are updated through the adaptive moment estimation optimizer (Adam optimizer) and the loss function is adjusted through regularization to improve the generalization ability of the luxury goods counterfeit detection model and prevent overfitting, so as to improve the training effect of the luxury goods counterfeit detection model. The initial luxury goods counterfeit detection model is trained and the parameters are updated through the loss function. The loss function uses the cross entropy loss function. The adaptive moment estimation optimizer uses the first-order moment estimation and second-order moment estimation of the gradient to dynamically adjust the parameters, accelerate convergence and improve the training results; regularization adjustment of the loss function can effectively prevent overfitting and ensure the generalization ability of the luxury goods counterfeit detection model.

[0094] On the other hand, the present invention also provides a method for identifying counterfeit luxury goods, the method comprising the following steps S201-S202:

[0095] Step S201: Acquire a luxury goods print image to be detected.

[0096] Step S202: inputting the luxury goods print image to be detected into the luxury goods counterfeit detection model trained by the luxury goods counterfeit detection model training method, and outputting the authenticity classification and identification result of the luxury goods print image to be detected.

[0097] On the other hand, the present invention also provides a luxury goods counterfeit detection model training device, including a processor, a memory, and a computer program / instructions stored in the memory, the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0098] On the other hand, the present invention also provides a luxury goods counterfeit detection device, comprising a processor, a memory and a computer program / instruction stored in the memory, wherein the processor is used to execute the computer program / instruction, and when the computer program / instruction is executed, the device implements the steps of the above method.

[0099] The present invention is described below in conjunction with a specific embodiment:

[0100] The present invention provides a training method for a luxury goods counterfeit detection model, a luxury goods counterfeit detection method and device, and designs a dual-path model of a micro-feature extraction network and a macro-feature extraction network. The micro-feature extraction network is responsible for extracting the micro-features of luxury goods, mainly capturing the micro-details of the image; the macro-feature extraction network extracts the macro-features of the image and obtains the overall structural information of the luxury goods. The features of the two paths are then fused through a multimodal fusion network, and the self-attention mechanism is used to enable the luxury goods counterfeit detection model to capture micro-details, and the cross-attention mechanism is used to enable the luxury goods model to pay attention to both micro-details and macro-structures, thereby improving the accuracy of classification.

[0101] The micro-feature extraction network is responsible for extracting the micro-features of luxury goods. The macro-feature extraction network first extracts the macro information of the image, and then inputs these macro-features into the multimodal fusion network. Finally, the multimodal fusion network fuses the micro-features and macro-features to achieve a comprehensive judgment of the authenticity of luxury goods. The micro-feature extraction network encodes visual information through a convolutional neural network to capture the micro-features of the image. The macro-feature extraction network first extracts edges through an overall nested edge detection layer to obtain the macro information of the image, and then uses a residual network to extract macro-features to provide an overall structural perspective. The self-attention mechanism of the multimodal fusion network is used to help capture micro details, and the cross-attention mechanism of the multimodal fusion network is used to fuse both macro-features and micro-features. Finally, the classification network is used to classify the target classification vector.

[0102] 1. The micro-feature extraction network consists of a convolutional neural network layer and an overlapping patch embedding layer.

[0103] 1.1 The convolutional neural network layer is divided into three stages:

[0104] (1) The first stage: First, a convolutional layer with a stride of 2 is used to downsample the input features to obtain the first feature map, and then the features are gradually extracted through three simple structures containing three layers of convolution.

[0105] (2) The second stage: The first feature map is further downsampled using a maximum pooling layer with a stride of 2 and a convolution kernel of 3 x 3 to obtain the second feature map, reducing the spatial resolution while retaining important feature information. The features are then further extracted through a moving inverted bottleneck convolution (MBConv) block with a squeeze excitation mechanism.

[0106] (3) The third stage: The feature extraction capability is further enhanced through the moving inversion bottleneck convolution block of the squeeze incentive mechanism.

[0107] 1.2 Overlapping patch embedding layer The second feature map of the input is divided into image patches with heavy regions by using a convolutional layer with a stride of 2, thereby extracting embedded features. This can reduce the resolution of the feature map while retaining more local information, providing a more effective high-dimensional representation for subsequent feature extraction and processing.

[0108] 2. The macro feature extraction network includes the overall nested edge detection layer and the residual network.

[0109] 2.1 Overall Nested Edge Detection Layer (HED) Edge Extraction: This part of the workflow includes input preprocessing, feature extraction, construction of edge detection branches, and the final fusion of edge detection results to obtain the edge detection map.

[0110] First, the input luxury print image is preprocessed. Then, the image is passed through a series of convolutional layers for feature extraction. These convolutional layers include multiple levels, gradually extracting rich information from low-level features to high-level features.

[0111] After feature extraction, the model constructs multiple edge detection branches. Each edge detection branch generates branch feature maps of different resolutions through branch convolution layers, and upsamples the branch feature maps to the same size as the input luxury print image through a bilinear upsampling layer. The edge detection maps output by these branches are fused to generate the final target edge detection map. Specifically, the edge detection maps of layers 1 and 4 are weighted and fused to form the final target edge detection map.

[0112] The loss function uses a class-balanced cross entropy loss, and introduces a class balance factor to deal with the class imbalance problem, so that the model can better focus on the minority class during training. During model training, an adaptive moment estimation optimizer is used, and regularization is performed to prevent overfitting.

[0113] 2.2 Residual network extracts macro features: This module uses the target edge detection map as input and performs coarse-grained feature extraction through the residual network. In order to improve the adaptability of the model, the weights of the residual network remain trainable during the feature extraction stage, so that the model can fine-tune the pre-trained weights during the training process of a specific task, thereby better adapting to data characteristics and improving classification performance.

[0114] 3. Multimodal fusion of micro-features and macro-features.

[0115] The micro-feature extraction network encodes visual information through the convolutional neural network layer to generate micro-features. At the same time, the macro path processes the target edge detection map obtained from the overall nested edge detection layer through the residual network to extract macro-features. The micro-visual features are processed by the image patch embedding layer and input into the Transformer model together with the classification vector for further processing.

[0116] In the Transformer model, the Multi-Head Self-Attention (MSA) module is first used, which can process features at different positions in parallel and capture the global context and the association between different features. Through the Multi-Head Self-Attention module, the luxury goods counterfeit detection model can better identify important areas in the image, while combining information from micro and macro features to improve classification accuracy.

[0117] Next, we enter the Multilayer Perceptron (MLP) module, which is responsible for nonlinear mapping of features to further enhance feature expression capabilities. The MLP module makes more complex combinations of the rich features extracted from the attention mechanism to help the model understand the interaction between micro and macro features.

[0118] Finally, the Layer Normalization (LN) module is used to normalize the input features to ensure the stability of model training. It can normalize between different feature channels and help alleviate the training instability problem caused by different data distribution.

[0119] The output of the Transformer model is sent to the cross-attention feature fusion module. Use the macro features to calculate Q, use the micro features after the attention weights are adjusted to calculate K and V to calculate the scaled dot product attention, and obtain the micro features enhanced by the macro features. The micro features are upsampled to the feature map size output by the Transformer model, and weightedly added with the output of the Transformer model to obtain the final output of the multimodal fusion network. The output is re-input into the multimodal fusion network, and the final target classification vector is obtained after multiple iterations of the multimodal fusion network.

[0120] This fusion process uses macro features to query micro features, which can achieve the effect of using macro features to strengthen the representation of micro features. As a result, the model can not only capture the microscopic details of the object, but also use macro information for more comprehensive classification, thereby improving the robustness and accuracy of the model in the task of luxury goods authentication. 4. Luxury goods authentication model training method.

[0121] The luxury print image is input into the overall nested edge detection layer to generate the corresponding target edge detection map, and then the target edge detection map is input into the residual network extraction. The luxury print image is input into the micro-feature extraction network to obtain micro-features. The macro-features and micro-features are input into the multimodal fusion network. The micro-visual features are processed by the image patch embedding layer and input into the Transformer model together with the classification vector to obtain the micro-features optimized by the attention mechanism. The macro-features are used to query the micro-features, and the scaled click attention is calculated to obtain the micro-features enhanced by the macro-features. The micro-features are upsampled to the feature map size output by the transformer model, and weighted addition is performed with the output of the transformer model to obtain the final output of the multimodal fusion network. The output is re-input into the multimodal fusion network. After multiple iterations of the multimodal fusion network, the final target classification vector is obtained. The classification network classifies the target classification network to produce a classification output.

[0122] First, the original image is input into the overall nested edge detection layer to generate the corresponding target edge detection map.

[0123] Next, the target edge detection map and the luxury print image are input into the initial luxury goods authentication model. The luxury print image is extracted through the micro-feature extraction network to extract micro-features. The target edge detection map is extracted through the macro-feature extraction network to extract macro-features. The macro-features and micro-features are input into the multimodal fusion network. The micro-visual features are processed by the image patch embedding layer and input into the Transformer model together with the classification vector to obtain the micro-features optimized by the attention mechanism. The macro-features are used to query the micro-features, and the scaled click attention is calculated to obtain the micro-features enhanced by the macro-features. The micro-features are upsampled to the feature map size output by the transformer model, and weighted addition is performed with the output of the transformer model to obtain the final output of the multimodal fusion network. The output is re-input into the multimodal fusion network. After multiple iterations of the multimodal fusion network, the final target classification vector is obtained. The classification network classifies the target classification network to produce a classification output.

[0124] The present invention identifies luxury print images after being processed in three ways: shielding the luxury print images by grid masking, replacing the ink background of the luxury print images by background replacement, and sharpening the counterfeits. Table 1 is a comparison table of the recognition results accuracy of samples processed by different methods.

[0125] Table 1 Comparison of recognition accuracy of samples processed by different methods

[0126] Only use micro information Combining macro and micro information Performance comparison Samples not processed 99.73％ 99.88％ Increase by 0.15% Grid Mask 96.02％ 96.46％ Increased by 0.44% Background Replacement 90.95％ 93.67％ 2.72% increase Introducing difficult samples 98.23％ 98.67％ Increased by 0.44%

[0127] In summary, the present invention provides a luxury goods counterfeit detection model training method, luxury goods counterfeit detection method and device, which trains an initial luxury goods counterfeit detection model including a micro feature extraction network, a macro feature extraction network, a multimodal fusion network and a classification network through a training sample set; the micro feature extraction network includes a convolutional neural network layer and an overlapping patch embedding layer; the macro feature extraction network includes an overall nested edge detection layer and a residual network; the multimodal fusion network includes a Transformer model and a cross-attention feature fusion module; the multi-head self-attention module guides the model to focus on key areas, so that the classification vector captures key features and obtains micro features after attention weight adjustment, the multi-layer perceptron module enhances feature expression capabilities, the layer norm module ensures the stability of model training, the cross-attention module uses macro features to query micro features, enhances the model's context perception capabilities, and realizes multi-scale feature fusion; the loss function uses a cross entropy loss function, and minimizes the loss function as the goal to update parameters and obtain a luxury goods counterfeit detection model, and the authenticity classification results of luxury goods are obtained through the luxury goods counterfeit detection model.

[0128] Corresponding to the above method, the present invention also provides a luxury goods counterfeit detection model training device, including a processor, a memory and a computer program / instructions stored in the memory, the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0129] An embodiment of the present invention also provides a luxury goods counterfeit detection device, comprising a processor, a memory, and a computer program / instruction stored in the memory, wherein the processor is used to execute the computer program / instruction, and when the computer program / instruction is executed, the device implements the steps of the above method.

[0130] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0131] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0132] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A luxury goods counterfeit detection model training method, characterized in that: The method comprises the following steps: Obtain a training sample set, wherein the training sample set includes multiple samples, each sample includes a luxury print image; the multiple samples in the training sample set have labels including true and false classification values; An initial luxury goods counterfeit detection model is obtained, and the initial luxury goods counterfeit detection model includes a micro feature extraction network, a macro feature extraction network, a multimodal fusion network and a classification network; the macro feature extraction network includes an overall nested edge detection layer and a residual network; the multimodal fusion network includes an image patch embedding layer, a Transformer model and a cross-attention feature fusion module; the micro feature network extraction network takes samples in the training sample set as input and outputs micro features; the macro feature network extraction network takes samples in the training sample set as input, obtains a target edge detection map through the overall nested edge detection layer, and then obtains a target edge detection map through the residual network. The difference network extracts and outputs macro features; the image patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, then inputs the micro features into the Transformer model for attention weight adjustment to obtain the first target micro features, and inputs the micro features together with the macro features into the cross attention feature fusion module to obtain the second target micro features, and performs weighted summation of the first target micro features and the second target micro features and fuses the classification vector, then inputs the multimodal fusion network for global feature capture to obtain the target classification vector; the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample; A loss function is constructed according to the deviation between the authenticity prediction result corresponding to each sample and the true value of the authenticity classification, and the parameters of the initial luxury goods authentication model are iteratively updated with the goal of minimizing the loss function to obtain a luxury goods authentication model.

2. The luxury goods counterfeit detection model training method according to claim 1 is characterized in that: The micro-feature network extraction network takes samples in the training sample set as input, and the step of outputting micro-features includes: Inputting samples in the training sample set into the first convolution layer for downsampling to obtain a first feature map, and the first feature map is gradually extracted from image features through three consecutive convolution blocks; the convolution block includes three consecutive convolution layers; The first feature map is downsampled using a maximum pooling layer to obtain a second feature map, and the second feature map is subjected to depthwise separable convolution through two moving inverted bottleneck convolutions with a squeeze excitation mechanism to further enhance the extracted image features; The second feature map is divided into a plurality of fixed-size image blocks with overlapping areas through an overlapping patch embedding layer, and feature vectors of the plurality of image blocks are obtained through an embedding operation and encoded to obtain the microscopic features; the microscopic features include the density and distribution of ink dots.

3. The luxury goods counterfeit detection model training method according to claim 1, characterized in that: The macro-feature network extraction network uses the samples in the training sample set as input, obtains the target edge detection map through the overall nested edge detection layer, and then extracts and outputs the macro-features through the residual network, including: Extracting macroscopic feature information of the preprocessed samples layer by layer through multiple convolutional layers; Constructing a plurality of edge detection branches including branch convolution layers, wherein each of the edge detection branches generates a plurality of branch feature maps with different resolutions through the corresponding branch convolution layer; Inputting the plurality of branch feature maps into a bilinear upsampling layer for upsampling to obtain a plurality of edge detection maps of the same size as the luxury print image, and performing weighted fusion on the plurality of edge detection maps to generate a target edge detection map; The target edge detection image is subjected to coarse-grained feature extraction through the residual network to obtain the macro features; the macro features include the patterns and materials of the luxury goods.

4. The luxury goods counterfeit detection model training method according to claim 1, characterized in that: The Transformer model also includes a multi-head self-attention module, a multi-layer perceptron module and a layer norm module which are arranged in succession.

5. The luxury goods counterfeit detection model training method according to claim 4, characterized in that: The image patch embedding layer in the multimodal fusion network divides the micro features and performs vector conversion, then inputs the micro features into the Transformer model for attention weight adjustment to obtain a first target micro feature, and inputs the micro features together with the macro features into the cross attention feature fusion module to obtain a second target micro feature, performs weighted summation of the first target micro feature and the second target micro feature, fuses the classification vector, and then inputs the multimodal fusion network for global feature capture to obtain a target classification vector, and the classification network classifies the target classification vector to obtain the authenticity prediction result of the sample, including: Divide the microscopic features into a plurality of image patches of fixed size, flatten each patch into a vector, concatenate the image patch vector and the classification vector initialized to all 0s into a feature matrix, and input the matrix into the multi-head self-attention module for feature processing and global feature capture; The feature matrix is ​​nonlinearly mapped by the multilayer perceptron module and then input into the layer norm module for normalization processing to obtain the first target micro-feature after attention weight adjustment; The cross-attention fusion module uses the macro features F macro Calculate the query vector Q using the first target micro-feature F micro Calculate the key vector K and value vector V, the expression is: Q=W Q ·F macro ; K=W K ·F micro ; V=W V ·F micro ; The cross attention score is calculated based on the query vector and the key vector and normalized by the softmax function to obtain the attention weight matrix, which is expressed as: The second target micro-feature F is obtained by weighting the value vector by the attention weight enhanced , the expression is: F enhanced =A·V; The second target micro-feature is expanded to the same size as the luxury print image, and a learnable dynamic weight parameter α is introduced to the first target micro-feature F self and the second target microscopic feature F expanded After weighted summation and fusion of the classification vectors, the multimodal fusion network is input to perform global feature capture to obtain the target classification vector, which is expressed as: F final =α·F self +(1-α)·F expanded ; The classification network classifies the target classification vector to obtain a true or false prediction result of the sample.

6. The luxury goods counterfeit detection model training method according to claim 5, characterized in that: The method further comprises: A softmax function layer is connected to the output end of the classification network to output the probability of each classification in the luxury goods authenticity prediction result.

7. The luxury goods counterfeit detection model training method according to claim 1, characterized in that: The expression of the loss function is: Loss=-[ylog(p)+(1-y)·log(1-p)]; Wherein, y represents the value of the label, and p represents the probability that the luxury print image is authentic.

8. A method for identifying counterfeit luxury goods, characterized in that: The method comprises the following steps: Obtain the luxury goods print image to be detected; The luxury goods print image to be detected is input into the luxury goods counterfeit detection model trained by the luxury goods counterfeit detection model training method as claimed in any one of claims 1 to 7, and the authenticity classification and identification result of the luxury goods print image to be detected is output.

9. A luxury goods counterfeit detection model training device, comprising a processor, a memory, and a computer program / instruction stored in the memory, characterized in that: The processor is used to execute the computer program / instructions. When the computer program / instructions are executed, the device implements the steps of the method according to any one of claims 1 to 7.

10. A luxury goods counterfeit detection device, comprising a processor, a memory, and a computer program / instruction stored in the memory, characterized in that: The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the method according to claim 8.

Images