Commodity image feature matching and recognition method based on deep learning

Abstract

The present application relates to the technical field of commodity identification, and in particular to a commodity image feature matching and identification method based on deep learning, comprising the following steps: performing illumination invariance preprocessing on an input commodity image to generate an enhanced image with a standardized brightness distribution; inputting the enhanced image into a multi-modal feature fusion network to synchronously generate multi-modal feature vectors, the multi-modal feature vectors including a geometric structure feature vector, a material semantic feature vector and a color distribution feature vector; performing nonlinear combination on the multi-modal feature vectors through dynamic weight distribution to output a final feature descriptor for similarity matching with a target commodity library; the present application improves the generalization capability and model stability under a complex commodity type system, and is particularly suitable for heterogeneous commodity image matching scenarios with significant differences in material and form.

Description

Technical Field

[0001] This invention relates to the field of product recognition technology, and in particular to a product image feature matching and recognition method based on deep learning. Background Technology

[0002] With the rapid development of e-commerce, retail management, and intelligent recommendation systems, automatic product image recognition and matching technology has become a crucial foundation for building product databases, enabling image search, and intelligent retrieval. Current mainstream methods mostly extract image features based on deep convolutional neural networks and calculate similarity through fully connected or metric learning modules. However, existing methods have significant limitations in the following aspects:

[0003] First, most product image recognition methods use single-modal features as the matching basis, often focusing on texture or color, failing to fully combine multi-dimensional information such as the product's geometric outline, material properties, and color distribution. This makes it difficult to effectively distinguish products with similar structures and little color variation, affecting the overall recognition accuracy.

[0004] Secondly, existing feature fusion strategies usually rely on fixed weights or simple splicing methods, ignoring the differences in the focus of visual features among different product categories. For example, the geometric outline of 3C products is more important, while clothing products are more sensitive to color and material. Fixed fusion strategies are difficult to adapt to the retrieval needs of multiple categories and multiple scenarios. Summary of the Invention

[0005] This invention provides a product image feature matching and recognition method based on deep learning. This method integrates multimodal features, has category perception capabilities, and supports online dynamic calibration, thereby improving the robustness, interpretability, and cross-scene adaptability of product image representation, thus meeting the performance requirements of large-scale intelligent product retrieval systems.

[0006] A deep learning-based product image feature matching and recognition method includes the following steps:

[0007] S1: Perform illumination invariance preprocessing on the input product image to generate an enhanced image with a standardized brightness distribution;

[0008] S2: The enhanced image is input into the multimodal feature fusion network, which simultaneously generates multimodal feature vectors. The multimodal feature vectors include geometric structure feature vectors (containing curvature distribution encoding), material semantic feature vectors (containing micro-surface scattering characteristics), and color distribution feature vectors (containing color gamut mapping relationships).

[0009] S3: By dynamically assigning weights, multimodal feature vectors are non-linearly combined to output the final feature descriptor that is matched with the target product database based on similarity.

[0010] Optionally, S1 specifically includes:

[0011] S11: A dual-discriminator adversarial generative network is used to construct an illumination compensation matrix. The first discriminator is used to identify local brightness uniformity, and the second discriminator is used to evaluate texture fidelity, generating a compensation coefficient matrix with the same resolution as the input product image.

[0012] S12: Perform pixel-level brightness compensation on the original image based on the compensation coefficient matrix, and map the compensated image to the standard illumination space through a differentiable histogram matching algorithm to generate an enhanced image that satisfies the third-level constraints.

[0013] Optionally, the third-level constraints include:

[0014] Global brightness variance ≤ 0.05;

[0015] Local texture contrast retention rate ≥95%;

[0016] Chromaticity coordinate offset ΔE ≤ 2.0 (CIE Lab standard).

[0017] Optionally, the first discriminator is used to identify local brightness uniformity by employing a dilated convolutional layer with a dilation rate of 4 to extract multi-scale brightness distribution features and output a 128-dimensional brightness discrimination feature vector.

[0018] The second discriminator is used to evaluate texture fidelity, integrating the oriented gradient histogram operator with a 3×3 deformable convolution kernel to quantify the degree of texture detail preservation.

[0019] Optionally, the multimodal feature fusion network in S2 includes constructing a three-branch heterogeneous feature extraction network, which includes a geometric structure branch, a material semantic branch, and a color distribution branch.

[0020] Optionally, the geometric structure branch employs a differentiable curvature estimation layer, generates a pixel-level curvature distribution map through Hessian matrix eigenvalue analysis, and compresses it into a 128-dimensional feature vector;

[0021] The material semantic branch deploys a multi-scale texture analysis module, which, combined with the Micro Surface Scattering Model Parameter Estimation Network (MSPEN), outputs a 256-dimensional feature vector including anisotropic reflection characteristics.

[0022] The color distribution branch implements dynamic color gamut pyramid encoding, and generates a 64-dimensional color gamut fingerprint feature vector through adaptive vector quantization in CIE-Lab space.

[0023] Optionally, S2 further includes establishing a cross-modal feature calibration mechanism, specifically including:

[0024] The curvature distribution map output by the geometry branch is used as a spatial attention weight to modulate the receptive field of the convolution kernel of the material branch;

[0025] The color gamut fingerprint vector generated by the color branch is used to generate a channel attention mask through a gated recurrent unit to control the feature activation intensity of the geometric branch.

[0026] Each branch exchanges feature statistics (mean / variance) at the output of each layer of the network, and performs batch normalization parameter synchronous updates.

[0027] Optionally, S3 includes constructing a dynamic hypernetwork based on product category awareness. Through the constructed dynamic hypernetwork, a conditional vector is generated based on the category label of the input product to guide the feature fusion strategy. The dynamic hypernetwork includes a category-aware branch, which extracts the corresponding prior knowledge representation by parsing the category information and adjusts the fusion parameters between different modalities accordingly. At the same time, a cross-attention mechanism is established to calculate the correlation degree of the features of the three modalities of geometry, material and color respectively, and generate a score matrix representing the interaction strength between modalities to measure the interdependence between different features.

[0028] Optionally, S3 further includes performing feature space topology fusion, using the score matrix generated by the dynamic hypernetwork to perform nonlinear transformation operations on the three types of modal features respectively, and according to the modality division, splitting the fusion weights and using them for the modulation processing of geometric, material and color features respectively. Geometric features are subjected to affine transformation through linear mapping, material features are combined with activation functions to achieve weighted enhancement, and color features are smoothed through normalization operations. At the same time, a gating mechanism is introduced to perform weighted fusion of multimodal feature vectors according to the generated gating coefficients to generate a unified fusion feature representation. The fusion feature representation is used for matching and discrimination tasks, and the fusion coefficients are adaptively output by the dynamic hypernetwork according to the characteristics of the product category to ensure the rationality and adaptability of the weights of different modalities in the final representation.

[0029] Optionally, S3 also introduces a dynamic calibration mechanism based on similarity feedback. It retrieves several items in the target product library that are most similar to the current product to be matched, and calculates the mean representation of their fused features as a reference benchmark for feature distribution. The difference between the fused descriptor of the current product and the reference benchmark is used as an offset and input to a lightweight correction network to generate a weight compensation factor. The fused descriptor is adaptively corrected according to the weight compensation factor, and finally a feature descriptor with similarity matching is obtained. This alleviates the impact of feature imbalance or distribution shift between modalities and improves matching stability and system robustness.

[0030] The beneficial effects of this invention are:

[0031] This invention constructs a heterogeneous three-branch feature extraction network to independently model the geometric structure information, material semantic attributes, and color distribution features of product images. It also introduces modules such as differentiable curvature estimation, micro-surface scattering parameter modeling, and dynamic color gamut encoding to improve the physical interpretability and representational ability of various modal features. On this basis, it dynamically constructs the correlation between features through a cross-modal attention mechanism and completes the topological nonlinear coupling between modalities by combining a gating fusion strategy. This makes the final descriptor more discriminative and adaptable to different scenarios, which is significantly better than traditional weighted summation or single-modal recognition methods.

[0032] This invention addresses the issue of differing focus areas for different product categories in image recognition tasks. It designs a product tag-driven hypernetwork architecture that dynamically adjusts modal fusion weights and feature modulation matrices based on category prior information, enabling on-demand optimization of feature combination structures. This mechanism effectively avoids the biases generated by static fusion methods in multi-category recognition scenarios, improves generalization ability and model stability in complex product type systems, and is particularly suitable for heterogeneous product image matching scenarios with significant differences in material and shape.

[0033] This invention addresses the inter-domain offset problem between the feature distribution of product images and the product database in practical applications. It introduces a lightweight similarity correction network, which calculates the offset of the current descriptor and generates a weight compensation factor based on the feature distribution of Top-K similar samples in the target product database. This allows for online fine-tuning of the final output features, significantly improving the model's adaptability in dynamic environments and the accuracy of similar product discrimination. It also fills the technical gap in existing solutions that lack a feedback adjustment mechanism for the static processing of fused features. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the identification method according to an embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of a three-branch heterogeneous feature extraction network according to an embodiment of the present invention. Detailed Implementation

[0037] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0038] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0039] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0040] like Figures 1-2 As shown, the product image feature matching and recognition method based on deep learning includes the following steps:

[0041] S1: Perform illumination invariance preprocessing on the input product image to generate an enhanced image with a standardized brightness distribution;

[0042] S2: The enhanced image is input into the multimodal feature fusion network, which simultaneously generates multimodal feature vectors. The multimodal feature vectors include geometric structure feature vectors (containing curvature distribution encoding), material semantic feature vectors (containing micro-surface scattering characteristics), and color distribution feature vectors (containing color gamut mapping relationships).

[0043] S3: By dynamically assigning weights, multimodal feature vectors are non-linearly combined to output the final feature descriptor that is matched with the target product database based on similarity.

[0044] S1 specifically includes:

[0045] S11: A dual-discriminator adversarial generative network is used to construct an illumination compensation matrix. The first discriminator is used to identify local brightness uniformity, and the second discriminator is used to evaluate texture fidelity, generating a compensation coefficient matrix with the same resolution as the input product image.

[0046] S12: Perform pixel-level brightness compensation on the original image based on the compensation coefficient matrix, and map the compensated image to the standard illumination space through the differentiable histogram matching algorithm to generate an enhanced image that meets the third-level constraints.

[0047] Differentiable histogram matching algorithms include:

[0048] Constructing a brightness distribution transformation function : ,in, For the compensated image, To compensate for image parameters Brightness histogram under control For the preset standard illumination histogram template, parameters A differentiable mapping function is achieved through optimization using backpropagation.

[0049] Level 3 constraints include:

[0050] Global brightness variance ≤ 0.05;

[0051] Local texture contrast retention rate ≥95%;

[0052] Chromaticity coordinate offset ΔE ≤ 2.0 (CIE Lab standard).

[0053] The global brightness variance is calculated using a sliding window mechanism with a window size of 32×32 and a step size of 16. The calculation is as follows: ,in, For the first The average brightness value within each sliding window. This is the average brightness of all windows. The total number of sliding windows is [value], and the size of each sliding window is [value]. The step size is 16.

[0054] Local texture contrast retention rate is calculated based on the structural similarity index (SSIM). The calculation is as follows: ;in, For the first One original image window, For the first An enhanced image window, This represents the function for calculating structural similarity. This represents the total number of windows, consistent with the brightness variance.

[0055] The chromaticity coordinate offset is calculated using the CIEDE2000 color difference formula to measure the color difference between the enhanced image and the original image in the Lab color space.

[0056] The first discriminator is used to identify local brightness uniformity. It employs a dilated convolutional layer with a dilation rate of 4 to extract multi-scale brightness distribution features and outputs a 128-dimensional brightness discrimination feature vector.

[0057] Represented as: ,in, This represents the brightness feature vector output by the first discriminator, with a dimension of 128. For the input image, , , These are the image's height, width, and number of channels, respectively. This is a dilated convolution operation with a spatial rate of 4, used to expand the receptive field and capture multi-scale brightness distribution features;

[0058] The second discriminator is used to evaluate texture fidelity, integrating the oriented gradient histogram operator and a 3×3 deformable convolution kernel to quantify the degree of texture detail retention;

[0059] Represented as: ,in, This represents the texture feature vector output by the second discriminator, with a dimension of 128. This indicates that the histogram of oriented gradients (HARQ) features are extracted from the input image, preserving edge and texture information. This represents a deformable convolution operation of size 3x3, used to adapt to the deformation of texture boundaries and enhance the ability to perceive local structure.

[0060] The multimodal feature fusion network in S2 includes the construction of a three-branch heterogeneous feature extraction network, which includes a geometric structure branch, a material semantic branch, and a color distribution branch.

[0061] The geometric structure branch uses a differentiable curvature estimation layer, which generates a pixel-level curvature distribution map through Hessian matrix eigenvalue analysis and compresses it into a 128-dimensional feature vector.

[0062] Differentiable curvature estimation layer is used to generate pixel-level Gaussian curvature distribution maps. The Gaussian curvature calculation is expressed as:

[0063] ;

[0064] in, Indicates the image in First derivative in the direction, Indicates the image in Second derivative in the direction, Represents pixels The Gaussian curvature value at point is obtained by calculating the derivative using a Sobel-Hessian differentiable convolution kernel;

[0065] Extracted curvature map of the entire image After global pooling and linear transformation, it is compressed into a 128-dimensional vector:

[0066] ,in, The geometric structure feature vector is MLP, which stands for Multilayer Perceptron and is used for nonlinear dimensionality reduction.

[0067] The material semantic branch deploys a multi-scale texture analysis module, which, combined with the microsurface scattering model parameter estimation network (MSPEN), outputs a 256-dimensional feature vector including anisotropic reflection characteristics.

[0068] Material semantic branch: MSPEN model

[0069] Output material parameter vector : ,in, For surface roughness, Diffuse reflectance For specular reflectivity, It is the anisotropy coefficient (controlling the directionality of reflection). The refractive index of the material.

[0070] Discriminator loss function (Physics-guided adversarial learning, employing a physics-guided adversarial training strategy, the discriminator receives synthetic material images generated based on rendering equations as input, improving the realism and interpretability of parameter estimation): ,in, For the parameter vector Synthetic images generated by rendering functions For real images, It is a material discriminator.

[0071] The final material features are mapped to a 256-dimensional vector through an encoding network: ;

[0072] The color distribution branch realizes dynamic color gamut pyramid encoding, and generates a 64-dimensional color gamut fingerprint feature vector through adaptive vector quantization in CIE-Lab space;

[0073] In the color distribution branch, the color information of the image in the CIE-Lab color space needs to be encoded in order to generate a structured feature vector (i.e., a gamut fingerprint vector). To this end, a vector quantization codebook is constructed to map the color information to a finite number of category labels. The size of this codebook is determined by the number of bits.

[0074] Dynamic adjustment of the number of bits is represented as: ,in, These represent the variances of the a and b channels of the image in the CIELab color space, respectively, where b is the number of quantization bits. Size of the grayscale quantization codebook; Color vector encoding process: ,in, This indicates that the image is represented in the CIELab color space. For a vector quantizer with a codebook size of N, This is the final output color feature vector.

[0075] S2 also includes establishing a cross-modal feature calibration mechanism, specifically including:

[0076] The curvature distribution map output by the geometry branch is used as a spatial attention weight to modulate the receptive field of the convolution kernel of the material branch;

[0077] Curvature modulation material receptive field (spatial attention mechanism): ,in, This represents the spatial attention weights generated from the curvature map. The modulated material feature map is represented by the spatial attention weights generated from the curvature map K(x,y) through a normalization + activation function process, using the Sigmoid function to map the curvature values ​​to the range [0, 1].

[0078] The color gamut fingerprint vector generated by the color branch is used to generate a channel attention mask through a gated recurrent unit to control the feature activation intensity of the geometric branch.

[0079] Color modulation geometric activation (channel attention mechanism): ,in, For color feature vectors, For the geometric channel attention mask generated by GRU, This indicates channel-by-channel multiplication. This is the modulated geometric feature vector.

[0080] Each branch exchanges feature statistics (mean / variance) at the output of each layer of the network, and performs batch normalization parameter synchronous updates.

[0081] The output features of each layer share statistics across branches:

[0082] ;

[0083] in, Indicates the first The branch in the The first channel Each sample value The channel mean. For channel standard deviation; all branches Synchronous updates during training , As a branch of geometric structure, This is a branch of material semantics. This is a branch of color distribution.

[0084] S3 specifically includes:

[0085] S31: Constructing a dynamic supernetwork based on product category awareness: Constructing a perception-driven dynamic supernetwork to adaptively adjust feature fusion weights based on product category information:

[0086] The hypernetwork includes category-based conditional branches and extracts prior knowledge vectors based on product labels. ,in, For product categories The corresponding embedding vector is used to adjust the supernetwork weight regeneration logic.

[0087] Establish a cross-attention mechanism for computational geometry ( ), material ( ) and color ( The correlation between the three types of features is used to generate an attention score matrix between patterns. Its element definition is as follows:

[0088] , ,in, Representing modes With mode Attention scores between them This represents the feature vector of the corresponding pattern. Represents the learnable query and key projection matrix. Indicates the feature dimension, which can be 128 or 256; This indicates a row-level normalization operation.

[0089] S32: Perform Feature Space Expansion and Fusion: The fusion process includes three parts: weight matrix generation, modal feature nonlinear transformation, and gated fusion.

[0090] S321: The hypernetwork generates a cross-modal weight matrix and splits it by modality: These are used for feature modulation of geometry, material, and color branches, respectively.

[0091] S322: Perform nonlinear transformations on each modal feature:

[0092] ;

[0093] ;

[0094] ;

[0095] in, This represents the modal eigenvector after nonlinear mapping. This represents an affine linear transformation (matrix multiplication). This indicates element-wise multiplication, Sigmoid() is the activation function, and LayerNorm() represents the layer normalization operation.

[0096] S323: Gated fusion generates the final descriptor : ,in, The gating coefficients output by the hypernetwork control the modal fusion ratio, ensuring the constraints are met. .

[0097] S33: Similarity-driven descriptor calibration mechanism: To alleviate feature space drift caused by cross-modal fusion, a target product database feedback mechanism is introduced to dynamically correct the final descriptor.

[0098] S331: Extract the feature mean of the Top-K similar products in the target database. : ,in, Indicates the first in the library The descriptor of the most similar product.

[0099] S332: Calculate descriptor offset : ;

[0100] S333: Generate compensation factors and update descriptors using a lightweight correction network: ,in, To correct the weight compensation coefficients of the network output, This is the feature descriptor ultimately used for similarity matching.

[0101] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0102] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A product image feature matching and recognition method based on deep learning, characterized in that, Includes the following steps: S1: Perform illumination invariance preprocessing on the input product image to generate an enhanced image with a standardized brightness distribution; S2: The enhanced image is input into the multimodal feature fusion network, which simultaneously generates multimodal feature vectors, including geometric structure feature vectors, material semantic feature vectors, and color distribution feature vectors; S3: By dynamically assigning weights, multimodal feature vectors are non-linearly combined to output the final feature descriptor that is matched with the target product database based on similarity. S1 specifically includes: S11: A dual-discriminator adversarial generative network is used to construct an illumination compensation matrix. The first discriminator is used to identify local brightness uniformity, and the second discriminator is used to evaluate texture fidelity, generating a compensation coefficient matrix with the same resolution as the input product image. S12: Perform pixel-level brightness compensation on the original image based on the compensation coefficient matrix, and map the compensated image to the standard illumination space through the differentiable histogram matching algorithm to generate an enhanced image that satisfies the third-level constraints; The third-level constraints include: Global brightness variance ≤ 0.05; Local texture contrast retention rate ≥95%; Chromaticity coordinate offset ΔE ≤ 2.

0.

2. The product image feature matching and recognition method based on deep learning according to claim 1, characterized in that, The first discriminator is used to identify local brightness uniformity. It employs a dilated convolutional layer with a dilation rate of 4 to extract multi-scale brightness distribution features and outputs a 128-dimensional brightness discrimination feature vector. The second discriminator is used to evaluate texture fidelity, integrating the oriented gradient histogram operator with a 3×3 deformable convolution kernel to quantify the degree of texture detail preservation.

3. The product image feature matching and recognition method based on deep learning according to claim 1, characterized in that, The multimodal feature fusion network in S2 includes the construction of a three-branch heterogeneous feature extraction network, which includes a geometric structure branch, a material semantic branch, and a color distribution branch.

4. The product image feature matching and recognition method based on deep learning according to claim 3, characterized in that, The geometric structure branch adopts a differentiable curvature estimation layer, generates a pixel-level curvature distribution map through Hessian matrix eigenvalue analysis, and compresses it into a 128-dimensional feature vector; The material semantic branch deploys a multi-scale texture analysis module, which, combined with a micro-surface scattering model parameter estimation network, outputs a 256-dimensional feature vector including anisotropic reflection characteristics. The color distribution branch implements dynamic color gamut pyramid encoding, and generates a 64-dimensional color gamut fingerprint feature vector through adaptive vector quantization in CIE-Lab space.

5. The product image feature matching and recognition method based on deep learning according to claim 3, characterized in that, The S2 also includes establishing a cross-modal feature calibration mechanism, specifically including: The curvature distribution map output by the geometry branch is used as a spatial attention weight to modulate the receptive field of the convolution kernel of the material branch; The color gamut fingerprint vector generated by the color branch is used to generate a channel attention mask through a gated recurrent unit to control the feature activation intensity of the geometric branch. Each branch exchanges feature statistics at the output of each layer of the network and performs batch normalization parameter synchronous updates.

6. The product image feature matching and recognition method based on deep learning according to claim 1, characterized in that, S3 includes constructing a dynamic hypernetwork based on product category awareness. Through the constructed dynamic hypernetwork, a conditional vector is generated based on the category label of the input product to guide the feature fusion strategy. The dynamic hypernetwork includes a category awareness branch, which extracts the corresponding prior knowledge representation by parsing the category information and adjusts the fusion parameters between different modalities accordingly. At the same time, a cross-attention mechanism is established to calculate the correlation degree of features of the three modalities of geometry, material and color, respectively, and generate a score matrix representing the interaction strength between modalities to measure the interdependence between different features.

7. The product image feature matching and recognition method based on deep learning according to claim 6, characterized in that, The S3 further includes performing feature space topology fusion. Using the score matrix generated by the dynamic hypernetwork, nonlinear transformation operations are performed on the three types of modal features respectively. Based on the modality division, the fusion weights are split and used for the modulation processing of geometric, material and color features respectively. Geometric features are subjected to affine transformation through linear mapping, material features are combined with activation functions to achieve weighted enhancement, and color features are smoothed through normalization operations. At the same time, a gating mechanism is introduced to perform weighted fusion of multimodal feature vectors according to the generated gating coefficients to generate a unified fusion feature representation. The fusion feature representation is used for matching and discrimination tasks. The fusion coefficients are adaptively output by the dynamic hypernetwork according to the characteristics of the product category to ensure the rationality and adaptability of the weights of different modalities in the final representation.

8. The product image feature matching and recognition method based on deep learning according to claim 7, characterized in that, The S3 also introduces a dynamic calibration mechanism based on similarity feedback. It retrieves several items in the target product library that are most similar to the current product to be matched, and calculates the mean representation of their fused features as a reference benchmark for feature distribution. The difference between the fused descriptor of the current product and the reference benchmark is used as an offset and input to a lightweight correction network to generate a weight compensation factor. The fused descriptor is adaptively corrected according to the weight compensation factor to finally obtain a feature descriptor with similarity matching.

Images