Commodity intelligent classification management method and system

Abstract

The invention discloses an intelligent classification management method and system for commodities. The method comprises the following steps: obtaining a to-be-classified target commodity image and associated text description information; extracting depth visual features of the target commodity image through an image feature extraction model; semantic features of the text description information are extracted through a text feature extraction model; the depth visual features and the semantic features are subjected to multi-modal feature alignment, a fusion feature vector is generated, the alignment process involves calculation of a projection matrix of visual and text feature spaces, and cross-modal feature association is established through an attention mechanism; and determining a fine-grained classification result of the commodity based on the fusion feature vector, wherein the fine-grained classification result comprises a three-level classification system of brands, models and specifications. According to the method, deep visual features and semantic features are combined, multi-modal information fusion is realized, and the commodity classification accuracy and efficiency are improved. According to the invention, diversified requirements of commodity classification management are met, and the intelligent process in the fields of e-commerce, logistics and the like is greatly promoted.

Description

Technical Field

[0001] The present invention relates to an application of artificial intelligence in commodity classification management, relates to graphic data reading, is classified as G06K, and specifically relates to an intelligent commodity classification management method and system. Background Art

[0002] In modern retail and e-commerce, product classification management is crucial for ensuring inventory accuracy, improving retrieval efficiency, and optimizing the customer shopping experience. With the increasing variety of products, traditional manual classification methods are no longer able to meet the demand for fast and accurate classification. Existing classification methods often rely on barcode or RFID technology. While these methods have improved efficiency to a certain extent, they still have many limitations, such as a strong reliance on physical product labels, susceptibility to physical damage and signal interference, and an inability to provide detailed product information and real-time updates.

[0003] With the development of artificial intelligence technology, intelligent classification methods based on machine learning and image recognition have begun to be applied to commodity management. However, when processing large quantities of commodities, these technologies still face problems such as low recognition accuracy, slow processing speed, and poor environmental adaptability. Patent application with publication number CN119540603A discloses an intelligent image recognition and commodity classification system, which includes an image acquisition module, an image processing and feature extraction module, a deep learning model training module, a commodity classification and recognition module, a user interface module, a database module, a system integration and deployment module, an update and maintenance module, and a security module. The system can accurately capture more complex nonlinear relationships in images, thereby more effectively expressing abstract features in images and significantly improving the system's ability to recognize complex patterns. It can be seen that image recognition has been applied in commodity classification.

[0004] By integrating advanced image processing technologies, machine learning algorithms, and big data analytics, intelligent product identification, classification, and management can be achieved, thereby improving overall operational efficiency and customer satisfaction in the retail and e-commerce industries. However, traditional product classification methods often rely on a single information modality, such as images or text, resulting in inaccurate and incomplete classification results. Furthermore, existing methods lack efficient information fusion mechanisms and sophisticated feature extraction capabilities when processing complex and ever-changing product information, making it difficult to meet the diverse and refined needs of product classification management in e-commerce, logistics, and other fields. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide an intelligent classification management method and system for commodities. By combining the deep visual features of the target commodities and the semantic features of text description information, the effective fusion of multimodal information is achieved to improve the accuracy and efficiency of commodity classification.

[0006] The object of the present invention is achieved through the following technical solutions:

[0007] A method for intelligent classification management of commodities, comprising:

[0008] Obtain the target product image to be classified and the associated text description information;

[0009] Extracting deep visual features of the target product image through an image feature extraction model;

[0010] Extracting semantic features of the text description information through a text feature extraction model;

[0011] Performing multimodal feature alignment on the deep visual features and the semantic features to generate a fused feature vector, wherein the alignment process includes: calculating a projection matrix between the visual feature space and the text feature space, and establishing cross-modal feature associations through an attention mechanism;

[0012] A fine-grained classification result of the product is determined based on the fused feature vector, where the fine-grained classification includes a three-level classification system of brand, model, and specification.

[0013] As a preferred embodiment, the text feature extraction model includes:

[0014] Input encoding layer, which generates word vector sequence after segmenting text description;

[0015] Position coding layer, which uses the sinusoidal position coding formula to inject position information;

[0016] Multi-head self-attention layer, which calculates the relevance of each word vector with the global context;

[0017] The knowledge graph enhancement layer encodes the entity relationships in the product knowledge graph as additional feature vectors.

[0018] As a preferred embodiment, the operations of the knowledge graph enhancement layer include:

[0019] Retrieve relevant entity nodes from the pre-built product knowledge graph;

[0020] Aggregate neighborhood node information through graph convolutional networks:

[0021]

[0022] in, represents the hidden state of node i in the l+1 layer, σ represents the activation function (such as ReLU, Sigmoid, etc.), introduces nonlinearity to enhance the model's expressiveness, ∑ j∈N(i)It represents the sum of all neighbors j of node i, N(i) is the set of neighbor nodes of node i, and aggregates the information of neighbor nodes, W (l) Represents the trainable weight matrix of the lth layer, performs linear transformation on the features of neighbor nodes, and learns the interaction of features at different levels. Represents the hidden state of node j in layer l, providing the original feature input of the neighbor nodes. |N(i)| represents the number of neighbors of node i and is a normalization factor to avoid numerical instability caused by differences in the number of neighbors.

[0023] The aggregated node embedding is gated and fused with the text features. The gated fusion process is represented by the following two steps: gate weight calculation and feature fusion.

[0024] Gating weight calculation:

[0025] g=sigmoid(W g ·[h text ;h kg ])

[0026] Among them, h text is the text feature vector, h kg is the node embedding vector after knowledge graph aggregation, [;] represents the vector splicing operation, h text and h kg Splicing in a certain dimension (such as channel or feature dimension) to generate a joint input vector, W g is a learnable weight matrix that maps the concatenated joint vector to a scalar space to generate gating weights; sigmoid is an activation function, and its mathematical expression is:

[0027]

[0028] The output after linear transformation is compressed to the interval [0,1] to represent the gating weight. g is a dynamically generated gating weight scalar that controls the contribution ratio of text features and knowledge graph features during fusion.

[0029] Feature fusion:

[0030] h fused =g·h text +(1-g)·h kg

[0031] Among them, h fused is the fused multimodal feature vector, which is gated by weights h text and h kg Perform weighted summation to achieve dynamic feature fusion.

[0032] As a preferred embodiment, the fine-grained classification process adopts a hierarchical classifier, including:

[0033] The first-level classifier performs coarse classification based on brand labels, using a fully connected layer and softmax activation;

[0034] The second-level classifier performs model recognition based on brand classification, using a multi-task learning framework;

[0035] The third-level classifier constructs a metric learning loss function based on product specifications:

[0036]

[0037] Among them, L represents the metric learning loss value of the third-level classifier, which is used to optimize the model to ensure that the feature distances of similar samples are closer and the feature distances of heterogeneous samples are farther. i,j Sum the sample pairs and traverse all anchor-positive sample pairs (f i ,f j + ) and anchor-negative sample pairs (f i ,f j - ), max(0,·) is the hinge loss function, m is the boundary threshold (a hyperparameter, generally set to a positive number), which controls the minimum interval between positive and negative samples and ensures the separation of the feature spaces of similar and heterogeneous samples, D(·,·) is the distance metric function, which calculates the similarity distance between two feature vectors, and f i is the feature vector of the anchor sample, f j + is the feature vector of the positive sample belonging to the same category as the anchor sample i, and different product samples of the same specification, f j - is the negative sample feature vector belonging to a different category from the anchor sample i, and is a product sample of different specifications.

[0038] As a preferred embodiment, the multi-scale convolutional neural network includes:

[0039] The first branch uses a 7×7 convolution kernel to extract global features;

[0040] The second branch uses a 3×3 convolution kernel group to capture local details;

[0041] The third branch uses dilated convolution to expand the receptive field;

[0042] The features of each branch are fused through the channel attention mechanism, including channel attention weight calculation and feature weighted fusion;

[0043] Channel attention weight calculation:

[0044] s=sigmoid(W·[F1;F2;F3])

[0045] Among them, F1, F2, and F3 come from feature maps of different branches (such as global features, local detail features, and large receptive field features). F1 is the global feature extracted by the 7×7 convolution kernel, F2 is the local detail feature extracted by the 3×3 convolution kernel group, and F3 is the feature extracted by expanding the receptive field through the void convolution. [;;] is the feature map spliced along the channel dimension; W is a learnable weight matrix, sigmoid is the activation function, s is the channel attention weight vector, which contains the weight coefficients s1, s2, and s3 of the three branches. Each weight value range is [0, 1]. Feature weighted fusion:

[0046] F fused =s1·F1+s2·F2+s3·F3

[0047] Among them, F fused It is a fused multi-scale feature map, which sums the features of each branch according to the weight coefficient to achieve adaptive feature fusion.

[0048] As a preferred method, an anomaly detection module is also included:

[0049] Calculate the Mahalanobis distance between the new product features and the existing categories in the knowledge graph:

[0050]

[0051] Among them, D M (x) is the Mahalanobis distance from sample x to the target category distribution, x is the feature vector of the new product, μ is the feature mean vector of the existing categories in the knowledge graph, ∑ is the covariance matrix of the existing categories in the knowledge graph, which describes the correlation and variance between features, ∑ -1 is the inverse matrix of the covariance matrix;

[0052] When D M (x) Trigger new category alerts when thresholds are exceeded;

[0053] Construct a generative adversarial network to generate potential abnormal samples and enhance classification robustness.

[0054] As a preferred approach, the feature alignment process introduces contrastive learning loss:

[0055]

[0056] Among them, L cont is the contrast loss value, which measures the model's ability to align visual and text features. The smaller the value, the better the cross-modal alignment effect. v is the visual feature vector, t + is the positive sample text feature, the embedding representation of the real product description text matching the current visual feature v, t′ is the negative sample text feature, is the training batch; s(v,t) is the similarity function, which calculates the degree of match between visual and text features. Common forms include:

[0057] Dot product similarity:

[0058] Cosine similarity:

[0059] Dot product after projection: Where W v ,W t is the learnable projection matrix;

[0060] τ is the temperature coefficient, Represents a set All negative samples t′ in are summed up, exp(s(v,t′) / τ) maps the similarity score to the positive range, amplifying the influence of high scores, and τ controls the sharpness of the probability distribution.

[0061] In a second aspect, the present invention provides an intelligent classification management system for commodities, comprising:

[0062] a data acquisition unit configured to collect product images and associated text descriptions;

[0063] Visual feature extraction unit, which includes a multi-scale convolutional neural network to generate deep visual features;

[0064] Text feature extraction unit, integrated with Transformer architecture, is used to generate semantic embedding vectors;

[0065] Cross-modal alignment unit, implementing feature projection and attention mechanisms;

[0066] The classification decision unit includes a hierarchical classifier group and outputs fine-grained classification results.

[0067] As a preferred embodiment, it also includes:

[0068] Visual review unit, which generates an interactive interface including heat maps and key terms;

[0069] Knowledge graph management unit, which maintains the product entity relationship database and supports incremental updates;

[0070] Model optimization unit, dynamically adjusts feature alignment parameters based on review feedback;

[0071] The cross-modal alignment unit includes:

[0072] The projection submodule configures a learnable matrix to map visual / text features to a common space;

[0073] Attention calculation submodule, which realizes the calculation of cross-modal feature correlation matrix;

[0074] The feature fusion submodule performs weighted concatenation operations to generate a joint feature vector.

[0075] As a preferred embodiment, the text feature extraction unit includes:

[0076] The word segmentation module uses Jieba word segmentation tool to process Chinese descriptions;

[0077] The word vector layer loads the pre-trained BERT model to generate context-sensitive embeddings;

[0078] The graph neural network processor connects to the product knowledge graph database to obtain enhanced features.

[0079] The present invention has at least the following beneficial effects:

[0080] The present invention can significantly improve the accuracy and efficiency of commodity classification. By combining the deep visual features of the target commodity with the semantic features of the textual description information, effective fusion of multimodal information is achieved. Through multimodal feature alignment technology, the correlation between visual and textual features is enhanced, providing strong support for the generation of high-quality fused feature vectors. Ultimately, the fine-grained classification results achieved based on this fused feature vector meet the diverse needs of commodity classification management. The application of this invention will greatly promote the intelligent development of e-commerce, logistics and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] To reveal the technical details of the embodiments of the present invention, the following is a brief introduction to the drawings involved in the embodiments. It should be emphasized that these drawings only illustrate several embodiments of the present invention and should not be considered as defining the scope of the invention. Those skilled in the art can deduce other relevant drawings based on these drawings without engaging in creative work.

[0082] Figure 1 The present invention is a flowchart showing a method for intelligent classification management of commodities;

[0083] Figure 2 Generate process diagrams for the visual review interface;

[0084] Figure 3 Schematic diagram of the multimodal feature alignment process. DETAILED DESCRIPTION

[0085] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0086] In the following, embodiments of the present disclosure are described in detail with the aid of accompanying drawings. However, please be aware that the present disclosure is not limited to the specific forms shown herein. Rather, it should be understood to encompass various variations, equivalents, and / or alternatives to the embodiments of the present disclosure. In describing the drawings, the same reference numerals will be used to indicate similar components.

[0087] In this disclosure, terms are used to illustrate specific embodiments and do not constitute limitations of this disclosure. In this context, the use of the singular also encompasses the plural, unless the text clearly indicates otherwise. In the process of explanation, it should be understood that terms such as "including" or "having" are intended to indicate the presence of a feature, quantity, step, operation, structural component, part, or combination thereof, and do not preclude the possibility or addition of one or more other features, quantities, steps, operations, structural components, parts, or combinations thereof.

[0088] It should be understood that while the following description provides extensive specific details intended to facilitate a comprehensive understanding of the example embodiments, those skilled in the art will appreciate that the example embodiments can be implemented without these specific details. For example, systems may be presented in block diagram form to avoid excessive detail that would obscure the clarity of the examples. In other cases, unnecessary details regarding well-known processes, structures, and techniques may be omitted to maintain clarity of the examples.

[0089] like Figure 1 As shown, a method for intelligent classification management of commodities includes:

[0090] S1: Obtain the target product image to be classified and the associated text description information;

[0091] S2: extracting deep visual features of the target product image through an image feature extraction model, wherein the image feature extraction model adopts a multi-scale convolutional neural network structure;

[0092] S3: The semantic features of the text description information are extracted through a text feature extraction model. The text feature extraction model adopts a pre-trained language model based on Transformer; the deep semantics of the text is captured through a self-attention mechanism and multi-layer encoding. For example, the text: "pure cotton round neck T-shirt, 2023 summer new style", its core process includes: (1) word segmentation and embedding, word segmentation: split the sentence into word units (such as "pure cotton", "round neck", "T-shirt", "2023", "summer", "new style"). Word embedding: each word is converted into a 768-dimensional vector (such as "pure cotton", the vector represents its material attribute; "2023", represents the time attribute). Use a pre-trained vocabulary (such as BERT's 30,000+ vocabulary) to process unregistered words (such as "T-shirt" is split into "T" and "shirt"). (2) Context encoding (Transformer layer), self-attention mechanism: analyze the dependency relationship between words. For example, "pure cotton" and "breathable" are implicitly associated, and "2023" and "new style" form a time logic. Multi-head attention, 8 "attention heads" focus on different semantic dimensions (such as material, season, style). Position encoding: inject word position information to distinguish the difference in word order between "2023 summer new style" and "new style 2023 summer". (3) Knowledge graph enhancement, identify key entities in the text (such as "pure cotton" to "material", "T-shirt" to "clothing category"). Graph embedding fusion: encode attributes such as "breathability" and "softness" associated with "pure cotton" in the knowledge base as additional features. The model combines knowledge graphs to understand the common uses of "pure cotton" in clothing and enhance semantic representation. (4) Dynamic weight allocation, attention weight: calculate word importance. "New style" and "2023" have higher weights (season attribute is key), while "round neck" may have lower weight (style is secondary). Output semantic features: 768-dimensional vector, integrating text description and external knowledge, such as: [material: pure cotton, season: 2023 summer, type: T-shirt, attribute: new style]. Contextual understanding: Capturing long-range dependencies (such as the relationship between "summer" and "new style") through self-attention. It also handles polysemy, such as dynamically distinguishing the meaning of "apple" (fruit vs. brand) in different scenarios. Knowledge enhancement: Incorporating a product knowledge base resolves text ambiguity (such as the different standards for "waterproof" in electronics and clothing). Through deep semantic encoding and external knowledge fusion, the model transforms text into semantic features rich in business attributes, providing a precise semantic foundation for subsequent multimodal alignment (such as image + text), ultimately improving the accuracy and interpretability of product classification.

[0093] S4: performing multimodal feature alignment on the deep visual features and the semantic features to generate a fused feature vector, wherein the alignment process includes: calculating a projection matrix between the visual feature space and the text feature space, and establishing cross-modal feature associations through an attention mechanism;

[0094] S5: Determine a fine-grained classification result of the product based on the fused feature vector, where the fine-grained classification includes a three-level classification system of brand, model, and specification.

[0095] The present invention achieves accurate classification by combining image and text information. First, the system obtains product images and text descriptions, uses a multi-level image feature extraction network to analyze the details of the product appearance, and uses advanced text understanding models to parse the semantic information in the text. Subsequently, the image features and text features are deeply associated through intelligent alignment technology, such as automatically identifying the intrinsic connection between the picture of "red dress" and keywords such as "red" and "chiffon material" in the text description. The integrated features after fusion will enter a three-level classification system: first determine the brand, then refine it to the model, and finally determine the specifications (42 yards / black and white color). If the system has doubts about the classification results, it will automatically generate a visual review interface, highlighting the key parts of the picture (such as the sole pattern) and the core parameters of the text (such as the waterproof grade) for manual confirmation. The review feedback will continuously optimize the system and update the verified classification results to the product knowledge base, so that the classification accuracy will continue to improve with use.

[0096] In one embodiment, when it is detected that the classification confidence is lower than a preset threshold, a visual review interface containing key feature areas is generated; the visual review interface generation process (when the classification confidence is lower than the threshold) (see Figure 2 ):

[0097] Step 1: Confidence Detection and Trigger Conditions. Input: The fine-grained classification results and corresponding confidence scores output by the classifier (e.g., brand classification confidence 0.85, model classification 0.72, specification classification 0.65). Threshold Judgment: A preset threshold (e.g., 0.7) is set. When the confidence level of any classification level falls below the threshold, the review process is triggered. For example, if the confidence level of the specification classification is 0.65 < 0.7, a review is triggered.

[0098] Step 2: Key Feature Location. ① Visual Key Region Location, Gradient Weighted Class Activation Map (Grad-CAM): Backpropagate classification gradients to the image feature map, calculate channel weights, and generate an activation heatmap. Output highlights the areas in the image that have the greatest impact on the classification decision (such as the specification label on the product packaging). ② Text Key Term Extraction, Attention Weight Analysis: Extract attention weights from text features, calculate term importance scores, and select the terms with the highest scores (such as "500ml" and "Waterproof Grade IP68"). Output a list of key terms in the text description that are related to low-confidence classifications.

[0099] Step 3: Cross-modal spatial alignment, cross-modal attention mechanism: Map the visual heatmap coordinates and text term position encodings to the same space, from image feature map coordinates to position encoding vectors, and from text term position indices to position encoding vectors. Calculate the alignment matrix and output the spatial association between the image heatmap and the text term (e.g., "500ml" corresponds to the bottle label area).

[0100] Step 4: Generate an interactive review interface. Interface elements include a visual heatmap overlay, which creates a semi-transparent heatmap overlaid on the original image, with red areas indicating high relevance. A hover tooltip for text terms displays the corresponding image area when the mouse hovers over a term (e.g., clicking "500ml" highlights the bottle label). Manual annotation controls provide drop-down menus or checkboxes for reviewers to modify classification labels or add additional attributes.

[0101] Step 5: Feedback Processing and Model Update. Review feedback collection and record manually corrected classification results and annotation information (e.g., correcting "500ml" to "1L"). Dynamically adjust parameters and update the feature alignment projection matrix. Optimize knowledge graph nodes, add a "1L" specification node, and associate it with the corresponding brand and model. Incremental training is then performed, adding the audited data to the training set, fine-tuning the classifier parameters, and improving subsequent classification confidence.

[0102] This implementation uses Grad-CAM and the attention mechanism to intuitively display the basis for model decisions. Cross-modal interaction and spatial alignment enable "image-text cross-references," reducing the complexity of manual review. Audit feedback feeds back into the model in real time, forming an iterative chain of detection, correction, and enhancement. Audit logs are recorded to meet the traceability requirements of quality management systems.

[0103] In one embodiment, feature alignment parameters are updated based on review feedback, and the verified classification results are associated and stored in the product knowledge graph database.

[0104] Audit feedback data collection, manually review and confirm or correct classification results (e.g., correcting "500ml mineral water" to "550ml mineral water"). Corrected classification labels include user-adjusted brand, model, and specification information, and manually annotated visual key areas (e.g., bottle labels) or text key terms (e.g., "550ml"). Structured log records include the original classification results, corrected results, and the audit timestamp.

[0105] Feature alignment parameters are updated to associate the corrected classification label with the original features (visual features V, text features T) to generate training sample pairs. The corrected "550ml" is bound to the bottle label area features in the original image. The supervised learning goal is to minimize the distance between the corrected features and the correct category and maximize the distance from the incorrect category. Parameter update formula: Where η is the learning rate, Wv ,W t is the projection matrix of the joint embedding space of vision and text. Attention mechanism optimization, according to the key areas marked by manual, adjust the cross-modal attention weights, A new =A 原始 +α·(A 人工标注 -A 原始 ), α is the attenuation coefficient, balancing the new and old weights. A 原始 is the original attention weight matrix automatically generated by the model, A 人工标注 The attention weight matrix is generated based on the key areas marked by manual review feedback. When the classification confidence is low, the manual review will mark the key areas (such as highlighting the bottle label), and the system will convert these marks into attention weights. For example, the weight of the manually marked label area may be set to 1, and the background is set to 0. The attention distribution of the model is modified to make it focus more on the important areas of manual verification. new is the adjusted new attention weight matrix. Combining the model's original judgment with human feedback, a more reliable attention distribution is generated for subsequent classification or feature alignment.

[0106] When the automatic classification result of a product is not certain enough (for example, "500ml of mineral water" is mistakenly classified as "550ml"), a visual review interface will be generated, highlighting key areas in the image (such as the bottle label) and keywords in the text (such as "500ml") for manual reviewers to check.

[0107] Update parameter process: The auditor confirms the correct classification (e.g., changing it to "500ml") and annotates key features (e.g., label location). Model learning: Based on the correction results, the system automatically adjusts the weights for feature alignment (e.g., focusing more on the visual features of the label area and the textual features of the "ml" unit). Optimize the model: The next time a similar product is encountered, the model can more accurately classify it based on the combination of image and text information.

[0108] The verified classification (e.g., brand "Nongfu Spring," model "mineral water," and size "500ml") is connected to existing nodes in the knowledge graph (e.g., "Nongfu Spring, bottled water, 500ml"). If a new size is discovered (e.g., "550ml"), a new node is added to the graph and an association is established (e.g., "Add a 550ml node, associated with the bottled water category"). Like a "smart map," the knowledge graph is continuously enriched with each review, helping the system quickly identify similar products in the future.

[0109] In one embodiment, the multimodal feature alignment process specifically includes (see Figure 3 ):

[0110] Constructing a joint visual-text embedding space:

[0111] V′=Wv V+b v

[0112] T′=W t T+b t

[0113] Among them, V and T are the original visual features and text features respectively, V is the visual feature vector output by the image feature extraction model, and T is the semantic feature vector output by the text feature extraction model. v and W t is a learnable projection matrix that maps visual and textual features into the same joint embedding space, resolving modality differences. v and b t is the bias vector, which adjusts the feature distribution after projection and enhances the flexibility of the model. V′ and T′ are the visual and text features after projection.

[0114] Calculate cross-modal attention weights:

[0115]

[0116] Among them, V′·T ′T It is the similarity matrix between visual features and text features. It calculates the correlation between each pair of visual-text features. If V′ corresponds to the “red” feature of the product image and T′ corresponds to the “red” description in the text, then the similarity score is high. is a scaling factor that prevents the dot product from being too large, which can cause the softmax gradient to vanish. Softmax is a normalized exponential function that converts the similarity matrix into attention weights, with the sum of the weights being 1. A is the cross-modal attention weight matrix.

[0117] Generate aligned fusion features:

[0118] F = concat(V′,A·T′)

[0119] Here, A·T′ is the weighted representation of the text features using the attention weights. The contribution of the text features is dynamically adjusted based on the importance of the image region. If the attention weight of the "bottle label" region in the image is high, the corresponding text feature (such as "500ml") is emphasized. concat is a vector concatenation operation that combines visual features with weighted text features to generate a multimodal joint feature. F is the fused multimodal feature vector.

[0120] Image feature extraction model structure: The image feature extraction model includes cascaded residual convolution blocks, each of which contains a 3×3 convolution layer, a batch normalization layer, and a ReLU activation function. Expressed as: residual convolution block = 3×3 convolution layer + batch normalization layer + ReLU activation function. The 3×3 convolution layer extracts local features (such as edges and textures). Batch normalization standardizes the feature distribution and accelerates model convergence. The ReLU activation function introduces nonlinearity and enhances the model's expressiveness. The residual convolution block alleviates gradient vanishing and enhances the model's ability to handle complex product images (such as occlusion and blur).

[0121] In a preferred embodiment, the text feature extraction model includes:

[0122] Input encoding layer, which generates word vector sequence after segmenting text description;

[0123] The position encoding layer uses the sinusoidal position encoding formula to inject position information:

[0124] Even dimensions (2i):

[0125]

[0126] Odd dimensions (2i+1):

[0127]

[0128] PE(pos,2i) and PE(pos,2i+1) are position encoding functions, representing the encoding values of position pos in the 2ith dimension (even dimension) and the 2i+1th dimension (odd dimension) of the embedding vector, respectively. By alternating the sine and cosine functions, a position representation with periodicity and smoothness is generated. pos is the position index of the current word in the input sequence, and i is the dimension index of the embedding vector. d is the total dimension of the position encoding, which is usually consistent with the dimension of the word embedding. 10000 2i / d Frequency adjustment factor, used to control the wavelength (period length) of different dimensions.

[0129] In one embodiment, a multi-head self-attention layer calculates the correlation between each word vector and the global context. The multi-head self-attention mechanism allows the model to "observe" each word in a sentence from multiple angles. The model analyzes the relationship between each word and other words through multiple independent "observation perspectives" (i.e., multiple "heads"). After each head analyzes independently, the model combines the results to form a more comprehensive semantic understanding. This approach allows the model to capture multiple association patterns between words simultaneously, avoiding the limitations of a single perspective, and thus more accurately understanding the context.

[0130] In one embodiment, the knowledge graph enhancement layer encodes entity relationships in the product knowledge graph as additional feature vectors. The purpose of the knowledge graph enhancement layer is to equip the model with an "industry knowledge base." When the system analyzes product description text (such as "pure cotton round neck T-shirt"), in addition to understanding the literal meaning, it also automatically connects to relevant information in the product knowledge graph. Entity retrieval: Searches for entities such as "pure cotton" (material) and "round neck" (style) in the graph and associates their attributes (such as "pure cotton - good breathability" and "round neck - casual style"). Relationship fusion: Through a graph neural network, the associated information around the entity (such as "pure cotton" is often associated with "summer clothing" and "suitable for sensitive skin") is aggregated into a feature vector. Dynamic enhancement: A gating mechanism is used to automatically adjust the weights of text features and knowledge features. If the word "waterproof" in the text is associated with "waterproof rating of electronic products" in the knowledge base, the weight of this feature is enhanced. This setting allows the model to recognize implicit information, for example, automatically associating "Apple brand" and "new model" with "iPhone" even if the text does not explicitly mention it.

[0131] In one embodiment, the operations of the knowledge graph enhancement layer include:

[0132] Retrieve relevant entity nodes from the pre-built product knowledge graph;

[0133] Aggregate neighborhood node information through graph convolutional networks:

[0134]

[0135] in, represents the hidden state of node i in the l+1 layer, and the node representation is updated by aggregating neighbor information. σ represents the activation function (such as ReLU, Sigmoid, etc.), which introduces nonlinearity to enhance the model's expressiveness. ∑ j∈N(i) It represents the sum of all neighbors j of node i, N(i) is the set of neighbor nodes of node i, and aggregates the information of neighbor nodes, W (l) Represents the trainable weight matrix of the lth layer, performs linear transformation on the features of neighbor nodes, and learns the interaction of features at different levels. Represents the hidden state of node j in layer l, providing the original feature input of the neighbor nodes. |N(i)| represents the number of neighbors of node i and is a normalization factor to avoid numerical instability caused by differences in the number of neighbors.

[0136] The above formula describes the update process of node features in the graph convolutional network (GCN). Aggregate neighbor features: Aggregate the features of all neighbors j of node i at layer l Through the weight matrix W (l)Perform linear transformation and then sum. Normalization: By dividing by the number of neighbors |N(i)|, the influence of the difference in the number of neighbors of different nodes is eliminated. Nonlinear activation: Apply activation function σ to the aggregation result to generate the feature representation of node i at layer l+1 If node i has 3 neighbors, the update process will first perform a linear transformation on the features of each neighbor, sum them up and divide them by 3, and then activate them through ReLU to finally get the new node features.

[0137] The aggregated node embedding is gated and fused with the text features. The gated fusion process is represented by the following two steps: gate weight calculation and feature fusion.

[0138] Gating weight calculation:

[0139] g=sigmoid(W g ·[h text ;h kg ])

[0140] Among them, h text is the text feature vector, the semantic embedding representation generated by the text feature extraction model (such as Transformer), h kg is the node embedding vector after knowledge graph aggregation, which is the embedding representation obtained by aggregating neighborhood information of related nodes in the knowledge graph through the graph convolutional network (GCN). [;] represents the vector splicing operation. text and h kg Splicing in a certain dimension (such as channel or feature dimension) to generate a joint input vector, W g is a learnable weight matrix that maps the concatenated joint vector to a scalar space to generate gating weights; sigmoid is an activation function, and its mathematical expression is:

[0141]

[0142] The output after linear transformation is compressed to the interval [0,1] to represent the gating weight. g is a dynamically generated gating weight scalar that controls the contribution ratio of text features and knowledge graph features during fusion.

[0143] Feature fusion:

[0144] h fused =g·h text +(1-g)·h kg

[0145] Among them, h fused is the fused multimodal feature vector, which is gated by weights h text and h kg Perform weighted summation to achieve dynamic feature fusion.

[0146] By concatenating text features h text and knowledge graph features h kg , using the learnable weight matrix W g The gating weight g is generated by the sigmoid function. The larger the g value, the more the current task depends on the text features; conversely, it depends more on the knowledge graph features. The text features and knowledge graph features are linearly combined according to the gating weights to generate the final fusion feature h fused This fusion approach adaptively balances the information contributions of the two modalities, improving the model's ability to model product classification. The gating mechanism automatically adjusts weights based on input features, eliminating the need for manually setting fixed ratios. Text features provide semantic information, while knowledge graph features supplement structured knowledge. The fusion of the two enhances classification robustness. Gating weights reflect the model's reliance on text or knowledge graphs, facilitating subsequent analysis.

[0147] In a preferred embodiment, the method for generating a visual audit interface includes:

[0148] Locate the region most relevant to the classification decision in the image feature map and calculate the gradient-weighted class activation map (Grad-CAM):

[0149]

[0150] A is the importance weight of the k-th feature map channel to the target category c, which is calculated by back-propagation gradient and indicates the contribution of the channel to the classification decision. k is the feature map of the kth channel output by the convolutional layer. ReLU is the rectified linear unit activation function.

[0151] Extract key terms from text features and calculate term importance scores:

[0152]

[0153] α i,j is the attention weight between the i-th term and the j-th text position, generated by the multi-head self-attention mechanism, indicating the degree of dependence of term i on position j. j is the hidden layer state vector of the jth text position, which comes from the output of the Transformer encoder and contains contextual semantic information. i is the comprehensive importance score of the i-th term.

[0154] Hybrid heatmap generation, spatial alignment operation: image feature map CAM and text term importance score s i Mapping to the same spatial coordinate system, an interactive hybrid heat map is generated, that is, the visual attention area is spatially aligned with the key terms to generate an interactive hybrid heat map.

[0155] In a preferred embodiment, the fine-grained classification process uses a hierarchical classifier, including:

[0156] The first-level classifier performs coarse classification based on brand labels, using a fully connected layer and softmax activation;

[0157] The second-level classifier performs model recognition based on brand classification, using a multi-task learning framework;

[0158] The third-level classifier constructs a metric learning loss function based on product specifications:

[0159]

[0160] Among them, L represents the metric learning loss value of the third-level classifier, which is used to optimize the model to ensure that the feature distances of similar samples are closer and the feature distances of heterogeneous samples are farther. i,j Sum the sample pairs and traverse all anchor-positive sample pairs (f i ,f j + ) and anchor-negative sample pairs (f i ,f j - ), max(0,·) is the hinge loss function, which calculates the loss only when the difference between the positive sample distance and the negative sample distance is less than the boundary threshold m; otherwise, the loss is 0, m is the boundary threshold (a hyperparameter, generally set to a positive number), which controls the minimum interval between positive and negative samples and ensures the separation of the feature spaces of similar and heterogeneous samples, D(·,·) is the distance metric function, which calculates the similarity distance between two feature vectors. Common methods include Euclidean distance or cosine distance. Taking Euclidean distance as an example, D(f i ,f j )=||f i -f j ||2,f i is the feature vector of the anchor sample, the fused feature vector generated by multimodal feature alignment (such as the product specification feature in brand-model-specification classification), f j + is the feature vector of the positive sample belonging to the same category as the anchor sample i, and different product samples of the same specification, f j - The loss function drives the model to learn the following characteristics by comparing the distance between the anchor point and the positive / negative samples. i ,f j +) should be as small as possible. Heterogeneous sample separation: the feature distance D(f i ,f j - ) should be at least m larger than the positive sample distance. Boundary constraint: If the difference between the positive and negative sample distances satisfies D(f i ,f j + )-D(f i ,f j - ) ≥ m, the loss is 0 and optimization stops. Fine-grained classification is suitable for classifying product specifications (such as "500ml" and "1L"), addressing the problem of small intra-class differences and ambiguous inter-class differences. Robustness: A boundary threshold m is used to prevent the model from overfitting to noisy samples. Scalability: Can be combined with multimodal features (images, text, knowledge graphs) to improve classification accuracy.

[0161] In a preferred embodiment, the construction of the product knowledge graph database includes: extracting entity relationship triples from the product description text and using the BiLSTM-CRF model for named entity recognition; establishing a pattern layer to define the product attribute ontology, including the inheritance relationship of brand, manufacturer, and material composition; implementing an incremental update mechanism to automatically expand the graph nodes when new products are put into the warehouse.

[0162] Building a product knowledge graph database involves the following steps: Step 1: Automatically extracting entity relationships. The system uses intelligent text analysis to extract key information from product descriptions. For example, it identifies three elements from the phrase "pure cotton Nike sneakers": entities ("Nike," "pure cotton") and relationships ("made of"). A two-layer neural network (BiLSTM-CRF) is used to automatically annotate key information such as brand and material in the text and establish relationships between them (e.g., "Nike - Production - Sneakers," "Sneakers - Material - Pure Cotton"). Step 2: Building a structured knowledge framework. The extracted information is organized into a tree structure based on business rules, forming a "product attribute dictionary." For example, brands are classified at the manufacturer level (e.g., "Nike - Nike, Inc."). Material composition is categorized by use (e.g., "pure cotton - clothing material," "polyester fiber - luggage material"). This structure allows for a clear classification system for disorganized information, similar to the classification of books in a library. Step 3: Dynamic Expansion and Maintenance. When new products are added to the inventory, the system automatically performs the following operations: New Entity Detection: If an unregistered brand or material (such as "graphene") is found, a node is automatically created. Relationship Verification: Checks whether "graphene" belongs to an existing material category. If not, it is placed in the "New Materials" branch. Conflict Resolution: If the same material has multiple names (such as "polyester fiber" and "polyester"), they are marked as aliases and associated.

[0163] In a preferred embodiment, the multi-scale convolutional neural network comprises:

[0164] The first branch uses a 7×7 convolution kernel to extract global features;

[0165] The second branch uses a 3×3 convolution kernel group to capture local details;

[0166] The third branch uses dilated convolution to expand the receptive field;

[0167] The features of each branch are fused through the channel attention mechanism, including channel attention weight calculation and feature weighted fusion;

[0168] Channel attention weight calculation:

[0169] s=sigmoid(W·[F1;F2;F3])

[0170] Among them, F1, F2, and F3 come from feature maps of different branches (such as global features, local detail features, and large receptive field features). F1 is the global feature extracted by the 7×7 convolution kernel, F2 is the local detail feature extracted by the 3×3 convolution kernel group, and F3 is the feature extracted by expanding the receptive field through the void convolution. [;;] is the feature map spliced along the channel dimension; W is a learnable weight matrix, sigmoid is the activation function, s is the channel attention weight vector, which contains the weight coefficients s1, s2, and s3 of the three branches. Each weight value range is [0, 1]. Feature weighted fusion:

[0171] F fused =s1·F1+s2·F2+s3·F3

[0172] Among them, F fused It is a fused multi-scale feature map, which sums the features of each branch according to the weight coefficient to achieve adaptive feature fusion.

[0173] By splicing the feature maps of different branches, using the learnable weight matrix W and the sigmoid function, a dynamic channel attention weight s is generated. The model automatically learns the importance of different branches. For example, the global feature (F1) may be more important in identifying brands. Local details (F2) are more critical in distinguishing model specifications. The features of the three branches are weighted and summed according to s1, s2, and s3 to generate the final fusion feature F. fused . Combine global, local and multi-scale information to improve classification robustness. Adaptively adjust weights to avoid manually setting fixed fusion ratios. When the brand logo is clear but the model text is blurred in a product image, the weight s1 of F1 (global features) is higher, and the model relies more on brand information. The weight s2 of F2 (local details) is lower because the model text is difficult to recognize. The final classification decision is made by F fused Medium and high weight features dominate to ensure classification accuracy.

[0174] In a preferred embodiment, it also includes an anomaly detection module:

[0175] Calculate the Mahalanobis distance between new product features and existing categories in the knowledge graph:

[0176]

[0177] Among them, D M (x) is the Mahalanobis distance from sample x to the target category distribution, which measures the statistical distance between the new product features and the existing categories in the knowledge graph, taking into account the correlation between features. x is the feature vector of the new product, which is assumed to be d×1, where d is the feature dimension (such as brand, model, specification, etc.), μ is the feature mean vector of the existing categories in the knowledge graph, which is obtained by taking the average of the features of all samples in this category, Σ is the covariance matrix of the existing categories in the knowledge graph, which describes the correlation and variance between features, Σ -1 is the inverse matrix of the covariance matrix,

[0178] Mahalanobis distance measures the difference between new products and existing categories and removes the correlation by ∑ -1 Eliminate the influence of correlation between features, standardization: unify features of different dimensions to the same scale, anomaly detection: if D M If (x) exceeds the threshold, the new product may be an anomaly or a new category. Compared to Euclidean distance, Mahalanobis distance is more suitable for multi-dimensional correlated data. The threshold can be adjusted according to business needs, flexibly controlling the sensitivity of anomaly detection.

[0179] When D M (x) Trigger new category alerts when thresholds are exceeded;

[0180] Construct a generative adversarial network to generate potential abnormal samples and enhance classification robustness.

[0181] In a preferred embodiment, the feature alignment process introduces contrastive learning loss:

[0182]

[0183] Among them, L cont is the contrast loss value, which measures the model's ability to align visual and text features. The smaller the value, the better the cross-modal alignment effect. v is the visual feature vector, which is the feature representation of the product image extracted by the convolutional neural network. + is the positive sample text feature, the embedding representation of the real product description text matching the current visual feature v, t′ is the negative sample text feature, the current training batch Divide by t + All other text features except are used to construct difficult negative samples. is a training batch containing N visual-text pairs, where each visual feature vi Corresponding to the only text The remaining N-1 texts are negative samples; s(v,t) is a similarity function that calculates the degree of match between visual and text features. Common forms include:

[0184] Dot product similarity:

[0185] Cosine similarity:

[0186] Dot product after projection: Where W v ,W t is the learnable projection matrix;

[0187] τ is the temperature coefficient, a hyperparameter with a value range of 0.01≤τ≤1.0. Its functions include adjusting the sharpness of the probability distribution, balancing the magnitude of the gradient, and preventing training instability. Represents a set All negative samples t′ in are summed up, exp(s(v,t′) / τ) maps the similarity score to the positive range, amplifying the influence of high scores, and τ controls the sharpness of the probability distribution.

[0188] The present invention also provides an intelligent classification management system for commodities, comprising:

[0189] a data acquisition unit configured to collect product images and associated text descriptions;

[0190] Visual feature extraction unit, which includes a multi-scale convolutional neural network to generate deep visual features;

[0191] Text feature extraction unit, integrated with Transformer architecture, is used to generate semantic embedding vectors;

[0192] Cross-modal alignment unit, implementing feature projection and attention mechanisms;

[0193] The classification decision unit includes a hierarchical classifier group and outputs fine-grained classification results.

[0194] This system achieves intelligent product classification through multimodal collaborative processing, closely matching the proposed method. The data acquisition unit collects product images (e.g., photos of bottled beverages) and associated text (e.g., the product description "500ml natural mineral water"). The visual feature extraction unit includes a multi-branch structure: The global branch (7×7 convolution) captures the overall product outline (e.g., the shape of the beverage bottle). The local branch (3×3 convolution group) identifies details (e.g., the cap threads and label text). The association branch (atrous convolution) analyzes cross-regional features (e.g., the relationship between the brand logo and the packaging color). Residual connections preserve original features through skip connections, preventing information loss in deep networks. The output is a deep visual feature vector (e.g., encoding "red bottle + round label"). The text feature extraction unit, BERT word embedding, segments text (e.g., "500ml" - "capacity parameter") into words and generates context-dependent vectors. The knowledge graph enhancement connects to the product knowledge base and associates "500ml" with the "standard beverage specifications" node. Multi-head self-attention: Identify the importance of keywords (e.g., "natural" has a higher weight than "mineral water"). Output semantic embedding vectors (e.g., encode "natural mineral water - 500ml - bottled"). Cross-modal alignment unit Joint embedding space projection: Visual features are projected through a matrix, and text features are projected so that the two are in the same space. Cross-modal attention calculation: Calculate the association weights between image regions and text terms (e.g., the label region and "500ml" have the highest attention score). Feature fusion: Concatenate visual features and weighted text features to generate a fusion vector. Classification decision unit Hierarchical classification process: Brand classification: Determine the brand based on fused features (e.g., "Nongfu Spring" vs. "Coca-Cola"). Model identification: Subdivide the model under the brand (e.g., "mineral water" vs. "juice drink"). Specification confirmation: Use metric learning to distinguish subtle differences such as "500ml" and "550ml."

[0195] In a preferred embodiment, it also includes:

[0196] Visual review unit, which generates an interactive interface including heat maps and key terms;

[0197] Knowledge graph management unit, which maintains the product entity relationship database and supports incremental updates;

[0198] Model optimization unit, dynamically adjusts feature alignment parameters based on review feedback;

[0199] The cross-modal alignment unit includes:

[0200] The projection submodule configures a learnable matrix to map visual / text features to a common space;

[0201] Attention calculation submodule, which realizes the calculation of cross-modal feature correlation matrix;

[0202] The feature fusion submodule performs weighted concatenation operations to generate a joint feature vector.

[0203] This visual audit detects anomalies by calculating the Mahalanobis distance between new product features and categories in the knowledge graph. If the distance exceeds a threshold, an audit is triggered. The interactive interface includes a visual heatmap that highlights key image areas (such as bottle labels). Text terms that highlight risk terms (such as "waterproof rating IP68"). Spatial alignment: Clicking "IP68" displays a heatmap of the product's sealing structure. Based on manual corrections, feature alignment parameters and knowledge graph nodes are updated.

[0204] The knowledge graph management unit maintains the product knowledge base and updates it dynamically. It automatically extracts key information (such as brand and material) from the product description, similar to a librarian labeling a new book. It establishes entity associations (such as "Nike-Sneakers-Breathable Mesh") according to the rules to form a tree-like classification system. When new products enter the warehouse, it automatically detects unregistered entities (such as the new brand "Yuanqi Forest"), creates nodes and associates relationships. If a conflict is found (such as "polyester" and "polyester fiber" are the same material), it is marked as an alias for unified management.

[0205] The model optimization unit optimizes the model based on manual review feedback. When a classification error is manually corrected (e.g., changing a misclassified "500ml" to "550ml"), the model automatically downweights the incorrect feature and increases the importance of the correct feature. Incremental learning: This corrected data is incorporated into training, allowing the model to gradually adapt to new products (e.g., recognizing new packaging designs).

[0206] The cross-modal alignment unit and projection submodule "translate" image and text features into the same space. The attention calculation submodule finds the connection points between the image and text (such as the matching degree between the bottle label area and the text "500ml"). The feature fusion submodule combines image and text features to generate a comprehensive judgment basis.

[0207] When a new product enters the inventory, the knowledge graph automatically adds a "graphene material" node and links it to the "new materials" branch. The model detects significant differences between "graphene" and existing materials, triggering a review interface to highlight this feature. After manual confirmation, the model optimization unit adjusts the feature weights, enabling subsequent similar products to be accurately classified. Through collaboration among these units, the system implements a closed loop of automatic learning, manual verification, and continuous optimization, significantly improving classification efficiency and accuracy.

[0208] In a preferred embodiment, the text feature extraction unit includes:

[0209] The word segmentation module uses Jieba word segmentation tool to process Chinese descriptions;

[0210] The word vector layer loads the pre-trained BERT model to generate context-sensitive embeddings;

[0211] The graph neural network processor connects to the product knowledge graph database to obtain enhanced features.

[0212] The text feature extraction unit converts product text descriptions into semantically rich vectors in three steps: The word segmentation module uses Chinese word segmentation tools (such as Jieba) to break the text into independent words. For example, "pure cotton round neck T-shirt" is broken down into "pure cotton," "round neck," and "T-shirt," providing clear semantic units for subsequent processing. The word embedding layer uses a pre-trained BERT model to convert each word into a context-sensitive vector. For example, "pure cotton" not only indicates the material but also dynamically adjusts its meaning based on other terms in the sentence, such as "breathability," to avoid polysemy. The knowledge graph enhancement layer connects to the product knowledge base and combines words with entity relationships in the library. For example, "pure cotton" is associated with attributes in the knowledge base, such as "material - high breathability" and "suitable for summer clothing," to generate an enhanced feature vector that incorporates industry knowledge. The system not only understands literal meaning but also uses the knowledge base to identify implicit information (for example, "iPhone 14" is automatically associated with "Apple brand" and "2022 new model"), significantly improving classification accuracy.

[0213] In a preferred embodiment, the visual audit unit comprises:

[0214] Class activation map generator, which calculates visual attention regions based on gradient backpropagation;

[0215] Term extractor, which analyzes the importance of keywords in text features;

[0216] Hybrid rendering engine that spatially aligns visual attention areas with key terms.

[0217] The visual audit unit generates an interactive audit interface to help humans quickly locate risk points. Class activation map generator: Highlights key areas in the image like a "highlighter". For example, when the system determines that a product belongs to "sports shoes", it automatically marks the sole pattern, brand logo and other areas with a red heat map, intuitively displaying the visual basis that affects the classification decision. Term extractor: Capture core risk words from text descriptions. For example, when analyzing "waterproof grade IP68", it automatically marks "waterproof" and "IP68" as key terms, calculates their importance scores, and displays high-risk words first. Hybrid rendering engine: Links the key points of images and text. For example, when clicking on "IP68" in the text, the corresponding product's sealing structure area in the image is highlighted simultaneously, forming an interactive verification of "text description-image area", which is convenient for auditors to quickly check.

[0218] In a preferred embodiment, the knowledge graph management unit includes: an entity recognizer that uses a BiLSTM-CRF model to extract product attributes; a relational reasoning engine that discovers implicit product associations based on a rule engine; and a version controller that records the graph structure change history and supports rollback.

[0219] The knowledge graph management unit's entity recognizer uses an intelligent model (BiLSTM-CRF) to automatically scan product description text and accurately capture key attributes (such as "Brand: Nike" and "Material: Cotton"). For example, it extracts "cotton" as the material entity from "cotton sneakers." The relational reasoning engine automatically establishes associations between entities based on preset business rules. For example, it finds a "production" relationship between "Nike" and "sneakers," or a "suitable scenario" relationship between "cotton" and "summer clothing." It can also infer implicit relationships based on rules, such as automatically associating "waterproof rating IP68" with the "electronic products" category. The version controller records all modifications to the knowledge base (such as adding new brands and adjusting categories) and supports one-click rollback to historical versions. For example, if "graphene" is mistakenly classified as "metal material," it can be quickly restored to the correct classification, ensuring data traceability. Through the collaborative operation of these three modules, the system dynamically maintains an accurate and structured product knowledge base, providing a reliable data foundation for classification and review.

[0220] In a preferred embodiment, it also includes an edge computing module: a lightweight inference engine that deploys quantized classification models; a local cache database that stores frequently accessed product feature vectors; and a differential update component that regularly synchronizes model parameters with the central server. The edge computing module allows the intelligent classification system to run efficiently on local devices. The lightweight inference engine compresses complex models into a "lite version," just like turning computer software into a mobile phone app. For example, if the product classification model is reduced to 1 / 10 of its original size, products can be identified in real time even on low-end devices. The local cache database automatically stores recently processed product information, and there is no need to query the Internet for the next identification. The differential update component automatically synchronizes the latest data from the cloud every week, but only downloads the "changed parts." For example, if a new "Sakura Cola" category is added, only the update package needs to be downloaded instead of the entire model, which saves traffic and ensures that the device always uses the latest classification rules.

[0221] The intelligent classification management system of the present invention achieves accurate commodity classification through a multimodal collaborative mechanism. The system first collects commodity images and text descriptions through a data acquisition unit. The visual feature extraction unit uses a multi-scale convolutional network to analyze the global contours and local details of the image. The text feature extraction unit combines the Transformer model with the knowledge graph to embed and parse semantic information. The cross-modal alignment unit projects the visual and text features into a common space and dynamically associates key areas (such as the bottle label and the "500ml" text) through an attention mechanism to generate a fused feature vector. The classification decision unit performs hierarchical classification (brand-model-specification) based on the fused features. If the classification confidence is insufficient, the visual review unit generates a heat map and a key term interactive interface to assist manual verification. Review feedback drives the knowledge graph management unit to update entity relationships and optimize the model feature alignment parameters, forming a closed-loop learning process of data input-classification-review-update. The edge computing module ensures efficient local reasoning and realizes real-time classification and incremental synchronization. The modules work together, combining dynamic knowledge enhancement and continuous self-optimization to comprehensively improve the accuracy and intelligence level of commodity classification.

[0222] Although the preferred embodiments of the present invention have been described in detail, those skilled in the art may still make further changes and adjustments to these embodiments after understanding their basic innovative concepts. Therefore, the appended claims are intended to cover these preferred embodiments, as well as all changes and modifications that fall within the scope of the present invention. The above content is only an elaboration of the preferred embodiments of the present invention and is not intended to limit its scope. It should be clear that any modifications, equivalent substitutions or improvements made under the guidance of the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for intelligent classification management of commodities, characterized in that: include: Obtain the target product image to be classified and the associated text description information; Extracting deep visual features of the target product image through an image feature extraction model; Extracting semantic features of the text description information through a text feature extraction model; Performing multimodal feature alignment on the deep visual features and the semantic features to generate a fused feature vector, wherein the alignment process includes: calculating a projection matrix between the visual feature space and the text feature space, and establishing cross-modal feature associations through an attention mechanism; A fine-grained classification result of the product is determined based on the fused feature vector, where the fine-grained classification includes a three-level classification system of brand, model, and specification.

2. The intelligent classification management method for commodities according to claim 1, characterized in that: The text feature extraction model includes: Input encoding layer, which generates word vector sequence after segmenting text description; Position coding layer, which uses the sinusoidal position coding formula to inject position information; Multi-head self-attention layer, which calculates the relevance of each word vector with the global context; The knowledge graph enhancement layer encodes the entity relationships in the product knowledge graph as additional feature vectors.

3. The intelligent classification management method for commodities according to claim 2, characterized in that: The operations of the knowledge graph enhancement layer include: Retrieve relevant entity nodes from the pre-built product knowledge graph; Aggregate neighborhood node information through graph convolutional networks: in, represents the hidden state of node i in the l+1 layer, σ represents the activation function, ∑ j∈N(i) represents the sum of all neighbors j of node i, N(i) is the set of neighbor nodes of node i, W (l) represents the trainable weight matrix of layer l, represents the hidden state of node j in layer l, |N(i)| represents the number of neighbors of node i; The aggregated node embedding is gated and fused with the text features. The gated fusion process is represented by the following two steps: gate weight calculation and feature fusion. Gating weight calculation: g=sigmoid(W g ·[h text ;h kg ]) Among them, h text is the text feature vector, h kg is the node embedding vector after knowledge graph aggregation, [;] represents the vector splicing operation, W g is a learnable weight matrix; sigmoid is the activation function, and g is a dynamically generated gating weight scalar; Feature fusion: h fused =g·h text +(1-g)·h kg Among them, h fused is the fused multimodal feature vector.

4. The intelligent classification management method for commodities according to claim 1, characterized in that: The fine-grained classification process uses a hierarchical classifier, including: The first-level classifier performs coarse classification based on brand labels, using a fully connected layer and softmax activation; The second-level classifier performs model recognition based on brand classification, using a multi-task learning framework; The third-level classifier constructs a metric learning loss function based on product specifications: Where L represents the metric learning loss value of the third-level classifier, ∑ i,j Sum the sample pairs and traverse all anchor-positive sample pairs (f i ,f j + ) and anchor-negative sample pairs (f i ,f j - ), max(0,·) is the hinge loss function, m is the boundary threshold, D(·,·) is the distance metric function, and the similarity distance between two feature vectors is calculated. i is the feature vector of the anchor sample, f j + is the positive sample feature vector belonging to the same category as the anchor sample i, f j - is the feature vector of negative samples belonging to different categories from the anchor sample i.

5. The intelligent classification management method for commodities according to claim 1, characterized in that: The multi-scale convolutional neural network comprises: The first branch uses a 7×7 convolution kernel to extract global features; The second branch uses a 3×3 convolution kernel group to capture local details; The third branch uses dilated convolution to expand the receptive field; The features of each branch are fused through the channel attention mechanism, including channel attention weight calculation and feature weighted fusion; Channel attention weight calculation: s=sigmoid(W·[F1;F2;F3]) Among them, F1, F2, and F3 come from feature maps of different branches. F1 is the global feature extracted by the 7×7 convolution kernel, F2 is the local detail feature extracted by the 3×3 convolution kernel group, and F3 is the feature extracted by expanding the receptive field through the void convolution. [;;] is the feature map spliced along the channel dimension; W is a learnable weight matrix, sigmoid is the activation function, s is the channel attention weight vector, which contains the weight coefficients s1, s2, and s3 of the three branches. The value range of each weight is [0, 1]. Weighted feature fusion: <h2 style=";text-align:left;direction:ltr">F<h2 style=";text-align:left;direction:ltr"> fused <h2 style=";text-align:left;direction:ltr"> =s1·F1+s2·F2+s3·F3 Among them, F fused It is a fused multi-scale feature map, which sums the features of each branch according to the weight coefficient to achieve adaptive feature fusion.

6. The intelligent classification management method for commodities according to claim 1, characterized in that: Also includes anomaly detection modules: Calculate the Mahalanobis distance between new product features and existing categories in the knowledge graph: Among them, D M (x) is the Mahalanobis distance from sample x to the target category distribution, x is the feature vector of the new product, μ is the feature mean vector of the existing categories in the knowledge graph, ∑ is the covariance matrix of the existing categories in the knowledge graph, ∑ -1 is the inverse matrix of the covariance matrix; When D M (x) Trigger new category alerts when thresholds are exceeded; Construct a generative adversarial network to generate potential abnormal samples and enhance classification robustness.

7. A method for intelligent classification management of commodities according to any one of claims 1 to 6, characterized in that: The feature alignment process introduces contrastive learning loss: Among them, L cont is the contrast loss value, which measures the model's ability to align visual and text features. The smaller the value, the better the cross-modal alignment effect. v is the visual feature vector, t + is the positive sample text feature, the embedding representation of the real product description text matching the current visual feature v, t′ is the negative sample text feature, is the training batch; s(v,t) is the similarity function; τ is the temperature coefficient, Represents a set All negative samples t′ in are summed up, exp(s(v,t′) / τ) maps the similarity score to the positive range, amplifying the influence of high scores, and τ controls the sharpness of the probability distribution.

8. An intelligent classification management system for commodities, characterized in that: include: a data acquisition unit configured to collect product images and associated text descriptions; Visual feature extraction unit, which includes a multi-scale convolutional neural network to generate deep visual features; Text feature extraction unit, integrated with Transformer architecture, is used to generate semantic embedding vectors; Cross-modal alignment unit, implementing feature projection and attention mechanisms; The classification decision unit includes a hierarchical classifier group and outputs fine-grained classification results.

9. The intelligent classification management system for commodities according to claim 8, characterized in that: Also includes: Visual review unit, which generates an interactive interface including heat maps and key terms; Knowledge graph management unit, which maintains the product entity relationship database and supports incremental updates; Model optimization unit, dynamically adjusts feature alignment parameters based on review feedback; The cross-modal alignment unit includes: The projection submodule configures a learnable matrix to map visual / text features to a common space; Attention calculation submodule, which realizes the calculation of cross-modal feature correlation matrix; The feature fusion submodule performs weighted concatenation operations to generate a joint feature vector.

10. An intelligent classification management system for commodities according to claim 8 or 9, characterized in that: The text feature extraction unit includes: The word segmentation module uses Jieba word segmentation tool to process Chinese descriptions; The word vector layer loads the pre-trained BERT model to generate context-sensitive embeddings; The graph neural network processor connects to the product knowledge graph database to obtain enhanced features.

Images