A commodity same-model identification method and system based on a large language model

Abstract

The present application belongs to the technical field of commodity information processing, and particularly relates to a commodity same-model identification method and system based on a large language model. The method obtains text information and image information of a to-be-built library commodity and a to-be-searched commodity, and after processing such as field segmentation, unit of measurement normalization, and abnormal character filtering, constructs a field text sequence and an image block sequence; then, in combination with field weights and field confidence masks, cross-modal alignment is performed on field text vectors, main image global vectors, and detail image local vectors to obtain fusion feature vectors and attribute sub-vectors; subsequently, same-model determination is realized through candidate rough screening, attribute difference recheck, and critical sample re-trial, and the model is updated through misjudgment sample back-feeding. The scheme can reduce the interference of noise fields and abnormal data on the identification result, and enhance the ability to distinguish commodities that are similar in appearance but different in model, specifications, or packaging quantity.

Description

Technical Field

[0001] This invention belongs to the technical field of commodity information processing, specifically relating to a method and system for identifying identical products based on a large language model. Background Technology

[0002] With the rapid development of e-commerce platforms, social media marketing channels, and cross-platform transaction scenarios, the forms of product information display are becoming increasingly diverse. There is often significant heterogeneity among product titles, attribute descriptions, main images, detail page text, and brand logos. In the circulation scenarios of apparel, shoes, bags, cosmetics, home furnishings, and appliances, the same product may be published under different names, specifications, or display templates. Therefore, the need for identifying consistency, correspondence, and relevance between products is constantly increasing. Especially in business processes such as product aggregation display, duplicate link management, cross-platform price comparison, supply chain management, content review, and intelligent recommendation, product matching identification has become one of the important foundational technologies supporting refined operations and data governance.

[0003] In existing technologies, the processing routes for identifying identical products typically include the following categories: one is to perform rule comparison or field matching based on structured fields such as product title, brand, model, specifications, and barcode information; another is to perform image similarity analysis based on the main image, detail images, and layout features of the product; and yet another is to fuse text features, image features, and some transaction attributes, and output the degree of similarity between product pairs through machine learning or deep learning models.

[0004] The aforementioned technical approach has a certain application foundation in scenarios involving standardized products, products with complete fields, or products with minimal image differences. However, in practical applications, product titles often contain abbreviations, aliases, mixed marketing terms, missing attributes, and inconsistent order of descriptions. Images may also be affected by factors such as shooting angle, lighting and background, cropping methods, watermark occlusion, and post-processing, making it difficult for single-modal features to stably represent the product itself. Furthermore, differences in category systems, attribute definitions, and product description habits across different platforms further increase the complexity of identifying identical products, leaving room for improvement in the accuracy and generalization ability of existing technologies under cross-platform, cross-merchant, and complex expression conditions.

[0005] Therefore, it is necessary to propose a product matching technology solution that is more suitable for the actual e-commerce environment, in order to meet the growing demand for refined product management and intelligent recognition, especially for application scenarios where product information is complex, cross-modal features are scattered, and the relationship between similar products is implicit. Summary of the Invention

[0006] To address the above problems, the present invention aims to propose a method for identifying identical products based on a large language model, comprising the following steps: S1. Standardization processing of multi-source commodity data: Obtain text and image information of commodities to be added to the database and commodities to be retrieved, and perform preprocessing to obtain field text sequences and image block sequences; S2. Constructing a text-image fusion vector with field credibility constraints: Encode the field text sequence and image block sequence respectively to obtain field text vectors, main image global vectors, and detail image local vectors. Assign field weights based on the distinguishability of brand, model, specification, material, color, and title fields. Generate field credibility masks based on field missing, abnormal characters, promotional words, packaging quantity conflicts, and abnormal field lengths. Perform bit-by-bit decay or zeroing processing on the field text vectors. Then input the processed field text vectors, main image global vectors, and detail image local vectors into a multimodal pre-trained model to perform cross-modal alignment and output fused feature vectors and attribute sub-vectors. S3. Construct a model for determining the same product: Use known pairs of the same product and pairs of different products to construct a training set. The pairs of different products include at least some difficult-to-distinguish samples that are similar in appearance but different in model, specifications or packaging quantity under the same category. Perform joint training of classification loss and metric loss on the fused feature vector to obtain the model for determining the same product. S4. Perform candidate coarse screening: Generate a fusion feature vector to be retrieved for the product to be retrieved, perform cosine similarity retrieval in the feature vector library and select the top N products as initial candidates, then calculate the Euclidean distance, and retain products with cosine similarity higher than the first threshold and Euclidean distance lower than the second threshold as candidate products. S5. Perform attribute difference verification: Construct attribute difference vectors based on the brand, model, specifications, material and color of the candidate product and the product to be retrieved. When the weighted norm of the attribute difference vector is not higher than the third threshold and at least two fields in the brand field, model field and specification field meet the consistency condition, it is determined to be the same product. For candidate products located in the critical interval, the multimodal pre-trained model is called to perform secondary semantic verification. S6. Perform error sample reinjection and update: Write the identification results into the sample library, and write highly similar but re-judged as different samples into the difficult-to-distinguish sample library. When the preset sample quantity or preset period condition is met, the same sample determination model is updated by mixing new samples and historical samples.

[0007] As a preferred technical solution, in step S1, the text information and image information include: title, brand, model, specifications, material, color, packaging quantity, and packaging description.

[0008] As a preferred technical solution, the preprocessing includes: field segmentation, unit of measurement normalization, synonym mapping, abnormal character removal and promotional word filtering, and size normalization, background suppression, main area extraction and local identifier area cropping for the main image and detail images.

[0009] As a preferred technical solution, in step S2, the field credibility mask is generated according to the proportion of valid characters, the proportion of abnormal characters, the proportion of promotional words, the result of unit of measurement normalization, and the consistency of packaging quantity. The mask value corresponding to missing fields is zero, the mask value corresponding to fields containing abnormal characters or promotional words but retaining valid information is 0.2 to 0.8, and the mask value corresponding to fields that meet the preset format after standardization is 1.

[0010] As a preferred technical solution, in step S2, the field weights of the brand field, model field, and specification field are all higher than the field weights of the color field and title field, and the weight of the model field is not lower than the weight of the specification field; before the field text vector is fused with the image vector, it is first multiplied bit by bit with the corresponding field credibility mask, and then a weighted sum is performed according to the field weight.

[0011] As a preferred technical solution, in step S2, the global vector of the main image represents the overall outline, overall color scheme and packaging shape of the product, and the local vector of the detail image represents the label text, interface structure, local texture, printing mark or assembly details; the local vector of the detail image is formed by splicing together at least two local mark areas after they are encoded separately.

[0012] As a preferred technical solution, in step S3, the loss measurement adopts triplet loss or contrast loss. During training, product pairs with similar text descriptions and similar main image appearances but different model fields, specification fields or packaging quantities are selected from the same category of products as difficult-to-distinguish samples, so as to constrain the vector distance between the same sample pairs to be less than the vector distance between difficult-to-distinguish sample pairs.

[0013] As a preferred technical solution, in step S4, the initial number of candidates N is 10 to 200; the initial candidate products are first sorted in descending order of cosine similarity, and then a second screening is performed based on Euclidean distance to screen out candidate products whose overall vector directions are similar and whose vector magnitude differences are greater than a preset value from the review stage.

[0014] As a preferred technical solution, in step S5, the attribute difference vector includes at least brand difference component, model difference component, specification difference component, material difference component and color difference component, wherein the weight of model difference component and specification difference component is higher than that of color difference component.

[0015] As a preferred technical solution, when the brand difference component exceeds a preset brand threshold, or the model difference component exceeds a preset model threshold, the candidate product is directly determined to be a different product.

[0016] The present invention also provides a product same-item recognition system based on a large language model, including a processor, a memory, an image preprocessing unit, a text standardization unit, a fusion encoding unit, a feature vector library, a candidate screening unit, an attribute verification unit, and a model update unit.

[0017] As a preferred technical solution, the memory stores program instructions executable by the processor; the image preprocessing unit is used to perform size normalization, background suppression, main area extraction, and local identifier area cropping on the main product image and detail images; the text normalization unit is used to perform field segmentation, unit of measurement normalization, synonym mapping, and abnormal character filtering on the product title, brand, model, specifications, material, color, packaging quantity, and packaging description; the fusion encoding unit is used to generate field text vectors, main image global vectors, and detail image local vectors, and output fused feature vectors and attribute sub-vectors based on field weights and field confidence masks.

[0018] As a preferred technical solution, the feature vector library stores the fused feature vectors of the products to be built into the library, the candidate screening unit outputs a set of candidate products based on cosine similarity and Euclidean distance, the attribute verification unit outputs the same product judgment result based on the attribute difference vector and field consistency condition, and the model update unit performs model update based on the newly added same product samples and difficult-to-distinguish samples.

[0019] As a preferred technical solution, the fusion encoding unit includes a field weight table, a field credibility calculation subunit, and a cross-modal alignment subunit. The field weight table stores at least the weight parameters corresponding to the brand field, model field, specification field, material field, color field, and title field. The field credibility calculation subunit generates a field credibility mask based on the field missing status, abnormal character ratio, promotional word ratio, unit of measurement normalization result, and packaging quantity consistency. The cross-modal alignment subunit inputs the masked field text vector, the main image global vector, and the detail image local vector into the multimodal pre-trained model and outputs a fused feature vector and attribute sub-vector.

[0020] As a preferred technical solution, the attribute verification unit includes a difference vector calculation subunit, a verification judgment subunit, and a critical interval verification subunit. The difference vector calculation subunit constructs an attribute difference vector composed of brand difference components, model difference components, specification difference components, material difference components, and color difference components. The verification judgment subunit outputs the same product judgment result based on the weighted norm of the attribute difference vector and the field consistency condition.

[0021] As a preferred technical solution, the critical interval re-judgment subunit only calls the multimodal pre-trained model to perform secondary semantic re-judgment for candidate products whose cosine similarity is within a preset critical interval and whose attribute difference vector does not exceed the negation threshold; the model update unit includes a difficult sample construction subunit and a retraining subunit. The difficult sample construction subunit collects product pairs that are highly similar but re-judged as different, and the retraining subunit performs model update when a preset number of samples or a preset period condition is met.

[0022] Beneficial effects This invention introduces a field credibility mask during the image-text fusion process and incorporates missing fields, abnormal characters, promotional word interference, packaging quantity conflicts, and abnormal field lengths into a unified constraint. This ensures that product text fields from different sources and of different quality are hierarchically suppressed before entering the fusion stage. Therefore, no longer treating noisy fields with valid fields equally reduces the pulling effect of dirty data on the fusion vector, allowing key fields such as brand, model, and specifications to play a more focused role in identifying similar products, thereby improving the stability of similar product representation in complex product data environments.

[0023] This invention does not rely solely on single image or text features for product identification. Instead, it collaboratively constructs fused features using field text vectors, global vectors from the main image, and local vectors from the detail images, and further combines attribute sub-vectors for constraint-based judgment. In particular, by incorporating the overall outline, color scheme, and packaging shape of the main image with label text, interface structure, printing markings, and assembly details from the detail images into the identification criteria, a more granular differentiation mechanism can be established between products that are highly similar in appearance but differ in details, thereby enhancing the ability to identify "similar appearance but different products" scenarios.

[0024] This invention incorporates attribute difference verification after candidate screening and reintroduces highly similar samples that are reclassified as non-identical to a difficult-to-distinguish sample library. These samples are then mixed with historical samples to update the same-item determination model, forming a closed-loop optimization path for scenarios prone to misjudgment. This technical solution moves beyond ordinary incremental training in model updates, focusing instead on continuously strengthening the discrimination boundaries around easily confused samples such as different models, specifications, and packaging quantities. This allows the identification rules to gradually align with the real confusion relationships within similar products, improving the specificity and transferability of subsequent same-item determination results. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0026] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0027] Example 1 In existing e-commerce product retrieval and merging scenarios, common methods for identifying identical products mainly fall into two categories: one is to perform rule-based comparison or text similarity calculation solely based on text fields such as product title, brand, and specifications; the other is to perform nearest neighbor retrieval solely based on image feature extraction from the product's main image. While the former method is simple to implement, it is easily affected by factors such as title stuffing, promotional word interference, inconsistent model abbreviations, and inconsistent unit writing. Although the latter method can utilize appearance information to some extent, it often lacks sufficient differentiation capabilities for scenarios involving packaging changes, changes in shooting angles, differences in local details, and products that are "similar in appearance but not identical."

[0028] Based on this, this embodiment adopts a joint modeling approach using textual and image information, and conducts collaborative design in aspects such as field credibility constraints, image-text fusion representation, candidate screening, attribute verification, and misjudgment feedback updates, in order to improve the stability and interpretability of the product same-item recognition results.

[0029] This embodiment uses small home appliances from an e-commerce platform as an example. The products to be added to the database come from the historical product database, while the products to be retrieved come from the newly listed product stream. Product data mainly includes title, brand, model, specifications, material, color, packaging quantity, packaging description, as well as the main product image and detailed images. Because different merchants have significantly different data entry habits when publishing products, the same product may exhibit variations such as "full model name," "abbreviated model name," "mixed use of specifications and units," "title containing promotional language," and "different number of detailed images." Therefore, it is necessary to standardize the multi-source product information before performing subsequent same-product identification. (Reference) Figure 1 As shown, the method steps in this embodiment include: S1. Standardization processing of multi-source product data: First, the text and image information of each product in the product set to be built and the product set to be retrieved are obtained. The text information includes title, brand, model, specifications, material, color, packaging quantity, and packaging description; the image information includes one main image and at least two detail images. For the text part, field segmentation is performed first to split the brand information, specification information, model information, and promotional phrases in the title; then, unit of measurement is normalized, for example, "500 ml", "500mL", and "0.5L" are uniformly converted to standard unit expressions, and "2 pieces", "2 boxes", and "×2" are uniformly mapped to the packaging quantity field; then, synonym mapping is performed, for example, "space gray", "grayish black", and "graphite gray" are uniformly mapped to the standard expressions in the preset color dictionary; then, abnormal characters are removed from fields containing garbled characters, special emoticons, repeated symbols, and characters without actual identification meaning; finally, promotional words such as "limited-time offer", "buy one get one free", "official genuine product", and "hot-selling recommendation" are filtered. The purpose of this approach is not simply to delete text, but to retain as much information as possible that is relevant to the product itself, while suppressing marketing descriptions that are not directly related to the product's identity.

[0030] For the image portion, the main image and detail images are first normalized to ensure the input images meet the uniform size requirements of the subsequent encoding network. Then, background suppression is used to reduce the interference of complex backgrounds, shop window backgrounds, or contextualized backgrounds on product recognition. Next, main body region extraction is performed, preserving as much of the product's outline, packaging front, and key structural areas as possible. For detail images, local identifier region cropping is further performed, extracting distinguishable local areas such as label text areas, interface structure areas, printed marking areas, and packaging corner areas. Compared to existing technologies that only use the main image thumbnail for direct modeling, this step additionally preserves local identifier information in the detail images, enabling the subsequent model to not only see "whether it looks similar" but also "whether the local structure and label are consistent." After the above processing, a field text sequence and an image block sequence are obtained, serving as input for the subsequent image-text fusion step.

[0031] S2. Construct a text-image fusion vector with field credibility constraints: After obtaining the standardized field text sequence and image block sequence, both are encoded separately. For the text portion, the brand, model, specification, material, color, and title fields are input into the text encoder to obtain the corresponding field text vectors. For the image portion, the main image is input into the global image encoder to obtain the main image global vector, which is used to represent the overall outline, color scheme, and packaging shape of the product. Two or more local identifier areas are input into the local image encoder to obtain multiple local representations, which are then stitched together to form the detail image local vector, used to represent label text, interface structure, local texture, printed markings, and assembly details. The reason for this processing is that the main image is more suitable for reflecting the overall appearance, while the detail image is more suitable for reflecting the fine-grained differences between the same and different products; both are indispensable.

[0032] Next, field weights and field credibility masks are introduced for each text field. Field weights reflect the differentiating effects of various fields in identifying similar products. For example, in this embodiment, brand, model, and specification fields are generally more representative of product identity than color and title fields; therefore, brand, model, and specification fields are assigned higher weights, with the model field weight not lower than the specification field weight. The field credibility mask describes the reliability of the field content itself. For example, when a product's model field is missing, the corresponding mask value is set to zero; when the title contains many promotional terms and abnormal characters, but still retains some valid product information, the corresponding mask value can be set between 0.2 and 0.8; when the field, after standardization, has a regular format, clear information, and no conflicting packaging quantities, the corresponding mask value is set to 1. Then, each field's text vector is multiplied bit-by-bit by its corresponding field credibility mask. Fields with low credibility are attenuated, and invalid fields are cleared to zero. Finally, a weighted sum is calculated according to the field weights to obtain the processed aggregated representation of the fields.

[0033] Building upon this foundation, the processed field text vectors, along with the global vector of the main image and the local vectors of the detail images, are input into a multimodal pre-trained model. Cross-modal alignment is then performed, outputting a fused feature vector and attribute sub-vectors. The fused feature vector primarily represents the overall multimodal identity features of the product, while the attribute sub-vectors preserve differences in attributes such as brand, model, specifications, material, and color. Compared to existing techniques that simply add text similarity and image similarity, this step, by applying credibility constraints before performing image-text joint encoding, mitigates the impact of dirty data fields on the representation results and allows image and text information to mutually correct each other within the same representation space. For example, when the main images are very similar but the model field differs significantly after cleaning, the attribute sub-vectors can retain this difference; conversely, when the title writing differs significantly but the main image and detail image are highly consistent, the fused feature vectors can compensate for the impact of inconsistent textual expression.

[0034] S3. Construct a model for determining if items are identical: During the model training phase, a training set is constructed using known pairs of identical and dissimilar products. Pairs of identical products can come from data on the same product being listed in different stores at different times. In addition to ordinary samples of different products, dissimilar products also include hard-to-distinguish samples. Hard-to-distinguish samples refer to product pairs within the same category, with similar main images and text descriptions, but which are actually not identical due to differences in model number, specifications, or packaging quantity. For example, "standard version" and "gift box version" electric toothbrushes from the same brand, "single pack" and "double pack" with the same appearance, or two products whose model numbers differ by only one suffix letter can all be considered hard-to-distinguish samples. The significance of this approach is that if the training set only contains "obviously different" dissimilar samples, the model can easily learn coarse-grained differentiation, but in real-world scenarios, it may still misclassify products that "look very similar."

[0035] During training, the fused feature vectors are jointly trained using both classification loss and metric loss. The classification loss learns the binary boundary for "whether they are the same product," while the metric loss constrains the vector distance between similar product pairs to be closer, and the vector distance between difficult-to-distinguish dissimilar product pairs to be farther. The metric loss can be either triplet loss or contrastive loss. In this embodiment, to enhance the ability to distinguish between difficult-to-distinguish scenarios, product pairs with similar text descriptions and main image appearances, but different model numbers, specifications, or packaging quantities, are preferably selected from the same category as key training samples. This allows the model to gradually learn that similar appearances do not necessarily mean they are the same product; key attributes such as model number and specifications still play a decisive role. After multiple rounds of training, a product-matching model is obtained, and the fused feature vectors of the products to be added to the feature vector library are written into the feature vector database.

[0036] S4. Perform preliminary candidate screening: When a new product arrives for retrieval, a fused feature vector for that product is first generated according to steps S1 and S2. Then, a nearest neighbor search is performed in the feature vector database, selecting the top N products as initial candidates based on cosine similarity from highest to lowest, where N can be set between 10 and 200. In this embodiment, to balance recall and computational cost, N is set to 80. Using cosine similarity for the first round of retrieval is primarily to quickly identify candidate vectors that are directionally close, which helps reduce search overhead in a large-scale product database.

[0037] However, relying solely on cosine similarity may still retain some candidate samples that are "similar in direction but differ significantly in amplitude." Therefore, after obtaining the initial candidates, the Euclidean distance between each candidate and the fused feature vector to be retrieved is calculated, and products with cosine similarity higher than the first threshold and Euclidean distance lower than the second threshold are retained as candidate products. In other words, this embodiment does not only consider whether the vector directions are similar, but also further examines the actual proximity of the vectors in the overall space. Compared with the existing technology that only performs single-threshold nearest neighbor search, this two-layer coarse screening method of "cosine similarity + Euclidean distance" can eliminate some products that are similar in appearance but have large deviations in overall characterization while maintaining recall, thus reducing the amount of invalid computation in the subsequent review stage.

[0038] S5. Perform attribute difference review: After initial screening, the remaining candidate products typically exhibit high similarity to the product being searched. However, they cannot be directly identified as identical products at this stage, as many "similar appearance but different styles" samples are concentrated in this phase in actual business operations. Therefore, this embodiment further constructs an attribute difference vector based on the brand, model, specifications, material, and color of the candidate products and the product being searched. The attribute difference vector includes at least brand difference components, model difference components, specification difference components, material difference components, and color difference components, with model difference and specification difference components having higher weights than color difference components. This is because color differences do not necessarily lead to different styles in certain categories, but model and specifications often directly determine the specific identity of the product.

[0039] Next, a weighted norm is calculated for the attribute difference vector. If the weighted norm of the attribute difference vector is not higher than the third threshold, and at least two of the brand, model, and specification fields meet the consistency condition, the products are considered the same. For example, if a product to be searched and a candidate product have the same brand, the model number differs only in capitalization after standardization, and the specification units are different in spelling but have the same converted value, then they can be considered the same product. Conversely, if the brand difference component exceeds a preset brand threshold, or the model difference component exceeds a preset model threshold, the candidate product is directly determined to be a different product to avoid misjudgment due to over-reliance on appearance similarity.

[0040] For candidate products located in the critical interval—that is, samples with high cosine similarity, low Euclidean distance, but whose attribute difference vectors are near the boundary—the multimodal pre-trained model is invoked to perform a secondary semantic re-judgment. The purpose of this secondary re-judgment is not to repeat the previous step, but rather to refocus on the local correspondence between text and image for boundary samples. For example, it checks whether the main image corresponds to the same packaging shape, whether the label text in the detail image matches the model number field, and whether the packaging quantity matches the combination form in the image. This further improves the processing accuracy of boundary samples.

[0041] S6. Perform error sample reinjection and update: After obtaining the final identification results, the similarity judgment results are written into the sample library. For product pairs that show high similarity in the initial candidate screening stage but are judged to be different after attribute difference verification or secondary semantic re-judgment, they are written into the difficult-to-distinguish sample library. These samples are of high value because they precisely represent the scenarios where the system is most prone to errors, such as different brands with different specifications, different models with the same packaging, or products from different generations with highly consistent appearances. If these samples are not fed back in for a long time, even if the model performs stably on ordinary samples, it may still repeatedly make the same mistakes in real business.

[0042] Therefore, in this embodiment, when the preset sample quantity or preset period condition is met, the same-product determination model is updated by mixing new samples with historical samples. Specifically, confirmed same-product samples, ordinary different-product samples, and difficult-to-distinguish samples can be periodically extracted from the sample library and combined into a new training set according to a preset ratio, and then the same-product determination model is updated and trained. The purpose of this process is to enable the model not only to remember general same-product features, but also to continuously strengthen the recognition boundary for scenarios with high misjudgment rates. Compared with the existing approach of "training the model once and using it in a fixed way for a long time", this step constructs a closed-loop update mechanism for misjudgment scenarios, enabling the same-product recognition model to be continuously adjusted with changes in product style, packaging evolution, and changes in platform data quality, making it more suitable for long-term deployment in actual business environments.

[0043] Through steps S1 to S6 described above, this embodiment achieves a complete process for identifying identical products, from inputting the original product text, main image, and detail images, to image-text fusion representation, candidate coarse screening, attribute difference verification, and error correction and updating. This process does not rely solely on title rule matching or main image nearest neighbor search. Instead, it improves the ability to identify identical product relationships in complex product scenarios and enhances the stability of results through techniques such as field credibility constraints, collaborative representation of the main image global vector and detail image local vectors, dual judgment using fused feature vectors and attribute difference vectors, and closed-loop updating of difficult-to-distinguish samples.

[0044] Example 2 This embodiment provides a product matching recognition system based on a multimodal pre-trained model, such as... Figure 2 As shown, it includes a processor, memory, image preprocessing unit, text normalization unit, fusion encoding unit, feature vector library, candidate selection unit, attribute verification unit, and model update unit.

[0045] The memory stores program instructions executable by the processor. When the processor executes these instructions, it controls the various units to collaboratively complete the processes of receiving product image and text data, standardization processing, fusion encoding, candidate retrieval, attribute verification, and model update, thereby outputting a result indicating whether the products are identical. This system can be deployed on the product governance server, retrieval service node, product data platform, or cloud-based recognition platform of an e-commerce platform for identifying identical products between products to be added to the database and products to be retrieved.

[0046] The image preprocessing unit receives the main image and detail images of the product and performs size normalization, background suppression, main region extraction, and local marker region cropping on the input images. Size normalization adjusts images with different resolutions and aspect ratios to a unified input format; background suppression reduces the interference of scene background, decorative background, and shooting environment differences on product recognition; main region extraction preserves the product itself, packaging main body, or main structural areas; and local marker region cropping extracts label text areas, interface structure areas, printed marking areas, packaging corner areas, or local texture areas from the detail images. After processing by this unit, an image block sequence is formed and output to the fusion encoding unit. Compared with common processing methods that only use the main image for recognition, this structure simultaneously preserves overall appearance information and local detail information, making it easier to distinguish between products that look similar but are not the same.

[0047] The text standardization unit is used to receive text information such as product title, brand, model, specifications, material, color, packaging quantity and packaging description, and to perform field segmentation, unit of measurement normalization, synonym mapping, abnormal character filtering and promotional word filtering.

[0048] Field segmentation separates brand, specifications, model, quantity, and marketing terms mixed in the title into corresponding fields; unit of measurement normalization converts different ways of writing capacity, weight, length, and quantity into a unified expression; synonym mapping converts similar descriptions used by different merchants into standard terms; abnormal character filtering removes garbled characters, duplicate punctuation, and meaningless symbols; and promotional word filtering weakens information such as "free shipping," "special offer," and "limited-time offer" that do not directly contribute to product identification. After processing by this unit, the field text sequence is obtained and output to the fusion encoding unit. Through the above processing, the system can reduce the impact of non-standard product text expression on the identification of similar products.

[0049] The fusion coding unit is connected to the image preprocessing unit and the text normalization unit, respectively, and is used to receive field text sequences and image patch sequences to generate fused feature vectors and attribute sub-vectors. This fusion coding unit includes a field weight table, a field credibility calculation sub-unit, and a cross-modal alignment sub-unit. The field weight table stores the weight parameters corresponding to the brand field, model field, specification field, material field, color field, and title field, reflecting the differentiating effects of different fields in identifying similar products. The field credibility calculation sub-unit generates a field credibility mask based on the field's missing state, the proportion of abnormal characters, the proportion of promotional words, the unit of measurement normalization result, and the consistency of packaging quantity. When a field is missing or obviously invalid, the corresponding mask value is reduced to zero; when a field contains some noise but still retains valid content, the corresponding mask value is set to a decay value; when the field format is complete, the semantics are clear, and there are no conflicts, the corresponding mask value is set to a higher value.

[0050] In the specific processing, the fusion encoding unit first encodes the field text sequence to obtain field text vectors; it then encodes the main image to obtain a global vector, which represents the overall outline, color scheme, and packaging shape of the product; and it encodes and concatenates the local identifier areas in the detail image to obtain local vectors, which represent label text, interface structure, local textures, printed markings, and assembly details. Next, the field text vectors are multiplied bit-by-bit by the field confidence mask, and low-confidence fields are attenuated before being weighted and aggregated according to the field weight table. The cross-modal alignment subunit further inputs the processed field text representations, along with the main image global vector and detail image local vectors, into a multimodal pre-trained model to perform cross-modal alignment, outputting a fused feature vector and attribute sub-vectors. The fused feature vector describes the overall image-text joint representation of the product, while the attribute sub-vectors preserve differences in attributes such as brand, model, specifications, material, and color. This structure avoids the information imbalance problem caused by simple concatenation of text and images.

[0051] The feature vector library is used to store the fused feature vectors corresponding to the products to be added to the database and supports subsequent retrieval. For products entering the database creation process, after completing the fused encoding, the system writes the product identification information and the corresponding fused feature vector into the feature vector library. The feature vector library can use a vector index storage method to support fast nearest neighbor retrieval in a large-scale product environment. By setting up this feature vector library, the system does not need to perform repeated encoding and one-by-one comparison for all historical products each time it is identified, thereby improving the overall processing efficiency.

[0052] The candidate filtering unit is connected to the feature vector library and receives the fused feature vectors of the products to be retrieved. It then filters the candidate product set from the feature vector library based on cosine similarity and Euclidean distance. The candidate filtering unit first performs a cosine similarity search, selecting the top N products with high similarity as initial candidates for rapid recall. Then, it calculates the Euclidean distance between each initial candidate and the product to be retrieved, retaining products with cosine similarity higher than a first threshold and Euclidean distance lower than a second threshold as candidate products. Through this two-layer filtering method, the system retains the efficiency advantage of nearest neighbor retrieval while eliminating samples that are similar in overall direction but have significant differences in their comprehensive characteristics, reducing the processing burden on the subsequent attribute verification unit.

[0053] The attribute verification unit is used to output a matching result based on the attribute difference vector and field consistency conditions between the candidate product and the product to be retrieved. This attribute verification unit includes a difference vector calculation subunit, a verification judgment subunit, and a critical interval verification subunit. The difference vector calculation subunit constructs an attribute difference vector composed of brand difference components, model difference components, specification difference components, material difference components, and color difference components. The verification judgment subunit outputs a matching result based on the weighted norm of the attribute difference vector and the field consistency conditions. The critical interval verification subunit calls a multimodal pre-trained model to perform a secondary semantic verification for candidate products whose cosine similarity is within a preset critical interval and whose attribute difference vector does not exceed a negation threshold. Specifically, when the weighted norm of the attribute difference vector is not higher than a third threshold, and at least two of the brand, model, and specification fields meet the consistency conditions, a matching result is output; when the brand difference component exceeds a preset brand threshold, or the model difference component exceeds a preset model threshold, a non-matching result is directly output. Through this structure, the system further introduces attribute-level verification logic on the basis of overall vector similarity, which is helpful in distinguishing products of the same brand but different specifications, products of the same appearance but different models, or products of the same packaging but different quantities.

[0054] The model update unit is connected to the attribute verification unit and is used to perform model updates based on newly added samples and hard-to-classify samples. The model update unit includes a hard-to-classify sample construction subunit and a retraining subunit.

[0055] The difficult-to-distinguish sample construction subunit is used to collect product pairs that show high similarity during the candidate screening stage but are judged as dissimilar after attribute verification or critical interval re-judgment, and write them into the difficult-to-distinguish sample library. The retraining subunit is used to retrieve newly added same-product samples, dissimilar samples, difficult-to-distinguish samples, and historical samples to form an updated training set when a preset number of samples or a preset period condition is met, and to perform updated training on the same-product judgment model. Through this update mechanism, the system can continuously feed back misjudgment scenarios exposed in actual business operations into the training process, so that the model's judgment boundary for complex product relationships can be continuously corrected.

[0056] During system operation, the products to be retrieved first enter the image preprocessing unit and text standardization unit to complete the normalization of image and text data. Then, the fusion encoding unit generates fusion feature vectors and attribute sub-vectors. The candidate screening unit recalls candidate products from the feature vector library. The attribute verification unit outputs the final judgment result based on the attribute difference vector and field consistency conditions. The model update unit organizes and re-injects new samples generated during the recognition process. For products to be added to the database, the system writes their fusion feature vectors into the feature vector library after completing fusion encoding to expand the subsequent retrieval scope. Thus, the system forms a complete processing chain from original product data input, unified image and text encoding, candidate recall, attribute verification to model update, suitable for application scenarios such as product merging, duplicate product identification, similar product retrieval, and product management.

[0057] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying identical products based on a large language model, characterized in that, Includes the following steps: S1. Standardization processing of multi-source commodity data: Obtain text and image information of commodities to be added to the database and commodities to be retrieved, and perform preprocessing to obtain field text sequences and image block sequences; S2. Constructing a text-image fusion vector with field credibility constraints: Encode the field text sequence and image block sequence respectively to obtain field text vectors, main image global vectors, and detail image local vectors. Assign field weights based on the distinguishability of brand, model, specification, material, color, and title fields. Generate field credibility masks based on field missing, abnormal characters, promotional words, packaging quantity conflicts, and abnormal field lengths. Perform bit-by-bit decay or zeroing processing on the field text vectors. Then input the processed field text vectors, main image global vectors, and detail image local vectors into a multimodal pre-trained model to perform cross-modal alignment and output fused feature vectors and attribute sub-vectors. S3. Construct a model for determining the same product: Use known pairs of the same product and pairs of different products to construct a training set. The pairs of different products include at least some difficult-to-distinguish samples that are similar in appearance but different in model, specifications or packaging quantity under the same category. Perform joint training of classification loss and metric loss on the fused feature vector to obtain the model for determining the same product. S4. Perform candidate coarse screening: Generate a fusion feature vector to be retrieved for the product to be retrieved, perform cosine similarity retrieval in the feature vector library and select the top N products as initial candidates, then calculate the Euclidean distance, and retain products with cosine similarity higher than the first threshold and Euclidean distance lower than the second threshold as candidate products. S5. Perform attribute difference verification: Construct attribute difference vectors based on the brand, model, specifications, material and color of the candidate product and the product to be retrieved. When the weighted norm of the attribute difference vector is not higher than the third threshold and at least two fields in the brand field, model field and specification field meet the consistency condition, it is determined to be the same product. For candidate products located in the critical interval, the multimodal pre-trained model is called to perform secondary semantic verification. S6. Perform error sample reinjection and update: Write the identification results into the sample library, and write highly similar but re-judged as different samples into the difficult-to-distinguish sample library. When the preset sample quantity or preset period condition is met, the same sample determination model is updated by mixing new samples and historical samples.

2. The product matching recognition method based on a large language model according to claim 1, characterized in that: In step S1, the text information and image information include: title, brand, model, specifications, material, color, packaging quantity, and packaging description; The preprocessing includes: field segmentation, unit of measurement normalization, synonym mapping, abnormal character removal, and promotional word filtering; and size normalization, background suppression, main area extraction, and local identifier area cropping for the main image and detail images.

3. The product matching recognition method based on a large language model according to claim 1, characterized in that: In step S2, the field credibility mask is generated according to the proportion of valid characters, the proportion of abnormal characters, the proportion of promotional words, the result of unit of measurement normalization, and the consistency of packaging quantity. The mask value corresponding to missing fields is zero. The mask value corresponding to fields containing abnormal characters or promotional words but retaining valid information is 0.2 to 0.

8. The mask value corresponding to fields that meet the preset format after standardization is 1.

4. The product matching recognition method based on a large language model according to claim 1, characterized in that: In step S2, the weights of the brand field, model field, and specification field are all higher than the weights of the color field and title field, and the weight of the model field is not lower than the weight of the specification field. Before the field text vector is fused with the image vector, it is multiplied bit by bit with the corresponding field credibility mask, and then a weighted sum is performed according to the field weight.

5. The product matching recognition method based on a large language model according to claim 1, characterized in that: In step S2, the global vector of the main image represents the overall outline, color scheme and packaging shape of the product, while the local vector of the detail image represents the label text, interface structure, local texture, printing marks or assembly details; the local vector of the detail image is formed by splicing together at least two local mark areas after they are encoded separately.

6. The product matching recognition method based on a large language model according to claim 1, characterized in that: In step S3, the loss is measured using triplet loss or contrastive loss. During training, product pairs with similar text descriptions and main images, but different model fields, specification fields, or packaging quantities are selected from the same category of products as hard-to-distinguish samples, so as to constrain the vector distance between the same sample pairs to be less than the vector distance between hard-to-distinguish sample pairs.

7. The product matching recognition method based on a large language model according to claim 1, characterized in that: In step S4, the initial number of candidates N is 10 to 200. The initial candidate products are first sorted in descending order of cosine similarity, and then a second screening is performed based on Euclidean distance to screen out candidate products whose overall vector directions are similar and whose vector magnitude differences are greater than a preset value from the review stage.

8. The product matching recognition method based on a large language model according to claim 1, characterized in that: In step S5, the attribute difference vector includes at least brand difference component, model difference component, specification difference component, material difference component and color difference component, among which the weight of model difference component and specification difference component is higher than that of color difference component; when the brand difference component exceeds the preset brand threshold, or the model difference component exceeds the preset model threshold, the candidate product is directly determined to be a different product.

9. A product matching recognition system based on a large language model, used to implement the method as described in any one of claims 1-8, characterized in that, It includes a processor, memory, image preprocessing unit, text normalization unit, fusion coding unit, feature vector library, candidate selection unit, attribute verification unit, and model update unit.

10. The product matching recognition system based on a large language model according to claim 9, characterized in that: The memory stores program instructions that can be executed by the processor; The image preprocessing unit is used to perform size normalization, background suppression, main area extraction, and local marker area cropping on the main product image and detail image. The text standardization unit is used to perform field segmentation, unit of measurement normalization, synonym mapping, and abnormal character filtering on product titles, brands, models, specifications, materials, colors, packaging quantities, and packaging descriptions. The fusion encoding unit is used to generate field text vectors, main image global vectors, and detail image local vectors, and outputs fusion feature vectors and attribute sub-vectors based on field weights and field confidence masks; The feature vector library stores the fused feature vectors of the products to be added to the database; The candidate selection unit outputs a set of candidate products based on cosine similarity and Euclidean distance. The attribute verification unit outputs the same judgment result based on the attribute difference vector and field consistency conditions; The model update unit performs model updates based on newly added identical samples and difficult-to-distinguish samples.

Images