Commodity representation model training method, same commodity retrieval method and device

Abstract

This application provides a product representation model training method, a similar product retrieval method, and an apparatus. The method includes: determining a first product as a query product from a first similar product cluster, and a second product as a positive sample corresponding to the first product from the first similar product cluster. A third product as a negative sample corresponding to the first product is determined from a second similar product cluster. A first product representation model is trained using a contrastive learning approach based on triplets constructed from the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product. Specifically, by selecting query products and positive samples from the same similar product cluster and negative samples from different similar product clusters to construct triplets as training data, the quality of the constructed training data is higher. Using a contrastive learning approach to train the model enables the embedded representation generated by the first product representation model to possess discriminative power.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a product representation model training method, a method and device for retrieving similar products. Background Technology

[0002] With the rapid development of e-commerce, the product databases of various e-commerce platforms have continued to expand, and the variety of goods has become increasingly rich. During user selection and merchant operations, it is often necessary to quickly and accurately identify the same product as the one being searched from a massive amount of goods. However, in practical applications, especially in specific scenarios such as wholesale and retail, and unbranded e-commerce, the product database contains a large number of generic products. These products generally lack unified brand identification and standardized attribute information, and different merchants often have significant differences in image photography, text descriptions, and attribute entries for the same product. Therefore, even genuinely identical products may exhibit significant differences in visual appearance or semantic expression due to factors such as shooting angle, lighting conditions, color variations, packaging, or inconsistent descriptive language. This greatly increases the difficulty of accurately identifying identical products from a massive product database in such scenarios, placing higher demands on the discriminative power and robustness of product representation models. Therefore, in order to accurately identify identical products from a complex and diverse product database in complex e-commerce scenarios containing a large number of non-standard generic products, a product representation model capable of learning discriminative product representations is urgently needed. Summary of the Invention

[0003] This application provides a product representation model training method, a similar product retrieval method, and a device, so that the product embedding representation generated by the trained first product representation model has discriminative power and can more reliably distinguish between similar and different products.

[0004] In a first aspect, embodiments of this application provide a method for training a commodity representation model, the method comprising: The first product is determined from the first cluster of identical products as the query product, and the second product is determined from the first cluster of identical products as the positive sample corresponding to the first product. From the second cluster of identical products, identify the third product as the negative sample corresponding to the first product; Obtain the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product; Based on the triplet consisting of the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product, a contrastive learning approach is used to train the representation model of the first product.

[0005] Secondly, embodiments of this application provide a commodity representation model training device, the device comprising: The first determining module is used to determine, from the first cluster of identical products, the first product to be queried and the second product to be the positive sample corresponding to the first product from the first cluster of identical products; The second determining module is used to determine a third product from the second cluster of identical products as a negative sample corresponding to the first product; The acquisition module is used to acquire the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product; The training module is used to train the representation model of the first product using a contrastive learning approach based on a triplet consisting of the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product.

[0006] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a communication interface; wherein, the memory stores a computer program, and when the computer program is executed by the processor, the processor can at least implement the commodity representation model training method as described in the first aspect.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor of an electronic device, enables the processor to at least implement the commodity representation model training method as described in the first aspect.

[0008] Fifthly, embodiments of this application provide a computer program product, including: a computer program or instructions, which, when executed by a processor of an electronic device, enable the processor to at least implement the commodity representation model training method as described in the first aspect.

[0009] Sixthly, embodiments of this application provide a method for searching for similar products, the method comprising: Obtain at least one modal information of the product to be searched; The embedding representation corresponding to at least one modal information of the product to be retrieved is determined according to the first product representation model, wherein the first product representation model is trained based on the method described in the first aspect; The target embedding representation that satisfies the similarity condition with the embedding representation corresponding to the at least one modality information is determined from the embedding representation vector library. The embedding representation vector library is obtained by encoding the embedding representation of multimodal information of the products in the product library through the first product representation model. Candidate similar products of the product to be retrieved are determined from the products corresponding to the target embedding representation.

[0010] Seventhly, embodiments of this application provide a device for searching for similar products, the device comprising: The acquisition module is used to acquire at least one modal information of the product to be retrieved; The first determining module is used to determine the embedding representation corresponding to at least one modal information of the product to be retrieved based on the first product representation model, wherein the first product representation model is trained based on the method described in the first aspect. The second determining module is used to determine, from the embedding representation vector library, a target embedding representation that satisfies the similarity condition to the embedding representation corresponding to the at least one modal information. The embedding representation vector library is obtained by encoding the embedding representation of multimodal information of the products in the product library through the first product representation model. The third determining module is used to determine candidate similar products of the product to be retrieved from the products corresponding to the target embedding representation.

[0011] Eighthly, embodiments of this application provide an electronic device, including: a memory, a processor, and a communication interface; wherein, the memory stores a computer program, and when the computer program is executed by the processor, the processor can at least implement the same product retrieval method as described in the sixth aspect.

[0012] Ninthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor of an electronic device, enables the processor to at least implement the same product retrieval method as described in the sixth aspect.

[0013] In a tenth aspect, embodiments of this application provide a computer program product, including: a computer program or instructions, which, when executed by a processor of an electronic device, enable the processor to at least implement the same product retrieval method as described in the sixth aspect.

[0014] In the solution provided in this application embodiment, before training the first product representation model, multiple product clusters of the same style can be constructed based on the multimodal information corresponding to the products in the product database. Each product cluster includes multiple products belonging to the same style, i.e., they are identical products. This allows the first product representation model to be trained using the multimodal information corresponding to the products in the constructed multiple product clusters of the same style. Specifically, when training the first product representation model, firstly, a first product is determined from the first product cluster as the query product, and a second product is selected from the first product cluster as the positive sample corresponding to the first product. Then, a third product is selected from the second product cluster as the negative sample corresponding to the first product. That is, the selected positive sample (the second product) belongs to the same style as the first product, while the negative sample (the third product) belongs to a different style than the first product. Next, the multimodal information of the first product, the second product, and the third product is obtained. Then, a triplet consisting of the multimodal information of the first, second, and third commodities is constructed, and the triplet is used as training data to train the representation model of the first commodity using a contrastive learning approach.

[0015] In the above scheme, a first product is selected from the same product cluster as the query product, and another second product is selected from it as a positive sample of the query product. At the same time, a third product is selected from different product clusters as a negative sample of the query product. This constructs a triplet for each query product as training data. Each triplet consists of the multimodal information of a query product, a positive sample of the same product, and a negative sample of the different product. The training data constructed in this way is of higher quality. Therefore, training the first product representation model based on this training data can enable the trained first product representation model to better represent the essential features of the product. Furthermore, when using this training data and employing a contrastive learning approach to train the first product representation model, the model can narrow the embedding distance between the query product and positive samples in the vector space, while simultaneously widening the embedding distance between it and negative samples. This results in the embedded representations of the products generated by the trained first product representation model possessing discriminative power, enabling more reliable differentiation between identical and dissimilar products. Consequently, in actual retrieval, the model can more accurately identify genuine identical products in the product database based on the generated embedded representations of the product to be retrieved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a product representation model training method provided in this application embodiment; Figure 2a This is a schematic diagram of training data for triples of the same product provided in an embodiment of this application; Figure 2b A schematic diagram of training data for another triplet of the same product provided in an embodiment of this application; Figure 2c A schematic diagram of training data for a triplet of the same product provided in an embodiment of this application; Figure 3 A schematic diagram of the architecture of a commodity representation model provided in this application embodiment; Figure 4 A flowchart illustrating another product representation model training method provided in this application embodiment; Figure 5 This is a schematic diagram illustrating the application of a commodity representation model training method provided in an embodiment of this application; Figure 6 A flowchart illustrating a method for searching for similar products provided in this application embodiment; Figure 7 A schematic diagram of the structure of a commodity representation model training device provided in an embodiment of this application; Figure 8 To and Figure 7 A schematic diagram of the electronic device corresponding to the commodity representation model training device provided in the embodiment shown. Figure 9 This is a schematic diagram of the structure of a product retrieval device provided in an embodiment of this application; Figure 10 To and Figure 9 The illustrated embodiment provides a schematic diagram of the electronic device corresponding to the same product retrieval device. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0020] The following describes some embodiments of this application in detail with reference to the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features can be combined with each other. Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0021] First, the terms or concepts involved in the embodiments of this application will be explained: Stock Keeping Unit (SKU): This refers to the smallest salable and traceable unit of a product. Products within the same SKU are completely identical in attributes such as brand, model, specifications, color, and size.

[0022] In the process of user selection or merchant operation, it is often necessary to quickly and accurately identify the same product as the product to be searched from a massive amount of goods. However, in practical applications, especially in specific scenarios such as wholesale and retail, the product database contains a large number of generic products. These products generally lack unified brand identification and standardized attribute information, and different merchants often have significant differences in the images, text descriptions, and attribute entries for the same product. Therefore, even genuinely identical products may show significant differences in visual appearance or semantic expression due to factors such as shooting angle, lighting conditions, color variations, packaging forms, or inconsistent descriptive language. This greatly increases the difficulty of accurately identifying identical products from a massive product database in this scenario, and places higher demands on the discriminative power and robustness of the product representation model.

[0023] Based on this, in order to obtain a product representation model capable of learning discriminative product representations, the inventors propose a solution. The basic idea is to pre-construct multiple product clusters of the same style based on the multimodal information corresponding to products in the product database. Each product cluster includes multiple products belonging to the same style, i.e., they are identical to each other. Based on the constructed product clusters, one product is selected as the query product from the same cluster, another product is selected as the positive sample from that cluster, and a product from a different product cluster is selected as the negative sample. Using triplet training data containing positive samples of the same style and negative samples of different styles, and employing a contrastive learning approach to train the first product representation model, the first product representation model, when generating product embedding representations, can narrow the embedding distance between the query product and the positive sample in the vector space, while widening the embedding distance between it and the negative sample. This makes the product embedding representations generated by the trained first product representation model discriminative, enabling it to more reliably distinguish between identical and different products.

[0024] The following provides a detailed description of the product representation model training scheme and the scheme for retrieving similar products using the trained product representation model, as provided in the embodiments of this application. The product representation model training method provided in the embodiments of this application can be executed by an electronic device, such as a PC, laptop, or smartphone, or a server. The server can be a physical server containing an independent host, a virtual server, a cloud server, or a server cluster.

[0025] Figure 1 A flowchart of a product representation model training method provided in this application embodiment is shown below. Figure 1 As shown, the method may specifically include the following steps: 101. Determine the first product from the first cluster of identical products as the query product, and determine the second product from the first cluster of identical products as the positive sample corresponding to the first product.

[0026] 102. Identify the third product from the second cluster of identical products as the negative sample corresponding to the first product.

[0027] 103. Obtain the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product.

[0028] 104. Based on the triplet consisting of the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product, a contrastive learning approach is used to train the representation model of the first product.

[0029] When constructing the training data for the first product representation model, multiple product clusters of the same type can be built based on the multimodal information of the products in the product database. Each product cluster includes multiple products belonging to the same style, meaning they are all the same type of product. For example, smartphones of the same brand and model sold by different merchants, despite differences in product image shooting style, copywriting descriptions, or attribute filling formats, should be grouped into the same product cluster because their core specifications are consistent.

[0030] This multimodal information includes product image information and product text description information. Specifically, product image information can include multi-view images uploaded by the merchant, such as front view, side view, close-up details, and packaging images. Product text description information includes text content such as product title, subtitle, feature introduction, and product attribute information. Product attribute information includes structured fields such as key attributes like material, style, size, color, brand, and model.

[0031] Furthermore, when constructing product clusters using multimodal information corresponding to products, a pre-trained second product representation model can be used to efficiently build high-quality product clusters within a large-scale product database. This second product representation model can fuse and encode the multimodal information of any product, generating a unified embedding representation. Based on this embedding representation, the semantic similarity between products can be effectively measured, thereby grouping truly similar products together in the vector space.

[0032] In practical implementation, the multimodal information corresponding to each product in the product library is first input into the second product representation model to obtain the embedded representation of each product generated by the second product representation model. This embedded representation is a feature representation that integrates multimodal information. Then, based on the embedded representations of each product, a preset clustering algorithm is used to cluster multiple products. This algorithm calculates the vector distance between any two product embedded representations, grouping products with a distance less than a set threshold into the same cluster, thus obtaining multiple clusters of similar products. Each cluster contains at least one product, and products within the same cluster are highly consistent in core attributes such as style, appearance, function, and specifications, thus being considered similar products. Different clusters of similar products exhibit significant attribute differences, thereby automatically distinguishing similar products in the product library. The clustering algorithm used can be density clustering, hierarchical clustering, or spectral clustering, etc.

[0033] Then, training data is constructed based on the established clusters of identical products. Specifically, the first product to be used as the query product is determined from the first cluster of identical products, and the second product to be used as the positive sample corresponding to the first product is also determined from the first cluster of identical products. The first cluster of identical products is any one of the multiple clusters of identical products obtained by clustering the product library using the second product representation model, and it contains multiple products of the same style that are identical in model, structure, and core function. That is, by selecting the first and second products that are identical from the same cluster of identical products, valid positive sample pairs are formed, providing a supervisory signal for the consistency of identical product features when training the first product representation model subsequently.

[0034] In actual sampling, two different products can be randomly selected from the first cluster of similar products, serving as the first product and the second product, respectively. Alternatively, the second product can be selected from the first cluster of similar products based on its corresponding embedding representation. Specifically, the second product can be selected based on the similarity between the first embedding representation of the first product and the embedding representations of other products within the first cluster of similar products, provided that the product meets preset conditions.

[0035] The first and second embedding representations are both determined by the trained second product representation model. That is, in the process of constructing multiple product clusters of the same product using the second product representation model, the embedding representations of all products in the product library output by the second product representation model can be pre-stored in a preset storage space (such as a vector database or feature cache file) so that they can be directly read in the subsequent positive and negative sample screening stage, avoiding repeated calculations and improving the sampling efficiency of training data.

[0036] However, in practical applications, even if all products within the same product cluster are identical, they may still exhibit significant differences in color, style details, shooting angle, lighting, background environment, or text description style. These differences lead to a certain distance in their embedding representations under the second product representation model, meaning their similarity is relatively low. Using these sample pairs with some similarity differences for training can improve the first product representation model's ability to perceive subtle differences and its resistance to interference, enabling the first product representation model to learn more robust representations of identical products.

[0037] If only highly similar pairs of identical samples are selected as training data to train the first product representation model, the model may over-rely on surface consistency and struggle to generalize to identical products with significant differences in appearance or description in real-world scenarios. In an optional embodiment, the preset condition can be set to select candidate products from the cluster of identical products that differ somewhat in representation from the first product as the second product. Although these second products are identical to the first product, differences in color variations, shooting angles, lighting conditions, background environments, or text description styles result in them being relatively far apart in the multimodal representation space, constituting difficult positive samples.

[0038] By introducing such positive samples, while maintaining semantic consistency, the robustness of the first product representation model to non-essential differences in product representation can be enhanced, making it more focused on essential attributes.

[0039] In a preferred embodiment, the preset condition can be set as follows: the similarity between the first embedded representation and the candidate product embedded representation is lower than a first preset threshold (but still within the same product cluster). That is, the first similarity between the first embedded representation of the first product and the second embedded representation of the second product is lower than the first preset threshold. The first preset threshold can be determined statistically based on the similarity distribution of embedded representations within the same product cluster. For example, the first preset threshold can be set as the lower 25% quantile of the similarity between all product pairs within the same product cluster, or it can be selected after evaluating the impact of different thresholds on the accuracy of same-product retrieval on a validation set.

[0040] This data sampling strategy proactively filters out sample pairs that are similar in appearance but have significant differences in representation, avoiding excessive homogeneity of samples and providing more discriminative supervisory signals for training the first product representation model. Therefore, by actively selecting sample pairs with low embedding similarity but still belonging to the same product category as positive samples, the first product representation model can learn to ignore irrelevant interfering factors such as color and shooting style during training, focusing instead on the essential attributes that determine the similarity relationship, such as structure, function, and model number. This improves the robustness and generalization ability of the first product representation model for similar products, enabling it to accurately identify truly similar products even when faced with a complex and diverse product database.

[0041] The research revealed that identical products in the product database often exhibit significant differences in visual appearance or textual descriptions due to factors such as shooting conditions, color variations, packaging differences, or non-standard attribute entries, affecting the accuracy of identical product identification. Simultaneously, some dissimilar products in the database are extremely similar to the target product in overall appearance, category, or style, easily leading to misjudgments by the first product representation model. The generated product embedding representation fails to effectively distinguish between identical and dissimilar products, causing inter-class confusion and ultimately reducing the accuracy of identical product identification.

[0042] Therefore, in this embodiment of the application, after determining the first product and its corresponding positive samples as the query product when constructing training data, a third product can be selected from the second cluster of similar products as a negative sample of the first product. The second cluster of similar products refers to another cluster of similar products that is different from the first cluster; that is, the products in the first cluster and the products in the second cluster are not considered to be similar products. Preferably, the second cluster of similar products can be selected as another cluster of similar products belonging to the same category as the first cluster, or the second cluster of similar products can be a cluster of similar products under the same leaf category as the first cluster.

[0043] Because products within the same leaf category are highly similar in type, function, or usage scenario—for example, all being women's crew neck short-sleeved T-shirts—their visual features such as appearance, color, and style are often quite similar. Therefore, selecting negative samples from these categories presents a greater challenge to the first product representation model, effectively simulating the confusion caused by similar appearances but not belonging to the same product category in actual searches. For instance, if the products in the first cluster of similar products are all "white slim-fit pure cotton T-shirt from brand A," then the second cluster of similar products can be selected from other similar product clusters under the same leaf category "women's pure cotton short-sleeved T-shirts," such as the similar product clusters corresponding to "beige loose-fit T-shirt from brand B" or "striped basic T-shirt from brand C."

[0044] Although these products are not the same as the first product, they are in the same subcategory and their overall appearance, style, color scheme, or style elements are highly similar. This can easily interfere with the discrimination process of the first product representation model, causing the first product representation model to incorrectly map different products to similar embedding areas, thereby reducing the accuracy of the same product search.

[0045] Therefore, by constructing negative samples within the same category and across product clusters of the same type, we can effectively simulate confusion in real-world scenarios while ensuring semantic rationality. This allows the first product representation model to learn more refined distinguishing abilities, focusing on essential product attributes that determine the same product relationship, such as brand, model, and structural details, rather than relying solely on coarse-grained category features.

[0046] Furthermore, in the specific implementation process, after determining the second cluster of identical products, a third product meeting preset conditions can be selected as a negative sample from this second cluster based on the similarity of its embedding representations. Specifically, products meeting the following conditions can be selected: the second similarity between the first embedding representation of the first product and the third embedding representation of the third product is greater than a second set threshold. The third embedding representation is generated by a pre-trained second product representation model, and its determination method is consistent with that of the first embedding representation. That is, during the construction of the identical product cluster, the embedding representations of all products have been calculated by the second product representation model and pre-stored in a vector database or feature cache, which can be directly read during subsequent sampling without repeated reasoning; this will not be elaborated further here.

[0047] Since the second similarity between the first embedded representation of the first product and the third embedded representation of the third product is relatively high, it means that the two are close in the embedding representation space. Although such negative samples are not the same as the first product, they are highly deceptive in multimodal semantics or visual appearance, thus constituting difficult negative samples.

[0048] The second threshold can be determined statistically based on the similarity distribution of dissimilar product pairs within the same category, or dynamically adjusted based on feedback from the same-product recognition performance on the validation set. Introducing such highly similar, challenging negative samples significantly increases the training difficulty and improves the sensitivity and robustness of the first product representation model to subtle differences. As described above, by introducing a third product as a negative sample into the training data, the first product representation model can be guided to learn a clear discriminative representation standard. For products of the same type, their embeddings are made closer; for dissimilar products, their embeddings are made more distant. This effectively reduces interference from products that appear similar but are not actually the same, allowing the first product representation model to more accurately represent different products.

[0049] After identifying the first product as the query product, the second product as the positive sample, and the third product as the negative sample, multimodal information for the first product, the second product, and the third product is obtained. This multimodal information includes product image information, text description information, etc. The multimodal information of the first product, the second product, and the third product is then combined into a triplet training data set.

[0050] In practice, the multimodal information for each of the three products can be obtained from pre-stored product data in the product database. Alternatively, the multimodal information can be read in real time from the e-commerce platform's product database, product detail page, or backend product management system.

[0051] If relying solely on single-modal information, the first product feature model struggles to capture the essential characteristics of the product. Therefore, in this embodiment, by organizing the multimodal information of the first, second, and third products into triples, a comparative supervision signal can be provided to the model. Simultaneously, the fusion of multimodal information complements and verifies each other, effectively suppressing single-modal noise. This allows the first product feature model to focus on core discriminative features in the vector space, generating more robust and generalizable product representations, thereby significantly improving the accuracy of identifying identical products in a complex and diverse product database.

[0052] Based on this, the constructed triples can be used as training data to train the first product representation model using a contrastive learning approach. Specifically, the multimodal information of the first, second, and third products in the triples is input into the first product representation model, which then outputs the corresponding embedding representations for each product. Based on these embedding representations, a contrastive loss function is calculated, and the first product representation model is trained using this loss function.

[0053] The embedding representation can include unimodal embedding representations corresponding to each unimodal information and hybrid embedding representations generated by fusing multimodal information. The unimodal embedding representation refers to the vector representation generated based solely on the single modal information of the product. The hybrid embedding representation refers to the vector representation generated by fusing the unimodal embedding representations corresponding to multimodal information such as images, text, and attributes. This two-level representation structure enables the first product representation model to utilize both unimodal discriminative cues and multi-source information to form a more comprehensive understanding of the product.

[0054] In this embodiment, not only is contrastive learning performed using hybrid embedding representations, but a multi-layered alignment mechanism is further introduced. First, within the same modality, constraints are imposed on the first product representation model, causing it to actively reduce the distance between the single-modal embedding representations of the same product in that modality, making the embedding representations of the same product closer together. Simultaneously, the first product representation model actively increases the distance between the corresponding single-modal embedding representations of the same product and dissimilar products in that modality, making the embedding representations of dissimilar products more distant, thereby enhancing the first product representation model's ability to distinguish between the same and dissimilar products.

[0055] Secondly, through cross-modal alignment mechanisms, different forms of representation (image embeddings, text embeddings) become closer in the same semantic space. For example, by maximizing the cosine similarity between different modal embeddings, semantic unification can be achieved. This avoids deviations in the representation of the product by the first product feature model due to mismatches between product images and text descriptions or errors in filling in product attributes.

[0056] Furthermore, alignment constraints between single-modal embeddings and hybrid embeddings are introduced. This mechanism requires that each single-modal embedding and its corresponding hybrid embedding remain sufficiently close in the representation space, thereby ensuring that the fused hybrid representation retains both the specific information of each modality and global semantic consistency, effectively preventing information dilution or excessive dominance of a single modality during the fusion process.

[0057] During model training, the various alignment constraints mentioned earlier are integrated into the contrastive learning framework. Specifically, during training, not only is the loss based on hybrid embedding representation calculated, but also the loss for each unimodal embedding is calculated simultaneously. In addition, cross-modal alignment loss and unimodal-hybrid embedding alignment loss are also included. All these losses are integrated together to form a multi-task joint optimization objective, thereby comprehensively optimizing the performance of the first commodity representation model.

[0058] Through this multi-granularity, multi-level joint training strategy, the first product representation model can collaboratively learn the same product relationship at different representation levels, which significantly improves the accuracy and stability of same product recognition in complex e-commerce environments, and is especially suitable for white-label products, non-standardized descriptions and other scenarios.

[0059] In summary, in this embodiment, by selecting a first product from the same product cluster as the query product, and selecting another second product from it as a positive sample of the query product, while selecting a third product from different product clusters as a negative sample of the query product, a triplet corresponding to each query product is constructed as training data. Each triplet consists of the multimodal information of a query product, a positive sample of the same product, and a negative sample of the different product. The training data constructed in this way is of higher quality. Therefore, training the first product representation model based on this training data can enable the trained first product representation model to better represent the essential features of the product. Furthermore, when using this training data and employing a contrastive learning approach to train the first product representation model, the model can narrow the embedding distance between the query product and positive samples in the vector space, while simultaneously widening the embedding distance between it and negative samples. This results in the embedded representations of the products generated by the trained first product representation model possessing discriminative power, enabling more reliable differentiation between identical and dissimilar products. Consequently, in actual retrieval, the model can more accurately identify genuine identical products in the product database based on the generated embedded representations of the product to be retrieved.

[0060] The above embodiments describe the construction ideas of training data and the model training process. During the model training process, in order to further improve the quality of training samples and enhance the generalization ability of the first product representation model, a variety of different sampling strategies were designed for the data cleaning and positive / negative sample sampling stages to adapt to the training needs at different stages. The following is a detailed explanation of the various data cleaning and sampling strategies used in the actual training process. The first sampling strategy is as follows: select the first product from the same product cluster as the query product; select the user's real-shot product image with the farthest embedding distance from the first product from the set of user real-shot product images that have a purchase relationship with the first product as the positive sample; and select a product image from different product clusters of the same product belonging to the same leaf category as the first product in the product library as the negative sample, thereby constructing the triplet required for training.

[0061] The positive samples were selected by filtering out the user photos that were furthest from the corresponding representation of the first product from among multiple user photos that showed a purchase relationship with the first product. The negative samples were selected by selecting products that did not belong to the same product cluster from among the products belonging to the same leaf category as the first product.

[0062] For example, Figure 2aAs shown, suppose the first product in the query is a white sneaker, with its corresponding embedding representation being item1. The set of user-uploaded product images related to the first product includes three images, with corresponding embedding representations Query1, Query2, and Query3. When constructing positive samples, the representation distance between item1 and the embedding representations Query1, Query2, and Query3 of each user-uploaded product image is calculated. Query3 has the greatest representation distance from item1, and its corresponding user-uploaded product image is selected as the second product. For the selection of negative samples, from the product database, products belonging to the same leaf category as the first product but in different product clusters are selected. The product images corresponding to products whose embedding representation item2 is greater than a set threshold than the embedding representation item1 of the first product are selected as negative samples.

[0063] However, in wholesale and retail scenarios, the focus is more on the same products, and there is no need to model the representation of user-taken product images. Furthermore, the data distribution and data quality storage of user-taken product images differ greatly from those in the product database. Therefore, the following sampling strategies can be preferred.

[0064] The second sampling strategy is to randomly select two different products from each product cluster to form a positive sample pair, and randomly select one product from other non-product clusters as a negative sample, and use the multimodal information of these three products to form triplet training data.

[0065] However, since the randomly selected samples are mostly easy samples, the first product representation model may have difficulty effectively distinguishing between products that are similar in appearance but not the same type, and the generated embedding representation lacks discriminative power. Therefore, in an optional specific embodiment, after determining the first product as the query product from the first cluster of similar products, a third sampling strategy can be adopted when selecting positive samples: randomly select a product from the first cluster of similar products as the second product, and require that the similarity between the embedding representation of the second product and the embedding representation of the first product is below a certain preset threshold (i.e., in a relatively low similarity range). Negative samples are selected by choosing products that do not belong to the same cluster of similar products from products belonging to the same leaf category as the first product. Here, the embedding representation can include unimodal embedding representation and hybrid embedding representation.

[0066] Furthermore, in an optional embodiment, a fourth sampling strategy can be adopted: after determining the first product as the query product from the first cluster of similar products, when selecting positive samples, the product whose embedding representation is furthest from the embedding representation of the first product in the first cluster of similar products can be selected as the second product, and the similarity between the embedding representation of the second product and the embedding representation of the first product is required to be higher than a set threshold. Negative samples are selected by choosing products that do not belong to the same cluster of similar products from products belonging to the same leaf category as the first product.

[0067] However, in practical applications, the title text description information in the multimodal information of a product may have problems such as non-standard descriptions. In another optional embodiment, a fifth sampling strategy can be adopted, which is to add cleaning of the text description information of the product based on the fourth sampling strategy.

[0068] For example, the original product title text description is: "Pure cotton gauze summer women's pajamas, cute camisole nightgown, cartoon summer thin long dress, home wear, suitable for outerwear." This can be cleaned to obtain the cleaned product title text description: "Gauze pajamas, camisole nightgown, long dress, home wear." In other words, when selecting the above three products, the text description information of each product cluster is first cleaned, then the cleaned text description information is encoded to obtain the corresponding text embedding representation. Then, based on other single-modal embedding representations and hybrid embedding representations, the embedding representation corresponding to each product is determined. In this first product cluster, the product whose embedding representation is furthest from the embedding representation of the first product is selected as the second product, and the similarity between the embedding representation of the second product and the embedding representation of the first product is required to be higher than a set threshold. The negative sample selection method is: from products belonging to the same leaf category as the first product, products that do not belong to the same product cluster are selected.

[0069] Furthermore, based on the fifth sampling strategy, the contrast loss function can be further optimized to remove the image-text alignment loss function value from the contrast loss function.

[0070] In another optional embodiment, a sixth sampling strategy can also be adopted: After determining the first product as the query product from the first cluster of similar products, when selecting positive samples, the product whose embedding representation is furthest from the embedding representation of the first product in the first cluster of similar products can be selected as the second product, and the similarity between the embedding representation of the second product and the embedding representation of the first product is not required to meet a certain preset threshold. When selecting negative samples, products that do not belong to the same cluster of similar products are selected from products belonging to the same leaf category as the first product.

[0071] For example, Figure 2bAs shown, assuming that product cluster 1 contains 4 products, when selecting a query product, the first product is randomly selected from product cluster 1 as the query product, and its corresponding embedding is item4. The embeddings of the other three products in product cluster 1 are item1, item2, and item3, respectively. When constructing positive samples, the representation distance between item1 and the embeddings of the other three products in product cluster 1 (item1, item2, and item3) is calculated. Among them, item3 has the greatest representation distance from item4, and its corresponding product is selected as the second product. For the selection of negative samples, from product clusters belonging to the same leaf category as the first product but belonging to different product clusters, the product image corresponding to the product whose embedding (item2) is greater than a set threshold than the embedding (item1) of the first product is selected as the negative sample.

[0072] In another optional embodiment, a first product can be selected from a first cluster of similar products as the query product. When selecting positive samples, a query cluster semantically similar to the first product is first determined. This query cluster includes user-photographed product images. A set of historical products for which the user made purchases when searching based on this query cluster is then selected. Next, from this set of historical products, the product whose embedding representation is furthest from the embedding representation of the first product, and whose similarity to the first product is less than a set threshold, is selected as the second product (i.e., a positive sample). Simultaneously, a product from the product library belonging to the same leaf category as the first product but to a different cluster of similar products is selected as a negative sample. Based on the multimodal information of the three selected products, triples required for training are constructed.

[0073] In other words, the selected positive sample second product originates from historical purchase records corresponding to query clusters containing user-photographed product images, and this second product has a purchase relationship with the first product. During the selection process, it is required that, within the semantic space of the first product cluster, the embedding representation of the second product is furthest from the embedding representation of the first product, and the similarity between them is below a preset threshold. That is, by utilizing purchase records generated from similar queries (such as user-photographed product searches) and selecting samples with the greatest feature differences, the positive sample sources for the current query product are enriched, thereby training the first product representation model to identify highly difficult similar products.

[0074] For example, such as Figure 2cAs shown, assuming that product cluster 1 contains 4 products, when selecting a query product, the first product is randomly selected from product cluster 1 as the query product, and its corresponding embedding is item2. The embeddings of the remaining products in product cluster 1 are item1, item4, and item5, respectively. First, a query cluster semantically similar to item2 is determined. This query cluster contains two user-photographed product images, and their corresponding embeddings are query1 and query2, respectively. The set of historical products for which users made purchases when searching based on queries in this query cluster is then selected, i.e., the set of historical products includes the product corresponding to item1.

[0075] Next, the representation distance between the embedding representation of the queried product item2 and the embedding representation of the corresponding item1 in the historical product set is calculated. Among them, the representation distance between item2 and item1 is the farthest, and the similarity between item1 and item2 is lower than a preset threshold. Therefore, the product corresponding to item1 is regarded as the second product (positive sample).

[0076] For negative sample selection, an embedding representation of a product (item3) belonging to the same leaf category as the first product but in a different cluster of similar products is chosen, such that the representation distance between item3 and the embedding representation of the first product (item2) is greater than a set threshold. In actual training, combining one or more of these sampling strategies can significantly improve the quality and diversity of the training data, thereby enhancing the first product representation model's ability to identify similar products in complex e-commerce environments.

[0077] The above embodiments primarily focus on constructing high-quality training data, enhancing the model's discriminative ability through refined positive and negative sample sampling strategies. Furthermore, the contrastive learning strategy itself can be optimized to more directly and efficiently achieve the learning objective of bringing the embedding representations of similar product samples closer together and pushing away the embedding representations of dissimilar product samples.

[0078] Before detailing the specific design of the contrastive learning loss function, the overall architecture of the first commodity representation model is first described. This model includes encoders for each of the multimodal information types and a hybrid encoder. Each encoder is responsible for extracting features from a specific type of input data and converting them into unimodal embedding representations. The hybrid encoder is used to fuse the unimodal embedding representations generated by multiple unimodal encoders to generate a single hybrid embedding representation. This architecture enables the first commodity feature model to not only learn effectively on unimodal information but also improve overall performance through the fusion of cross-modal information.

[0079] Based on the above model architecture, this application further designs a multi-level contrastive learning loss function to train the first product representation model. Specifically, through encoders corresponding to various information modalities, single-modal embedding representations corresponding to the multimodal information of the first product, the single-modal embedding representations corresponding to the multimodal information of the second product, and the single-modal embedding representations corresponding to the multimodal information of the third product are obtained; through a hybrid encoder, hybrid embedding representations corresponding to the multimodal information of the first product, the hybrid embedding representations corresponding to the multimodal information of the second product, and the hybrid embedding representations corresponding to the multimodal information of the third product are obtained; based on the single-modal and hybrid embedding representations corresponding to the multimodal information of the first product, the single-modal and hybrid embedding representations corresponding to the multimodal information of the second product, and the single-modal and hybrid embedding representations corresponding to the multimodal information of the third product, multiple first multimodal cross-contrast loss function values ​​for the first product, the second product, and the third product are determined; based on the sum of the multiple first multimodal cross-contrast learning loss function values, the first product representation model is trained.

[0080] The first multimodal cross-contrast loss function calculates the contrast loss for both single-modal embedding representations and hybrid embedding representations, and then sums the loss values ​​corresponding to each modality. In other words, it calculates the contrast loss for the first product versus the second product, and the first product versus the third product, in each single modality (image modality, text modality, etc.), and also calculates the contrast loss for both in the hybrid modality. All these loss values ​​together constitute multiple values ​​of the first multimodal cross-contrast loss function. By summing these loss values, multi-dimensional alignment of the first product representation model can be achieved, ensuring that the first product representation model simultaneously learns the feature discrimination ability of each single modality and the discriminative ability of multimodal fusion features. Thus, when a user searches for the same product in different scenarios, regardless of whether the input is the product image information, text information, or just a single product image or text information, the trained first product representation model can accurately search for the same product.

[0081] Furthermore, in an optional embodiment, each first multimodal cross-contrast loss function value can be calculated using contrastive loss functions such as triplet loss, contrastive loss, cosine contrastive loss, circle loss, and circle loss.

[0082] In addition, to further improve the robustness and generalization ability of the first commodity representation model and solve the problems of insufficient negative sample diversity and easy overfitting of the first commodity representation model during training, this application also proposes a spatiotemporal negative sample enhancement strategy, that is, based on the time and space dimensions during training, more discriminative negative samples are mined to enrich the distribution of negative samples.

[0083] In an optional embodiment, when constructing training data for training triples, the spatiotemporal negative sample strategy can also be used to determine multiple augmented negative samples corresponding to the first product. These multiple augmented negative sample products include products obtained through at least one of the following methods: products within the same training batch of the triple located on the same processing chip as the triple; products within other triples in historical training batches located on the same processing chip as the triple; and products within other triples in the same training batch located on different processing chips than the triple.

[0084] In other words, this spatiotemporal negative sample augmentation strategy breaks the limitation of traditional negative samples being selected only from the same batch and chip. It takes into account both the diversity of training data in different processing chips in space and the sample characteristics of historical training batches in time. This makes the feature differences between the augmented negative samples and the first product (queried product) more diverse, while ensuring the rationality of the negative samples. As a result, the first product representation model learns more detailed and essential product features, avoiding overfitting to training data from specific batches and chips.

[0085] Next, the unimodal and hybrid embedding representations corresponding to the multimodal information of multiple enhanced negative sample items are obtained. These unimodal and hybrid embedding representations are determined using the first item representation model. In other words, the unimodal and hybrid embedding representations corresponding to the multimodal information of multiple enhanced negative sample items are obtained in the same way as the embedding representations of the first, second, and third items, ensuring consistency and comparability of the embedding representations.

[0086] Furthermore, in practical applications, after determining the unimodal and hybrid embedding representations corresponding to the multimodal information of multiple enhanced negative sample products using the first product representation model, the identification information of these enhanced negative sample products, along with their corresponding unimodal and hybrid embedding representations, can be stored together in a pre-defined storage space (such as a vector database or feature cache file). An association index between the enhanced negative sample product identifiers and their corresponding embedding representations can be established for convenient and rapid retrieval later. Simultaneously, the stored embedding representations can be updated periodically to ensure their timeliness and accuracy, preventing them from becoming invalid due to product feature updates or model parameter optimizations.

[0087] In this way, when training the first product representation model and calculating the multimodal cross-comparison loss function value between the first product and multiple enhanced negative sample products, the single-modal embedding representation and hybrid embedding representation corresponding to multiple enhanced negative sample products can be quickly obtained directly from the embedding representation vector library or the preset storage space through the associated index, without having to repeatedly encode the multimodal information of the enhanced negative sample products through the first product representation model.

[0088] Then, based on the single-modal embedding representation and mixed embedding representation corresponding to the multimodal information of the first product, the single-modal embedding representation and mixed embedding representation corresponding to the multimodal information of the second product, the single-modal embedding representation and mixed embedding representation corresponding to the multimodal information of the third product, and the single-modal embedding representation and mixed embedding representation corresponding to the multimodal information of multiple enhanced negative sample products, the values ​​of multiple first multimodal cross-comparison loss functions for the first product and the second product, the third product and multiple enhanced negative sample products are determined.

[0089] The calculation logic for the first multimodal cross-contrast loss function value here is the same as that for the first multimodal cross-contrast loss function value without the addition of augmented negative samples. The only difference is that the calculation of the contrast loss between the first product and each augmented negative sample is added. That is, the contrast loss between the first product and each augmented negative sample in each single mode and mixed mode is calculated separately. These loss values ​​are then added to the contrast loss values ​​between the first product and the second and third products to obtain multiple first multimodal cross-contrast loss function values. This ensures that the augmented negative samples can fully play a supervisory role and further optimize the model parameters of the first product representation model.

[0090] Finally, the first product representation model is trained by summing the values ​​of multiple first multimodal cross-contrast learning loss functions. The spatiotemporal negative sample augmentation strategy enriches the diversity of negative samples. Training the first product representation model using this method, combined with multi-dimensional alignment of multi-level contrastive loss, can effectively improve the recognition accuracy of the first product representation model for the same product, enhance its adaptability to complex scenarios, and improve its generalization ability.

[0091] Furthermore, in practical applications, a bidirectional contrastive loss function can be used to train the first product representation model, further enhancing the model's feature discrimination ability. In other words, by breaking away from the contrast logic of single-direction queries and constructing bidirectional contrastive supervision, the contrastive loss function described above—using the first product as the query product, the second product as the positive sample, and the third product and enhanced negative samples as the negative samples—can be calculated in one direction. Conversely, the contrastive loss function can be calculated in the other direction, using the second product as the query product, the first product as the positive sample, and the third product and enhanced negative samples as the negative samples. This bidirectional supervision allows the model to more comprehensively learn the feature relationships between similar products.

[0092] For the calculation of the contrastive loss function in the second direction, specifically, the second product is determined to be the query product again, and the first product is determined to be the positive sample corresponding to the second product, and multiple enhanced negative sample products are determined to be the negative samples corresponding to the second product; based on the single-modal embedding representation and mixed embedding representation corresponding to the multimodal information of the first product, the single-modal embedding representation and mixed embedding representation corresponding to the multimodal information of the second product, and the single-modal embedding representation and mixed embedding representation corresponding to the multimodal information of the multiple enhanced negative sample products, multiple second multimodal cross-contrast loss function values ​​of the second product with the first product and the multiple enhanced negative sample products are determined; based on the multiple first multimodal cross-contrast loss function values ​​and the multiple second multimodal cross-contrast loss function values, the representation model of the first product is trained.

[0093] The calculation method of the second multimodal cross-comparison loss function is completely consistent with that of the first multimodal cross-comparison loss function. Both calculate the comparison loss for single mode and mixed mode separately and then sum them up to ensure the consistency and supervision effectiveness of bidirectional comparison. Through bidirectional comparison, the feature aggregation ability between the same products is strengthened, while further widening the feature distance with negative samples and improving the discrimination accuracy of the first product representation model.

[0094] Furthermore, in practical applications, in order to highlight the core role of non-textual modalities in identifying the same product and to avoid the interference of redundant modifiers and descriptive differences in the textual modalities on the training of the first product representation model, the multimodal cross-comparison loss function value of the first product representation model includes the comparison loss function value corresponding to the textual modalities and the comparison loss function value corresponding to the non-textual modalities. Moreover, the weighting coefficient of the comparison loss function value corresponding to the textual modalities is smaller than the weighting coefficient of the comparison loss function value corresponding to the non-textual modalities.

[0095] As described above, this application's embodiments construct a comprehensive and efficient training scheme for the first product representation model through a multi-level contrastive learning loss function, a spatiotemporal negative sample augmentation strategy, a bidirectional contrastive loss function, and modality weighted adjustment. This scheme, through multi-dimensional supervision and sample augmentation, allows the model to focus on the essential features of the product (especially the core features of non-textual modalities), effectively reducing the influence of redundant and interfering information, improving the model's representation ability and discrimination accuracy for the same product, and simultaneously enhancing the robustness and generalization ability of the first product representation model.

[0096] It should be noted that, in practical implementation, the above model architecture can be flexibly configured according to actual task requirements and data formats. In practical applications, the multimodal information of goods is usually mainly in the form of images and text. Therefore, in practical implementation, the first product representation model may include at least one image encoder, one text encoder, and one hybrid encoder. In a preferred embodiment of this application, to further enhance the model's representation capability and training stability, a dual-branch symmetric structure is adopted, that is, independent encoding paths are set to process query products, positive samples, and negative samples respectively, thereby avoiding optimization interference caused by feature coupling.

[0097] Combination Figure 3 This will illustrate the model training process under the model architecture of the first commodity representation model. For example... Figure 3 As shown, the first product representation model includes two image encoders, two text encoders, and two hybrid encoders. One set of encoders (including one image encoder, one text encoder, and one hybrid encoder) processes the query product (first product) in the triple, while the other set processes positive samples (second product) and negative samples (third product), respectively. Through this dual-branch design, the first product representation model can perform fair and consistent representation learning for the three products within the triple in a unified vector space, providing a reliable foundation for the subsequent calculation of the contrastive loss.

[0098] During training, multiple triplet training data are acquired. Based on the multiple triplet training data, the first product representation model is trained using a contrastive learning approach. Since the training process for each training data is roughly the same, we will take a triplet containing two identical products (denoted as product A and product B) and one different product (denoted as product C) as an example to illustrate the specific implementation process of training the first product representation model.

[0099] Specifically, such as Figure 3As shown, multimodal information for products A, B, and C is obtained. This multimodal information includes product images, product attributes, and text titles. The product image for product A is input into an image encoder to generate its image embedding representation. The text title and product attributes for product A are input into a text encoder to generate its text embedding representation. The image and text embedding representations for product A are then input into a hybrid encoder to generate its hybrid embedding representation. Following the same process, the multimodal information for products B and C is processed separately to obtain image, text, and hybrid embedding representations for product B, and three types of embedding representations for product C. Simultaneously, a spatiotemporal negative sample enhancement strategy is used to retrieve non-identical products from historical batches and other chip batches from the memory bank, generating multiple enhanced negative samples and obtaining the three types of embedding representations corresponding to these enhanced negative samples.

[0100] Next, using product A as the query product, product B as the positive sample, and product C and each enhanced negative sample as the negative sample, as the contrast loss in one direction, we calculate multiple first multimodal cross-contrast loss function values ​​between product A and product B, and between product A and each negative sample.

[0101] To make it easier to understand, the following example illustrates the calculation of contrast loss across six different dimensions: .in, subscript , , These represent image embedding, hybrid embedding, and text embedding, respectively. 1 represents the query product (i.e., product A), and 2 represents positive and negative samples (i.e., product B, product C, and each augmented negative sample). The first multimodal cross-comparison loss function value is calculated by comparing the embedding representation of a certain dimension of the queried product with the embedding representations of a certain dimension of the positive and negative samples.

[0102] In other words, the contrast loss is calculated in six different dimensions: the first multimodal cross-contrast loss function value between the image embedding representations of product A and product B, the first multimodal cross-contrast loss function value between the image embedding representation of product A and the mixed embedding representation of product B, the first multimodal cross-contrast loss function value between the image embedding representation of product A and the text embedding representation of product B, the first multimodal cross-contrast loss function value between the mixed embedding representation of product A and the image embedding representation of product B, the first multimodal cross-contrast loss function value between the mixed embedding representation of product A and the mixed embedding representation of product B, and the first multimodal cross-contrast loss function value between the text embedding representation of product A and the image embedding representation of product B. Simultaneously, the contrast loss between product A and product C, and each enhanced negative sample, is calculated in these six dimensions. All these losses together constitute multiple first multimodal cross-contrast loss function values ​​in the first direction.

[0103] Then, using product B as the query product, product A as the positive sample, and product C and each enhanced negative sample as the negative sample, as the contrast loss in another direction, we calculate multiple second multimodal cross-contrast loss function values ​​between product B and product A, and between product B and each negative sample.

[0104] Following the example above, specifically, we calculate the contrast loss for six different dimensions: .in, subscript , These represent image embedding, hybrid embedding, and text embedding, respectively. 1 represents the query product (i.e., product B), and 2 represents positive and negative samples (i.e., product A and various augmented negative samples). The second multimodal cross-comparison loss function value is calculated by comparing the embedding representation of a certain dimension of the queried product with the embedding representations of a certain dimension of the positive and negative samples.

[0105] In other words, the contrast loss is calculated in reverse along six different dimensions: the second multimodal cross-contrast loss function value between the image embedding representation of product B and the image embedding representation of product A; the second multimodal cross-contrast loss function value between the image embedding representation of product B and the mixed embedding representation of product A; the second multimodal cross-contrast loss function value between the image embedding representation of product B and the text embedding representation of product A; the second multimodal cross-contrast loss function value between the mixed embedding representation of product B and the image embedding representation of product A; the second multimodal cross-contrast loss function value between the mixed embedding representation of product B and the mixed embedding representation of product A; and the second multimodal cross-contrast loss function value between the text embedding representation of product B and the image embedding representation of product A. Simultaneously, the contrast loss between product B and product C, and each enhanced negative sample, is calculated along these six dimensions, forming multiple second multimodal cross-contrast loss function values ​​in the second direction.

[0106] Then, based on multiple first multimodal cross-contrast loss function values ​​and multiple second multimodal cross-contrast loss function values, the first commodity representation model is trained.

[0107] Continuing with the example above, specifically, based on multiple first multimodal cross-contrast loss function values ​​and multiple second multimodal cross-contrast loss function values, the total loss function value is calculated, i.e. In other words, the six types of losses in the first direction and the six types of losses in the second direction, totaling 12 types of losses, are weighted and summed to obtain the total loss function value for training the first commodity representation model. The first commodity representation model is then trained based on the total loss function value.

[0108] Furthermore, each type of loss can be calculated using circular loss; for example, each loss can be calculated using... .in, This represents the similarity score between the i-th positive sample pair (the query product and the positive sample). This represents the similarity score between the j-th negative sample pair (the query product and the negative sample, where the negative sample includes each enhanced negative sample). This represents the target lower bound for positive sample similarity. This represents the target upper limit for negative sample similarity. , These represent adaptive weights, which are not fixed constants but are dynamically calculated based on the similarity of the current samples. This represents the scaling factor, similar to the reciprocal of the temperature coefficient, used to control the magnitude of the gradient. K and L represent the number of positive and negative samples, respectively.

[0109] That is, each multimodal cross-loss function value is calculated using Circle Loss, and adaptive weights can be used to achieve the effect of similarity for positive samples. It's already quite large, indicating that the model has learned very well. Therefore, we should assign its corresponding weights... To reduce the gradient, if the similarity is smaller... The small value indicates that this is a difficult positive sample; therefore, its corresponding weight is... Increase the gradient; for negative samples, if the similarity... The values ​​are already very small, indicating that the model has separated the elements sufficiently. Therefore, the corresponding weights should be... To reduce the gradient, if the similarity is smaller... The large value indicates that this is a difficult negative sample; therefore, its corresponding weight should be... Increase the gradient to maximize the similarity of positive samples and minimize the similarity of negative samples.

[0110] By calculating the first and second multimodal cross-loss function values ​​using Circle Loss, the similarity between many similar-looking but not identical products in e-commerce data can be forcibly reduced. This not only improves the discriminative power of the first product representation model but also increases model training efficiency. Furthermore, through backpropagation of the total loss, the model parameters of the image encoder, text encoder, and hybrid encoder are updated, making the embedding representations of identical products closer together in the vector space, while the embedding representations of dissimilar products are further apart.

[0111] As described above, the total loss function for training the first product representation model is obtained by weighted summation of 12 types of losses. Each type of loss consists of multiple multimodal cross-loss function values, and each multimodal cross-loss function value corresponds to a preset weight. Due to the high noise level in text descriptions, which can lead to redundant descriptions and image conflicts, after multiple experiments, the weighting coefficient of the contrastive loss function value corresponding to the text modality can be made smaller than the weighting coefficient of the contrastive loss function value corresponding to the non-text modality. For example, the weighting coefficient of the contrastive loss function value corresponding to the text modality can be set to 0.1, and the weighting coefficient of the contrastive loss function value corresponding to the non-text modality can be set to 1. This reduces the interference of text noise on model training, allowing the first product representation model to focus more on core essential features such as product images and mixed features, thus improving the robustness of identifying similar products.

[0112] The first product representation model described in the above embodiments is mainly aimed at identifying identical products at the product granularity, and is suitable for matching image modalities and text modalities (matching image information and product title information). However, in application scenarios at the SKU granularity, such as price comparison and product exchange, it is also necessary to determine whether the specification attributes are semantically consistent. For example, it is necessary to further determine whether the product color, size, and other information are consistent. To this end, after completing the training of the first product representation model, this application further introduces a third product representation model to focus on SKU attribute label semantic modeling, in order to improve the discrimination accuracy at the SKU level. The specific implementation process of this third product representation model and its training will be described in detail below.

[0113] Figure 4 A flowchart illustrating another product representation model training method provided in this application embodiment. For example... Figure 4 As shown, based on the above embodiments, the method may further include the following steps: 401. Obtain multiple identical product sample pairs for training the third product representation model, as well as the corresponding supervision signal sequences for these sample pairs. Each identical product sample pair includes textual descriptions of the inventory units corresponding to the two identical products. The corresponding supervision signals are used to describe whether the textual descriptions of the two corresponding inventory units match.

[0114] 402. Input multiple identical product sample pairs into the third product representation model to obtain the embedding representation pairs corresponding to each of the multiple identical product sample pairs.

[0115] 403. Based on the corresponding embedding pairs and supervision signal sequences of multiple identical product samples, determine the loss function value, and train the third product representation model based on the loss function value.

[0116] To ensure that this third product representation model can accurately determine the semantic consistency between different SKU attributes, it needs to undergo specialized supervised training. Specifically, this involves combining... Figure 5 To illustrate the training process of the third product model, the first step is to obtain multiple pairs of identical product samples used to train the third product representation model, as well as the corresponding supervision signal sequences for these pairs.

[0117] Each training data set consists of a pair of product samples and its corresponding supervision signal. Each pair of identical product samples includes textual descriptions of the inventory units of the two identical products. The corresponding supervision signal is used to describe whether the textual descriptions of the two corresponding inventory units match.

[0118] For example, such as Figure 5The example of three identical product sample pairs shows two different SKUs. The first pair has the following text descriptions: SKU1: "150*50, Harmony Brings Prosperity, Dazzling Black (Aluminum Alloy Frame + Crystal Ceramic Surface), Independent"; and SKU2: "150*50 [Tempered Crystal Ceramic Surface + Black Aluminum Alloy Frame], Harmony Brings Prosperity." While they differ in the order of product attributes, use of parentheses, and terminology, their core specifications—size, brand, material combination, and color (black)—are identical. Therefore, the monitoring signal for this first identical product sample pair should be marked as "Match," indicating that the two text descriptions of the inventory unit are semantically equivalent and can be considered variations of the same specification.

[0119] In the second sample pair of the same product, the inventory unit text description SKU1 is "3XL", and the inventory unit text description SKU2 is "XXXL". In practical applications, "3XL" and "XXXL" are defined as equivalent sizes, and their corresponding supervision signals can be marked as "matched".

[0120] The third sample of the same product has two inventory unit text description SKUs: SKU1 is "Camellia Pattern - Sea Breeze Lime" and SKU2 is "Camellia Pattern - White Peach Oolong". Both have the same camellia pattern theme, but there are significant differences in color or fragrance descriptions, so their corresponding monitoring signals can be marked as "mismatch".

[0121] Furthermore, in order to facilitate the third product representation model to effectively learn and identify the supervision signals corresponding to the same product sample pairs, these signals can be converted into machine-recognizable numerical labels. If the semantics of the two SKU text descriptions match, they are marked as 1; if the semantics of the two SKU text descriptions do not match, they are marked as 0.

[0122] In addition, when constructing training data, text description information of inventory units corresponding to the same product from multiple different e-commerce platforms can be collected. Furthermore, by using preset algorithm rules, multi-source and diverse sample pairs of the same product can be constructed. These samples can cover various real-world scenarios, such as different specification description orders, inconsistent terminology, custom naming, and some attributes being the same but key attributes being different. This not only improves the generalization and robustness of the third product representation model in practical applications but also enhances the accuracy of the third product representation model in identifying the same product across e-commerce platforms.

[0123] As can be seen from the above examples, the third product representation model needs to be able to identify synonyms, brand-specific naming habits, and key attribute differences, so as to achieve accurate semantic alignment and matching judgment in complex and diverse SKU texts.

[0124] During training, multiple pairs of identical product samples are input into a third product representation model to obtain corresponding embedding pairs for each sample. This third product representation model includes a text encoder and an output module. The text encoder primarily encodes the text description information of the product's corresponding inventory unit. The output module maps the encoded vectors from the text encoding to a unified vector space, generating SKU-level embedding representations of the product.

[0125] Specifically, such as Figure 5 As shown, the text description information of inventory units sku1 and sku2 from the same product sample pair can be input into the text encoder, which outputs the encoding vectors corresponding to sku1 and sku2. Then, the encoding vectors corresponding to sku1 and sku2 are input into the output module to generate the embedding representation emb1 corresponding to sku1 and the embedding representation emb2 corresponding to sku2.

[0126] During the model training phase, after obtaining the embedding representation pairs corresponding to each sample of the same product through the third product representation model, the loss function value is determined based on the embedding representation pairs and supervision signal sequences corresponding to multiple samples of the same product, so as to train the third product representation model based on the loss function value.

[0127] In an optional embodiment, the cosine similarity between the corresponding embedding pairs of each pair of identical product samples can be calculated, and based on the cosine similarity and the supervision signal sequence, the loss function value can be calculated using the CoSENT (Consistent SENTenceembedding Loss) loss function, so as to train the third product representation model based on the loss function value.

[0128] In this context, although the SKU negative samples in the same product sample pair are semantically unrelated, they often have a lot of overlap in wording (such as co-occurring words like size and model), resulting in a high degree of similarity. If these superficially similar but semantically different negative samples are forcibly pushed apart in the vector space, it is easy for the third product representation model to overfit the literal differences, which will impair the generalization ability. Therefore, in this embodiment, the CoSENT loss function can be used to optimize the relative ranking between the same product sample pairs, rather than directly optimizing the distance between the sample pairs, so as to guide the third product representation model to learn more discriminative text representations, so that the embedding representations of the same product are closer in the vector space and the embedding representations of different products are farther apart, thereby improving the third product representation model's ability to represent SKU-level product text and the accuracy of same-product representation.

[0129] In summary, in this embodiment, by constructing diverse pairs of identical product samples and determining the loss function value based on the corresponding embedding representation pairs and supervision signal sequences for each of the multiple identical product sample pairs, a third product representation model is trained based on the loss function value. This enables the embedded representations of the inventory unit text description information of the two products generated by the trained third product representation model to have strong discriminative power, achieving high aggregation of identical product embeddings and effective separation of non-identical product embeddings. Furthermore, the embedded representations can be used to accurately determine whether two inventory unit text descriptions refer to the same product.

[0130] The first and third product representation models trained using the above method can be effectively applied to the scenario of searching for similar products. The first product representation model can generate an image and text fusion representation focusing on the essential features of the product based on its multimodal information, while the third product representation model can output a highly discriminative SKU-level text representation based on the product's inventory unit text description information. Based on the two trained representation models, this application also provides a method for searching for similar products, which can quickly and accurately match products identical to the target product in the product database. The following is a detailed description of this method for searching for similar products.

[0131] Figure 6 A flowchart illustrating a method for searching for similar products provided in an embodiment of this application. Figure 6 As shown, based on the above embodiments, the method may further include the following steps: 601. Obtain at least one modal information of the product to be searched.

[0132] 602. Determine the embedding representation corresponding to at least one modal information of the product to be retrieved based on the first product representation model.

[0133] 603. Determine the target embedding representation from the embedding representation vector library that satisfies the similarity condition for the embedding representation corresponding to at least one modality information. The embedding representation vector library is obtained by encoding the embedding representation of multimodal information of the products in the product library through the first product representation model.

[0134] 604. Identify candidate similar products of the product to be retrieved from the products corresponding to the target embedding representation.

[0135] When performing a search for similar products, the process first involves acquiring at least one modal information about the product to be searched. This modal information can be image information or text description information. The image information can be the main image of the product, and the text description information can be the title. For example, in a specific application, when a user browses a product they like on an e-commerce platform and wants to search for similar products, they can use that product as the search target. The user can then extract the product's theme and copy its text title, product attributes, and other information to perform the search for similar products.

[0136] In practice, based on the search information entered by the user, at least one modal information of the product to be searched can be obtained from the product database, or the same product can be directly searched based on the search information entered by the user. There are no restrictions on this.

[0137] After obtaining at least one modal information of the product to be retrieved, the at least one modal information is input into the trained first product representation model to obtain the embedded representation of the at least one modal information of the product to be retrieved output by the first product representation model.

[0138] Next, a target embedding representation that satisfies the similarity condition for the embedding representation corresponding to at least one modality of information is determined from the embedding representation vector library. The embedding representation vector library stores single-modal embedding representations and hybrid embedding representations corresponding to the multimodal information of multiple products. Furthermore, the embedding representation vector library is obtained by encoding the multimodal information of products in the product library using a first product representation model.

[0139] The similarity criteria can be pre-set according to the retrieval requirements. For example, it can be set as follows: the cosine similarity between the hybrid embedding representation of the product to be retrieved and the hybrid embedding representation of products in the vector library is higher than a preset similarity threshold, or the similarity between the single-modal embedding representation of the product to be retrieved and the single-modal embedding representation of the corresponding modality in the vector library meets a preset range. Through this similarity filtering, a set of candidate products highly related to the features of the product to be retrieved can be quickly preliminarily screened from a massive product library, greatly improving retrieval efficiency.

[0140] Then, candidate similar products are determined from the products corresponding to the target embedding representation. Alternatively, the products corresponding to the target embedding representation can be directly identified as candidate similar products, or they can be sorted by similarity from high to low, and a predetermined number of products at the top of the sorting can be selected as candidate similar products.

[0141] To further improve the accuracy of searching for similar products and avoid mismatches caused by relying solely on multimodal features (such as products with similar appearance but different models or functions being misjudged as similar products), in an optional embodiment, after identifying candidate similar products, a trained third product representation model can be used to perform a second precise screening of the candidate similar products.

[0142] Specifically, the process involves obtaining the textual descriptions of the inventory units of candidate similar products and the textual descriptions of the inventory units to be retrieved input by the user. A third product representation model is used to determine the embedded representations corresponding to the textual descriptions of the inventory units of both candidate and retrieved products. Based on the similarity between the embedded representations of the inventory units of candidate and retrieved products, target similar products matching the textual descriptions of the retrieved inventory units are selected from the candidate similar products.

[0143] The third product representation model, trained using the CoSENT loss function, is the SKU-level text representation model described earlier. Based on the embedded representations of the inventory unit text descriptions of candidate similar products and the inventory unit text descriptions of the product to be retrieved, it accurately determines whether the inventory unit text descriptions of the candidate similar products and the product to be retrieved refer to the same product. This further determines whether the candidate similar products and the product to be retrieved are the same product. In other words, by combining multimodal information coarse screening and SKU text fine screening, the retrieval method can significantly improve the accuracy of identifying similar products while ensuring retrieval efficiency, effectively solving the identification problems in scenarios such as white-label products, different colors of the same product, and differences in shooting conditions.

[0144] In summary, in this embodiment, a first product representation model is used to encode the multimodal information of products, construct an embedded representation vector library, and quickly filter candidate similar products based on vector similarity, achieving efficient coarse-level filtering. A third product representation model is then used to perform fine-grained semantic matching on the SKU text description information of candidate similar products, completing a secondary, precise filtering. This approach improves the accuracy of similar product identification while maintaining retrieval efficiency, effectively addressing issues such as diverse product styles, complex shooting conditions, redundant text descriptions, or inconsistent terminology in real e-commerce scenarios. This provides users with accurate and efficient similar product retrieval processing.

[0145] The following will describe in detail one or more embodiments of the commodity representation model training apparatus of this application. Those skilled in the art will understand that these apparatuses can all be configured using commercially available hardware components through the steps taught in this solution.

[0146] Figure 7This is a schematic diagram of the structure of a commodity representation model training device provided in an embodiment of this application, as shown below. Figure 7 As shown, the device includes: a first determining module 11, a second determining module 12, an acquisition module 13, and a training module 14.

[0147] The first determining module 11 is used to determine, from the first cluster of identical products, the first product to be queried and the second product to be a positive sample corresponding to the first product.

[0148] The second determining module 12 is used to determine the third product from the second cluster of identical products as the negative sample corresponding to the first product.

[0149] The acquisition module 13 is used to acquire the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product.

[0150] Training module 14 is used to train the representation model of the first product using a contrastive learning method based on the triplet formed by the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product.

[0151] Optionally, each product in the first and second identical product clusters corresponds to the same category.

[0152] Optionally, the first similarity between the first embedded representation of the first product and the second embedded representation of the second product is lower than a first set threshold, and the second similarity between the first embedded representation of the first product and the third embedded representation of the third product is greater than a second set threshold, wherein the first embedded representation, the second embedded representation, and the third embedded representation are all determined by a trained second product representation model.

[0153] Optionally, the first product representation model includes encoders corresponding to each of the multiple information modalities and a hybrid encoder; the training module 14 is specifically used to: obtain the single-modal embedding representations corresponding to the multimodal information of the first product, the single-modal embedding representations corresponding to the multimodal information of the second product, and the single-modal embedding representations corresponding to the multimodal information of the third product through the encoders corresponding to the multimodal information; obtain the hybrid embedding representations corresponding to the multimodal information of the first product, the hybrid embedding representations corresponding to the multimodal information of the second product, and the hybrid embedding representations corresponding to the multimodal information of the third product through the hybrid encoder; determine multiple first multimodal cross-contrast loss function values ​​for the first product, the second product, and the third product based on the single-modal and hybrid embedding representations corresponding to the multimodal information of the first product, the single-modal and hybrid embedding representations corresponding to the multimodal information of the second product, and the single-modal and hybrid embedding representations corresponding to the multimodal information of the third product; and train the first product representation model based on the sum of the multiple first multimodal cross-contrast learning loss function values.

[0154] Optionally, the training module 14 is further configured to: determine a plurality of augmented negative sample products corresponding to the first product, wherein the plurality of augmented negative sample products include products obtained in at least one of the following ways: products within other triples in the same training batch located on the same processing chip as the triple, products within other triples in a historical training batch located on the same processing chip as the triple, and products within other triples in the same training batch located on different processing chips as the triple; obtain single-modal embedding representations and hybrid embedding representations corresponding to the multimodal information of the plurality of augmented negative sample products, wherein the multimodal information of the plurality of augmented negative sample products... The single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information are determined by the first product representation model; based on the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the first product, the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the second product, the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the third product, and the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the multiple enhanced negative sample products, the values ​​of multiple first multimodal cross-comparison loss functions for the first product, the second product, the third product, and the multiple enhanced negative sample products are determined.

[0155] Optionally, the training module 14 is further configured to: determine that the second product is re-used as the query product and that the first product is used as the positive sample corresponding to the second product, and that the plurality of enhanced negative sample products are used as the negative samples corresponding to the second product; determine a plurality of second multimodal cross-contrast loss function values ​​for the second product, the first product, and the plurality of enhanced negative sample products based on the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the first product, the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the second product, and the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the plurality of enhanced negative sample products; and train the first product representation model based on the plurality of first multimodal cross-contrast loss function values ​​and the plurality of second multimodal cross-contrast loss function values.

[0156] Optionally, the multimodal cross-contrast loss function value of the first product representation model includes the contrast loss function value corresponding to the text mode and the contrast loss function value corresponding to the non-text mode, wherein the weighting coefficient of the contrast loss function value corresponding to the text mode is less than the weighting coefficient of the contrast loss function value corresponding to the non-text mode.

[0157] Optionally, the training module 14 is further configured to: acquire multiple identical product sample pairs for training the third product representation model and the supervision signal sequence corresponding to the multiple identical product sample pairs; wherein, each identical product sample pair includes textual description information of inventory units corresponding to two identical products, and the corresponding supervision signal is used to describe whether the corresponding two textual description information of inventory units match; input the multiple identical product sample pairs into the third product representation model to obtain the embedding representation pairs corresponding to each of the multiple identical product sample pairs; determine the loss function value according to the embedding representation pairs corresponding to each of the multiple identical product sample pairs and the supervision signal sequence, so as to train the third product representation model according to the loss function value.

[0158] Figure 7 The device shown can perform the steps described in the foregoing embodiments. For detailed execution process and technical effects, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0159] In one possible design, the above Figure 7 The structure of the commodity representation model training device shown can be implemented as an electronic device, such as... Figure 8 As shown, the electronic device may include: a memory 21, a processor 22, and a communication interface 23. The memory 21 stores a computer program, which, when executed by the processor 22, enables the processor 22 to at least implement the commodity representation model training method provided in the foregoing embodiments.

[0160] The aforementioned memory 21 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0161] Figure 9 This is a schematic diagram of the structure of a product retrieval device provided in an embodiment of this application, as shown below. Figure 9 As shown, the device includes: an acquisition module 31, a first determination module 32, a second determination module 33, and a third determination module 34.

[0162] The acquisition module 31 is used to acquire at least one modal information of the product to be retrieved.

[0163] The first determining module 32 is used to determine the embedding representation corresponding to at least one modal information of the product to be retrieved according to the first product representation model, wherein the first product representation model is trained based on the method of any one of claims 1-8.

[0164] The second determining module 33 is used to determine, from the embedding representation vector library, a target embedding representation that satisfies the similarity condition to the embedding representation corresponding to the at least one modal information. The embedding representation vector library is obtained by encoding the embedding representation of multimodal information of the products in the product library through the first product representation model.

[0165] The third determining module 34 is used to determine candidate similar products of the product to be retrieved from the products corresponding to the target embedded representation.

[0166] Optionally, the third determining module 34 is further configured to: obtain the inventory unit text description information of the candidate similar products and the inventory unit text description information to be retrieved input by the user; determine the embedding representation corresponding to the inventory unit text description information of the candidate similar products and the embedding representation corresponding to the inventory unit text description information to be retrieved through a third product representation model, wherein the third product representation model is trained based on the method described in claim 8; and select target similar products that match the inventory unit text description information to be retrieved from the candidate similar products based on the similarity between the embedding representation corresponding to the inventory unit text description information of the candidate similar products and the embedding representation corresponding to the inventory unit text description information to be retrieved.

[0167] Figure 9 The device shown can perform the steps described in the foregoing embodiments. For detailed execution process and technical effects, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0168] In one possible design, the above Figure 9 The structure of the product retrieval device shown can be implemented as an electronic device, such as... Figure 10 As shown, the electronic device may include: a memory 41, a processor 42, and a communication interface 43. The memory 41 stores a computer program, which, when executed by the processor 42, enables the processor 42 to at least implement the same product retrieval method provided in the foregoing embodiments.

[0169] The aforementioned memory 41 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0170] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, digital video disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium.

[0171] Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0172] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. This application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] Finally, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0175] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for training a commodity representation model, characterized in that, include: The first product is determined from the first cluster of identical products as the query product, and the second product is determined from the first cluster of identical products as the positive sample corresponding to the first product. From the second cluster of identical products, identify the third product as the negative sample corresponding to the first product; Obtain the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product; The representation model of the first product is trained using a contrastive learning approach based on a triplet consisting of the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product.

2. The method according to claim 1, characterized in that, Each product in the first and second identical product clusters corresponds to the same category.

3. The method according to claim 2, characterized in that, The first similarity between the first embedded representation of the first product and the second embedded representation of the second product is lower than a first set threshold, and the second similarity between the first embedded representation of the first product and the third embedded representation of the third product is greater than a second set threshold, wherein the first embedded representation, the second embedded representation and the third embedded representation are all determined by a trained second product representation model.

4. The method according to claim 1, characterized in that, The first commodity representation model includes encoders corresponding to various information modalities and a hybrid encoder; The step of training a representation model for the first product using a contrastive learning approach based on a triplet formed from the multimodal information of the first product, the multimodal information of the second product, and the multimodal information of the third product includes: The single-modal embedding representations corresponding to the multimodal information of the first product, the single-modal embedding representations corresponding to the multimodal information of the second product, and the single-modal embedding representations corresponding to the multimodal information of the third product are obtained through the encoders corresponding to the multimodal information of each product. The hybrid encoder is used to obtain the hybrid embedding representations corresponding to the multimodal information of the first product, the hybrid embedding representations corresponding to the multimodal information of the second product, and the hybrid embedding representations corresponding to the multimodal information of the third product. Based on the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the first product, the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the second product, and the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the third product, determine multiple first multimodal cross-comparison loss function values ​​for the first product, the second product, and the third product; The first commodity representation model is trained by summing the values ​​of the multiple first multimodal cross-contrast learning loss functions.

5. The method according to claim 4, characterized in that, The method further includes: Determine multiple enhanced negative sample products corresponding to the first product. The multiple enhanced negative sample products include products obtained in at least one of the following ways: products in other triplets in the same training batch of the same processing chip as the triplet, products in other triplets in historical training batches of the same processing chip as the triplet, and products in other triplets in the same training batch of different processing chips as the triplet. Obtain the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the multiple enhanced negative sample products. The single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the multiple enhanced negative sample products are determined by the first product representation model. The determination of multiple first multimodal cross-comparison loss function values ​​for the first product, the second product, and the third product includes: Based on the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the first product, the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the second product, the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the third product, and the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the multiple enhanced negative sample products, determine multiple first multimodal cross-comparison loss function values ​​for the first product, the second product, the third product, and the multiple enhanced negative sample products.

6. The method according to claim 5, characterized in that, The method further includes: The second product is determined to be used as the query product again, and the first product is determined as the positive sample corresponding to the second product. The plurality of enhanced negative sample products are used as negative samples corresponding to the second product. Based on the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the first product, the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the second product, and the single-modal embedding representation and hybrid embedding representation corresponding to the multimodal information of the multiple enhanced negative sample products, determine multiple second multimodal cross-comparison loss function values ​​of the second product, the first product, and the multiple enhanced negative sample products; The first commodity representation model is trained based on the multiple first multimodal cross-contrast loss function values ​​and the multiple second multimodal cross-contrast loss function values.

7. The method according to any one of claims 4-6, characterized in that, The multimodal cross-comparison loss function value of the first product representation model includes the comparison loss function value corresponding to the text mode and the comparison loss function value corresponding to the non-text mode. The weighting coefficient of the comparison loss function value corresponding to the text mode is smaller than the weighting coefficient of the comparison loss function value corresponding to the non-text mode.

8. The method according to any one of claims 1-6, characterized in that, The method further includes: Acquire multiple identical product sample pairs for training the third product representation model and the corresponding supervision signal sequence of the multiple identical product sample pairs; wherein, each identical product sample pair includes the text description information of the inventory quantity unit corresponding to the two identical products, and the corresponding supervision signal is used to describe whether the corresponding two inventory quantity unit text description information match. The multiple identical product sample pairs are input into the third product representation model to obtain the embedding representation pairs corresponding to each of the multiple identical product sample pairs; Based on the embedding pairs corresponding to the multiple identical product samples and the supervision signal sequence, a loss function value is determined, and the third product representation model is trained based on the loss function value.

9. A method for searching for similar products, characterized in that, include: Obtain at least one modal information of the product to be searched; The embedding representation corresponding to at least one modal information of the product to be retrieved is determined according to the first product representation model, wherein the first product representation model is trained based on the method described in any one of claims 1-8; The target embedding representation that satisfies the similarity condition with the embedding representation corresponding to the at least one modality information is determined from the embedding representation vector library. The embedding representation vector library is obtained by encoding the embedding representation of multimodal information of the products in the product library through the first product representation model. Candidate similar products of the product to be retrieved are determined from the products corresponding to the target embedding representation.

10. The method according to claim 9, characterized in that, The method further includes: Obtain the text description information of the inventory quantity unit of the candidate similar products and the text description information of the inventory quantity unit to be searched input by the user; The third product representation model is used to determine the embedding representation of the text description information of the inventory unit of the candidate product and the embedding representation of the text description information of the inventory unit to be retrieved. The third product representation model is trained based on the method described in claim 8. Based on the similarity between the embedded representation corresponding to the textual description information of the inventory unit of the candidate similar products and the embedded representation corresponding to the textual description information of the inventory unit to be retrieved, target similar products that match the textual description information of the inventory unit to be retrieved are selected from the candidate similar products.

11. An electronic device, characterized in that, include: The device includes a memory, a processor, and a communication interface; wherein the memory stores a computer program, which, when executed by the processor, causes the processor to perform the product representation model training method as described in any one of claims 1 to 8 or the same product retrieval method as described in any one of claims 9 to 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor of an electronic device, causes the processor to perform the product representation model training method as described in any one of claims 1 to 8 or the same product retrieval method as described in any one of claims 9 to 10.

13. A computer program product, characterized in that, include: A computer program or instruction that, when executed by a processor of an electronic device, causes the processor to perform the product representation model training method as described in any one of claims 1 to 8 or the same product retrieval method as described in any one of claims 9 to 10.

Images