Method and apparatus for merchandise matching
By generating product vector pairs and combining structured and semantic features, the problem of low recall and accuracy in product matching in existing technologies is solved, and efficient cross-platform product matching is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING JINGDONG IND PRODUCTS TRADING CO LTD
- Filing Date
- 2026-03-03
- Publication Date
- 2026-06-09
AI Technical Summary
Existing technologies suffer from insufficient recall when processing unstructured descriptions of goods and cross-platform matching of comparable prices, resulting in the omission of a large number of potential matching goods and low product matching efficiency and accuracy.
By extracting the structured attribute features and semantic features of the products, product vector pairs are generated. These vector pairs are then deeply fused using an encoding model to determine whether the products match, thus avoiding reliance on matching rules.
It improved the recall rate and accuracy of product matching, reduced the maintenance cost of matching rules, and enhanced the matching effect across different e-commerce platforms.
Smart Images
Figure CN122175667A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of e-commerce and supply chain management technology, and in particular to a method and apparatus for product matching. Background Technology
[0002] In e-commerce and supply chain management, product price comparison is a core element of corporate procurement decisions. With the explosive growth in the number of products on e-commerce platforms, corporate procurement faces new pain points: how to quickly identify truly comparable products from a database of millions to accurately calculate price differences. Currently, mainstream technologies primarily rely on structured data matching or keyword-based rule engines to match comparable products from the product database.
[0003] However, existing technologies are prone to problems such as insufficient recall, missing a large number of potential matching products, and low product matching efficiency and accuracy when dealing with unstructured descriptions of products and cross-platform matching of comparable products. Summary of the Invention
[0004] In view of this, embodiments of the present invention provide a product matching method and apparatus that can perform product matching based on the semantic features and structured attribute features of the products. This allows for product matching based on the semantic features of the products, improving the recall rate, avoiding the omission of potential matching products, and enhancing the product matching effect across different e-commerce platforms, thereby improving the efficiency and accuracy of product matching.
[0005] To achieve the above objectives, according to one aspect of the present invention, a product matching method is provided, comprising: The products to be matched are combined in pairs to generate product pairs to be detected, and a product description text pair corresponding to the product pair is generated based on the product description text of each product. The product description text includes structured attribute data. The product description text pairs are encoded to obtain product vector pairs, which are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data. The matching product pair is determined based on the product vector pairs of the product pair.
[0006] Optionally, the product description text further includes unstructured description text; encoding the product description text pair to obtain the product vector pair includes: concatenating the structured attribute data and the unstructured description text to obtain concatenated description text; encoding the structured attribute data to obtain attribute vectors of the structured attribute data; semantically encoding the concatenated description text to obtain semantic vectors of the product description text; and concatenating the attribute vectors and the semantic vectors to obtain the product vector pair.
[0007] Optionally, concatenating the structured attribute data and the unstructured description text to obtain the concatenated description text includes: concatenating the field names and field values in the structured attribute data in a specified format to obtain concatenated attribute data; and concatenating the concatenated attribute data and the unstructured description text in sequence to obtain the concatenated description text.
[0008] Optionally, encoding the structured attribute data to obtain the attribute vector of the structured attribute data includes: inputting the structured attribute data into an encoding model for encoding to obtain the attribute vector of the structured attribute data; semantically encoding the concatenated description text to obtain the semantic vector of the product description text includes: segmenting the concatenated description text to obtain at least one word, and converting each word into a number sequence; inputting the number sequence of each word into an encoding model for semantic encoding to obtain the semantic vector of the product description text.
[0009] Optionally, before segmenting the concatenated description text to obtain at least one word, the method further includes: obtaining the length of the concatenated description text; truncating the concatenated description text if the length exceeds a set threshold; and segmenting the concatenated description text to obtain at least one word, including: segmenting the truncated concatenated description text to obtain at least one word.
[0010] Optionally, before segmenting the concatenated description text to obtain at least one word, the method further includes: obtaining the length of the concatenated description text; if the length of the concatenated description text exceeds a set threshold, dividing the concatenated description text into multiple concatenated description text segments according to a set window length; segmenting the concatenated description text to obtain at least one word, and converting each word into a numerical sequence, including: segmenting the multiple concatenated description text segments to obtain at least one word corresponding to each concatenated description text segment, and converting each word into a numerical sequence; inputting the numerical sequence of each word into an encoding model for semantic encoding to obtain a semantic vector of the product description text, including: for each concatenated description text segment, inputting the numerical sequence of at least one word corresponding to the concatenated description text segment into an encoding model for semantic encoding to obtain a semantic vector corresponding to the concatenated description text segment; concatenating the semantic vectors corresponding to the multiple concatenated description text segments to obtain a semantic vector of the product description text.
[0011] Optionally, the encoding model is trained by: obtaining historical product description text and matching tags of product pairs from a product database; performing data augmentation on the historical product description text and generating positive and negative sample pairs in combination with the matching tags; and performing supervised training on a pre-trained language model based on the positive and negative sample pairs to obtain the encoding model.
[0012] Optionally, the pre-trained language model is subjected to supervised training based on the positive sample pairs and the negative sample pairs to obtain the encoding model, including: encoding the positive sample pairs and the negative sample pairs using the pre-trained language model to obtain semantic vector pairs of the positive sample pairs and semantic vector pairs of the negative sample pairs; performing binary classification operations based on the semantic vector pairs of the positive sample pairs and the semantic vector pairs of the negative sample pairs to obtain binary classification probabilities of the positive sample pairs and the negative sample pairs; and performing supervised training on the pre-trained language model based on a preset loss function, the binary classification probabilities of the positive sample pairs and the binary classification probabilities of the negative sample pairs to obtain the encoding model.
[0013] According to another aspect of the present invention, an apparatus for product matching is provided, comprising: The data combination module is used to combine the products to be matched in pairs to generate product pairs to be detected, and to generate product description text pairs corresponding to the product pairs based on the product description text of each product, wherein the product description text includes structured attribute data. The data encoding module is used to encode the product description text pairs to obtain product vector pairs of the product pairs, wherein the product vector pairs are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data; The product matching module is used to determine matching product pairs based on the product vector pairs of the product pairs.
[0014] According to another aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the product matching method provided in the embodiments of the present invention.
[0015] According to another aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the product matching method provided in the embodiments of the present invention.
[0016] According to another aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the product matching method provided in the embodiments of the present invention.
[0017] One embodiment of the above invention has the following advantages or beneficial effects: By combining the products to be matched in pairs to generate product pairs to be detected, and generating corresponding product description text pairs based on the product description text of each product, the product description text includes structured attribute data; encoding the product description text pairs to obtain product vector pairs, which are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data; the technical solution of determining matching product pairs based on the product vector pairs, by extracting structured attribute features and semantic features of the product description text, deeply integrates the features of structured data with the features of unstructured data, accurately understanding the attribute features of the products, and thus determining whether the products match, without relying on matching rules, thereby saving the maintenance cost of matching rules. By performing product matching based on the semantic features and structured attribute features of the products, product matching can be performed based on the semantic features of the products, improving the recall rate, avoiding the omission of potential matching products, and improving the product matching effect across different e-commerce platforms, thereby improving the efficiency and accuracy of product matching.
[0018] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description
[0019] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein: Figure 1 This is a schematic diagram illustrating the main steps of a product matching method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a product matching process according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the main modules of a product matching device according to an embodiment of the present invention; Figure 4 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied; Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation
[0020] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0021] It should be noted that the technical solutions disclosed in this invention, regarding the collection, updating, analysis, processing, use, transmission, and storage of user personal information, all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.
[0022] In e-commerce and supply chain management, product price comparison is a core element of corporate procurement decisions. With the explosive growth in the number of products on e-commerce platforms, corporate procurement faces new pain points: how to quickly identify truly comparable products from a database of millions to accurately calculate price differences. Currently, mainstream technologies primarily rely on structured data matching or keyword-based rule engines to match comparable products from the product database.
[0023] Traditional rule-based product matching systems typically require pre-defining a large number of thesaurus entries and matching rules. For example, they necessitate manual maintenance of mappings such as "power tools" = "electric equipment" and "high-pressure pump" = "pressure pump." This approach is not only costly to maintain but also struggles to adapt to description differences across different platforms. This maintenance is particularly challenging when dealing with a large number of non-standardized industrial products. When faced with different descriptions of the same product, such as "100L / min industrial pump" and "100 liters per minute factory pump," rule-based matching systems often fail to accurately identify whether the product matches. Some platforms have attempted to improve product matching by using machine learning methods. These methods extract textual features and combine them with cluster analysis to group products. While these methods improve generalization to some extent, they still have significant drawbacks: firstly, they cannot understand semantically similar but differently expressed modifiers such as "high-efficiency" and "energy-saving"; secondly, they lack precise handling of numerical comparisons of key product attributes (such as flow rate and pressure). More advanced solutions attempt to utilize product coding for precise matching. This approach works reasonably well in the highly standardized retail sector, but faces serious challenges in B2B (business-to-business) e-commerce scenarios. Industry data shows that over 60% of industrial products and industrial supplies lack standardized coding, and different suppliers have vastly different classification and description systems for the same product. For example, a certain type of bearing might be classified as "mechanical parts" on platform A, while on platform B it might be called "industrial parts," making it difficult for existing product matching systems to establish effective associations.
[0024] In summary, existing product matching systems primarily rely on keyword matching and simple rules, failing to truly understand the semantics of product descriptions. This leads to frequent misjudgments when handling complex product descriptions. Furthermore, they require continuous and significant manpower to maintain the thesaurus and matching rules; whenever a new product category or description method emerges, technical personnel must manually add rules. While strict matching rules ensure high accuracy, they result in the omission of many potential comparable products, while relaxing matching conditions causes a surge in mismatch rates, severely impacting the usability of the product matching results. Moreover, significant differences in product classification systems and description specifications across different e-commerce platforms greatly reduce the effectiveness of cross-platform product matching. Therefore, existing technologies are prone to insufficient recall, missing many potential matching products, and exhibiting low product matching efficiency and accuracy when handling unstructured product descriptions and cross-platform matching of comparable products.
[0025] To address at least one of the aforementioned technical problems, this invention provides a method and apparatus for product matching. By extracting structured attribute features and semantic features from product description text, and deeply fusing the features of structured and unstructured data, the method can accurately understand the attribute features of products and determine whether a product matches, without relying on matching rules, thus saving the maintenance costs of matching rules. By performing product matching based on the semantic features and structured attribute features of products, the method improves recall, avoids missing potential matching products, and enhances the product matching effect across different e-commerce platforms, thereby improving product matching efficiency and accuracy.
[0026] Figure 1 This is a schematic diagram illustrating the main steps of a product matching method according to an embodiment of the present invention. Figure 1 As shown, the product matching method of this invention mainly includes the following steps S101 to S103.
[0027] Step S101: Combine the items to be matched in pairs to generate item pairs to be detected, and generate item description text pairs corresponding to the item pairs based on the item description text of each item. The item description text includes structured attribute data. According to an embodiment of the present invention, the items to be matched are, for example, items of the same category, such as items of the same brand, model, or purpose. When combining the items to be matched in pairs, for example, every two items can be combined to generate an item pair to be detected; alternatively, the main item among the items to be matched can be determined first, and the other items can be combined with the main item to generate item pairs to be detected.
[0028] After obtaining the product pairs, a corresponding product description text pair is generated for each product pair based on the product description text of the two products included in each product pair. The product description text includes, for example, multi-source information about the product, such as the product title, details, attributes, brand, and model.
[0029] In embodiments of the present invention, after obtaining the product description text, the product description text is first preprocessed, including removing duplicate or missing values, word segmentation, removing invalid information, etc., to ensure the accuracy of subsequent semantic analysis. Then, the preprocessed product description text is used for subsequent processing.
[0030] In embodiments of the present invention, the product description text includes structured attribute data, such as: Brand: AA; Model: BB; Power: CC; Input Voltage: EE; Color: FF, etc. It may also include unstructured descriptive text, such as: Details: Supports multiple control modes, has high efficiency and energy-saving characteristics, suitable for various industrial applications, etc.
[0031] Step S102: Encode the product description text pairs to obtain product vector pairs. The product vector pairs are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data.
[0032] According to one embodiment of the present invention, the product description text further includes unstructured description text. Encoding the product description text pairs to obtain product vector pairs may specifically include: concatenating structured attribute data and unstructured description text to obtain concatenated description text; encoding the structured attribute data to obtain attribute vectors of the structured attribute data; semantically encoding the concatenated description text to obtain semantic vectors of the product description text; and concatenating the attribute vectors and semantic vectors to obtain product vector pairs. In embodiments of the present invention, in order to better extract semantic features from the product description text and maximize the utilization of information in the product description text, the present invention first concatenates the product description text in an orderly manner to generate concatenated description text, and then performs semantic encoding on the concatenated description text to obtain semantic vectors.
[0033] According to one embodiment, concatenating structured attribute data and unstructured descriptive text to obtain concatenated descriptive text can specifically include: concatenating field names and values from structured attribute data in a specified format to obtain concatenated attribute data; and concatenating the concatenated attribute data with unstructured descriptive text in sequence to obtain the concatenated descriptive text. For structured attribute data, the field names and values included are first concatenated in a specified format, such as "field name: field value". This concatenated attribute data enhances the model's ability to understand attribute information. Then, according to a preset field concatenation order, the concatenated attribute data is concatenated with unstructured descriptive text in sequence to finally obtain the concatenated descriptive text.
[0034] In one embodiment of the present invention, suppose the product description text of a certain product A is as follows: "Product A: Brand: AA, Model: BB, Color: CC, Title: AA brand BB model PLC controller, Details: Supports multiple communication protocols, modular design, suitable for automation control". Then, by concatenating the structured attribute data and unstructured description text, we can obtain "Brand: AA; Model: BB; Color: CC; Title: AA brand BB model PLC controller; Details: Supports multiple communication protocols, modular design, suitable for automation control". Correspondingly, the same processing can be applied to the product description text of other products.
[0035] Then, the structured attribute data can be encoded to obtain the attribute vector of the structured attribute data, and the concatenated description text can be semantically encoded to obtain the semantic vector of the product description text; finally, the attribute vector and the semantic vector are concatenated to obtain the product vector pair of the product pair.
[0036] According to one embodiment of the present invention, encoding structured attribute data to obtain an attribute vector of the structured attribute data can specifically include: inputting the structured attribute data into an encoding model for encoding to obtain the attribute vector of the structured attribute data. Furthermore, semantically encoding the concatenated descriptive text to obtain a semantic vector of the product description text can specifically include: segmenting the concatenated descriptive text to obtain at least one word, and converting each word into a numerical sequence; inputting the numerical sequence of each word into an encoding model for semantic encoding to obtain the semantic vector of the product description text. Specifically, the encoding model is, for example, trained based on a pre-trained language model (such as BERT), or trained based on text feature extraction algorithms such as TF-IDF (termfrequency–inverse document frequency, a commonly used weighting technique for information retrieval and data mining), Word2Vec (an algorithm for generating lexical vector representations), and FastText (an efficient text processing tool).
[0037] In one embodiment, assuming the encoding model is trained based on a pre-trained language model, when performing semantic encoding on the concatenated descriptive text based on the encoding model, a tokenizer matched with the pre-trained language model can be used to segment the concatenated descriptive text into words and convert it into a numerical sequence. Then, the numerical sequence of each word obtained from word segmentation can be input into the encoding model for semantic encoding, resulting in the semantic vector of the product description text.
[0038] According to one embodiment of the present invention, before segmenting the concatenated description text to obtain at least one word, the method may further include: obtaining the length of the concatenated description text; and truncating the concatenated description text if its length exceeds a set threshold. Furthermore, segmenting the concatenated description text to obtain at least one word may specifically include: segmenting the truncated concatenated description text to obtain at least one word. The set threshold is, for example, set based on the maximum number of characters that the encoding model can process. When the length of the concatenated description text exceeds the set threshold, it needs to be truncated. Specifically, this can be done by truncating from back to front, retaining only the first set threshold number of words and discarding the later ones; or by uniformly truncating the concatenated description texts of product A and product B respectively, ensuring that information about both products is retained.
[0039] According to another embodiment of the present invention, before segmenting the concatenated description text to obtain at least one word element, the method further includes: obtaining the length of the concatenated description text; if the length of the concatenated description text exceeds a set threshold, dividing the concatenated description text into multiple concatenated description text segments according to a set window length. Furthermore, segmenting the concatenated description text to obtain at least one word element and converting each word element into a numerical sequence can specifically include: segmenting multiple concatenated description text segments to obtain at least one word element corresponding to each concatenated description text segment, and converting each word element into a numerical sequence. The numerical sequence of each word element is input into an encoding model for semantic encoding to obtain a semantic vector of the product description text. Specifically, this can include: for each concatenated description text segment, inputting the numerical sequence of at least one word element corresponding to the concatenated description text segment into an encoding model for semantic encoding to obtain a semantic vector corresponding to the concatenated description text segment; concatenating the semantic vectors corresponding to multiple concatenated description text segments to obtain a semantic vector of the product description text.
[0040] The window length can be set as needed. For example, assuming a threshold of 512 tokens, and windows can overlap, a window length of 400 tokens can be set. Then, based on the set window length, the concatenated description text can be divided into multiple concatenated description text segments. The numerical sequence of tokens corresponding to each concatenated description text segment is input into the encoding model for semantic encoding, yielding the semantic vector for each segment. Finally, the semantic vectors from multiple concatenated description text segments are concatenated and fused to obtain the semantic vector for the product description text. In one embodiment, assuming the concatenated description text for product A has 1200 tokens, it can be divided into three segments, with token numbers 1-512, 401-912, and 801-1200 respectively. Each segment is then semantically encoded, and the semantic vectors of each segment are combined to obtain the semantic vector for the product description text.
[0041] According to one embodiment of the present invention, the encoding model is trained as follows: historical product description text and matching tags for product pairs are obtained from a product database; data augmentation is performed on the historical product description text, and positive and negative sample pairs are generated by combining the matching tags; based on the positive and negative sample pairs, a pre-trained language model is subjected to supervised training to obtain the encoding model. The matching tags for product pairs are, for example, tags set by purchasing personnel indicating which product pairs can be matched. Based on these matching tags, matching and non-matching product pairs can be determined. For example, for a matching product pair, the matching tag is "label=1"; for a non-matching product pair, the matching tag is "label=0".
[0042] In one embodiment, when augmenting historical product description text with data, operations such as synonym replacement, parameter equivalence replacement (meaning the meaning remains unchanged before and after replacement, e.g., replacing 1m with 1 meter), brand alias replacement, shuffling the description order, and splicing attributes can be performed to improve the model's robustness to the diversity of expressions. Simultaneously, negative sample pairs that cannot be matched between the main product and extended products such as accessories can be generated based on positive sample pairs, and product pairs with similar descriptions but no actual match can be collected from historical product description texts to enhance the model's discriminative ability.
[0043] According to an embodiment of the present invention, a pre-trained language model is subjected to supervised training based on positive sample pairs and negative sample pairs to obtain an encoding model. Specifically, this may include: using the pre-trained language model to encode positive sample pairs and negative sample pairs respectively to obtain semantic vector pairs of positive sample pairs and semantic vector pairs of negative sample pairs; performing binary classification operations based on the semantic vector pairs of positive sample pairs and semantic vector pairs of negative sample pairs to obtain binary classification probabilities of positive sample pairs and negative sample pairs; and performing supervised training on the pre-trained language model based on a preset loss function, the binary classification probabilities of positive sample pairs and negative sample pairs to obtain an encoding model.
[0044] In one embodiment of the present invention, semantic encoding is performed on positive sample pairs by inputting them into a pre-trained language model, resulting in semantic vector pairs for positive sample pairs; similarly, semantic encoding is performed on negative sample pairs by inputting them into the pre-trained language model, resulting in semantic vector pairs for negative sample pairs. Then, the semantic vector pairs of positive and negative sample pairs are input into a binary classification model or a fully connected layer for binary classification, yielding binary classification probabilities for positive and negative sample pairs. Finally, supervised training is performed on the pre-trained language model based on a preset loss function, the binary classification probabilities of positive and negative sample pairs, to obtain an encoding model. The preset loss function is, for example, a binary cross-entropy loss, which combines the matching labels of positive or negative sample pairs with the binary classification probabilities output by the binary classification model to calculate the binary cross-entropy loss for adjusting model parameters. For example: ; Where y is the matching label of a positive or negative sample pair, and p is the binary classification probability output by the classification model.
[0045] By performing the above supervised training, we can both adjust the parameters of the pre-trained language model to obtain the encoding model and adjust the parameters of the binary classification model.
[0046] In other embodiments of the present invention, similarity calculation can be performed on the two semantic vectors in the semantic vector pair of positive sample pairs, and similarity calculation can be performed on the two semantic vectors in the semantic vector pair of negative sample pairs. The model parameters can be adjusted by combining a preset similarity threshold and a loss function. In this scenario, the loss function can be, for example, contrastive loss or triplet loss, etc.
[0047] In other embodiments of the present invention, the product vector pairs obtained by concatenating the semantic vector pairs and attribute vector pairs of positive sample pairs, and the product vector pairs obtained by concatenating the semantic vector pairs and attribute vector pairs of negative sample pairs, can be input together into a binary classification model or a fully connected layer to adjust the model parameters in conjunction with the corresponding loss function.
[0048] In embodiments of the present invention, after obtaining the trained model, the model can be converted to a specified format for deployment to improve inference efficiency. This specified format is, for example, the ONNX (Open Neural Network Exchange) format. ONNX is an open-source standard format for exchanging and inferring deep learning models, allowing models to be exported from one framework to ONNX format and loaded and run in another framework or inference engine. Furthermore, after model deployment, a service interface for the model can be built to facilitate model invocation. This service interface supports not only HTTP (Hypertext Transfer Protocol, a simple request-response protocol) but also RPC (Remote Procedure Call Protocol) to meet high-performance invocation requirements.
[0049] According to one embodiment of the present invention, after obtaining the product vector pairs for each product pair, the product vectors can be normalized to ensure the accuracy of similarity calculation. Furthermore, an index is constructed based on the normalized product vectors to balance query speed and memory usage, and a persistent storage mechanism is established to periodically back up the index files. When constructing the vector storage database, each product vector needs to be associated with product description text and a timestamp needs to be added to record the last update time.
[0050] Step S103: Determine matching product pairs based on the product vector pairs of the product pairs. For each product pair, the similarity of the product vectors of the two included products is measured, and a set similarity threshold is used to determine whether the two products match. Alternatively, the product vectors of the two included products are input into a binary classification model to obtain the probability of the two products matching, and a set probability threshold is used to determine whether the two products match. If the two products in a product pair can match, then the product pair is a matching product pair. In this way, matching product pairs can be determined from multiple product pairs.
[0051] Subsequently, the product attribute information can be compared based on the matched product pairs. The attribute information includes, for example, product parameters, product price, etc.
[0052] Figure 2 This is a schematic diagram of a product matching process according to an embodiment of the present invention. Figure 2As shown, in one embodiment of the present invention, after receiving a product matching request, computing resources are allocated to the request based on a load balancing strategy to perform product matching using the allocated computing resources. Then, product description texts of the products to be matched are obtained from the product pool and preprocessed. Next, based on the preprocessed product description texts of the products to be matched, pairs of products to be detected and corresponding product description text pairs are generated, wherein each product description text pair includes structured attribute data and unstructured description text. Then, for each product description text pair, the structured attribute data and unstructured description text in the product description text of each product are concatenated to obtain a concatenated description text. The structured attribute data is encoded to obtain an attribute vector, and the concatenated description text is semantically encoded to obtain a semantic vector. The attribute vector and the semantic vector are concatenated to obtain a product vector pair for the product pair.
[0053] Next, based on the product vector pairs of each product pair, it is determined whether the two products in the product pair match, thus obtaining the matched product pairs. Finally, attribute comparison is performed on the two products in the matched product pairs, and the results are output.
[0054] Figure 3 This is a schematic diagram of the main modules of a product matching device according to an embodiment of the present invention. Figure 3 As shown, the product matching device 300 of this embodiment mainly includes a data combination module 301, a data encoding module 302, and a product matching module 303.
[0055] The data combination module 301 is used to combine the products to be matched in pairs to generate product pairs to be detected, and to generate product description text pairs corresponding to the product pairs based on the product description text of each product. The product description text includes structured attribute data. The data encoding module 302 is used to encode the product description text pairs to obtain product vector pairs. The product vector pairs are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data. The product matching module 303 is used to determine the matching product pairs based on the product vector pairs of the product pairs.
[0056] According to one embodiment of the present invention, the product description text further includes unstructured description text; the data encoding module 302 is specifically used to: concatenate structured attribute data and unstructured description text to obtain concatenated description text; encode the structured attribute data to obtain attribute vectors of the structured attribute data, semantically encode the concatenated description text to obtain semantic vectors of the product description text; and concatenate the attribute vectors and semantic vectors to obtain product vector pairs of product pairs.
[0057] According to one embodiment of the present invention, the data encoding module 302 can be specifically used to: concatenate the field names and field values in the structured attribute data in a specified format to obtain concatenated attribute data; and concatenate the concatenated attribute data with unstructured description text in sequence to obtain concatenated description text.
[0058] According to one embodiment of the present invention, the data encoding module 302 can be specifically used to: input structured attribute data into an encoding model for encoding to obtain an attribute vector of the structured attribute data; segment the concatenated description text to obtain at least one word, and convert each word into a number sequence; input the number sequence of each word into an encoding model for semantic encoding to obtain a semantic vector of the product description text.
[0059] According to an embodiment of the present invention, the product matching device 300 may further include a text truncation processing module (not shown in the figure), which is used to: obtain the length of the concatenated description text before segmenting the concatenated description text to obtain at least one word; truncate the concatenated description text if the length of the concatenated description text exceeds a set threshold; and the data encoding module 302 may specifically be used to: segment the truncated concatenated description text to obtain at least one word.
[0060] According to one embodiment of the present invention, the product matching apparatus 300 may further include a text segmentation processing module (not shown in the figure), configured to: obtain the length of the concatenated description text before segmenting the concatenated description text to obtain at least one word; if the length of the concatenated description text exceeds a set threshold, divide the concatenated description text into multiple concatenated description text segments according to a set window length; and the data encoding module 302 may specifically be configured to: segment the multiple concatenated description text segments to obtain at least one word corresponding to each concatenated description text segment, and convert each word into a number sequence; for each concatenated description text segment, input the number sequence of at least one word corresponding to the concatenated description text segment into an encoding model for semantic encoding to obtain a semantic vector corresponding to the concatenated description text segment; and concatenate the semantic vectors corresponding to multiple concatenated description text segments to obtain a semantic vector of the product description text.
[0061] According to one embodiment of the present invention, the product matching apparatus 300 may further include an encoding model training module (not shown in the figure), which is used to train the encoding model by: obtaining historical product description text and matching tags of product pairs from a product database; performing data augmentation on the historical product description text and generating positive sample pairs and negative sample pairs in combination with the matching tags; and performing supervised training on the pre-trained language model based on the positive sample pairs and negative sample pairs to obtain the encoding model.
[0062] According to one embodiment of the present invention, the encoding model training module (not shown in the figure) can also be used to: encode positive sample pairs and negative sample pairs using a pre-trained language model to obtain semantic vector pairs of positive sample pairs and semantic vector pairs of negative sample pairs; perform binary classification operation based on the semantic vector pairs of positive sample pairs and semantic vector pairs of negative sample pairs to obtain the binary classification probability of positive sample pairs and the binary classification probability of negative sample pairs; and perform supervised training on the pre-trained language model based on a preset loss function, the binary classification probability of positive sample pairs, and the binary classification probability of negative sample pairs to obtain the encoding model.
[0063] According to the technical solution of this invention, the products to be matched are combined in pairs to generate product pairs to be detected, and a corresponding product description text pair is generated based on the product description text of each product. The product description text includes structured attribute data. The product description text pairs are encoded to obtain product vector pairs, which are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data. The technical solution of determining matching product pairs based on the product vector pairs extracts structured attribute features and semantic features of product description text, and deeply fuses the features of structured data with the features of unstructured data. This allows for an accurate understanding of the attribute features of the products, thereby determining whether the products match without relying on matching rules, thus saving the maintenance cost of matching rules. By performing product matching based on the semantic features and structured attribute features of the products, product matching can be performed based on the semantic features of the products, improving the recall rate, avoiding the omission of potential matching products, and improving the product matching effect across different e-commerce platforms, thus improving the efficiency and accuracy of product matching.
[0064] Figure 4 An exemplary system architecture 400 is shown, in which the product matching method or apparatus of embodiments of the present invention can be applied.
[0065] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, and 403, a network 404, and a server 405. Network 404 serves as the medium for providing communication links between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.
[0066] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, and 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).
[0067] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.
[0068] Server 405 can be a server providing various services, such as a backend management server supporting shopping websites browsed by users using terminal devices 401, 402, and 403 (for example only). The backend management server can process received data such as product matching requests by combining product pairs, encoding product description text, and matching products, and then feed back the processing results (such as determined matching product pairs - for example only) to the terminal device.
[0069] It should be noted that the product matching method provided in this embodiment of the invention is generally executed by server 405, and correspondingly, the product matching device is generally set in server 405.
[0070] It should be understood that Figure 4 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.
[0071] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing terminal devices or servers of the present invention. Figure 5 The terminal device or server shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.
[0072] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0073] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.
[0074] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.
[0075] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0076] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0077] The units or modules described in the embodiments of the present invention can be implemented in software or hardware. The described units or modules can also be housed in a processor; for example, a processor may be described as including a data combination module, a data encoding module, and a product matching module. The names of these units or modules do not necessarily limit the specific unit or module itself; for example, a product matching module may also be described as "a module for determining matching product pairs based on product vector pairs of product pairs."
[0078] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to include: combining items to be matched in pairs to generate item pairs to be detected; generating item description text pairs corresponding to the item pairs based on item description text for each item, the item description text including structured attribute data; encoding the item description text pairs to obtain item vector pairs for the item pairs, the item vector pairs being generated based on the semantic vector of the item description text and the attribute vector of the structured attribute data; and determining matching item pairs based on the item vector pairs of the item pairs.
[0079] According to the technical solution of this invention, the products to be matched are combined in pairs to generate product pairs to be detected, and a corresponding product description text pair is generated based on the product description text of each product. The product description text includes structured attribute data. The product description text pairs are encoded to obtain product vector pairs, which are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data. The technical solution of determining matching product pairs based on the product vector pairs extracts structured attribute features and semantic features of product description text, and deeply fuses the features of structured data with the features of unstructured data. This allows for an accurate understanding of the attribute features of the products, thereby determining whether the products match without relying on matching rules, thus saving the maintenance cost of matching rules. By performing product matching based on the semantic features and structured attribute features of the products, product matching can be performed based on the semantic features of the products, improving the recall rate, avoiding the omission of potential matching products, and improving the product matching effect across different e-commerce platforms, thus improving the efficiency and accuracy of product matching.
[0080] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A method for product matching, characterized in that, include: The products to be matched are combined in pairs to generate product pairs to be detected, and a product description text pair corresponding to the product pair is generated based on the product description text of each product. The product description text includes structured attribute data. The product description text pairs are encoded to obtain product vector pairs, which are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data. The matching product pair is determined based on the product vector pairs of the product pair.
2. The method according to claim 1, characterized in that, The product description text also includes unstructured description text; Encoding the product description text pairs to obtain product vector pairs for the product pairs includes: The structured attribute data and the unstructured descriptive text are concatenated to obtain the concatenated descriptive text; The structured attribute data is encoded to obtain the attribute vector of the structured attribute data, and the concatenated description text is semantically encoded to obtain the semantic vector of the product description text; The attribute vector and the semantic vector are concatenated to obtain the product vector pair of the product pair.
3. The method according to claim 2, characterized in that, The concatenated description text is obtained by concatenating the structured attribute data and the unstructured description text, including: The field names and field values in the structured attribute data are concatenated in a specified format to obtain concatenated attribute data; The splicing attribute data and the unstructured description text are spliced together in sequence to obtain the spliced description text.
4. The method according to claim 2, characterized in that, Encoding the structured attribute data to obtain the attribute vector of the structured attribute data includes: The structured attribute data is input into an encoding model for encoding to obtain the attribute vector of the structured attribute data; Semantic encoding of the concatenated description text to obtain the semantic vector of the product description text includes: The concatenated description text is segmented to obtain at least one word element, and each word element is converted into a number sequence; The numerical sequence of each word is input into the encoding model for semantic encoding to obtain the semantic vector of the product description text.
5. The method according to claim 4, characterized in that, Before segmenting the concatenated description text to obtain at least one word, the method further includes: Obtain the length of the concatenated description text; If the length of the spliced description text exceeds a set threshold, the spliced description text will be truncated. The concatenated description text is segmented to obtain at least one word element, including: The truncated and concatenated descriptive text is segmented to obtain at least one word element.
6. The method according to claim 4, characterized in that, Before segmenting the concatenated description text to obtain at least one word, the method further includes: Obtain the length of the concatenated description text; If the length of the spliced description text exceeds a set threshold, the spliced description text is divided into multiple spliced description text fragments according to the set window length. The concatenated description text is segmented to obtain at least one word element, and each word element is converted into a number sequence, including: Each of the multiple concatenated descriptive text fragments is segmented into words to obtain at least one word element corresponding to each concatenated descriptive text fragment, and each word element is converted into a number sequence; The numerical sequence of each word is input into the encoding model for semantic encoding to obtain the semantic vector of the product description text, including: For each concatenated descriptive text fragment, the numerical sequence of at least one word corresponding to the concatenated descriptive text fragment is input into the encoding model for semantic encoding to obtain the semantic vector corresponding to the concatenated descriptive text fragment; The semantic vector of the product description text is obtained by concatenating the semantic vectors corresponding to the multiple concatenated description text fragments.
7. The method according to any one of claims 4-6, characterized in that, The encoding model was trained in the following way: Retrieve historical product description text and matching tags for product pairs from the product database; Data augmentation is performed on the historical product description text, and positive and negative sample pairs are generated by combining the matching tags; Based on the positive sample pairs and the negative sample pairs, the pre-trained language model is subjected to supervised training to obtain the encoding model.
8. The method according to claim 7, characterized in that, Based on the positive sample pairs and the negative sample pairs, the pre-trained language model is subjected to supervised training to obtain the encoding model, including: The positive sample pairs and the negative sample pairs are encoded using pre-trained language models to obtain semantic vector pairs for the positive sample pairs and semantic vector pairs for the negative sample pairs. A binary classification operation is performed based on the semantic vector pairs of the positive sample pairs and the semantic vector pairs of the negative sample pairs to obtain the binary classification probability of the positive sample pairs and the binary classification probability of the negative sample pairs. Based on a preset loss function, the binary classification probabilities of the positive sample pairs and the binary classification probabilities of the negative sample pairs, the pre-trained language model is subjected to supervised training to obtain the encoding model.
9. A product matching device, characterized in that, include: The data combination module is used to combine the products to be matched in pairs to generate product pairs to be detected, and to generate product description text pairs corresponding to the product pairs based on the product description text of each product, wherein the product description text includes structured attribute data. The data encoding module is used to encode the product description text pairs to obtain product vector pairs of the product pairs, wherein the product vector pairs are generated based on the semantic vector of the product description text and the attribute vector of the structured attribute data; The product matching module is used to determine matching product pairs based on the product vector pairs of the product pairs.
10. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.
11. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.
12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.