Congeneric product normalization method and system based on large model, storage medium and computer equipment

Abstract

The invention discloses a same commodity normalization method and system based on a large model, a storage medium and computer equipment. The method comprises the following steps: obtaining commodity titles and self-filling commodity codes corresponding to a plurality of commodities; clustering the commodity titles to determine a commodity class, determining a target commodity code corresponding to the commodity class according to the self-filling commodity code of each commodity in the commodity class, and filtering the commodities, of which the self-filling commodity codes belong to a commodity code cluster, in each commodity of the commodity class according to the commodity code cluster to which the target commodity code belongs, commodities corresponding to the commodity codes in the commodity code cluster are the same; and determining a first to-be-normalized commodity of which the self-filling commodity code does not belong to the commodity code cluster in each commodity of the commodity class, and according to a commodity title corresponding to the first to-be-normalized commodity and a standard title in a standard library, performing same-commodity normalization on the first to-be-normalized commodity through a large model so as to determine a correct commodity code corresponding to the first to-be-normalized commodity. And efficient and accurate same-product normalization of the commodities is realized.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, system, storage medium and computer device for normalizing the same product based on a large model. Background Art

[0002] In the e-commerce industry, product management is a complex and critical task that is directly related to the merchant's operational efficiency, inventory management, and effective implementation of market strategies. As an important part of product management, product normalization aims to classify products with the same or similar attributes into the same standard product for unified management and sales. Traditional methods, such as clustering and normalization based on CPV (Category, Property, Value) attributes, although they have achieved preliminary classification of products to a certain extent, have many limitations and cannot meet the growing demand for accurate management and efficient operations.

[0003] First, the CPV normalization method is highly dependent on the upstream structured understanding results. This process involves complex algorithms such as named entity recognition (NER), relationship recognition, synonym normalization, and regular matching in natural language processing (NLP). Each step may introduce errors, and the errors will accumulate as the processing flow deepens, resulting in a decrease in the accuracy of the final normalization result. Secondly, CPV normalization is essentially a clustering algorithm, which aggregates at the granularity of commodities and ultimately generates the ID of the cluster rather than directly associating it with the standard barcode in the standard product library. This indirect association method results in a lack of effective direct mapping between the normalization results and the barcodes in the standard product library in actual application scenarios, which increases the complexity and uncertainty of subsequent processing. At the same time, it is also difficult to intervene and adjust the clustering results, making it difficult to meet flexible and changing service needs.

[0004] In view of the many problems existing in the above-mentioned prior art, there is an urgent need for a more efficient, accurate and flexible method for normalizing the same products to solve the limitations of the prior art in commodity management. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a method, system, storage medium and computer device for normalizing the same products based on a large model, which realizes efficient and accurate normalization of the same products.

[0006] According to one aspect of the present application, a method for normalizing the same product based on a large model is provided, the method comprising:

[0007] Get the product titles and self-filled product codes corresponding to multiple products;

[0008] Clustering the commodity titles to determine a commodity category, determining a target commodity code corresponding to the commodity category according to the self-filled commodity code of each commodity in the commodity category, and filtering commodities whose self-filled commodity codes belong to the commodity code cluster among the commodities in the commodity category, wherein the commodities corresponding to each commodity code in the commodity code cluster are the same;

[0009] Determine the first commodity to be normalized among the commodities in the commodity category whose self-filled commodity code does not belong to the commodity code cluster, and normalize the first commodity to be normalized by a large model according to the commodity title corresponding to the first commodity to be normalized and the standard product title in the standard product library, so as to determine the correct commodity code corresponding to the first commodity to be normalized.

[0010] In the embodiment of the present application, optionally, after filtering whether the self-filled product code in each product of the product category belongs to the product code cluster, the method further includes:

[0011] The commodity whose self-filled commodity code belongs to the commodity code cluster among the commodities in the commodity category is taken as a first normalized commodity, and the self-filled commodity code corresponding to the first normalized commodity is determined to be a correct commodity code.

[0012] In the embodiment of the present application, optionally, according to the commodity title corresponding to the first commodity to be normalized and the standard commodity title in the standard commodity library, the first commodity to be normalized is normalized by a large model to determine the correct commodity code corresponding to the first commodity to be normalized, including:

[0013] Using the product title corresponding to the first product to be normalized as the first product title, and obtaining the first standard product title corresponding to the self-filled product code of the first product to be normalized in the standard product library;

[0014] Using the large model, the first commodity title and the first standard product title are used to identify whether the first commodity to be normalized and the standard commodity corresponding to the standard product title are the same product;

[0015] Determine the first commodity to be normalized that is the same commodity as the standard commodity corresponding to the standard commodity title as the second normalized commodity, and determine that the self-filled commodity code corresponding to the second normalized commodity is the correct commodity code;

[0016] Determine that the first commodity to be normalized which is not the same commodity as the standard commodity corresponding to the standard product title is the second commodity to be normalized, and based on the commodity title of the second commodity to be normalized and the standard product titles in the standard product library, normalize the second commodity to be normalized through a large model to determine the correct commodity code of the second commodity to be normalized.

[0017] In the embodiment of the present application, optionally, by using a large model, performing same-product discrimination according to the first product title and the first standard product title, and identifying whether the first product to be normalized and the standard product corresponding to the standard product title are the same product, includes:

[0018] Acquire a first category corresponding to the first commodity to be normalized, and acquire first key attribute dimension information corresponding to the first commodity to be normalized according to the key attribute dimension corresponding to the first category;

[0019] Acquire the second category to which the standard product corresponding to the first standard product title belongs and the second key attribute dimension information under the key attribute dimension;

[0020] Based on the first product title, the first category, the first key attribute dimension information, the first standard product title, the second category and the second key attribute dimension information, self-filled information judgment prompt information is constructed, and a large model is used to identify whether the first product to be normalized and the standard product corresponding to the first standard product title are the same product based on the self-filled information judgment prompt information.

[0021] In the embodiment of the present application, optionally, according to the product title of the second product to be normalized and the standard product title in the standard product library, the second product to be normalized is normalized by a large model to determine the correct product code of the second product to be normalized, including:

[0022] The product title of the second product to be normalized is used as the second product title, and title recall is performed based on the second product title vector corresponding to the second product title and the standard product title vector corresponding to each standard product title in the standard product library, and at least one recalled standard product title is obtained in the recall result;

[0023] According to the second product title and the recalled standard product title, the large model is used to identify the same product standard product title corresponding to the second product title in the recalled standard product title, and the product code corresponding to the same product standard product title is obtained as the correct product code of the second product to be normalized.

[0024] In the embodiment of the present application, optionally, obtaining at least one recalled standard product title in the recall result includes:

[0025] taking a preset number of recall results ranked first in the recall results as the first recalled standard product title, wherein the recalled standard product title includes the first recalled standard product title; and / or,

[0026] In the recall results, the recall results whose similarity with the second product title vector is greater than a preset threshold are obtained as the second recalled standard product title, wherein the recalled standard product title includes the second recalled standard product title.

[0027] In the embodiment of the present application, optionally, according to the second product title and the recalled standard product title, identifying the same product standard product title corresponding to the second product title in the recalled standard product title through a large model, and obtaining the product code corresponding to the same product standard product title as the correct product code of the second product to be normalized includes:

[0028] Identify through a large model whether the recalled standard product title has a standard product title of the same product corresponding to the second product title;

[0029] If it exists, obtaining the product code corresponding to the title of the same product standard product as the correct product code of the second product to be normalized;

[0030] If it does not exist, then obtain the standard product title corresponding to the target product code in the standard product library as the target standard product title, and use the large model to identify whether the second product title and the target standard product title are the same product title, and when the identification result is yes, use the target product code as the correct product code of the second product to be normalized; otherwise, determine that the self-filled product code of the second product to be normalized is the correct product code.

[0031] In the embodiment of the present application, optionally, identifying whether there is a standard product title of the same product corresponding to the second product title in the recalled standard product title through a large model includes:

[0032] According to the recalled standard product title and the second product title, vector recall discrimination prompt information is constructed, and through a large model, it is judged whether there is a standard product title of the same product corresponding to the second product title in the recalled standard product title according to the vector recall discrimination prompt information.

[0033] In the embodiment of the present application, optionally, identifying whether the second product title and the target standard product title are the same product title through the large model includes:

[0034] According to the target standard product title and the second product title, normalized title discrimination prompt information is constructed, and through a large model, according to the normalized title discrimination prompt information, it is determined whether the second product title and the target standard product title are the same product title.

[0035] In the embodiment of the present application, optionally, clustering the product titles to determine the product categories includes:

[0036] The product title is parsed to determine CPV attribute information corresponding to the product title, and the product title is clustered based on the CPV attribute information to obtain the product category.

[0037] In the embodiment of the present application, optionally, determining the target product code corresponding to the product category according to the self-filled product code of each product in the product category includes:

[0038] The self-filled commodity code of each commodity in the commodity category is counted, and the self-filled commodity code with the largest number of occurrences is determined as the target commodity code corresponding to the commodity category.

[0039] According to another aspect of the present application, a large model-based homogeneous product normalization system is provided, the system comprising:

[0040] Self-filled information module, used to obtain product titles and self-filled product codes corresponding to multiple products;

[0041] a normalization filtering module, configured to cluster the commodity titles to determine a commodity category, determine a target commodity code corresponding to the commodity category according to the self-filled commodity code of each commodity in the commodity category, and determine whether the self-filled commodity code in each commodity in the commodity category belongs to the commodity code cluster according to the commodity code cluster to which the target commodity code belongs, wherein the commodities corresponding to each commodity code in the commodity code cluster are the same;

[0042] The large model identification module is used to determine the first commodity to be normalized among the commodities in the commodity category whose self-filled commodity code does not belong to the commodity code cluster, and to normalize the first commodity to be normalized by the large model according to the commodity title corresponding to the first commodity to be normalized and the standard product title in the standard product library, so as to determine the correct commodity code corresponding to the first commodity to be normalized.

[0043] In the embodiment of the present application, optionally, the normalization filtering module is further used to:

[0044] The commodity whose self-filled commodity code belongs to the commodity code cluster among the commodities in the commodity category is taken as a first normalized commodity, and the self-filled commodity code corresponding to the first normalized commodity is determined to be a correct commodity code.

[0045] In the embodiment of the present application, optionally, the large model discrimination module includes:

[0046] A self-filled information identification module is used to use the product title corresponding to the first product to be normalized as the first product title, and obtain the first standard product title corresponding to the self-filled product code of the first product to be normalized in the standard product library; through a large model, perform same-product identification based on the first product title and the first standard product title, and identify whether the first product to be normalized and the standard product corresponding to the standard product title belong to the same product; determine the first product to be normalized that belongs to the same product as the standard product corresponding to the standard product title as the second normalized product, and determine that the self-filled product code corresponding to the second normalized product is the correct product code;

[0047] A vector recall discrimination module is used to determine that the first commodity to be normalized which is not the same commodity as the standard commodity corresponding to the standard product title is the second commodity to be normalized, and based on the commodity title of the second commodity to be normalized and the standard product titles in the standard product library, the second commodity to be normalized is normalized with the same commodity through a large model to determine the correct commodity code of the second commodity to be normalized.

[0048] In the embodiment of the present application, optionally, the self-filled information determination module is further used to:

[0049] Acquire a first category corresponding to the first commodity to be normalized, and acquire first key attribute dimension information corresponding to the first commodity to be normalized according to the key attribute dimension corresponding to the first category;

[0050] Acquire the second category to which the standard product corresponding to the first standard product title belongs and the second key attribute dimension information under the key attribute dimension;

[0051] Based on the first product title, the first category, the first key attribute dimension information, the first standard product title, the second category and the second key attribute dimension information, self-filled information judgment prompt information is constructed, and a large model is used to identify whether the first product to be normalized and the standard product corresponding to the first standard product title are the same product based on the self-filled information judgment prompt information.

[0052] In the embodiment of the present application, optionally, the vector recall discrimination module is further used to:

[0053] The product title of the second product to be normalized is used as the second product title, and title recall is performed based on the second product title vector corresponding to the second product title and the standard product title vector corresponding to each standard product title in the standard product library, and at least one recalled standard product title is obtained in the recall result;

[0054] According to the second product title and the recalled standard product title, the large model is used to identify the same product standard product title corresponding to the second product title in the recalled standard product title, and the product code corresponding to the same product standard product title is obtained as the correct product code of the second product to be normalized.

[0055] In the embodiment of the present application, optionally, the vector recall discrimination module is further used to:

[0056] taking a preset number of recall results ranked first in the recall results as the first recalled standard product title, wherein the recalled standard product title includes the first recalled standard product title; and / or,

[0057] In the recall results, the recall results whose similarity with the second product title vector is greater than a preset threshold are obtained as the second recalled standard product title, wherein the recalled standard product title includes the second recalled standard product title.

[0058] In the embodiment of the present application, optionally, the vector recall discrimination module is further used to:

[0059] Identify through the large model whether there is a standard product title of the same product corresponding to the second product title in the recalled standard product title; if so, obtain the product code corresponding to the standard product title of the same product as the correct product code of the second product to be normalized;

[0060] The system also includes: a normalized title identification module, which is used to obtain the standard product title corresponding to the target product code in the standard product library as the target standard product title if it does not exist, identify through a large model whether the second product title and the target standard product title are the same product title, and use the target product code as the correct product code of the second product to be normalized if the identification result is yes, otherwise determine that the self-filled product code of the second product to be normalized is the correct product code.

[0061] In an embodiment of the present application, optionally, the vector recall discrimination module is also used to construct vector recall discrimination prompt information based on the recalled standard product title and the second product title, and through a large model, to discriminate whether there is a standard product title of the same product corresponding to the second product title in the recalled standard product title based on the vector recall discrimination prompt information.

[0062] In an embodiment of the present application, optionally, the normalized title identification module is also used to construct normalized title identification prompt information based on the target standard product title and the second product title, and through a large model, determine whether the second product title and the target standard product title are the same product title based on the normalized title identification prompt information.

[0063] In an embodiment of the present application, optionally, the normalization filtering module is also used to parse the product title, determine the CPV attribute information corresponding to the product title, and cluster the product title based on the CPV attribute information to obtain the product category.

[0064] In the embodiment of the present application, optionally, the normalization and filtering module is further used to count the self-filled product codes of each product in the product category, and determine the self-filled product code with the largest number of occurrences as the target product code corresponding to the product category.

[0065] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned large model-based same-product normalization method is implemented.

[0066] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned large-model-based same-product normalization method when executing the program.

[0067] By means of the above technical solution, the embodiment of the present application provides a method, system, storage medium and computer device for normalizing the same product based on a large model. First, the product title and self-filled product code of the product are obtained, and then the product title is grouped by cluster analysis to determine the product category, and the target product code representing the product category is determined by counting the self-filled product codes corresponding to each product title in the product category, and the normalized products are filtered by the product code cluster where the target product code is located in each product category. For the remaining products to be normalized, that is, the products whose self-filled product codes do not belong to the product code cluster, the large model is used for analysis, and finally the correct product codes corresponding to these products are determined to achieve normalization of the same product. The embodiment of the present application introduces a large model to perform normalization analysis on product titles, which can more accurately identify the same product relationship between products and avoid the error accumulation problem introduced by the complex algorithm process in the traditional method. This method does not completely rely on the upstream structured understanding results, but combines cluster analysis and product code cluster filtering and other means to make the normalization process more flexible and adaptable to the needs of different scenarios. In the preliminary stage, cluster analysis and product code cluster filtering are used to effectively reduce the number of products that need to be processed by the large model, thereby reducing the computing cost and improving the processing efficiency. In summary, this technical solution achieves efficient and accurate same-product normalization of products by introducing a large model and combining a variety of technical means, providing merchants with more reliable and efficient product management services.

[0068] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0070] Figure 1 A schematic diagram of a process of a method for normalizing the same product based on a large model provided in an embodiment of the present application is shown;

[0071] Figure 2 A schematic diagram of the process of another method for normalizing the same product based on a large model provided in an embodiment of the present application is shown;

[0072] Figure 3 A schematic diagram of the process of another method for normalizing the same product based on a large model provided in an embodiment of the present application is shown;

[0073] Figure 4 A schematic diagram of the process of another method for normalizing the same product based on a large model provided in an embodiment of the present application is shown;

[0074] Figure 5 A schematic diagram of the structure of a large-model-based same-product normalization system provided in an embodiment of the present application is shown;

[0075] Figure 6 A structural schematic diagram of a same-product normalization system based on a large model provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0076] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0077] The same products are normalized, that is, the same products with the same objective description or similar human perception from different channels are aggregated and identified. Typical service application scenarios include:

[0078] Price comparison: Nowadays, competition in the lower-tier markets of e-commerce is becoming increasingly fierce, because consumers in these markets are more sensitive to commodity prices. Therefore, the core capability of identifying the same item can be used to compare prices of commodities in the same channel and across channels. Commodities with high cost-effectiveness can be given more traffic, and some price marketing labels can also be added, such as "same item, low price", to improve traffic conversion efficiency.

[0079] Shopping guide to find similar products: When consumers are searching for a certain product, they can view similar products of that product and compare them in terms of price, quality and store services.

[0080] Shopping guide breaks up the same items: In today's e-commerce scenarios with rich supply, when users have a specific demand, front-end service scenarios such as search often return a large number of visually identical products, which not only affects the consumer experience, but also hinders consumers' decision-making and conversion to a certain extent. Therefore, breaking up the same products at the front end through the same item recognition capability can enhance user experience and improve conversion efficiency.

[0081] Product selection for investment: Look for products with the potential to become hits, and find local equivalents based on popular products in competing markets.

[0082] By normalizing the same products, merchants can be effectively helped to manage their products, assist the operation of circle products in landing, etc., and serve multiple downstream service scenarios. Among them, standard products have product barcodes (i.e., product codes). The goal of normalizing the same products in the embodiment of this application is to map the products to the product codes of the standard products corresponding to the products in the standard product library, so as to correct the errors of the merchants in filling in the product codes themselves.

[0083] In this embodiment, a method for normalizing the same product based on a large model is provided, such as Figure 1 As shown, the method includes:

[0084] Step 101, obtaining product titles and self-filled product codes corresponding to a plurality of products.

[0085] In an embodiment of the present application, first, obtain the product titles and self-filled product codes corresponding to multiple products that need to be unified in the e-commerce platform. The product title is the title filled in by the merchant for the product, and the self-filled product code is the product code filled in by the merchant for the product. The product title usually includes key information such as the name and attributes of the product, while the self-filled product code is a unique identifier assigned by the merchant to the product according to its own rules or habits.

[0086] Step 102, clustering the product titles to determine the product category, determining the target product code corresponding to the product category based on the self-filled product code of each product in the product category, and filtering the products in the product category whose self-filled product codes belong to the product code cluster based on the product code cluster to which the target product code belongs, wherein the products corresponding to each product code in the product code cluster are the same product.

[0087] Secondly, by clustering the commodity titles, commodities with similar or identical attributes are classified into the same category (commodity class). Through clustering analysis, a large number of commodities can be quickly grouped according to similar attributes, thereby improving processing efficiency. In each commodity class, a target commodity code is determined according to the frequency of occurrence of the self-filled commodity code or other rules. The target commodity code should be able to represent the main characteristics of the commodity class. Then, the commodities are filtered according to the commodity code cluster to which the target commodity code belongs (i.e., a set of commodity codes known to be the same product), and those commodities whose self-filled commodity codes already belong to the commodity code cluster are regarded as normalized commodities, and no further processing is performed, thereby using the commodity code cluster for preliminary filtering, reducing the number of commodities that need to be processed by a large model and reducing the computational cost. Among them, due to changes in packaging and version, the same commodity can correspond to multiple commodity codes, and each commodity code can be set with its own standard product title. The commodity code cluster to which the target commodity code belongs can be obtained through a pre-constructed one-product-multiple-code mapping table. In the one-product-multiple-code mapping table, one commodity corresponds to one or more standard product titles and the commodity code corresponding to each standard product title. Compared with the traditional CPV normalization method, this method does not completely rely on the upstream structured understanding results, but assists in normalization through cluster analysis and product code clusters, thereby improving the flexibility and accuracy of the normalization process.

[0088] Step 103, determine the first commodity to be normalized among the commodities in the commodity category whose self-filled commodity code does not belong to the commodity code cluster, and normalize the first commodity to be normalized by a large model according to the commodity title corresponding to the first commodity to be normalized and the standard product title in the standard product library, so as to determine the correct commodity code corresponding to the first commodity to be normalized.

[0089] Finally, the products whose self-filled product codes do not belong to the product code cluster (i.e., the first products to be normalized) are determined. These products need to be further processed to determine their correct product codes. A large model (such as a large language model or a multimodal model) is used to compare and analyze the product titles of these products with the standard product titles in the standard product library, and the most matching standard product titles are found through semantic understanding, similarity calculation, etc., so as to determine the correct product code corresponding to the first product to be normalized. The large model has strong semantic understanding and generalization capabilities, and can handle complex and changeable product title information, improving the accuracy and robustness of the normalization of the same product. The normalization of the same product by the large model avoids the error accumulation problems in the algorithm flow such as synonym normalization and regular matching that may exist in traditional methods.

[0090] By applying the technical solution of this embodiment, first obtain the product title and self-filled product code of the product, then use cluster analysis to group the product title to determine the product category, and determine the target product code representing the product category by counting the self-filled product code corresponding to each product title in the product category, and filter the normalized products in each product category through the product code cluster where the target product code is located. For the remaining products to be normalized, that is, the products whose self-filled product codes do not belong to the product code cluster, a large model is used for analysis, and finally the correct product codes corresponding to these products are determined to achieve the normalization of the same products. The embodiment of the present application can more accurately identify the same product relationship between products by introducing a large model for normalization analysis of product titles, avoiding the error accumulation problem introduced by the traditional method due to the complexity of the algorithm flow. This method does not completely rely on the upstream structured understanding results, but combines a variety of means such as cluster analysis and product code cluster filtering, so that the normalization process is more flexible and changeable, and can adapt to the needs of different scenarios. In the preliminary stage, cluster analysis and product code cluster filtering are used to effectively reduce the number of products that need to be processed by the large model, thereby reducing the calculation cost and improving the processing efficiency. In summary, this technical solution achieves efficient and accurate normalization of goods by introducing a large model and combining a variety of technical means, providing merchants with more reliable and efficient product management services.

[0091] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another method of normalizing the same product based on a large model is provided, such as Figure 2 As shown, the method includes:

[0092] Step 201, obtaining product titles and self-filled product codes corresponding to a plurality of products.

[0093] Step 202, clustering the product titles to determine the product category, determining the target product code corresponding to the product category based on the self-filled product code of each product in the product category, and filtering the products in the product category whose self-filled product codes belong to the product code cluster based on the product code cluster to which the target product code belongs, wherein the products corresponding to each product code in the product code cluster are the same product.

[0094] In this embodiment, similar product titles are grouped into the same category through a clustering algorithm, and then in each category, products that have been normalized (i.e., the product codes have been correctly matched) are filtered out according to the product code cluster (i.e., a known set of product codes of the same product). Clustering technology is used to improve processing efficiency, and at the same time, normalized products are quickly identified and filtered out, reducing the complexity of subsequent processing.

[0095] In an embodiment of the present application, optionally, in step 202, clustering the product titles to determine the product category includes: parsing the product titles, determining CPV attribute information corresponding to the product titles, and clustering the product titles based on the CPV attribute information to obtain the product category.

[0096] In the above embodiment, before clustering, the product title is first parsed to extract the CPV attribute information of the product. The CPV system includes categories, properties, and attribute values. Categories are the basic units of product classification, properties are attributes that describe products, and attribute values ​​are specific numerical values ​​of attributes. These attribute information provide a standardized way of classifying products according to the CPV system, which helps to more accurately understand the similarities and differences between products. By parsing the product title and determining the CPV attribute information, the accuracy and efficiency of clustering can be improved. Specifically, clustering can be performed based on the CPV attribute information and the text information of the product title, so that the clustering process not only depends on the text similarity of the title, but also takes into account the actual attributes and classification relationships between the products, thereby avoiding the misleading that may be caused by simple text similarity.

[0097] In an embodiment of the present application, optionally, in step 202, a target product code corresponding to the product category is determined based on the self-filled product code of each product in the product category, including: counting the self-filled product code of each product in the product category, and determining the self-filled product code with the largest number of occurrences as the target product code corresponding to the product category.

[0098] In the above embodiment, after the commodity category is determined, the self-filled commodity codes of all commodities in the category are counted, and the self-filled commodity code with the largest number of occurrences is used as the target commodity code of the commodity category. By counting the number of occurrences of the self-filled commodity code to determine the target commodity code, the influence of noise data can be reduced to a certain extent, and the stability and reliability of commodity code determination can be improved. At the same time, this also provides a clear benchmark for the subsequent filtering process, so that commodities that have been normalized (i.e., the commodity codes have been correctly matched) can be more effectively identified.

[0099] Step 203: The product whose self-filled product code belongs to the product code cluster among the products of the product category is taken as the first normalized product, and the self-filled product code corresponding to the first normalized product is determined to be a correct product code.

[0100] In this embodiment, the commodities screened out in step 202 are regarded as normalized commodities, and it is confirmed that the self-filled commodity codes are correct, so that the normalized commodities are quickly confirmed and marked, providing a clear boundary for subsequent processing.

[0101] Step 204, determine the first commodity to be normalized whose self-filled commodity code does not belong to the commodity code cluster among the commodities in the commodity category, and use the commodity title corresponding to the first commodity to be normalized as the first commodity title, and obtain the first standard product title corresponding to the self-filled commodity code of the first commodity to be normalized in the standard product library; through the large model, perform same-product discrimination based on the first commodity title and the first standard product title, and identify whether the first commodity to be normalized and the standard commodity corresponding to the standard product title are the same product.

[0102] In this embodiment, for products that have not passed the product code cluster filtering (i.e., the first products to be normalized), the large model is used to compare the similarity between their product titles and the standard product titles in the standard product library to determine whether they are the same product. The deep semantic understanding ability of the large model is used to improve the accuracy and flexibility of same-product discrimination, which helps to achieve same-product normalization of complex or ambiguous product titles.

[0103] In an embodiment of the present application, optionally, in step 204, a large model is used to perform product identification based on the first product title and the first standard product title to identify whether the first product to be unified and the standard product corresponding to the standard product title are the same product, including: obtaining the first category corresponding to the first product to be unified, and obtaining the first key attribute dimension information corresponding to the first product to be unified based on the key attribute dimension corresponding to the first category; obtaining the second category to which the standard product corresponding to the first standard product title belongs and the second key attribute dimension information under the key attribute dimension; constructing self-filled information identification prompt information based on the first product title, the first category, the first key attribute dimension information, the first standard product title, the second category and the second key attribute dimension information, and identifying whether the first product to be unified and the standard product corresponding to the first standard product title are the same product based on the self-filled information identification prompt information through a large model.

[0104] In this embodiment, in order to more accurately determine whether the first commodity to be normalized and the standard commodity in the standard product library belong to the same product, consideration of commodity categories and key attribute dimension information can be introduced to enhance the accuracy and reliability of the same product identification. First, the commodity category corresponding to the first commodity to be normalized is obtained as the first category. Then, according to the key attribute dimensions corresponding to the category (such as brand, model, specification, etc.), the first key attribute dimension information of the first commodity to be normalized is further obtained. For example, the category is office supplies_binding supplies_staples, and the corresponding key attribute dimensions are commodity body, brand and staple type. For another example, the category is clothing shoes and bags_men's and women's underwear / home clothes_nightgowns / nightgowns, and the corresponding key attribute dimensions are commodity body, material, brand and applicable gender. Similarly, the category to which the standard commodity corresponding to the first standard product title belongs is obtained as the second category, and the second key attribute dimension information of the standard commodity under the key attribute dimension is queried. Next, the first product title, the first category, the first key attribute dimension information, the first standard product title, the second category and the second key attribute dimension information are integrated to construct a self-filled information discrimination prompt information containing rich context information. This prompt information not only contains the text description of the product (product title), but also contains the classification and key attribute information of the product, providing a more comprehensive and in-depth judgment basis for the large model. Thus, by constructing the self-filled information discrimination prompt information, the large model can more accurately understand the intrinsic attributes and classification relationships of the product, so as to more accurately judge whether the two products belong to the same product. This method effectively overcomes the limitations of simply relying on text similarity for the same product discrimination, and improves the accuracy and robustness of the discrimination. Finally, the constructed self-filled information discrimination prompt information is input into the large model, and the deep learning and semantic understanding capabilities of the large model are used to discriminate the same product. The large model will comprehensively judge the similarity of the two products in multiple dimensions based on the input information, thereby giving a conclusion on whether they belong to the same product. This embodiment makes full use of its advantages in processing complex data and semantic relationships by introducing a large model for the same product discrimination. Large models can process large amounts of data and learn patterns and rules in them, thereby more accurately identifying similarities and differences between products. This approach not only improves the accuracy of product identification, but also greatly reduces the need for human intervention, improving processing efficiency and automation.

[0105] As an example, the self-filled information identification prompt information may be:

[0106] Now there are two retail related products, giving you the relevant product titles, categories, and key attribute dimensions:

[0107] - Product 1: Brand A No. 12 staples / box;

[0108] - Product 2: Brand A No. 10 staples;

[0109] - Both categories are: Office supplies_Binding supplies_Staples

[0110] The key attributes are: product entity, brand, and staple type

[0111] Please judge whether Product 1 and Product 2 are the same product based on the listed attribute dimensions and product titles? If there are other attribute dimensions that you think are important, please also consider them. Please directly answer "yes" or "no" or "cannot judge".

[0112] In addition, in addition to performing same-product discrimination based on the above self-filled information discrimination prompt information, the inference ability of the large model can also be used to optimize the key attribute dimensions. For example, the cause analysis content is added to the self-filled information discrimination prompt information of the large model, and the large model is used to analyze which attribute dimension is inconsistent and leads to the conclusion that the product is not the same product. The key attribute dimensions are adjusted according to the reasoning conclusions of the large model. Specifically, when the large model performs same-product analysis on products in a certain category, it can output multiple times that the same-product analysis conclusion is that it does not belong to the same product due to inconsistency in a certain attribute dimension, and the attribute dimension does not belong to the key attribute dimension that has been defined for the category. In this case, this attribute dimension is also used as the key attribute dimension under the category. Furthermore, in the above example, a discrimination example can be added to provide a reference for the large model. As another example, the self-filled information discrimination prompt information can be:

[0113] Now there are two retail related products, giving you the relevant product titles, categories, and key attribute dimensions:

[0114] - Product 1: Brand A No. 12 staples / box;

[0115] - Product 2: Brand A No. 10 staples;

[0116] - Both categories are: Office supplies_Binding supplies_Staples

[0117] The key attributes are: product entity, brand, and staple type

[0118] Please judge whether Product 1 and Product 2 are the same product based on the listed attribute dimensions and product titles? If there are other attribute dimensions that you think are important, please also consider them. Please directly answer "yes" or "no" or "cannot judge". If the answer is "no", please analyze which attribute dimension is inconsistent.

[0119] ###

[0120] Here is an example:

[0121] enter:

[0122] -Product 1: Honey Peanuts 260g / box;

[0123] - Product 2: Honey peanuts, about 300g / portion;

[0124] - The categories described in both are: Snacks_Nuts and Roasted Seeds_Peanuts in Shell / Peanut Kernels

[0125] -Key attribute dimensions are: commodity, whether imported, brand, net content

[0126] Output:

[0127] No. The inconsistent attribute dimensions are: Net content

[0128] ###

[0129] Step 205, determining the first commodity to be normalized that is the same product as the standard commodity corresponding to the standard product title as the second normalized commodity, and determining that the self-filled commodity code corresponding to the second normalized commodity is the correct commodity code.

[0130] In this embodiment, for the commodities that the large model determines to be the same, i.e., the second normalized commodities, it is confirmed that the self-filled commodity codes are correct. Through the judgment of the large model, the scope of the normalized commodities is further expanded, and the coverage and accuracy of commodity normalization are improved.

[0131] Step 206, determine that the first commodity to be normalized that does not belong to the same product as the standard commodity corresponding to the standard product title is the second commodity to be normalized, and based on the commodity title of the second commodity to be normalized and the standard product titles in the standard product library, normalize the second commodity to be normalized through a large model to determine the correct commodity code of the second commodity to be normalized.

[0132] In this embodiment, for the products that the large model determines are not the same product, i.e., the second product to be normalized, the large model is further used to normalize the same products according to the product title of the second product to be normalized and the titles of each standard product in the standard product library. For example, the large model is used to find the title in the standard product title that is closest to the product title of the second product to be normalized, and the product code corresponding to the title is used as the correct product code of the second product to be normalized, so as to achieve the same product normalization.

[0133] In the embodiment of the present application, optionally, step 206 includes:

[0134] Step 206-1, taking the product title of the second product to be normalized as the second product title, and performing title recall based on the second product title vector corresponding to the second product title and the standard product title vector corresponding to each standard product title in the standard product library, and obtaining at least one recalled standard product title in the recall result;

[0135] Step 206-2: Based on the second product title and the recalled standard product title, identify the same product standard product title corresponding to the second product title in the recalled standard product title through a large model, and obtain the product code corresponding to the same product standard product title as the correct product code of the second product to be normalized.

[0136] In the above embodiment, the second product title of the product to be normalized, that is, the second product title, is converted into a vector form, and a vector recall is performed in the standard product library to obtain standard product titles that may be similar to the product to be normalized, so that the scope of possible same products can be quickly narrowed down through vector recall technology, thereby improving the efficiency and accuracy of subsequent processing. In the recall results, a large model is used to further determine which standard product titles and the product to be normalized are truly the same product, and the corresponding product code is obtained as the correct product code of the product to be normalized. Finally, through the fine discrimination of the large model, the correct product code is determined for all products to be normalized, and comprehensive normalization of the same product is achieved.

[0137] In an embodiment of the present application, optionally, in step 206-1, at least one recalled product title is obtained in the recall results, including: taking a preset number of recall results ranked first in the recall results as the first recalled product title, wherein the recalled product title includes the first recalled product title; and / or, taking a recall result in the recall results whose similarity with the second product title vector is greater than a preset threshold as the second recalled product title, wherein the recalled product title includes the second recalled product title.

[0138] In the above embodiment, a preset number of recall results ranked first in the recall results are taken as the first recalled standard product title. The "preset number" here is a threshold value set according to actual needs, which is used to control the number of recalled standard product titles returned, for example, the first one is taken, so as to quickly screen out the recall results that are most likely to be related to the second product title, reducing the complexity and amount of calculation of subsequent processing. In addition, the recall results whose similarity with the second product title is greater than the preset threshold can also be obtained from the recall results as the second recalled standard product title. The "preset threshold" here is set according to actual needs, and is used to determine whether the similarity between the two title vectors is high enough to ensure that the selected recalled standard product title is highly similar to the second product title in semantics, thereby improving the accuracy of subsequent same product discrimination. In practical applications, the two strategies can be adopted at the same time, and the recalled standard product titles obtained by the two methods can be merged and processed to further improve the accuracy and efficiency of same product discrimination.

[0139] In an embodiment of the present application, optionally, step 206-2 includes: identifying through a large model whether there is a same-product standard product title corresponding to the second product title in the recalled standard product title; if so, obtaining the product code corresponding to the same-product standard product title as the correct product code of the second product to be normalized; if not, obtaining the standard product title corresponding to the target product code in the standard product library as the target standard product title, identifying through a large model whether the second product title and the target standard product title are the same product titles, and if the identification result is yes, using the target product code as the correct product code of the second product to be normalized, otherwise determining that the self-filled product code of the second product to be normalized is the correct product code.

[0140] In the above embodiment, the large model is first used to check whether there is a standard product title of the same product corresponding to the title of the second product in the title of the recalled standard product. When the second product to be normalized and the standard product belong to the same product, the titles of the two belong to the same product title. If the large model recognizes that there is such a standard product title of the same product, the system will obtain the product code corresponding to the standard product title of the same product and regard it as the correct product code of the second product to be normalized. If the corresponding standard product title of the same product is not found in the title of the recalled standard product, the system will turn to the standard product library for search. The standard product library is a database containing standardized product information, including the correspondence between standard product titles and product codes. The standard product title corresponding to the target product code is found in the standard product library as the target standard product title, and then the large model is used again to identify whether the second product title and the target standard product title are the same product titles. If the recognition result is yes, that is, the second product title and a standard product title in the standard product library describe the same product, then the system will use the product code corresponding to the standard product title as the correct product code of the second product to be normalized. If the above steps fail to successfully find a matching product title or product code, that is, the big model has neither found the same product title in the recalled product title nor the same product title in the product library, then the system will default to the self-filled product code of the second product to be normalized as the correct product code. This process fully utilizes the recognition ability of the big model and the standardized information of the product library, aiming to improve the accuracy and efficiency of product code normalization.

[0141] In an embodiment of the present application, optionally, a large model is used to identify whether there is a standard product title of the same product corresponding to the second product title in the recalled product title, including: constructing vector recall discrimination prompt information based on the recalled product title and the second product title, and using the large model to discriminate whether there is a standard product title of the same product corresponding to the second product title in the recalled product title based on the vector recall discrimination prompt information.

[0142] In the above embodiment, the recalled standard product title and the second product title vector (which may also include the product attribute information of the standard product corresponding to the recalled standard product title and the product attribute information of the second product to be normalized) are integrated together to construct a standard product title recall discrimination prompt information containing rich context information. This prompt information is intended to provide a comprehensive and accurate judgment basis for the large model. The constructed standard product title recall discrimination prompt information is input into the large model so that after the large model receives the discrimination prompt information, it will use its internal learning algorithm and parameters to process and analyze the information. For example, compare the similarity between the recalled standard product title and the second product title vector, consider the category and attribute information of the product, and output a discrimination result, that is, whether there is a standard product title of the same product corresponding to the second product title vector in the recalled standard product title. This result is usually a binary value (yes / no) used to indicate whether the same product title is found, and if the same product title is found, the same product standard product title (or same product title) is also output. By constructing the standard product title recall discrimination prompt information and using the large model for discrimination, the standard product title of the same product corresponding to the second product title vector in the recalled standard product title can be identified more accurately and efficiently. This method not only improves the accuracy of product code normalization, but also speeds up the processing speed, providing strong support for subsequent inventory management and sales optimization.

[0143] In an embodiment of the present application, optionally, a large model is used to identify whether the second product title and the target product title are the same product title, including: constructing a normalized title discrimination prompt information based on the target product title and the second product title, and using the large model to determine whether the second product title and the target product title are the same product title based on the normalized title discrimination prompt information.

[0144] In the above embodiment, by constructing a normalized title discrimination prompt information and using a large model for discrimination, the same product relationship between the second product title and the target standard product title can be identified more accurately and efficiently, and the need for human intervention is reduced.

[0145] In a specific application scenario, such as Figure 3As shown, the product normalization method based on the large model can be regarded as a funnel process: First, CPV normalization filtering is performed: for products that need to be normalized, the original product title of the product (that is, the product title filled in by the merchant) is structured to understand in order to determine the CPV attribute information of the product, and the CPV clustering of the product is performed to determine the product category (the product category can be represented by the MD5 clustering id), and the self-filled product code that appears most times in the product category is counted as barcode 1, and the barcode cluster corresponding to barcode 1 is determined based on the one-product-multiple-code table, and it is determined whether the self-filled product code (barcode 0) of the product belongs to the barcode cluster. If it does, the barcode 0 of the product is kept unchanged. If it does not, the next step of self-filled information judgment (that is, self-filled judgment in the figure) process is carried out. Secondly, self-filled information is identified: for products that have not been normalized in the previous process, the barcode 0 corresponding to the standard product title in the standard product library is found according to the barcode 0, that is, the barcode 0 title, and the LLM (large language model) same product identification is performed according to the barcode 0 title and the original product title. If it is determined to be the same product, the barcode 0 of the product is kept unchanged. If it is determined not to be the same product, the next step of the vector recall identification process is performed. Next, the vector recall identification is performed: the original product title is vectorized to obtain the title vector, and the title vector is recalled and retrieved according to the standard product vector library obtained by vectorizing each standard product title in the standard product library. Specifically, the TopK similar standard product titles can be taken as the recalled standard product titles, and the original product title and the recalled standard product title are LLM same product identification. It is determined whether there is the same product as the original product title in the recalled standard product title. If so, the product code of the standard product corresponding to the recalled standard product title that belongs to the same product as the original product title, that is, barcode 2, is used as the correct product code of the product, otherwise the next step of the same product title identification process based on CPV normalization is entered. Finally, the same product title based on CPV normalization (i.e., CPV normalization judgment in the figure) is judged: for the products that have not been normalized in the previous process, LLM same product judgment is performed based on the original product title of the product and the standard product title of barcode 1 corresponding to the product category to which the product belongs (i.e., barcode 1 title). Whether they are the same product is determined based on the original product title and the barcode 1 title. If they are the same product, barcode 1 is used as the correct product title of the product, otherwise barcode 0 is used as the correct title of the product.

[0146] In the above process, considering issues such as computing resources, the overall process adopts a funnel form, and proceeds layer by layer according to the confidence level / barcode source, and the predicted amount gradually decreases. Since CPV normalization is a clustering algorithm (it is impossible to directly obtain the product code of the standard product library) and lacks flexibility (structured understanding of error transmission), the confidence level is relatively low, and it is used as a bottom-up algorithm; but through the high-frequency self-filled barcodes of merchants, a secondary judgment can be made, so it is used as a pre-filter item for LLM judgment. In the actual application process of the same product normalization, the total number of products on a certain e-commerce platform exceeds 790 million per day. After the first step of CPV normalization filtering, about 420 million products are normalized, accounting for about 53%. After the second step of self-filled information judgment, about 150 million products are normalized, accounting for about 19%. After the third step of vector recall judgment, about 70 million products are normalized, accounting for about 8%. After the fourth step of same product title judgment, about 8.86 million products are normalized, accounting for about 1%. Figure 4 As shown in the figure, the whole process realizes the funnel-like normalization of the same product.

[0147] By applying the technical solution of this embodiment and introducing a large model to identify the same product title, it is possible to more accurately determine whether different product titles describe the same or similar products, thereby improving the accuracy of product code normalization, and helping to reduce problems such as chaotic inventory management and inaccurate sales data caused by incorrect product codes. The solution adopts multiple strategies to obtain the recalled standard product titles, which can quickly screen out the recall results that are most likely to be related to the product to be normalized, reducing the complexity and calculation amount of subsequent processing, and improving the overall processing efficiency.

[0148] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a system for normalizing the same product based on a large model, such as Figure 5 As shown, the system includes:

[0149] Self-filled information module, used to obtain product titles and self-filled product codes corresponding to multiple products;

[0150] a normalization filtering module, configured to cluster the commodity titles to determine a commodity category, determine a target commodity code corresponding to the commodity category according to the self-filled commodity code of each commodity in the commodity category, and determine whether the self-filled commodity code in each commodity in the commodity category belongs to the commodity code cluster according to the commodity code cluster to which the target commodity code belongs, wherein the commodities corresponding to each commodity code in the commodity code cluster are the same;

[0151] The large model identification module is used to determine the first commodity to be normalized among the commodities in the commodity category whose self-filled commodity code does not belong to the commodity code cluster, and to normalize the first commodity to be normalized by the large model according to the commodity title corresponding to the first commodity to be normalized and the standard product title in the standard product library, so as to determine the correct commodity code corresponding to the first commodity to be normalized.

[0152] In the embodiment of the present application, optionally, the normalization filtering module is further used to:

[0153] The commodity whose self-filled commodity code belongs to the commodity code cluster among the commodities in the commodity category is taken as a first normalized commodity, and the self-filled commodity code corresponding to the first normalized commodity is determined to be a correct commodity code.

[0154] In the embodiment of the present application, optionally, Figure 6 As shown, the large model discrimination module includes:

[0155] A self-filled information identification module is used to use the product title corresponding to the first product to be normalized as the first product title, and obtain the first standard product title corresponding to the self-filled product code of the first product to be normalized in the standard product library; through a large model, perform same-product identification based on the first product title and the first standard product title, and identify whether the first product to be normalized and the standard product corresponding to the standard product title belong to the same product; determine the first product to be normalized that belongs to the same product as the standard product corresponding to the standard product title as the second normalized product, and determine that the self-filled product code corresponding to the second normalized product is the correct product code;

[0156] A vector recall discrimination module is used to determine that the first commodity to be normalized which is not the same commodity as the standard commodity corresponding to the standard product title is the second commodity to be normalized, and based on the commodity title of the second commodity to be normalized and the standard product titles in the standard product library, the second commodity to be normalized is normalized with the same commodity through a large model to determine the correct commodity code of the second commodity to be normalized.

[0157] In the embodiment of the present application, optionally, the self-filled information determination module is further used to:

[0158] Acquire a first category corresponding to the first commodity to be normalized, and acquire first key attribute dimension information corresponding to the first commodity to be normalized according to the key attribute dimension corresponding to the first category;

[0159] Acquire the second category to which the standard product corresponding to the first standard product title belongs and the second key attribute dimension information under the key attribute dimension;

[0160] Based on the first product title, the first category, the first key attribute dimension information, the first standard product title, the second category and the second key attribute dimension information, self-filled information judgment prompt information is constructed, and a large model is used to identify whether the first product to be normalized and the standard product corresponding to the first standard product title are the same product based on the self-filled information judgment prompt information.

[0161] In the embodiment of the present application, optionally, the vector recall discrimination module is further used to:

[0162] The product title of the second product to be normalized is used as the second product title, and title recall is performed based on the second product title vector corresponding to the second product title and the standard product title vector corresponding to each standard product title in the standard product library, and at least one recalled standard product title is obtained in the recall result;

[0163] According to the second product title and the recalled standard product title, the large model is used to identify the same product standard product title corresponding to the second product title in the recalled standard product title, and the product code corresponding to the same product standard product title is obtained as the correct product code of the second product to be normalized.

[0164] In the embodiment of the present application, optionally, the vector recall discrimination module is further used to:

[0165] taking a preset number of recall results ranked first in the recall results as the first recalled standard product title, wherein the recalled standard product title includes the first recalled standard product title; and / or,

[0166] In the recall results, the recall results whose similarity with the second product title vector is greater than a preset threshold are obtained as the second recalled standard product title, wherein the recalled standard product title includes the second recalled standard product title.

[0167] In the embodiment of the present application, optionally, the vector recall discrimination module is further used to:

[0168] Identify through the large model whether there is a standard product title of the same product corresponding to the second product title in the recalled standard product title; if so, obtain the product code corresponding to the standard product title of the same product as the correct product code of the second product to be normalized;

[0169] The system also includes: a normalized title identification module, which is used to obtain the standard product title corresponding to the target product code in the standard product library as the target standard product title if it does not exist, identify through a large model whether the second product title and the target standard product title are the same product title, and use the target product code as the correct product code of the second product to be normalized if the identification result is yes, otherwise determine that the self-filled product code of the second product to be normalized is the correct product code.

[0170] In an embodiment of the present application, optionally, the vector recall discrimination module is also used to construct vector recall discrimination prompt information based on the recalled standard product title and the second product title, and through a large model, to discriminate whether there is a standard product title of the same product corresponding to the second product title in the recalled standard product title based on the vector recall discrimination prompt information.

[0171] In an embodiment of the present application, optionally, the normalized title identification module is also used to construct normalized title identification prompt information based on the target standard product title and the second product title, and through a large model, determine whether the second product title and the target standard product title are the same product title based on the normalized title identification prompt information.

[0172] In an embodiment of the present application, optionally, the normalization filtering module is also used to parse the product title, determine the CPV attribute information corresponding to the product title, and cluster the product title based on the CPV attribute information to obtain the product category.

[0173] In the embodiment of the present application, optionally, the normalization and filtering module is further used to count the self-filled product codes of each product in the product category, and determine the self-filled product code with the largest number of occurrences as the target product code corresponding to the product category.

[0174] It should be noted that for other corresponding descriptions of the functional units involved in the large-model-based same-product normalization system provided in the embodiment of the present application, reference can be made to Figure 1 to Figure 2 The corresponding description in the method will not be repeated here.

[0175] The embodiment of the present application also provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes a bus, a processor, a memory and a communication interface, and can also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0176] Those skilled in the art will appreciate that the structure of the above-mentioned computer device is only a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components, or combine certain components, or have a different arrangement of components.

[0177] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0178] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0179] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0180] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0181] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0182] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for normalizing the same product based on a large model, characterized in that: The method comprises: Get the product titles and self-filled product codes corresponding to multiple products; Clustering the commodity titles to determine a commodity category, determining a target commodity code corresponding to the commodity category according to the self-filled commodity code of each commodity in the commodity category, and filtering commodities whose self-filled commodity codes belong to the commodity code cluster among the commodities in the commodity category, wherein the commodities corresponding to each commodity code in the commodity code cluster are the same; Determine the first commodity to be normalized among the commodities in the commodity category whose self-filled commodity code does not belong to the commodity code cluster, and normalize the first commodity to be normalized by a large model according to the commodity title corresponding to the first commodity to be normalized and the standard product title in the standard product library, so as to determine the correct commodity code corresponding to the first commodity to be normalized.

2. The method according to claim 1, characterized in that After filtering whether the self-filled product code in each product of the product category belongs to the product code cluster, the method further includes: The commodity whose self-filled commodity code belongs to the commodity code cluster among the commodities in the commodity category is taken as a first normalized commodity, and the self-filled commodity code corresponding to the first normalized commodity is determined to be a correct commodity code.

3. The method according to claim 1 or 2, characterized in that: According to the product title corresponding to the first product to be normalized and the standard product title in the standard product library, the first product to be normalized is normalized by a large model to determine a correct product code corresponding to the first product to be normalized, including: Using the product title corresponding to the first product to be normalized as the first product title, and obtaining the first standard product title corresponding to the self-filled product code of the first product to be normalized in the standard product library; Using the large model, the first commodity title and the first standard product title are used to identify whether the first commodity to be normalized and the standard commodity corresponding to the standard product title are the same product; Determine the first commodity to be normalized that is the same commodity as the standard commodity corresponding to the standard commodity title as the second normalized commodity, and determine that the self-filled commodity code corresponding to the second normalized commodity is the correct commodity code; Determine that the first commodity to be normalized which is not the same commodity as the standard commodity corresponding to the standard product title is the second commodity to be normalized, and based on the commodity title of the second commodity to be normalized and the standard product titles in the standard product library, normalize the second commodity to be normalized through a large model to determine the correct commodity code of the second commodity to be normalized.

4. The method according to claim 3, characterized in that Using the large model, performing product identification according to the first product title and the first standard product title to identify whether the first product to be normalized and the standard product corresponding to the standard product title are the same product, including: Acquire a first category corresponding to the first commodity to be normalized, and acquire first key attribute dimension information corresponding to the first commodity to be normalized according to the key attribute dimension corresponding to the first category; Acquire the second category to which the standard product corresponding to the first standard product title belongs and the second key attribute dimension information under the key attribute dimension; Based on the first product title, the first category, the first key attribute dimension information, the first standard product title, the second category and the second key attribute dimension information, self-filled information judgment prompt information is constructed, and a large model is used to identify whether the first product to be normalized and the standard product corresponding to the first standard product title are the same product based on the self-filled information judgment prompt information.

5. The method according to claim 3, characterized in that: According to the product title of the second product to be normalized and the standard product title in the standard product library, the second product to be normalized is normalized by a large model to determine a correct product code of the second product to be normalized, including: The product title of the second product to be normalized is used as the second product title, and title recall is performed based on the second product title vector corresponding to the second product title and the standard product title vector corresponding to each standard product title in the standard product library, and at least one recalled standard product title is obtained in the recall result; According to the second product title and the recalled standard product title, the large model is used to identify the same product standard product title corresponding to the second product title in the recalled standard product title, and the product code corresponding to the same product standard product title is obtained as the correct product code of the second product to be normalized.

6. The method according to claim 5, characterized in that Get at least one recalled product title in the recall results, including: taking a preset number of recall results ranked first in the recall results as the first recalled standard product title, wherein the recalled standard product title includes the first recalled standard product title; and / or, In the recall results, the recall results whose similarity with the second product title vector is greater than a preset threshold are obtained as the second recalled standard product title, wherein the recalled standard product title includes the second recalled standard product title.

7. The method according to claim 5, characterized in that According to the second product title and the recalled standard product title, identifying the same product standard product title corresponding to the second product title in the recalled standard product title through a large model, and obtaining the product code corresponding to the same product standard product title as the correct product code of the second product to be normalized, including: Identify through a large model whether the recalled standard product title has a standard product title of the same product corresponding to the second product title; If it exists, obtaining the product code corresponding to the title of the same product standard product as the correct product code of the second product to be normalized; If it does not exist, then obtain the standard product title corresponding to the target product code in the standard product library as the target standard product title, and use the large model to identify whether the second product title and the target standard product title are the same product title, and when the identification result is yes, use the target product code as the correct product code of the second product to be normalized; otherwise, determine that the self-filled product code of the second product to be normalized is the correct product code.

8. A system for normalizing products of the same type based on a large model, characterized in that: The system comprises: Self-filled information module, used to obtain product titles and self-filled product codes corresponding to multiple products; a normalization filtering module, configured to cluster the commodity titles to determine a commodity category, determine a target commodity code corresponding to the commodity category according to the self-filled commodity code of each commodity in the commodity category, and determine whether the self-filled commodity code in each commodity in the commodity category belongs to the commodity code cluster according to the commodity code cluster to which the target commodity code belongs, wherein the commodities corresponding to each commodity code in the commodity code cluster are the same; The large model identification module is used to determine the first commodity to be normalized among the commodities in the commodity category whose self-filled commodity code does not belong to the commodity code cluster, and to normalize the first commodity to be normalized by the large model according to the commodity title corresponding to the first commodity to be normalized and the standard product title in the standard product library, so as to determine the correct commodity code corresponding to the first commodity to be normalized.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images