Method and system for identifying commodities of same type based on large language model, and electronic equipment

Abstract

The invention discloses a same-type commodity identification method and system based on a large language model, and electronic equipment. The method comprises the following steps: receiving a same-type commodity identification request initiated by a user; obtaining commodity data; carrying out structured processing on the commodity data; configuring instruction prompt words and necessary analysis attributes; combined cue words are generated by combining the same-type commodity recognition request, the commodity data, the instruction cue words and the analysis attributes; calling a large language model by using the combined cue word to generate standard attribute structure commodity data; carrying out grouping and similarity judgment on the commodities; configuring calculation attributes; grouping and calculating the commodity data forming the standard attribute structure; judging the same type of commodities according to the calculation result and the calculation attribute; and storing the data of the commodities of the same type in a database. Compared with the prior art, the method has the advantages that the requirement for manual intervention can be lowered, the method adapts to complex and diversified data, the recognition accuracy is improved, and the processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of commodity identification, and more specifically, to a method for identifying identical commodities based on a large language model. Background Art

[0002] The management and identification of product information is a key task in the field of e-commerce and retail. As the variety and quantity of products increase, the difficulty and importance of identifying identical products become more important. Accurately identifying identical products is crucial for inventory management, price comparison and consumer experience. Traditional methods usually rely on manual labeling or simple text matching. However, these methods have problems such as low efficiency and low accuracy when dealing with large-scale and diversified products.

[0003] At present, the common same-item product identification technologies in the market mainly include rule-based matching and simple machine learning models. These methods lack flexibility, especially perform poorly when faced with complex and changeable product information, and have poor adaptability to the complexity and diversity of data.

[0004] Therefore, how to provide a method for identifying identical products based on a large language model that can reduce the need for manual intervention, adapt to complex and diverse data, improve recognition accuracy, and improve processing efficiency has become a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method for identifying identical products based on a large language model, which can reduce the need for manual intervention, adapt to complex and diverse data, improve recognition accuracy, and enhance processing efficiency.

[0006] The technical solution provided by the present invention is as follows:

[0007] The present invention provides a method for identifying identical products based on a large language model. Compared with the prior art, the method comprises the following steps: receiving an identical product identification request initiated by a user; obtaining product data; performing structured processing on the product data: configuring instruction prompt words and necessary parsing attributes; generating a combined prompt word by combining the identical product identification request, the product data, the instruction prompt word and the parsing attribute; using the combined prompt word to call a large language model to generate standard attribute structure product data; grouping products and judging similarity: configuring calculation attributes; grouping and calculating product data forming a standard attribute structure; judging identical products according to calculation results and calculation attributes; saving the identical product data in a database; and providing an interface for obtaining identical product data to the outside.

[0008] Furthermore, in a preferred embodiment of the present invention, the use of combined prompt words to call a large language model to generate standard attribute structure product data includes: inputting the combined prompt words into a large language model interface; recording a conversion log; determining whether the product attribute data returned by the large language model interface complies with the standard attribute structure specification; normalizing the product attribute data that does not comply with the specification into a standard attribute structure; and saving the standard attribute structure product data into a database.

[0009] Furthermore, in a preferred embodiment of the present invention, the standard attribute structure is an attribute structure in JSON format.

[0010] Furthermore, the configuration calculation attributes include: setting key attributes, and determining that products without key attributes are not the same products; setting the minimum attribute matching number of the same products and the minimum attribute matching number of similar products; setting the minimum attribute matching threshold of the same products and the minimum attribute matching threshold of similar products.

[0011] Furthermore, in a preferred embodiment of the present invention, the grouping and calculation of commodity data forming a standard attribute structure includes: grouping the commodity data forming a standard attribute structure according to attributes; performing similarity calculation within the group using a pre-trained model; obtaining the similarity calculation results from the model; and storing the similarity calculation results in a database.

[0012] Furthermore, the pre-trained model is a model trained using a machine learning algorithm.

[0013] Furthermore, in a preferred embodiment of the present invention, using a pre-trained model to perform similarity calculation within a group includes: using a labeled data set to train the model, the model applying a machine learning algorithm to learn the importance and similarity of different attributes; the model performs similarity scoring on the product data within the group based on the calculated attributes; and generating a similarity calculation result.

[0014] Furthermore, the determining of identical products based on calculation results and calculated attributes includes: obtaining similarity calculation results from a database; determining products lacking key attributes as non-identical products based on the calculation results; determining products that reach a minimum attribute matching number for similar products and a minimum attribute matching threshold for similar products as similar products based on the calculation results; determining products that reach a minimum attribute matching number for identical products and a minimum attribute matching threshold for identical products as identical products based on the calculation results.

[0015] Furthermore, the obtaining of commodity data specifically involves obtaining commodity data through various data sources such as e-commerce platforms and supplier systems, and the commodity data includes brand, model, category, specification, and description.

[0016] Furthermore, the steps of configuring instruction prompt words, configuring necessary parsing attributes, and configuring calculation attributes are configured through SQL statements.

[0017] In addition, the present invention also provides a system for identifying identical products based on a large language model, the system comprising:

[0018] A first receiving module, the first receiving module is used to receive a same-item product identification request initiated by a user;

[0019] A first acquisition module, wherein the first acquisition module is used to acquire commodity data;

[0020] A first processing module, the first processing module is used to perform structured processing on the commodity data;

[0021] A second processing module, the second processing module is used to group the commodities and determine similarity;

[0022] A first storage module, the first storage module is used to save the data of the same product in a database;

[0023] The first interface module is used to provide an interface for obtaining data of the same product.

[0024] In addition, the present invention also provides an electronic device, comprising:

[0025] A computer program, wherein the computer program is used to execute the method for identifying identical products based on a large language model as described above;

[0026] A memory, the memory being used to store the computer program;

[0027] A processor is used to execute the computer program.

[0028] The invention provides a method, system and electronic device for identifying identical products based on a large language model. Compared with the prior art, the method comprises the following steps: receiving an identical product identification request initiated by a user; obtaining product data; performing structured processing on the product data: configuring instruction prompt words and necessary parsing attributes; generating a combined prompt word by combining the identical product identification request, the product data, the instruction prompt word and the parsing attribute; using the combined prompt word to call a large language model to generate product data with a standard attribute structure; grouping products and judging similarity: configuring calculation attributes; grouping and calculating product data with a standard attribute structure; judging identical products according to calculation results and calculation attributes; saving the identical product data in a database; and providing an interface for obtaining identical product data. Among them, configuring instruction prompts helps the large language model to accurately decompose and extract the attributes of product data, configuring necessary parsing attributes prevents missing of the large language model in extracting product data, calling the large language model to generate standard attribute structure product data reduces the need for manual intervention, and configuring calculation attributes provides standards, so that the method involved in the present invention can adapt to complex and diverse data, and provides an interface for obtaining the same product data for users to obtain information about the same product. In the above process, the tedious step of generating standard attribute structure product data is handed over to the large language model for processing, which greatly improves the processing efficiency of product data. In summary, the technical solution involved in the present invention, compared with the prior art, can reduce the need for manual intervention, adapt to complex and diverse data, improve recognition accuracy, and significantly improve processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 A flow chart of a method for identifying identical products based on a large language model provided in an embodiment of the present invention;

[0031] Figure 2 A flowchart of the steps of using a combination of prompt words to call a large language model to generate standard attribute structure commodity data in a method for identifying identical commodities based on a large language model provided by an embodiment of the present invention;

[0032] Figure 3 A flowchart of the steps of grouping and calculating commodity data with a standard attribute structure in a method for identifying commodities of the same type based on a large language model provided in an embodiment of the present invention;

[0033] Figure 4 A flowchart of the steps of calculating similarity within a group using a pre-trained model in a method for identifying identical products based on a large language model provided by an embodiment of the present invention;

[0034] Figure 5 A schematic diagram of an example of an instruction prompt SQL statement for a method for identifying identical products based on a large language model provided in an embodiment of the present invention;

[0035] Figure 6 A schematic diagram of a sample configuration of instruction prompt words for a method for identifying identical products based on a large language model provided in an embodiment of the present invention;

[0036] Figure 7 A schematic diagram of an example of an SQL statement for necessary parsing attributes of a method for identifying identical products based on a large language model provided in an embodiment of the present invention;

[0037] Figure 8 A schematic diagram of a sample configuration of necessary parsing attributes for a method for identifying identical products based on a large language model provided in an embodiment of the present invention;

[0038] Fig. 9 A schematic diagram of a sample code of an interface for obtaining data of identical products in a method for identifying identical products based on a large language model provided in an embodiment of the present invention;

[0039] Fig.10 A schematic diagram of a sample SQL statement for calculating attributes of a method for identifying identical products based on a large language model provided in an embodiment of the present invention;

[0040] Fig.11 A schematic diagram of a sample configuration of calculation attributes of a method for identifying identical products based on a large language model provided in an embodiment of the present invention;

[0041] Fig.12 A schematic diagram of a sample SQL statement for grouping products in a method for identifying products of the same type based on a large language model provided in an embodiment of the present invention;

[0042] Fig.13 A schematic diagram of an example of SQL statement format for same-item product data in a method for identifying same-item products based on a large language model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention.

[0044] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0045] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" and "several" is two or more, unless otherwise clearly and specifically defined.

[0046] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical contents disclosed by the present invention.

[0047] Please Figures 1 to 13 As shown, the method for identifying identical products based on a large language model provided by an embodiment of the present invention includes the following steps: receiving an identical product identification request initiated by a user; obtaining product data; performing structured processing on the product data: configuring instruction prompt words and necessary parsing attributes; generating a combined prompt word by combining the identical product identification request, the product data, the instruction prompt word and the parsing attribute; using the combined prompt word to call a large language model to generate product data with a standard attribute structure; grouping products and judging similarity: configuring calculation attributes; grouping and calculating product data with a standard attribute structure; judging identical products based on calculation results and calculation attributes; saving the identical product data in a database; and providing an interface for obtaining identical product data to the outside.

[0048] The same product identification request initiated by the user provides the SKU of the product. Configuring instruction prompts helps the large language model to accurately decompose and extract the attributes of product data. All prompts in the step will be sent to the large language model according to the sorting field. The instruction prompts can be configured through SQL statements such as Figure 5In the embodiment of the present invention, a configuration example is provided. Figure 6 The necessary parsing attributes are the attributes that must be parsed for all products to prevent missing data from being extracted by the large language model. This can be configured through SQL statements such as Figure 7 In the embodiment of the present invention, a configuration example is provided. Figure 8 , configuring the calculation attribute provides a standard, so that the method of the present invention can adapt to complex and diverse data, and finally provides an interface for obtaining the same product data for users to obtain the same product information. This interface provides a sample code in the embodiment of the present invention as follows Fig. 9 In the above process, the tedious step of generating standard attribute structure commodity data is handed over to the large language model, which greatly improves the processing efficiency of commodity data.

[0049] Therefore, after determining the same product based on the calculation results and calculation attributes, the same product data format is obtained as follows Fig.13 , and save the obtained data of the same product into the database.

[0050] Specifically, the use of combined prompt words to call a large language model to generate standard attribute structure product data includes: inputting the combined prompt words into a large language model interface; recording a conversion log; determining whether the product attribute data returned by the large language model interface conforms to the standard attribute structure specification; normalizing the product attribute data that does not conform to the specification into a standard attribute structure; and saving the standard attribute structure product data into a database.

[0051] It should be noted that in the embodiment of the present invention, recording the conversion log can monitor the attribute extraction process. When a problem occurs in the program, the time, cause and context of the problem can be quickly located. Since the commodity attribute data returned by the large language model is not necessarily standardized, it is necessary to determine whether the commodity attribute data returned by the large language model interface conforms to the standard attribute structure specification and process it. After processing, each commodity code corresponds to its attributes and attribute values.

[0052] Specifically, in the embodiment of the present invention, the standard attribute structure is an attribute structure in JSON format.

[0053] It should be noted that the JSON format is a lightweight data exchange format that is easy for humans to read and write, can be read and processed by a variety of programming languages, and is also easy for machines to parse and generate. In the embodiments of the present invention, the use of the JSON format can facilitate the transfer of data between different steps and facilitate the processing of subsequent steps.

[0054] Specifically, in an embodiment of the present invention, the configuration calculation attributes include: setting key attributes, and determining that products that lack key attributes are not the same products; setting the minimum attribute matching number of the same products and the minimum attribute matching number of similar products; setting the minimum attribute matching threshold of the same products and the minimum attribute matching threshold of similar products.

[0055] It should be noted that in the embodiment of the present invention, the configuration of calculated attributes includes: key attributes. Goods that lack key attributes are judged as non-identical goods. When making preliminary judgments, filtering the product data by key attributes can reduce the amount of calculation; the number of attribute matches, that is, the number of attributes with the same value. If the minimum attribute match number is reached, the goods may be similar or identical. If it is not reached, they are non-identical goods. Adding this attribute parameter can improve the accuracy of identifying identical goods; the threshold value of similar goods and identical goods. If the result of the calculation of the goods attribute reaches this value, it can be regarded as the same or similar goods. In the embodiment of the present invention, the calculation attributes can be configured through SQL statements such as Fig.10 , configuration data such as Fig.11 .

[0056] Specifically, in an embodiment of the present invention, the grouping and calculation of commodity data forming a standard attribute structure includes: grouping commodity data forming a standard attribute structure according to attributes; performing similarity calculation within the group using a pre-trained model; obtaining similarity calculation results from the model; and storing the similarity calculation results in a database.

[0057] It should be noted that since calculating the similarity of the same product requires cross-comparison of all products, the amount of calculation is large. Grouping first makes it impossible for the products in the group to be the same as the products in other groups, which can save a lot of computing resources. In the embodiment of the present invention, the grouping standard can be selected to group by brand and category, but this standard is not unique, and the grouping selection can be changed according to the actual situation. In the embodiment of the present invention, the grouping can be performed through SQL statements such as Fig.12 .

[0058] Specifically, in an embodiment of the present invention, the pre-trained model is a model trained using a machine learning algorithm.

[0059] Specifically, in an embodiment of the present invention, using a pre-trained model to perform similarity calculation within a group includes: using a labeled data set to train a model, the model applying a machine learning algorithm to learn the importance and similarity of different attributes; the model performs similarity scoring on the product data within the group based on the calculated attributes; and generating a similarity calculation result.

[0060] Therefore, similarity scoring of commodity data in a group based on calculated attributes is also completed by a pre-trained model, which greatly reduces manual intervention and reduces labor costs. The model uses machine learning algorithms to learn the importance and similarity of different attributes, which can achieve accurate processing of complex data and improve the accuracy and reliability of similarity calculation results.

[0061] Specifically, in an embodiment of the present invention, determining identical products based on calculation results and calculated attributes includes: obtaining similarity calculation results from a database; determining products that lack key attributes as non-identical products based on the calculation results; determining products that reach a minimum attribute matching number for similar products and a minimum attribute matching threshold for similar products as similar products based on the calculation results; determining products that reach a minimum attribute matching number for identical products and a minimum attribute matching threshold for identical products as identical products based on the calculation results.

[0062] Specifically, in the embodiment of the present invention, the obtaining of commodity data specifically involves obtaining commodity data through various data sources such as e-commerce platforms and supplier systems, and the commodity data includes brand, model, category, specification, and description.

[0063] Specifically, in the embodiment of the present invention, the steps of configuring instruction prompt words, configuring necessary parsing attributes, and configuring calculation attributes are configured through SQL statements.

[0064] In addition, an embodiment of the present invention further provides a system for identifying identical products based on a large language model, the system comprising:

[0065] A first receiving module, the first receiving module is used to receive a same-item product identification request initiated by a user;

[0066] A first acquisition module, wherein the first acquisition module is used to acquire commodity data;

[0067] A first processing module, the first processing module is used to perform structured processing on the commodity data;

[0068] A second processing module, the second processing module is used to group the commodities and determine similarity;

[0069] A first storage module, the first storage module is used to save the data of the same product in a database;

[0070] The first interface module is used to provide an interface for obtaining data of the same product.

[0071] In addition, an embodiment of the present invention further provides an electronic device, including:

[0072] A computer program, wherein the computer program is used to execute the method for identifying identical products based on a large language model as described above;

[0073] A memory, the memory being used to store the computer program;

[0074] A processor is used to execute the computer program.

[0075] In summary, the technical solutions involved in the embodiments of the present invention involve the following important technical contents: the first is the disassembly of commodity attributes based on a large language model: using a large model to accurately disassemble and extract the attributes of commodity information. And automatically identify key attributes, such as brand, model, and specification; the second is attribute comparison and scoring: that is, applying a machine learning algorithm to compare and score the disassembled attributes, and learn the importance and similarity of different attributes; the third is a threshold judgment mechanism: that is, setting a similarity score threshold to judge the same commodity. Ensure high accuracy and reliability. The present invention uses a large language model and a machine learning algorithm to improve the ability to extract and compare commodity attributes, reduce misjudgments and missed judgments, automatically process commodity information, reduce the complexity of manual intervention and rule setting, adapt to large-scale commodity data, and improve the flexibility and adaptability of the recognition system for diversified commodity information. It is suitable for commodities of different categories and complexities, and has significant technical effects compared with the prior art.

[0076] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying identical products based on a large language model, characterized in that: The following steps are involved: Receive a request from a user to identify the same product; Acquire commodity data, and perform structured processing on the commodity data; Configure command prompt words and necessary parsing attributes; Generate a combined prompt word by combining the same-item product identification request, the product data, the instruction prompt word and the parsed attribute; Use combined prompt words to call the large language model to generate standard attribute structure product data; Grouping and similarity determination of products: Configure calculated properties; Group and calculate the commodity data that forms the standard attribute structure; Determine the same product based on the calculation results and calculation attributes; The data of the same product is saved in the database, and an interface for obtaining the data of the same product is provided externally.

2. The method for identifying identical products based on a large language model according to claim 1, characterized in that: Steps: Using combined prompt words to call a large language model to generate standard attribute structure product data includes the following steps: Input the combined prompt words into the large language model interface; Record conversion logs; Determine whether the product attribute data returned by the large language model interface conforms to the standard attribute structure specification; Normalize the non-compliant product attribute data into a standard attribute structure; Save the standard attribute structure product data into the database.

3. The method for identifying identical products based on a large language model according to claim 1, characterized in that: The standard attribute structure is an attribute structure in JSON format; the configuration calculation attribute includes: Set key attributes. Products without key attributes will be considered as different products. Set the minimum attribute matching number for the same product and the minimum attribute matching number for similar products; Set the minimum attribute matching threshold for identical products and the minimum attribute matching threshold for similar products.

4. The method for identifying identical products based on a large language model according to claim 3, characterized in that: The steps of grouping and calculating the commodity data forming the standard attribute structure include: Grouping the commodity data that forms the standard attribute structure according to the attributes; Use a pre-trained model to calculate similarity within a group; Get the similarity calculation results from the model; The similarity calculation result is stored in a database.

5. The method for identifying identical products based on a large language model according to claim 4, characterized in that: The pre-trained model is a model trained using a machine learning algorithm.

6. The method for identifying identical products based on a large language model according to claim 5, characterized in that: The method of using the pre-trained model to perform similarity calculation within a group includes: The model is trained using a labeled dataset and applies machine learning algorithms to learn the importance and similarity of different attributes; The model scores the similarity of the product data in the group based on the calculated attributes and generates similarity calculation results.

7. The method for identifying identical products based on a large language model according to claim 6, characterized in that: Determining the same products based on calculation results and calculation attributes includes: Get similarity calculation results from the database; According to the calculation results, the products that lack key attributes are judged as different products; According to the calculation results, the products that meet the minimum attribute matching number of similar products and the minimum attribute matching threshold of similar products are determined as similar products; According to the calculation results, the products that meet the minimum attribute matching number and the minimum attribute matching threshold of the same product are determined to be the same product.

8. The method for identifying identical products based on a large language model according to claim 1, characterized in that: The obtaining of commodity data specifically includes obtaining commodity data through various data sources such as e-commerce platforms and supplier systems; the commodity data includes: brand, model, category, specification, and description; And / or the steps of configuring instruction prompt words, configuring necessary parsing attributes, and configuring calculation attributes are configured through SQL statements.

9. A system for identifying identical products based on a large language model, characterized in that: The system is used to execute the same-item product identification method based on a large language model as described in any one of claims 1 to 8, comprising: A first receiving module, the first receiving module is used to receive a same-item product identification request initiated by a user; A first acquisition module, wherein the first acquisition module is used to acquire commodity data; A first processing module, the first processing module is used to perform structured processing on the commodity data; A second processing module, the second processing module is used to group the commodities and determine similarity; A first storage module, the first storage module is used to save the data of the same product in a database; The first interface module is used to provide an interface for obtaining data of the same product.

10. An electronic device, characterized in that: include: A computer program, the computer program being used to execute the method for identifying identical products based on a large language model as described in any one of claims 1 to 8; A memory, the memory being used to store the computer program; A processor is used to execute the computer program.

Images