Commodity attribute intelligent processing method, device and equipment based on large model

Abstract

The invention discloses a commodity attribute intelligent processing method, device and equipment based on a large model. The method comprises the following steps: acquiring a pre-constructed commodity attribute knowledge graph; the knowledge graph comprises mapping relation data of standardized description and multiple non-standardized description of attributes, mapping relation data of multiple synonyms and synonyms of the standardized description of the attributes and the standardized description of the attributes, and attribute value standardized conversion rules corresponding to the standard attributes; for a to-be-processed commodity description text, extracting key attribute information by using a first method, and obtaining an extraction attribute of the commodity description text and a corresponding extraction attribute value; and for the extracted attributes and the extracted attribute values, performing mapping, matching and conversion by utilizing the commodity attribute knowledge graph, and determining standard attributes and standard attribute values in the commodity description text. According to the invention, the accuracy and efficiency of intelligent matching and standardization of attributes and attribute values are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device and equipment for intelligently processing commodity attributes based on a large model. Background Art

[0002] With the rapid development of e-commerce, the scale of commodity data has exploded. However, commodity data has serious heterogeneity and non-standardization problems, such as: 1. The brand name is not uniform. For example, "Huawei" may be described as "HUAWEI", "HUAWEI (HUAWEI)", etc.; 2. The product type names are confusing, such as "laptop" may be described as "laptop", "notebook", etc.; 3. The description of attributes is not uniform, for example, "dimension" may be described as "size", "specification", etc.; 4. The attribute value format is confusing, such as "1.5L" and "1500ml" represent the same capacity; Existing rule-based methods have difficulty handling such complex semantic changes, while traditional machine learning methods have limited generalization capabilities and are difficult to adapt to new categories, while manual standardization is costly and inefficient. Summary of the invention

[0003] In view of the problems existing in the above-mentioned prior art, the present invention provides a method, device and equipment for intelligent processing of commodity attributes based on a large model, which improves the efficiency and accuracy of commodity attribute extraction, matching and standardization. The technical solution of the present invention is as follows: In a first aspect, a method for intelligently processing commodity attributes based on a large model is provided, comprising the following steps: Step 1, obtaining a pre-constructed commodity attribute knowledge graph, wherein the attributes in the knowledge graph are described in a standardized manner and are recorded as standard attributes, and the knowledge graph includes a hierarchical relationship between multiple standard attributes; the knowledge graph includes: mapping relationship data between a standardized description of an attribute and multiple non-standard descriptions, mapping relationship data between multiple synonyms and near synonyms of the standardized description of an attribute and the standardized description of the attribute, and attribute value standardization conversion rules corresponding to standard attributes; Step 2: for the product description to be processed, extract key attribute information using the first method to obtain the extracted attributes and corresponding extracted attribute values ​​of the product description; Step 3: For extracting attributes and extracting attribute values, use the product attribute knowledge graph to perform mapping, matching and conversion, determine the standard attributes and standard attribute values ​​in the product description, and obtain standardized results.

[0004] In some embodiments, the first method is a named entity recognition algorithm based on a pre-trained language model, which is used to input a product description and output product attributes and product attribute value information in the product description.

[0005] In some embodiments, the step 3 comprises: Based on the mapping relationship data and hierarchical relationship, the extracted attributes are mapped into standardized descriptions of the attributes; For the extracted attributes that cannot be mapped to standardized descriptions through mapping relationship data and hierarchical relationship mapping, vector similarity analysis is performed based on the extracted attributes and the standardized descriptions of the attributes in the knowledge graph, and a standardized description matching the extracted attributes is determined based on the vector similarity; For the extracted attribute value corresponding to the extracted attribute, a standardized description of the attribute value is determined based on the attribute value standardized conversion rule.

[0006] In some implementations, mapping the extracted attributes to standardized descriptions of the attributes based on the mapping relationship data and the hierarchical relationship includes: Use pre-built synonyms and near-synonyms semantic relationships for precise mapping; Perform association mapping through hierarchical relationships between attributes; The standard attributes in the knowledge graph obtained based on the mapping are used as the standardized description of the extracted attributes; The determining, based on vector similarity, a standardized description matching the extracted attribute comprises: Obtaining a first vector representation of the extracted attribute and a second vector representation of the standard attribute in the knowledge graph; Calculating the similarity between the first vector representation and the second vector representation; The standard attribute corresponding to the second vector representation having the greatest similarity to the first vector representation and a similarity exceeding a preset threshold is taken as the matching result.

[0007] In some embodiments, the method for intelligently processing commodity attributes based on a large model further includes: step 4, optimizing the knowledge graph in step 1 and the first method in step 2 based on historical normalization result feedback, wherein step 4 includes: Step 41, using the methods of steps 1 to 3 to obtain standardized results of the M product descriptions; obtaining the attribute matching accuracy, standardized consistency, and coverage statistics of the M product descriptions; Step 42, comprehensively characterizing the evaluation results based on the attribute matching accuracy, standardization consistency, and coverage statistics; when the evaluation result is lower than a preset condition, analyzing the error statistics of the standardization results of the M product descriptions and the theoretical standardization results; Step 43, feedback and correct the hierarchical relationship, mapping relationship data and standardized conversion rules in step 1 according to the error statistical data of the standardized result, and feedback and correct the first method in step 2.

[0008] In some implementations, in step 42, the error statistics include error types and the frequency of each error type; the error types include: semantic understanding errors, attribute mapping errors, and attribute value standardization conversion errors.

[0009] In some embodiments, the first method is obtained by training based on samples in a sample library; The step 43, correcting the first method in step 2, comprises: Expand the product descriptions whose error type is semantic understanding error in the error statistics data of step 42 to the training sample library; Perform sample category balancing on the expanded sample library; The first method is retrained based on the sample library after the equalization process to obtain a revised first method.

[0010] In some embodiments, the method for obtaining the attribute matching accuracy rate includes: for M product descriptions, obtaining the correct matching value of each attribute in the manually annotated product description as a reference answer; analyzing whether the attributes in the standardized results obtained by using steps 1 to 3 are consistent with the reference answer, and if they are consistent, adding 1 to the attribute matching number count; and taking the ratio of the attribute matching number to the total number of attributes in the product description as the attribute matching accuracy rate; The method for obtaining standardized consistency includes: for M product descriptions, analyzing whether the attribute values ​​of the attributes in the standardized results obtained by using steps 1 to 3 conform to the attribute value standardized conversion rules, and if so, adding 1 to the consistency value count; and taking the ratio of the consistency value to the total number of attributes in the product description as the standardized consistency result; The method for obtaining the coverage statistics result includes: for M product descriptions, if a product description can extract at least one standard attribute using steps 1 to 3, the coverage number count is increased by 1, and the coverage statistics result is taken as the ratio of the coverage number to the total number of product descriptions.

[0011] In a second aspect, a commodity attribute intelligent processing device based on a large model is provided, comprising: A commodity attribute knowledge graph acquisition unit is used to acquire a pre-built commodity attribute knowledge graph, wherein the attributes in the knowledge graph are described in a standardized manner and are recorded as standard attributes, and the commodity attribute knowledge graph includes a hierarchical relationship between multiple standard attributes; the commodity attribute knowledge graph includes: mapping relationship data between a standardized description of an attribute and multiple non-standardized descriptions, mapping relationship data between multiple synonyms and near synonyms of the standardized description of an attribute and the standardized description of the attribute, and attribute value standardization conversion rules corresponding to standard attributes; An attribute extraction unit, configured to extract key attribute information from the commodity description to be processed by using a first method, and obtain extracted attributes of the commodity description and corresponding extracted attribute values; The attribute matching and standardization unit is used to map, match and convert the extracted attributes and attribute values ​​using the product attribute knowledge graph, determine the standard attributes and standard attribute values ​​in the product description, and obtain the standardized results.

[0012] According to a third aspect, an electronic device is provided, the electronic device comprising: processor; a memory for storing processor-executable instructions; The processor runs the executable instructions to implement the intelligent processing method of commodity attributes based on a large model as described in the first aspect above.

[0013] The method, device and equipment for intelligently processing commodity attributes based on a large model of the present invention have the following beneficial effects: 1. The first method enables accurate analysis of diversified product descriptions and improves the accuracy of attribute matching. The powerful semantic understanding capability of the pre-trained large language model enables the extraction of diversified product attributes and improves the efficiency and intelligence of attribute extraction. 2. Based on the standard attributes, hierarchical relationships, mapping relationships, conversion rules, etc. in the knowledge graph, standardized management of attributes and attribute values ​​is achieved, improving the quality of product data; 3. An adaptive mechanism based on feedback optimization has been designed, and the intelligent processing method of product attributes can be continuously learned and improved, reducing maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flowchart of a method for intelligently processing commodity attributes based on a large model according to an embodiment of the present application; Figure 2 is a schematic diagram of the method flow of step 3 of the embodiment of the present application; Figure 3 is a schematic diagram of the method flow of step 4 of the embodiment of the present application; Figure 4It is a structural diagram of a device for intelligently processing commodity attributes based on a large model according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0016] The embodiment of the present application provides a method for intelligently processing commodity attributes based on a large model, comprising the following steps: Step 1, obtaining a pre-built commodity attribute knowledge graph, where the attributes in the knowledge graph are described in a standardized manner and are recorded as standard attributes; In this knowledge graph, a hierarchical relationship of multiple standard attributes is included; In the knowledge graph, there are provided: mapping relationship data between a standardized description of an attribute and a plurality of non-standardized descriptions, and mapping relationship data between a plurality of synonyms and near synonyms of the standardized description of an attribute and the standardized description of the attribute; In this knowledge graph, standard attribute correspondence can set corresponding attribute value standardization conversion rules.

[0017] Step 2: for the product description to be processed, extract key attribute information using a first method to obtain extracted attributes and corresponding extracted attribute values ​​of the product description. In one embodiment, the first method is a named entity recognition algorithm based on a pre-trained language model, which is used to input a product description and output product attributes and product attribute value information in the product description. The product attributes in the product description include multiple categories such as product name, product specification, and product color. Step 3: For extracting attributes and extracting attribute values, use the product attribute knowledge graph to perform mapping, matching and conversion, determine the standard attributes and standard attribute values ​​in the product description, and obtain standardized results.

[0018] In the embodiments of the present application, the first method is used to accurately parse the descriptions of diversified commodities, thereby improving the accuracy of attribute matching. The powerful semantic understanding capability of the pre-trained large language model is used to extract the attributes of diversified commodities, thereby improving the efficiency and intelligence of attribute extraction. In one embodiment, the pre-trained language model adopts a BERT model.

[0019] Based on the standard attributes, hierarchical relationships, mapping relationships, conversion rules, etc. in the knowledge graph, standardized management of attributes and attribute values ​​is achieved, improving data quality.

[0020] In one embodiment, the above step 3, for extracting attributes and extracting attribute values, uses the product attribute knowledge graph to perform mapping, matching and conversion, determines the standard attributes and standard attribute values ​​in the product description, and obtains the standardized results, including: Step 31, mapping the extracted attributes into standardized descriptions of the attributes based on the mapping relationship data and the hierarchical relationship; Step 32, for the extracted attributes that cannot be mapped to standardized descriptions through mapping relationship data and hierarchical relationships, perform vector similarity analysis based on the extracted attributes and the standardized descriptions of the attributes in the knowledge graph, and determine the standardized description that matches the extracted attributes based on the vector similarity; Step 33: for the extracted attribute value corresponding to the extracted attribute, determine a standardized description of the attribute value based on the attribute value standardized conversion rule.

[0021] In an embodiment of the present application, the extracted attributes are first mapped to the standard attributes in the knowledge graph through the hierarchical relationship and mapping relationship data in the knowledge graph to obtain a standardized description of the extracted attributes, and further, vector similarity matching is considered.

[0022] In one embodiment, the above step 31, based on the mapping relationship data and the hierarchical relationship, maps the extracted attributes into a standardized description of the attributes, including: Step 311, using pre-built semantic relationships of synonyms and near-synonyms to perform precise mapping; Step 312, performing association mapping through the hierarchical relationship between attributes; Step 313, using the standard attributes in the knowledge graph obtained based on the mapping as the standardized description of the extracted attributes; In one embodiment, the above step 32, determining the standardized description matching the extracted attribute based on the vector similarity, includes: Step 321, obtaining a first vector representation of the extracted attribute and a second vector representation of the standard attribute in the knowledge graph; Step 322, calculating the similarity between the first vector representation and the second vector representation; Step 323: taking the standard attribute corresponding to the second vector representation having the greatest similarity to the first vector representation and a similarity exceeding a preset threshold as the matching result.

[0023] Specifically, in the embodiment of the present application, on the one hand, mapping matching (the above step 31) includes: Use pre-built synonyms and near-synonyms semantic relationships for precise mapping; Through the hierarchical relationship between attributes, we can understand that the attributes in the knowledge graph have a hierarchical relationship, such as clothing attributes > style features > collar type > V-neckline, so when mapping and matching, we can map a whole set of standard categories according to the hierarchical relationship; The standard attributes in the knowledge graph obtained based on the mapping are used as the standardized description of the extracted attributes; For example, "capacity: 4L (liters)" → is mapped to the standard attribute "capacity: 4 liters" through a synonymous relationship; "voltage: 220V" → is directly mapped to the standard attribute "voltage: 220V" through a synonymous relationship; On the other hand, there is vector similarity matching (the above step 32), including: Calculate the vector representation of the attribute through the built-in graph database (GDS) module of the knowledge graph to obtain the first vector representation of the extracted attribute and the second vector representation of the standard attribute in the knowledge graph, or calculate the vector representation of the attribute based on the pre-trained model; Calculate the similarity between the first vector representation and the second vector representation, and use an algorithm such as cosine similarity; The standard attribute with the highest similarity and exceeding the preset threshold is selected as the matching result.

[0024] For example, "color: rose gold" → is mapped to the standard attribute "color: gold" through similarity calculation (similarity 0.92).

[0025] In one embodiment, the above step 3, for extracting attributes and extracting attribute values, uses the product attribute knowledge graph to perform mapping, matching and conversion, determines the standard attributes and standard attribute values ​​in the product description, and obtains the standardized results, including: Step 301, for the product description to be processed, the extracted attributes and corresponding extracted attribute values ​​obtained by the first method are used to form an extracted attribute sequence and an extracted attribute value sequence corresponding to the product description; Step 302, mapping, matching and converting the extracted attributes representing the commodity category in the extracted attribute sequence as keywords in the commodity attribute knowledge graph to determine the node corresponding to the commodity category in the commodity attribute knowledge graph, this step is implemented based on the method of the above steps 31 to 32, or based on the method of the above steps 311-313 and steps 321-323; Step 303, after determining in step 302 that the commodity category corresponds to a unique node in the commodity attribute knowledge graph, execute step 304; when it is determined in step 302 that the node corresponding to the commodity category in the commodity attribute knowledge graph is not unique and the corresponding node data is less than a first preset threshold, execute step 304; when it is determined in step 302 that the node corresponding to the commodity category in the commodity attribute knowledge graph does not exist or the corresponding node data is greater than the first preset threshold, execute step 305; Step 304, mapping, matching and converting are performed based on the extracted attribute sequence corresponding to the product description and the knowledge and data stored in all associated nodes that have a hierarchical relationship with the corresponding node in the knowledge graph, determining the correspondence between all extracted attributes corresponding to the product description and the nodes in the knowledge graph, and obtaining the standardized description of the extracted attributes stored in the nodes in the knowledge graph; in the embodiment of the present application, through steps 302 to 304, the product category is used as a keyword to search the entire knowledge graph, determine the node corresponding to the product category in the product attribute knowledge graph, and map, match and convert the knowledge and data stored in all associated nodes that have a hierarchical relationship with the corresponding node, first based on a keyword search and positioning, and then using the association relationship between the nodes of the knowledge graph to determine a target area in the knowledge graph, in the target area, the extracted attribute sequence and the extracted attribute value sequence corresponding to the product description are mapped, matched and converted, effectively improving the retrieval speed. In addition, in this case, instead of searching the entire knowledge graph based on a single extracted attribute, the entire extracted attribute sequence and the extracted attribute value sequence are searched in the entire knowledge graph as a whole, which effectively utilizes the information combination and information dependency between multiple extracted attributes, and improves the accuracy of mapping, matching and conversion of extracted attributes and extracted attribute values ​​in the commodity attribute knowledge graph; it should be noted that in step 304, all associated nodes having a hierarchical relationship with the corresponding node include neighboring nodes having a direct connection relationship with the corresponding node, and also include the next neighboring node transmitted through the neighboring node; Step 305, based on the existing commodity knowledge graph, obtain the local knowledge graph corresponding to each commodity category, the local knowledge graph corresponding to the commodity category includes the commodity category as the core point, the attributes having a connection relationship with the core point in the knowledge graph and their corresponding connection relationship. In another embodiment, in step 305, the local knowledge graph corresponding to the commodity category includes the commodity category as the core point, the attributes having a connection relationship with the core point in the knowledge graph and their corresponding connection relationship, and the attribute values ​​of the attributes. It can be understood that the local knowledge graph is consistent with the entire knowledge graph in step 1, has a hierarchical relationship of standard attributes and multiple standard attributes, and carries relevant mapping relationship data, attribute value standardization conversion rules, etc.; including mapping relationship data of standardized description of attributes and multiple non-standard descriptions, mapping relationship data of multiple synonyms and near synonyms of the standardized description of the attributes and the standardized description of the attributes, and attribute value standardization conversion rules corresponding to standard attributes; Step 306, feature encoding is performed on the local knowledge graph corresponding to each product category, with the goal that the feature encoding results of the local knowledge graph corresponding to each product category are completely different and the feature encoding results of the local knowledge graph corresponding to each product category after randomly deleting nodes and edges are similar, training a local knowledge graph feature encoding network model; Step 307, use the local knowledge graph feature coding network model to feature encode the local knowledge graph corresponding to each product category to obtain the first feature coding result of each local knowledge graph; use the local knowledge graph feature coding network model to feature encode the extracted attribute sequence corresponding to the product description to obtain the second feature coding result, and perform a comparative analysis on the first feature coding result based on the second feature coding result, determine the local knowledge graph corresponding to the first feature coding result that is most similar to the second feature coding result as the target area, and map, match and convert the extracted attributes and extracted attribute values ​​corresponding to the product description based on the knowledge and data stored in the knowledge graph nodes of the target area. In the embodiments of the present application, it is taken into consideration that, on the one hand, for the description of the product to be processed, using the first method to extract key attribute information, there may be inaccurate extraction results, which may lead to the inability to find the corresponding standard knowledge in the knowledge graph for mapping, matching and conversion; on the other hand, the amount of knowledge stored in the knowledge graph is large, and each standard attribute is also provided with multiple synonym mapping databases and multiple non-standardized description databases. There is a problem of low efficiency and slow search for the extracted attributes of the description of the product to be processed in the knowledge graph. In the embodiments of the present application, the knowledge graph is divided into local knowledge graphs and feature encoding is performed separately, so that the local knowledge graph containing each product category has a unique feature encoding result, and the local knowledge graph used to map, match and convert the extracted attributes and extracted attribute values ​​corresponding to the product description is quickly determined by feature encoding the extracted attribute sequence corresponding to the product description. It can be understood that the local knowledge graph feature encoding network model in the present application is used to encode the local commodity knowledge graph. Generally speaking, the feature encoding of the knowledge graph can be performed by using a graph neural network algorithm to process the graph. However, in the embodiment of the present application, considering that all the edges in the commodity knowledge graph represent the connection relationship between two attributes, such as the upper and lower level inclusion relationship, the knowledge and data stored in the nodes in the local knowledge graph can be converted into text matrix content in sequence according to the connection relationship and hierarchical relationship of the nodes in the local knowledge graph in the embodiment of the present application. The text matrix contains text content with a sequence (different rows above and below or different columns left and right in the matrix) and text content with the same level (the same row or the same column in the matrix). Then, a natural language processing algorithm (i.e., a text data processing model) is used for the text matrix content corresponding to the local knowledge graph as a local knowledge graph feature encoding network model for feature encoding. The local knowledge graph feature encoding network model can be a text data processing model, such as a recursive neural network RNN, a long short-term memory network LSTM, a converter model Transformer, BERT, etc., of course, not limited to the above models.

[0026] In one embodiment, after the above step 3 is performed for standardization, the method further includes: step 4, using a preset number of product descriptions, adopting the methods of steps 1 to 3 to obtain standard attributes and standard attribute values ​​in the product descriptions, recording them as standardization results of the product descriptions, evaluating and analyzing the standardization results, and optimizing the method process of steps 1 to 3 based on the feedback of the evaluation and analysis results; The step 4 comprises: Step 41, using the methods of steps 1 to 3 to obtain standardized results of the M product descriptions; obtaining the attribute matching accuracy, standardized consistency, and coverage statistics of the M product descriptions; Step 42, comprehensively characterizing the evaluation results based on the attribute matching accuracy, standardization consistency, and coverage statistics; when the evaluation result is lower than a preset condition, analyzing the error statistics of the standardization results of the M product descriptions and the theoretical standardization results; Step 43, feedback and correct the hierarchical relationship, mapping relationship data and standardized conversion rules in step 1 according to the error statistical data of the standardized result, and feedback and correct the first method in step 2.

[0027] In an embodiment of the present application, the evaluation results are comprehensively characterized based on the statistical results of attribute matching accuracy, standardization consistency, and coverage. The three can be directly integrated for characterization, or corresponding data indicator requirements can be set for the three respectively. For example, it is required that the attribute matching accuracy is greater than a first value, the standardization consistency is greater than a second value, and the coverage is greater than a third value. When the error of the attribute standardization result obtained by the method of steps 1 to 3 of the evaluation result characterized by the comprehensive characterization of the three is large, an error analysis is performed based on the error generated, and then feedback is given to correct the method process of steps 1 to 3.

[0028] In the embodiment of the present application, the performance of the product attribute matching and standardization in the above steps 1 to 3 is continuously optimized through a feedback mechanism. Through the feedback mechanism, the mapping rules, matching rules, model parameters and semantic relationships in the knowledge graph can be dynamically adjusted, and the attribute relationships in the knowledge graph (such as synonymous relationships, hierarchical relationships, etc.) can be corrected to improve the accuracy of attribute matching, thereby improving the accuracy of attribute matching and the consistency of standardization.

[0029] In one embodiment, in the above step 42, the error statistics include error types and the frequency of each error type; the error types include: semantic understanding errors, attribute mapping errors, and attribute value standardization conversion errors.

[0030] In the embodiment of the present application, the error types include: Semantic understanding errors: For example, due to the complexity of product descriptions (such as synonyms, abbreviations, multiple languages, etc.), the BERT model fails to correctly understand the semantics. For example, due to the BERT model's insufficient understanding of vocabulary in certain fields, semantic understanding errors occur.

[0031] Attribute mapping errors: For example, due to incomplete mapping rules in the attribute knowledge graph, product attributes are mapped to incorrect standard attributes.

[0032] Attribute value standardization conversion errors: For example, the system fails to correctly standardize attribute values ​​due to inconsistent formats of attribute values.

[0033] Specifically, error cases can be collected from historical standardization results, each error case can be labeled to indicate the type of error, the frequency of occurrence of each error type can be counted, and high-frequency error patterns can be identified. Each error pattern can be deeply analyzed to find out the specific cause; the analysis results can be recorded in detail and a detailed error pattern report can be generated, including the specific description, type, cause and impact of the error sample; the error samples can be classified for targeted optimization.

[0034] In one embodiment, the above step 43 includes: Step 431, feedback and correct the hierarchical relationship, mapping relationship data and standardized conversion rules in step 1 according to the error statistical data of the standardized result, and feedback and correct the first method in step 2; Step 432, use the revised hierarchical relationship, mapping relationship data and standardized conversion rules, and the revised first method to re-obtain the standardized results of M product descriptions. If the error statistics of the standardized results are less than the error statistics of step 42, the revised mapping relationship data and standardized conversion rules, and the revised first method are determined to be valid corrections.

[0035] In one embodiment, the first method of extracting key attributes in step 2 is obtained through training based on samples in the sample library; The step 43, correcting the first method in step 2, comprises: Step 433, expanding the product descriptions whose error type is semantic understanding error in the error statistics data of step 42 to the training sample library; Step 434, performing sample category balancing processing on the expanded sample library; Step 435: retrain the first method based on the sample library after the equalization processing to obtain a revised first method.

[0036] In the embodiment of the present application, taking the first method using the BERT model as an example, the first method in step 2 is corrected in the above step 43, including: optimizing the parameters of the BERT model using gradient descent method, adaptive learning rate and other methods; updating the weights of the BERT model through the back propagation algorithm to reduce semantic understanding errors.

[0037] Furthermore, the erroneous cases in the historical standardization results can be expanded to the original sample library, and the model used in the first method can be retrained. At the same time, when expanding the sample library, the balance of sample categories needs to be considered to ensure that the samples of various attributes in the sample library are evenly distributed to avoid the model being biased towards certain specific attributes.

[0038] In one implementation, the method for obtaining the attribute matching accuracy, standardization consistency, and coverage in the above step 41 is as follows: 1) A method for obtaining attribute matching accuracy, including: for M product descriptions, constructing a labeled data set: obtaining the correct matching value of each attribute in the manually labeled product description as a benchmark answer; analyzing the attribute matching accuracy: analyzing whether the attributes in the standardized results obtained by using steps 1 to 3 are consistent with the benchmark answer, and if they are consistent, adding 1 to the attribute matching number count; and taking the ratio of the attribute matching number to the total number of attributes in the product description as the attribute matching accuracy; 2) A method for obtaining standardized consistency, comprising: for M product descriptions, analyzing whether the attribute values ​​of the attributes in the standardized results obtained by using steps 1 to 3 conform to the attribute value standardized conversion rules, and if so, adding 1 to the consistency value count; and taking the ratio of the consistency value to the total number of attributes in the product description as the standardized consistency result; 3) A method for obtaining coverage, including: for M product descriptions, if a product description can extract at least one standard attribute using steps 1 to 3, the number of coverages is counted plus 1, and the ratio of the number of coverages to the total number of product descriptions is used as the coverage statistical result.

[0039] The embodiment of the present application provides a commodity attribute intelligent processing device based on a large model, comprising: A commodity attribute knowledge graph acquisition unit is used to acquire a pre-built commodity attribute knowledge graph, wherein the attributes in the knowledge graph are described in a standardized manner and are recorded as standard attributes, and the commodity attribute knowledge graph includes a hierarchical relationship between multiple standard attributes; the commodity attribute knowledge graph includes: mapping relationship data between a standardized description of an attribute and multiple non-standardized descriptions, mapping relationship data between multiple synonyms and near synonyms of the standardized description of an attribute and the standardized description of the attribute, and attribute value standardization conversion rules corresponding to standard attributes; An attribute extraction unit, configured to extract key attribute information from the commodity description to be processed by using a first method, and obtain extracted attributes of the commodity description and corresponding extracted attribute values; The attribute matching and standardization unit is used to map, match and convert the extracted attributes and attribute values ​​using the product attribute knowledge graph, determine the standard attributes and standard attribute values ​​in the product description, and obtain the standardized results.

[0040] For the specific definition of the intelligent processing device for commodity attributes based on the big model, please refer to the definition of the intelligent processing method for commodity attributes based on the big model above, which will not be repeated here. Each unit in the above-mentioned intelligent processing device for commodity attributes based on the big model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned units can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned units.

[0041] An embodiment of the present application provides an electronic device, the electronic device comprising: a processor; a memory for storing processor executable instructions; wherein the processor implements the intelligent processing method for commodity attributes based on a large model as described in the above method embodiment by running the executable instructions. The memory and the processor are connected via a bus, and when the executable instructions are executed by the processor, the various processes of the above-mentioned embodiment of the intelligent processing method for commodity attributes based on a large model are implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The bus represents one or more of any bus structure of several types of bus structures, including a memory bus and a memory controller, a peripheral bus, a processor, or a local bus using any bus structure in various bus architectures. The processor can be a microprocessor or any conventional processor. The method steps disclosed in conjunction with the above method embodiment can be directly executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module may be located in a readable storage medium known in the art, such as a random access memory (RAM), a flash memory (Flash Memory), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), or a register. The readable storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above-mentioned large-model-based intelligent processing method for commodity attributes in combination with its hardware. The memory in the embodiment of the present application may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory.

[0042] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.

Claims

1. A method for intelligently processing commodity attributes based on a large model, characterized in that: The steps include: Step 1, obtaining a pre-constructed commodity attribute knowledge graph, wherein the attributes in the knowledge graph are described in a standardized manner and are recorded as standard attributes, and the knowledge graph includes a hierarchical relationship between multiple standard attributes; the knowledge graph includes: mapping relationship data between a standardized description of an attribute and multiple non-standard descriptions, mapping relationship data between multiple synonyms and near synonyms of the standardized description of an attribute and the standardized description of the attribute, and attribute value standardization conversion rules corresponding to standard attributes; Step 2: for the product description to be processed, extract key attribute information using the first method to obtain the extracted attributes and corresponding extracted attribute values ​​of the product description; Step 3: For extracting attributes and extracting attribute values, use the product attribute knowledge graph to perform mapping, matching and conversion, determine the standard attributes and standard attribute values ​​in the product description, and obtain standardized results.

2. The method for intelligently processing commodity attributes based on a large model according to claim 1 is characterized in that: The first method is a named entity recognition algorithm based on a pre-trained language model, which is used to input a product description and output product attributes and product attribute value information in the product description.

3. The method for intelligently processing commodity attributes based on a large model according to claim 1 is characterized in that: The step 3 comprises: Based on the mapping relationship data and hierarchical relationship, the extracted attributes are mapped into standardized descriptions of the attributes; For the extracted attributes that cannot be mapped to standardized descriptions through mapping relationship data and hierarchical relationship mapping, vector similarity analysis is performed based on the extracted attributes and the standardized descriptions of the attributes in the knowledge graph, and a standardized description matching the extracted attributes is determined based on the vector similarity; For the extracted attribute value corresponding to the extracted attribute, a standardized description of the attribute value is determined based on the attribute value standardized conversion rule.

4. The method for intelligently processing commodity attributes based on a large model according to claim 3 is characterized in that: The step of mapping the extracted attributes into standardized descriptions of the attributes based on the mapping relationship data and the hierarchical relationship includes: Use pre-built synonyms and near-synonyms semantic relationships for precise mapping; Perform association mapping through hierarchical relationships between attributes; The standard attributes in the knowledge graph obtained based on the mapping are used as the standardized description of the extracted attributes; The determining, based on vector similarity, a standardized description matching the extracted attribute comprises: Obtaining a first vector representation of the extracted attribute and a second vector representation of the standard attribute in the knowledge graph; Calculating the similarity between the first vector representation and the second vector representation; The standard attribute corresponding to the second vector representation having the greatest similarity to the first vector representation and a similarity exceeding a preset threshold is taken as the matching result.

5. The method for intelligently processing commodity attributes based on a large model according to claim 1 is characterized in that: Also includes: Step 4, based on the historical normalization result feedback, optimizes the knowledge graph in step 1 and the first method of step 2, wherein step 4 includes: Step 41, using the methods of steps 1 to 3 to obtain standardized results of the M product descriptions; obtaining the attribute matching accuracy, standardized consistency, and coverage statistics of the M product descriptions; Step 42, comprehensively characterizing the evaluation results based on the attribute matching accuracy, standardization consistency, and coverage statistics; when the evaluation result is lower than a preset condition, analyzing the error statistics of the standardization results of the M product descriptions and the theoretical standardization results; Step 43, feedback and correct the hierarchical relationship, mapping relationship data and standardized conversion rules in step 1 according to the error statistical data of the standardized result, and feedback and correct the first method in step 2.

6. The method for intelligently processing commodity attributes based on a large model according to claim 5 is characterized in that: In step 42, the error statistics data include error types and the frequency of each error type; the error types include: semantic understanding errors, attribute mapping errors, and attribute value standardization conversion errors.

7. The method for intelligently processing commodity attributes based on a large model according to claim 6 is characterized in that: The first method is obtained through training based on samples in the sample library; The step 43, correcting the first method in step 2, comprises: Expand the product descriptions whose error type is semantic understanding error in the error statistics data of step 42 to the training sample library; Perform sample category balancing on the expanded sample library; The first method is retrained based on the sample library after the equalization process to obtain a revised first method.

8. The method for intelligently processing commodity attributes based on a large model according to claim 6 is characterized in that: The method for obtaining the attribute matching accuracy rate includes: for M product descriptions, obtaining the correct matching value of each attribute in the manually annotated product description as a reference answer; analyzing whether the attributes in the standardized results obtained by using steps 1 to 3 are consistent with the reference answer, and if they are consistent, adding 1 to the attribute matching number count; and taking the ratio of the attribute matching number to the total number of attributes in the product description as the attribute matching accuracy rate; The method for obtaining standardized consistency includes: for M product descriptions, analyzing whether the attribute values ​​of the attributes in the standardized results obtained by using steps 1 to 3 conform to the attribute value standardized conversion rules, and if so, adding 1 to the consistency value count; and taking the ratio of the consistency value to the total number of attributes in the product description as the standardized consistency result; The method for obtaining the coverage statistics result includes: for M product descriptions, if a product description can extract at least one standard attribute using steps 1 to 3, the coverage number count is increased by 1, and the coverage statistics result is taken as the ratio of the coverage number to the total number of product descriptions.

9. A commodity attribute intelligent processing device based on a large model, characterized in that: include: A commodity attribute knowledge graph acquisition unit is used to acquire a pre-built commodity attribute knowledge graph, wherein the attributes in the knowledge graph are described in a standardized manner and are recorded as standard attributes, and the commodity attribute knowledge graph includes a hierarchical relationship between multiple standard attributes; the commodity attribute knowledge graph includes: mapping relationship data between a standardized description of an attribute and multiple non-standardized descriptions, mapping relationship data between multiple synonyms and near synonyms of the standardized description of an attribute and the standardized description of the attribute, and attribute value standardization conversion rules corresponding to standard attributes; An attribute extraction unit, configured to extract key attribute information from the commodity description to be processed by using a first method, and obtain extracted attributes of the commodity description and corresponding extracted attribute values; The attribute matching and standardization unit is used to map, match and convert the extracted attributes and attribute values ​​using the product attribute knowledge graph, determine the standard attributes and standard attribute values ​​in the product description, and obtain the standardized results.

10. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

Images