Multiple semantic analysis commodity name pairing and mutual recognition method

Abstract

The invention discloses a multiple semantic analysis commodity name pairing and mutual recognition method, which comprises the following steps: S1, creating a synonym mapping library containing standard commodity names and synonyms corresponding to the standard commodity names; s2, inputting a commodity name, and if matching succeeds, extracting a standard name; s3, if matching is not carried out in S2, character string similarity calculation is carried out, and an editing distance and word vector similarity algorithm is involved; s4, if matching does not occur in the step S3, semantic analysis is conducted through a deep learning model, and if the similarity is larger than or equal to 0.9 or classification labels are consistent, it is judged that the commodities are the same commodities; s5, if the input name is not matched in S4, listing the input name into a to-be-audited list, and performing manual auditing; and S6, updating the mapping library and the model parameters according to the auditing result or the business data, including adding the commodity name association relationship to the mapping library and updating the model. The problem that commodity names are difficult to pair and recognize in large-scale commodity information management is effectively solved, and then a new research direction and application prospects are provided for the field of commodity data processing in the future.

Description

Technical Field

[0001] The present application relates to the technical field of e-commerce data processing, and in particular to a method for mutual recognition of product names through multiple semantic analysis. Background Art

[0002] In e-commerce platforms, supply chain management, and data integration scenarios, product names contain a large number of synonyms, abbreviations, aliases, and expression variations, making cross-system data matching difficult. Existing solutions mainly rely on keyword matching or a single similarity algorithm, such as calculating edit distance, which is ineffective when faced with semantic differences and contextual diversity, making it difficult to achieve high-precision automatic recognition and matching. Therefore, in response to this technical difficulty, technicians in this field urgently need to develop a multi-semantic analysis product name matching and mutual recognition method that can improve the accuracy and efficiency of product name matching to meet actual usage needs. Summary of the Invention

[0003] The main purpose of this application is to provide a method for mutual recognition of product names through multiple semantic analysis, aiming to solve the above-mentioned technical problems.

[0004] A method for mutual recognition of product names through multiple semantic analysis, comprising the following steps:

[0005] S1: Collect synonym pairs of product names and pre-build a synonym mapping library, wherein the synonym mapping library includes at least one standard product name and its corresponding multiple synonym variants;

[0006] S2: Enter the product name. If the pairing is successful, the standard product name will be obtained.

[0007] S3: When S2 fails to match, a string similarity calculation is performed between the input product name and all standard product names in the synonym mapping library. The similarity calculation includes at least one of the following:

[0008] Edit distance algorithm: Calculates the edit distance between the input product name and the standard name. If the edit distance is less than or equal to the preset edit distance threshold, the product is considered to be the same and the corresponding standard product name is obtained.

[0009] Word vector similarity algorithm: Generates vector representations of the input product name and the standard product name based on a pre-trained word vector model, calculates cosine similarity, and if the similarity is ≥ the preset similarity threshold, it is determined to be the same product and the corresponding standard product name is obtained;

[0010] S4: If S3 is not matched, a deep learning model is used to perform semantic analysis on the input product name and the standard product name. If the semantic similarity of the output is ≥ 0.9, or the classification labels are the same, they are both determined to be the same product, and the corresponding standard product name is obtained;

[0011] S5: If S4 still fails to match, the input product name will be added to the pending review list for manual review to confirm whether it is the same product;

[0012] S6: Automatically update synonym mapping library and model parameters based on audit results or business data, including:

[0013] Add the association between the approved product names and existing standard product names to the synonym mapping library;

[0014] Update word vector models and deep learning models through incremental training or online learning.

[0015] In the above-mentioned method for mutual recognition of product names through multiple semantic analysis, the synonym mapping library in step S1 is constructed in the following manner:

[0016] The initial synonym mapping library consists of high-frequency product names and their synonyms;

[0017] Dynamic expansion is achieved through at least one of the following methods:

[0018] Based on user feedback and business data analysis, synonym pairs are regularly updated;

[0019] Clustering algorithms are used to cluster the word vectors of product names and automatically discover potential synonyms.

[0020] Based on the rule engine, names containing specific keywords are extracted as candidate synonyms.

[0021] In the multi-semantic analysis product name pairing and mutual recognition method described above, the preset edit distance threshold in step S3 is set as a dynamic variable, and the specific calculation formula is: threshold = min(3, fl oor(0.2*L)), where L is the character length of the input product name. When the name length exceeds 15 characters, the threshold is fixed at 3.

[0022] In the multi-semantic analysis product name pairing and mutual recognition method described above, the edit distance algorithm and the word vector similarity algorithm in step S3 adopt a cascade calculation method: first, the edit distance algorithm is used to screen the candidate set with a similarity greater than 0.7, and then the candidate set is calculated twice using the word vector similarity algorithm, and the final similarity score is weightedly fused according to a weight ratio of 0.4:0.6.

[0023] In the multi-semantic analysis product name pairing and mutual recognition method described above, the word vector model adopts multi-granularity embedding technology: it simultaneously generates vector representations at the character level, word level, and phrase level, and dynamically integrates different granularity features through the attention mechanism, wherein the phrase-level vector is generated by a Bi-LSTM encoder.

[0024] In the multi-semantic analysis product name pairing and mutual recognition method described above, the deep learning model in step S4 adopts a pre-trained language model based on the Transformer architecture, and the pre-trained language model includes at least one of BERT or RoBERTa, and an attention mechanism is introduced during model training to calculate the semantic relevance between product names.

[0025] In the above-mentioned method for mutual recognition of product name pairing based on multiple semantic analysis, the classification label determination in step S4 adopts a hierarchical decision-making process: first, the third-level category labels are matched, and if no agreement is reached, the second-level categories are returned to. When the second-level categories match and the semantic similarity is greater than 0.85, they are still determined to be the same product.

[0026] In the above-mentioned method for mutual recognition of product name pairing based on multiple semantic analysis, the manual review interface in step S5 is provided with a double-blind verification mechanism: comparison information between the input product name and the standard product name is displayed to at least two reviewers in parallel, and the review result is adopted only when both reviewers confirm a match.

[0027] The comparison information is presented in a visual way by marking the difference morphemes with colors, and displaying the main product image and specification parameter comparison table in conjunction with each other. At the same time, a similarity distribution histogram of historical review cases is provided as a reference for decision-making.

[0028] In the aforementioned method for mutual recognition of product name pairing using multiple semantic analysis, the incremental training in step S6 adopts a dynamic sampling strategy: stratified sampling is performed on the newly added synonym pair data according to the product category distribution, ensuring that the sample increment for each category does not exceed 15% of the total training data, and the underlying embedding layer parameters are frozen during model updates;

[0029] The dynamic sampling strategy introduces a category balancing factor: an oversampling coefficient α=1.5 is set for low-frequency categories, an undersampling coefficient β=0.8 is set for high-frequency categories, and a SMOTE algorithm is used to generate synthetic samples to fill the long-tail distribution gaps.

[0030] The multi-semantic analysis product name pairing mutual recognition method described above establishes a version rollback mechanism in step S6: when a newly added mapping relationship causes the accuracy of existing product pairing to drop by more than 5%, the model is automatically triggered to roll back to the previous stable version, and an exception report is generated for manual verification.

[0031] Compared with the prior art, the above application has the following advantages:

[0032] The multiple semantic analysis product name matching and mutual recognition method of this application realizes autonomous and intelligent multi-level matching by establishing a synonym mapping library, string similarity calculation, semantic analysis of deep learning models, manual review and automatic update mechanism and other multiple semantic analysis matching means, thereby significantly improving the accuracy and efficiency of product name recognition. This not only optimizes the management process of product data, but also enables the system to adapt to changes in product names through a continuous learning mechanism, ensuring the real-time and adaptability of semantic analysis, and effectively solving the problem of difficult product name matching and mutual recognition technology in large-scale product information management, thereby providing new research directions and application prospects for the future field of product data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flowchart of the method for mutual recognition of product names based on multiple semantic analysis for this application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0035] It should be noted that all directional indications in the embodiments of the present application (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0036] In addition, the descriptions of "first" and "second" in this application are for descriptive purposes only and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0037] like Figure 1 As shown, a method for mutual recognition of product names through multiple semantic analysis includes the following steps:

[0038] S1: Collect synonym pairs of product names and pre-build a synonym mapping library, wherein the synonym mapping library includes at least one standard product name and its corresponding multiple synonym variants;

[0039] S2: Enter the product name. If the pairing is successful, the standard product name will be obtained.

[0040] S3: When S2 fails to match, a string similarity calculation is performed between the input product name and all standard product names in the synonym mapping library. The similarity calculation includes at least one of the following:

[0041] Edit distance algorithm: Calculates the edit distance between the input product name and the standard name. If the edit distance is less than or equal to the preset edit distance threshold, the product is considered to be the same and the corresponding standard product name is obtained.

[0042] Word vector similarity algorithm: Generates vector representations of the input product name and the standard product name based on a pre-trained word vector model, calculates cosine similarity, and if the similarity is ≥ the preset similarity threshold, it is determined to be the same product and the corresponding standard product name is obtained;

[0043] S4: If S3 is not matched, a deep learning model is used to perform semantic analysis on the input product name and the standard product name. If the semantic similarity of the output is ≥ 0.9, or the classification labels are the same, they are both determined to be the same product, and the corresponding standard product name is obtained;

[0044] S5: If S4 still fails to match, the input product name will be added to the pending review list for manual review to confirm whether it is the same product;

[0045] S6: Automatically update synonym mapping library and model parameters based on audit results or business data, including:

[0046] Add the association between the approved product names and existing standard product names to the synonym mapping library;

[0047] Update word vector models and deep learning models through incremental training or online learning.

[0048] The multiple semantic analysis product name matching and mutual recognition method of this application realizes autonomous and intelligent multi-level matching by establishing a synonym mapping library, string similarity calculation, semantic analysis of deep learning models, manual review and automatic update mechanism and other multiple semantic analysis matching means, thereby significantly improving the accuracy and efficiency of product name recognition. This not only optimizes the management process of product data, but also enables the system to adapt to changes in product names through a continuous learning mechanism, ensuring the real-time and adaptability of semantic analysis, and effectively solving the problem of difficult product name matching and mutual recognition technology in large-scale product information management, thereby providing new research directions and application prospects for the future field of product data processing.

[0049] Furthermore, the synonym mapping library in step S1 is constructed in the following manner:

[0050] The initial synonym mapping library consists of high-frequency product names and their synonyms;

[0051] Dynamic expansion is achieved through at least one of the following methods:

[0052] Based on user feedback and business data analysis, synonym pairs are regularly updated;

[0053] Clustering algorithms are used to cluster the word vectors of product names and automatically discover potential synonyms.

[0054] Based on the rule engine, names containing specific keywords are extracted as candidate synonyms.

[0055] Furthermore, the preset edit distance threshold in step S3 is set as a dynamic variable, specifically calculated as follows: threshold = min(3, fl oor(0.2*L)), where L is the character length of the input product name. When the name length exceeds 15 characters, the threshold is fixed at 3. This dynamic threshold mechanism effectively balances the high precision requirement of short names with the generalization ability of long names, significantly improving the accuracy and robustness of cross-category product matching.

[0056] Furthermore, in step S3, the edit distance algorithm and the word vector similarity algorithm are calculated in cascade: the edit distance algorithm is first used to screen a candidate set with a similarity greater than 0.7, and then the word vector similarity algorithm is used to perform a secondary calculation on the candidate set. The final similarity score is weighted and fused with a weight ratio of 0.4:0.6. The weight ratio is dynamically adjusted based on the product category. For example, the edit distance weight is increased to 0.6 for highly standardized 3C products, while the word vector weight is increased to 0.7 for complex descriptions of clothing products.

[0057] Furthermore, the word vector model adopts multi-granularity embedding technology: it simultaneously generates vector representations at the character level, word level and phrase level, and dynamically integrates features of different granularities through the attention mechanism, where the phrase-level vector is generated by a Bi-LSTM encoder.

[0058] The word vector model used in this application uses multi-granularity embedding technology to capture semantic information of text data at different levels. First, through character-level embedding, the model can capture subtle semantic differences at the level of individual characters, which is very useful for dealing with spelling errors, rare characters or root information; second, through word-level embedding, the model can understand the basic semantics of vocabulary, which is crucial for conventional word meaning analysis and understanding; third, phrase-level embedding is generated through the Bi-LSTM encoder, which can not only capture the contextual dependencies of phrases or phrases, but also better understand the meaning of phrases in specific contexts.

[0059] In addition, this application dynamically integrates features of different granularities through the attention mechanism. The model can automatically adjust its dependence on information of different granularities according to the context, thereby extracting the most relevant semantic information in different situations, and helping to improve the performance of natural language processing tasks such as text classification, sentiment analysis, machine translation, etc.

[0060] Furthermore, the deep learning model in step S4 adopts a pre-trained language model based on the Transformer architecture, and the pre-trained language model includes at least one of BERT or RoBERTa, and an attention mechanism is introduced during model training to calculate the semantic relevance between product names.

[0061] The pre-trained language model is preferably the BERT model, which has the advantage that the BERT model can adopt a contrastive learning strategy in the fine-tuning stage, that is, when constructing positive sample pairs, in addition to synonyms, abbreviations, initials and common misspellings of product names can also be added as data enhancement samples.

[0062] Furthermore, the classification label determination in step S4 uses a hierarchical decision-making process: first, the third-level category labels are matched. If no agreement is reached, the second-level category labels are reverted to. If the second-level categories match and the semantic similarity is greater than 0.85, the products are still considered the same. This has the advantage of improving the accuracy and flexibility of product classification through a hierarchical decision-making mechanism.

[0063] Furthermore, the manual review interface in step S5 is provided with a double-blind verification mechanism: comparison information of the input product name and the standard product name is displayed to at least two reviewers in parallel, and the review result is adopted only when both reviewers confirm a match.

[0064] The comparison information is presented in a visual way by marking the difference morphemes with colors, and displaying the main product image and specification parameter comparison table in conjunction with each other. At the same time, a similarity distribution histogram of historical review cases is provided as a reference for decision-making.

[0065] This application ensures the accuracy of the manual review process by implementing a double-blind verification mechanism. At least two reviewers independently compare the input product name with the standard product name. The review result will only be adopted when both reviewers confirm a match. This mechanism effectively avoids possible subjective bias or errors of a single reviewer and improves the reliability of the review.

[0066] The visual presentation of differences further enhances the intuitiveness and efficiency of the review process. By color-coding the different morphemes, reviewers can quickly identify the differences between the input product name and the standard product name. Furthermore, the linked display of the product's main image and specification comparison table allows reviewers to comprehensively assess the consistency of product information.

[0067] In addition, this application can also provide a similarity distribution histogram of historical audit cases as a reference for decision-making, which will help auditors make more accurate judgments based on the experience of past cases.

[0068] The above-mentioned way of displaying comprehensive information not only improves the audit efficiency, but also greatly improves the audit quality.

[0069] Furthermore, the incremental training in step S6 adopts a dynamic sampling strategy: the newly added synonym pair data is stratified and sampled according to the product category distribution, ensuring that the sample increment for each category does not exceed 15% of the total training data, and the underlying embedding layer parameters are frozen when the model is updated;

[0070] The dynamic sampling strategy introduces a category balancing factor: an oversampling coefficient α=1.5 is set for low-frequency categories, an undersampling coefficient β=0.8 is set for high-frequency categories, and a SMOTE algorithm is used to generate synthetic samples to fill the long-tail distribution gaps.

[0071] This application implements a dynamic sampling strategy to ensure that the sample increment of each commodity category is reasonably controlled during the incremental training process and does not exceed 15% of the total training data. This stratified sampling method helps to maintain the balance of data distribution and prevent the model from overfitting to certain categories during training, thereby improving the generalization ability of the model. At the same time, freezing the parameters of the underlying embedding layer helps to retain the existing knowledge base, avoid negative impacts on the learned feature representations when the model is updated, and ensure the stability and continuity of the model. The purpose of introducing the category balance factor is to further optimize the sample distribution between categories by setting the oversampling coefficient α = 1.5 for low-frequency categories and the undersampling coefficient β = 0.8 for high-frequency categories. This balancing measure helps to alleviate the problem of category imbalance, allowing the model to pay more attention to low-frequency categories with less data and improve the model's recognition ability for these categories.

[0072] Using the SMOTE algorithm to generate synthetic samples to fill gaps in long-tail distributions effectively addresses the problem of insufficient class samples in long-tail distributions. The SMOTE algorithm generates new samples by interpolating between minority class samples, increasing data diversity. This helps the model better learn and generalize to new, unseen data, thereby improving the model's performance when processing long-tail distribution data.

[0073] Furthermore, a version rollback mechanism is established in step S6: when the newly added mapping relationship causes the accuracy of existing product matching to drop by more than 5%, the model is automatically triggered to roll back to the previous stable version, and an exception report is generated for manual verification.

[0074] The version rollback mechanism established in this application is intended to ensure the accuracy and stability of data information, avoid paralysis of the entire mechanism due to a single error, and facilitate maintenance personnel to conduct verification and maintenance based on abnormal reports.

[0075] The above is an implementation method provided in conjunction with specific content, and the specific implementation of this application is not limited to these descriptions. Any similarity or similarity with the methods and structures of this application, or any technical deduction or replacement based on the concept of this application, shall be considered within the scope of protection of this application.

Claims

1. A method for mutual recognition of product names through multiple semantic analysis, characterized in that: The following steps are involved: S1: Collect synonym pairs of product names and pre-build a synonym mapping library, wherein the synonym mapping library includes at least one standard product name and its corresponding multiple synonym variants; S2: Enter the product name. If the pairing is successful, the standard product name will be obtained. S3: When S2 fails to match, a string similarity calculation is performed between the input product name and all standard product names in the synonym mapping library. The similarity calculation includes at least one of the following: Edit distance algorithm: Calculates the edit distance between the input product name and the standard name. If the edit distance is less than or equal to the preset edit distance threshold, the product is considered to be the same and the corresponding standard product name is obtained. Word vector similarity algorithm: Generates vector representations of the input product name and the standard product name based on a pre-trained word vector model, calculates cosine similarity, and if the similarity is ≥ the preset similarity threshold, it is determined to be the same product and the corresponding standard product name is obtained; S4: If S3 is not matched, a deep learning model is used to perform semantic analysis on the input product name and the standard product name. If the semantic similarity of the output is ≥ 0.9, or the classification labels are the same, they are both determined to be the same product, and the corresponding standard product name is obtained; S5: If S4 still fails to match, the input product name will be added to the pending review list for manual review to confirm whether it is the same product; S6: Automatically update synonym mapping library and model parameters based on audit results or business data, including: Add the association between the approved product names and existing standard product names to the synonym mapping library; Update word vector models and deep learning models through incremental training or online learning.

2. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: The synonym mapping library in step S1 is constructed in the following manner: The initial synonym mapping library consists of high-frequency product names and their synonyms; Dynamic expansion is achieved through at least one of the following methods: Based on user feedback and business data analysis, synonym pairs are regularly updated; Clustering algorithms are used to cluster the word vectors of product names and automatically discover potential synonyms. Based on the rule engine, names containing specific keywords are extracted as candidate synonyms.

3. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: The preset edit distance threshold in step S3 is set as a dynamic variable, and the specific calculation formula is: threshold = min(3, floor(0.2*L)), where L is the character length of the input product name. When the name length exceeds 15 characters, the threshold is fixed at 3.

4. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: In step S3, the edit distance algorithm and the word vector similarity algorithm are calculated in cascade: the edit distance algorithm is first used to screen the candidate set with a similarity greater than 0.7, and then the word vector similarity algorithm is used to perform a secondary calculation on the candidate set. The final similarity score is weightedly fused at a weight ratio of 0.4:0.

6.

5. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: The word vector model adopts multi-granularity embedding technology: it simultaneously generates vector representations at the character level, word level, and phrase level, and dynamically integrates features of different granularities through an attention mechanism, where the phrase-level vector is generated by a Bi-LSTM encoder.

6. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: The deep learning model in step S4 adopts a pre-trained language model based on the Transformer architecture, and the pre-trained language model includes at least one of BERT or RoBERTa, and an attention mechanism is introduced during model training to calculate the semantic relevance between product names.

7. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: The classification label determination in step S4 adopts a hierarchical decision: first match the third-level category label, and if no agreement is reached, fall back to the second-level category. When the second-level category matches and the semantic similarity is greater than 0.85, it is still determined to be the same product.

8. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: The manual review interface in step S5 is provided with a double-blind verification mechanism: comparison information of the input product name and the standard product name is displayed to at least two reviewers in parallel, and the review result is adopted only when both reviewers confirm a match. The comparison information is presented in a visual way by marking the difference morphemes with colors, and displaying the main product image and specification parameter comparison table in conjunction with each other. At the same time, a similarity distribution histogram of historical review cases is provided as a reference for decision-making.

9. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: The incremental training in step S6 adopts a dynamic sampling strategy: stratified sampling is performed on the newly added synonym pair data according to the product category distribution, ensuring that the sample increment for each category does not exceed 15% of the total training data, and freezing the underlying embedding layer parameters when the model is updated; The dynamic sampling strategy introduces a category balancing factor: an oversampling coefficient α=1.5 is set for low-frequency categories, an undersampling coefficient β=0.8 is set for high-frequency categories, and a SMOTE algorithm is used to generate synthetic samples to fill the long-tail distribution gaps.

10. The method for mutual recognition of product names through multiple semantic analysis according to claim 1, characterized in that: In step S6, a version rollback mechanism is established: when the newly added mapping relationship causes the accuracy of existing product matching to drop by more than 5%, the model is automatically triggered to roll back to the previous stable version, and an exception report is generated for manual verification.

Images