A multi-semantic analysis commodity name pairing mutual recognition method

Abstract

The application discloses a multiple semantic analysis commodity name pairing mutual recognition method, comprising the following steps: S1, creating a synonym mapping library containing standard commodity names and their corresponding synonyms; S2, inputting commodity names, if matching is successful, extracting standard names; S3, if S2 is not matched, performing string similarity calculation, involving edit distance and word vector similarity algorithm; S4, if S3 is not matched, performing semantic analysis by using a deep learning model, if the similarity is greater than or equal to 0.9 or the classification labels are consistent, the same commodity is determined; S5, if S4 is not matched, inputting the name into a list to be audited for manual auditing; S6, updating the mapping library and model parameters according to the auditing results or business data, including adding commodity name association to the mapping library and updating the model. The application effectively solves the problem of difficult pairing and mutual recognition of commodity names in large-scale commodity information management, and further provides a new research direction and application prospect for the future commodity data processing field.

Description

Technical Field

[0001] This application relates to the technical field of e-commerce data processing, and in particular to a method for matching and recognizing product names through multiple semantic analysis. Background Technology

[0002] In e-commerce platforms, supply chain management, and data integration scenarios, product names contain numerous synonyms, abbreviations, aliases, and variations, making cross-system data matching difficult. Existing solutions primarily rely on keyword matching or single similarity algorithms, such as calculating edit distance. This is ineffective when faced with semantic differences and contextual diversity, hindering high-precision automatic identification and pairing. Therefore, to address this technical challenge, those skilled in the art urgently need to develop a multi-semantic analysis-based product name pairing and mutual recognition method that can improve the accuracy and efficiency of product name pairing to meet practical application needs. Summary of the Invention

[0003] The main purpose of this application is to provide a method for mutual recognition of product name pairing based on multiple semantic analysis, which aims to solve the above-mentioned technical problems.

[0004] A method for pairing and recognizing product names using multiple semantic analysis includes the following steps:

[0005] S1: Collect synonym pairs of product names and pre-build a synonym mapping library, which includes at least one standard product name and its corresponding multiple synonym variants;

[0006] S2: Enter the product name. If the match is successful, the standard product name will be obtained.

[0007] S3: When S2 is not paired, perform string similarity calculation between the input product name and all standard product names in the thesaurus. The similarity calculation includes at least one of the following:

[0008] Edit distance algorithm: Calculate the edit distance between the input product name and the standard name. If the edit distance is less than or equal to the preset edit distance threshold, it is determined to be the same product, and the corresponding standard product name is obtained.

[0009] Word vector similarity algorithm: Generate vector representations of input product name and standard product name based on pre-trained word vector model, calculate cosine similarity, and if the similarity is greater than or equal to the preset similarity threshold, it is determined to be the same product and the corresponding standard product name is obtained;

[0010] S4: When S3 is not paired, the input product name and the standard product name are semantically analyzed by a deep learning model. When the semantic similarity of the output is ≥0.9, or the category label is the same category, they are all determined to be the same product, and the corresponding standard product name is obtained.

[0011] S5: If S4 still does not match, the entered product name will be added to the list of products awaiting review, and will be manually reviewed to confirm whether they are the same product;

[0012] S6: Automatically update the thesaurus and model parameters based on audit results or business data, including:

[0013] Add the associations between approved product names and existing standard product names to the thesaurus;

[0014] Update word vector models and deep learning models through incremental training or online learning.

[0015] In the multi-semantic analysis product name pairing and mutual recognition method described above, the thesaurus in step S1 is constructed in the following way:

[0016] The initial thesaurus consists of high-frequency product names and their synonyms;

[0017] Dynamic expansion is achieved through at least one of the following methods:

[0018] We regularly update synonym pairs based on user feedback and business data analysis.

[0019] By clustering the word vectors of product names using a clustering algorithm, potential synonyms can be automatically discovered.

[0020] The rule engine extracts names containing specific keywords as candidate synonyms.

[0021] In the multi-semantic analysis product name pairing and mutual recognition method described above, the preset edit distance threshold in step S3 is set as a dynamic variable, and the specific calculation formula is: threshold = min(3, floor(0.2*L)), where L is the character length of the input product name. When the name length exceeds 15 characters, the threshold is fixed at 3.

[0022] As described above, the multi-semantic analysis product name pairing and mutual recognition method uses a multi-granularity embedding technique in its word vector model: it simultaneously generates character-level, word-level, and phrase-level vector representations, and dynamically fuses features of different granularities through an attention mechanism. The phrase-level vectors are generated by a Bi-LSTM encoder.

[0023] As described above, in the multi-semantic analysis product name pairing and mutual recognition method, the deep learning model in step S4 adopts a pre-trained language model based on the Transformer architecture. The pre-trained language model includes at least one of BERT or RoBERTa, and an attention mechanism is introduced during model training to calculate the semantic correlation between product names.

[0024] As described above, in the multi-semantic analysis product name matching and mutual recognition method, the determination of the category tag in step S4 adopts a hierarchical decision: first, the third-level category tag is matched. If no agreement is reached, it falls back to the second-level category. When the second-level category matches and the semantic similarity is >0.85, it is still determined to be the same product.

[0025] As described above, in the multi-semantic analysis product name matching and mutual recognition method, the manual review interface in step S5 is equipped with a double-blind verification mechanism: the comparison information between the input product name and the standard product name is displayed to at least two reviewers in parallel, and the review result is adopted only when both reviewers confirm the match.

[0026] The comparison information is presented using a difference visualization method: different words are marked with colors, and the main product image and specification parameter comparison table are displayed in conjunction with the comparison. At the same time, a histogram of similarity distribution of historical review cases is provided as a decision reference.

[0027] As described above, in the multi-semantic analysis product name pairing and mutual recognition method, the incremental training in step S6 adopts a dynamic sampling strategy: the newly added synonym pair data are stratified and sampled according to the product category distribution to ensure that the sample increment of each category does not exceed 15% of the total training data, and the parameters of the underlying embedding layer are frozen when the model is updated.

[0028] The dynamic sampling strategy introduces a category balance factor: an oversampling coefficient α=1.5 is set for low-frequency categories, and an undersampling coefficient β=0.8 is set for high-frequency categories. The SMOTE algorithm is used to generate synthetic samples to fill the gaps in the long-tail distribution.

[0029] As described above, the multi-semantic analysis product name matching and mutual recognition method establishes a version rollback mechanism in step S6: when the addition of a new mapping relationship causes the accuracy of existing product matching to drop by more than 5%, the model is automatically triggered to roll back to the previous stable version, and an anomaly report is generated for manual verification.

[0030] Compared with the prior art, the above application has the following advantages:

[0031] This application's multi-semantic analysis-based product name pairing and mutual recognition method utilizes multiple semantic analysis pairing techniques, including establishing a synonym mapping library, calculating string similarity, semantic analysis using a deep learning model, and a manual review and automatic update mechanism. This achieves autonomous and intelligent multi-level pairing, significantly improving the accuracy and efficiency of product name recognition. It not only optimizes the management process of product data but also allows the system to adapt to changes in product names through a continuous learning mechanism, ensuring the real-time nature and adaptability of semantic analysis. This effectively solves the problem of difficult product name pairing and mutual recognition in large-scale product information management, and provides a new research direction and application prospect for the future field of product data processing. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the multi-semantic analysis product name pairing and mutual recognition method of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0034] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0035] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0036] like Figure 1 As shown, a multi-semantic analysis product name pairing and mutual recognition method includes the following steps:

[0037] S1: Collect synonym pairs of product names and pre-build a synonym mapping library, which includes at least one standard product name and its corresponding multiple synonym variants;

[0038] S2: Enter the product name. If the match is successful, the standard product name will be obtained.

[0039] S3: When S2 is not paired, perform string similarity calculation between the input product name and all standard product names in the thesaurus. The similarity calculation includes at least one of the following:

[0040] Edit distance algorithm: Calculate the edit distance between the input product name and the standard name. If the edit distance is less than or equal to the preset edit distance threshold, it is determined to be the same product, and the corresponding standard product name is obtained.

[0041] Word vector similarity algorithm: Generate vector representations of input product name and standard product name based on pre-trained word vector model, calculate cosine similarity, and if the similarity is greater than or equal to the preset similarity threshold, it is determined to be the same product and the corresponding standard product name is obtained;

[0042] S4: When S3 is not paired, the input product name and the standard product name are semantically analyzed by a deep learning model. When the semantic similarity of the output is ≥0.9, or the category label is the same category, they are all determined to be the same product, and the corresponding standard product name is obtained.

[0043] S5: If S4 still does not match, the entered product name will be added to the list of products awaiting review, and will be manually reviewed to confirm whether they are the same product;

[0044] S6: Automatically update the thesaurus and model parameters based on audit results or business data, including:

[0045] Add the associations between approved product names and existing standard product names to the thesaurus;

[0046] Update word vector models and deep learning models through incremental training or online learning.

[0047] This application's multi-semantic analysis-based product name pairing and mutual recognition method utilizes multiple semantic analysis pairing techniques, including establishing a synonym mapping library, calculating string similarity, semantic analysis using a deep learning model, and a manual review and automatic update mechanism. This achieves autonomous and intelligent multi-level pairing, significantly improving the accuracy and efficiency of product name recognition. It not only optimizes the management process of product data but also allows the system to adapt to changes in product names through a continuous learning mechanism, ensuring the real-time nature and adaptability of semantic analysis. This effectively solves the problem of difficult product name pairing and mutual recognition in large-scale product information management, and provides a new research direction and application prospect for the future field of product data processing.

[0048] Furthermore, the thesaurus in step S1 is constructed in the following manner:

[0049] The initial thesaurus consists of high-frequency product names and their synonyms;

[0050] Dynamic expansion is achieved through at least one of the following methods:

[0051] We regularly update synonym pairs based on user feedback and business data analysis.

[0052] By clustering the word vectors of product names using a clustering algorithm, potential synonyms can be automatically discovered.

[0053] The rule engine extracts names containing specific keywords as candidate synonyms.

[0054] Furthermore, in step S3, the preset edit distance threshold is set as a dynamic variable, specifically calculated as: threshold = min(3, floor(0.2*L)), where L is the character length of the input product name. When the name length exceeds 15 characters, the threshold is fixed at 3. This dynamic threshold mechanism effectively balances the high precision requirement of short names with the generalization ability of long names, significantly improving the accuracy and robustness of cross-category product matching.

[0055] Furthermore, in step S3, the edit distance algorithm and the word vector similarity algorithm are calculated in a cascaded manner: first, the edit distance algorithm is used to filter candidate sets with similarity > 0.7; then, the word vector similarity algorithm is used to perform a second calculation on the candidate sets; finally, the similarity scores are weighted and fused according to a weight ratio of 0.4:0.6. The weight ratio is dynamically adjusted according to the product category. For example, for 3C products with a high degree of standardization, the edit distance weight is increased to 0.6; for clothing products with complex descriptions, the word vector weight is increased to 0.7.

[0056] Furthermore, the word vector model employs multi-granularity embedding technology: simultaneously generating character-level, word-level, and phrase-level vector representations, and dynamically fusing features of different granularities through an attention mechanism, wherein phrase-level vectors are generated by a Bi-LSTM encoder.

[0057] This application employs a word vector model with multi-granularity embedding technology, which can capture semantic information of text data at different levels. First, through character-level embedding, the model can capture subtle semantic differences at the individual character level, which is very useful for handling spelling errors, rare characters, or word root information. Second, through word-level embedding, the model can understand the basic semantics of words, which is crucial for routine word semantic analysis and understanding. Third, through phrase-level embedding generated by a Bi-LSTM encoder, it can not only capture the contextual dependencies of phrases or groups of words, but also better understand the meaning of phrases in specific contexts.

[0058] In addition, this application dynamically fuses features of different granularities through an attention mechanism. The model can automatically adjust its dependence on information of different granularities according to the context, thereby extracting the most relevant semantic information in different situations and helping to improve the performance of natural language processing tasks such as text classification, sentiment analysis, and machine translation.

[0059] Furthermore, the deep learning model in step S4 adopts a pre-trained language model based on the Transformer architecture, which includes at least one of BERT or RoBERTa, and an attention mechanism is introduced during model training to calculate the semantic correlation between product names.

[0060] The pre-trained language model is preferably the BERT model, which has the advantage that the BERT model can adopt a contrastive learning strategy during the fine-tuning stage. That is, when constructing positive sample pairs, in addition to synonyms, abbreviations of product names, initial abbreviations, and common spelling errors can also be added as data augmentation samples.

[0061] Furthermore, the classification label determination in step S4 adopts a hierarchical decision-making approach: first, it matches the third-level category labels; if no agreement is reached, it reverts to the second-level category. When the second-level category matches and the semantic similarity is >0.85, it is still determined to be the same product. Its advantage lies in improving the accuracy and flexibility of product classification through a hierarchical decision-making mechanism.

[0062] Furthermore, the manual review interface in step S5 is equipped with a double-blind verification mechanism: the input product name and the standard product name are displayed to at least two reviewers in parallel, and the review result is adopted only when both reviewers confirm a match.

[0063] The comparison information is presented using a difference visualization method: different words are marked with colors, and the main product image and specification parameter comparison table are displayed in conjunction with the comparison. At the same time, a histogram of similarity distribution of historical review cases is provided as a decision reference.

[0064] This application implements a double-blind verification mechanism to ensure the accuracy of the manual review process. At least two reviewers independently compare the input product name with the standard product name. The review result is only adopted when both reviewers confirm a match. This mechanism effectively avoids possible subjective bias or errors by a single reviewer and improves the reliability of the review.

[0065] The visualization of differences further enhances the intuitiveness and efficiency of the review process. By using color-coding to highlight distinguishing morphemes, reviewers can quickly identify the differences between the entered product name and the standard product name. Simultaneously, the linked display of the product main image and specification comparison table allows reviewers to comprehensively assess the consistency of product information.

[0066] In addition, this application may provide a histogram of similarity distribution of historical review cases as a reference for decision-making, which can help reviewers make more accurate judgments based on their experience with past cases.

[0067] The way the above comprehensive information is displayed not only improves the efficiency of the review process, but also significantly enhances the quality of the review.

[0068] Furthermore, in step S6, the incremental training adopts a dynamic sampling strategy: the newly added synonym pair data is stratified and sampled according to the product category distribution to ensure that the sample increment of each category does not exceed 15% of the total training data, and the parameters of the underlying embedding layer are frozen when the model is updated.

[0069] The dynamic sampling strategy introduces a category balance factor: an oversampling coefficient α=1.5 is set for low-frequency categories, and an undersampling coefficient β=0.8 is set for high-frequency categories. The SMOTE algorithm is used to generate synthetic samples to fill the gaps in the long-tail distribution.

[0070] This application implements a dynamic sampling strategy to ensure that the sample increment for each product category is reasonably controlled during incremental training, and does not exceed 15% of the total training data. This stratified sampling method helps maintain the balance of data distribution, preventing the model from overfitting to certain categories during training, thereby improving the model's generalization ability. Simultaneously, freezing the parameters of the underlying embedding layer helps preserve existing knowledge, avoiding negative impacts on learned feature representations during model updates, and ensuring the model's stability and continuity. The introduction of a category balancing factor aims to further optimize the sample distribution between categories by setting an oversampling coefficient α=1.5 for low-frequency categories and an undersampling coefficient β=0.8 for high-frequency categories. This balancing measure helps alleviate the category imbalance problem, allowing the model to focus more on low-frequency categories with less data, improving the model's ability to identify these categories.

[0071] When the SMOTE algorithm is used to generate synthetic samples to fill gaps in the long-tail distribution, the problem of insufficient category samples in the long-tail distribution can be effectively solved. The SMOTE algorithm generates new samples by interpolating between minority class samples, which increases data diversity and helps the model learn and generalize better to new and unseen data, thereby improving the model's performance when processing long-tail distribution data.

[0072] Furthermore, a version rollback mechanism is established in step S6: when the addition of a new mapping relationship causes the accuracy of existing product matching to drop by more than 5%, the model is automatically rolled back to the previous stable version, and an anomaly report is generated for manual verification.

[0073] The version rollback mechanism established in this application ensures the accuracy and stability of the retained data information, avoids the paralysis of the entire mechanism due to a single error, and facilitates maintenance personnel to conduct verification and maintenance based on anomaly reports.

[0074] The above description is one implementation method provided in conjunction with specific content, and does not imply that the specific implementation of this application is limited to these descriptions. Any methods or structures that are similar to or identical to those of this application, or any technical deductions or substitutions made based on the concept of this application, should be considered within the scope of protection of this application.

Claims

1. A method for pairing and recognizing product names using multiple semantic analysis, characterized in that, Includes the following steps: S1: Collect synonym pairs of product names and pre-build a synonym mapping library, which includes at least one standard product name and its corresponding multiple synonym variants; S2: Enter the product name. If the match is successful, the standard product name will be obtained. S3: When S2 is not paired, perform string similarity calculation between the input product name and all standard product names in the thesaurus. The similarity calculation includes at least one of the following: Edit distance algorithm: Calculate the edit distance between the input product name and the standard name. If the edit distance is less than or equal to the preset edit distance threshold, it is determined to be the same product, and the corresponding standard product name is obtained. Word vector similarity algorithm: Generate vector representations of input product name and standard product name based on pre-trained word vector model, calculate cosine similarity, and if the similarity is greater than or equal to the preset similarity threshold, it is determined to be the same product and the corresponding standard product name is obtained; S4: When S3 is not paired, the input product name and the standard product name are semantically analyzed by a deep learning model. When the semantic similarity of the output is ≥0.9, or the category label is the same category, they are all determined to be the same product, and the corresponding standard product name is obtained. S5: If S4 still does not match, the entered product name will be added to the list of products awaiting review, and will be manually reviewed to confirm whether they are the same product; S6: Automatically update the thesaurus and model parameters based on audit results or business data, including: Add the associations between approved product names and existing standard product names to the thesaurus; Update word vector models and deep learning models through incremental training or online learning; In step S4, the classification label determination adopts a hierarchical decision-making method: first, the third-level category label is matched. If no agreement is reached, it is back to the second-level category. When the second-level category matches and the semantic similarity is >0.85, it is still determined to be the same product.

2. The multi-semantic analysis product name pairing and mutual recognition method according to claim 1, characterized in that, The thesaurus map in step S1 is constructed in the following manner: The initial thesaurus consists of high-frequency product names and their synonyms; Dynamic expansion is achieved through at least one of the following methods: We regularly update synonym pairs based on user feedback and business data analysis. By clustering the word vectors of product names using a clustering algorithm, potential synonyms can be automatically discovered. The rule engine extracts names containing specific keywords as candidate synonyms.

3. The method for pairing and recognizing product names using multiple semantic analysis according to claim 1, characterized in that, In step S3, the preset edit distance threshold is set as a dynamic variable. The specific calculation formula is: threshold = min(3, floor(0.2*L)), where L is the character length of the input product name. When the name length exceeds 15 characters, the threshold is fixed at 3.

4. The method for pairing and recognizing product names using multiple semantic analysis according to claim 1, characterized in that, The word vector model employs multi-granularity embedding technology: it simultaneously generates character-level, word-level, and phrase-level vector representations, and dynamically fuses features of different granularities through an attention mechanism. The phrase-level vectors are generated by a Bi-LSTM encoder.

5. The method for pairing and recognizing product names using multiple semantic analysis according to claim 1, characterized in that, The deep learning model in step S4 adopts a pre-trained language model based on the Transformer architecture. The pre-trained language model includes at least one of BERT or RoBERTa, and an attention mechanism is introduced during model training to calculate the semantic correlation between product names.

6. The method for pairing and recognizing product names using multiple semantic analysis according to claim 1, characterized in that, In step S5, the manual review interface is equipped with a double-blind verification mechanism: the comparison information between the input product name and the standard product name is displayed to at least two reviewers in parallel, and the review result is adopted only when both reviewers confirm a match; The comparison information is presented using a difference visualization method: different words are marked with colors, and the main product image and specification parameter comparison table are displayed in conjunction with the comparison. At the same time, a histogram of similarity distribution of historical review cases is provided as a decision reference.

7. The method for pairing and recognizing product names using multiple semantic analysis according to claim 1, characterized in that, In step S6, incremental training adopts a dynamic sampling strategy: the newly added synonym pair data is stratified and sampled according to the product category distribution to ensure that the sample increment of each category does not exceed 15% of the total training data, and the parameters of the underlying embedding layer are frozen when the model is updated. The dynamic sampling strategy introduces a category balance factor: an oversampling coefficient α=1.5 is set for low-frequency categories, and an undersampling coefficient β=0.8 is set for high-frequency categories. The SMOTE algorithm is used to generate synthetic samples to fill the gaps in the long-tail distribution.

8. The method for pairing and recognizing product names using multiple semantic analysis according to claim 1, characterized in that, In step S6, a version rollback mechanism is established: when the addition of a new mapping relationship causes the accuracy of existing product matching to drop by more than 5%, the model is automatically rolled back to the previous stable version, and an anomaly report is generated for manual verification.

Images