A method and system for product title clustering

Abstract

The application discloses a kind of method and system for commodity title clustering, specifically related to text clustering technical field, for solving the technical scheme of general data is generally aimed at existing disclosure technology, there is also scheme is aimed at such data, but due to different application scenarios, such scheme is not completely applicable problem, its method includes data crawling, commodity title normalization, semantic vector conversion, similarity analysis, clustering analysis and search recommendation of similar goods, system is composed of hardware and software, software includes crawler module, processing module, semantic vector module, similarity calculation module, clustering module and recommendation module, hardware includes CPU, memory bank, storage and GPU;It is through the competitive product analysis of online commodity transaction information data, obtains the similarity of commodity title and carries out clustering analysis and search recommendation to the similarity of commodity title, to improve the accuracy of clustering and search recommendation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text clustering, and more particularly, to a method and system for product title clustering. BACKGROUND

[0002] The knowledge graph constructed around an enterprise is a knowledge representation form that links knowledge, information, entities, etc. inside and outside the enterprise, which is used to help the enterprise better manage, analyze and apply knowledge. It presents the relationship between the data inside and outside the enterprise in the form of a graph by collecting, integrating and analyzing the data, thereby forming a comprehensive, visual and queryable knowledge network. The knowledge graph constructed around an enterprise can contain various information of the enterprise, such as the organization structure, business processes, products and services, employee information, etc. inside the enterprise, and also includes market information, competitor information, industry reports, etc. outside the enterprise. The knowledge graph abstracts, models and expresses the relationship between these information and entities, so that the enterprise can better understand the connection between knowledge, better identify the value and potential application scenarios of knowledge, and thus make better decisions. Enterprise competitor analysis refers to the process of conducting a comprehensive and in-depth analysis of competitors' products, services, marketing strategies, etc. to better understand the competitive advantages and disadvantages of the enterprise in the market, and to develop more scientific and effective market strategies and product development strategies. In competitor analysis, the enterprise will analyze competitors' products and services in detail to better understand competitors' market positioning and marketing strategies.

[0003] In the knowledge graph construction around an enterprise, competitor analysis is a very important part. There are various data sources for obtaining enterprise-related information, and an important source of data information about the enterprise's products is online commodity transaction information data. These data usually include product titles, and the core information of the product is obtained by clustering the product titles, so as to further obtain competitor-related data. However, existing public technologies are usually technical solutions for general data, and there are also solutions for such data. However, due to different application scenarios, such solutions are not completely applicable.

[0004] In order to solve the above problems, a technical solution is provided. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present application provide a method and system for commodity title clustering, which obtains the similarity of commodity titles by performing competitive product analysis on online commodity transaction information data, performs clustering analysis on the similarity of commodity titles, aggregates similar commodities together, thereby obtaining the relationship between competitive products, processes the commodity title through word segmentation and word frequency statistics, obtains a more accurate commodity title normalization word containing core content segments and core content supplement segments, thereby improving the accuracy of clustering and search recommendation, converts the commodity title into a semantic vector, calculates the cos distance, obtains the similarity index between commodities, thereby realizing the clustering and search recommendation of commodities, so as to solve the problems proposed in the above background art.

[0006] To achieve the above object, the present application provides the following technical scheme:

[0007] A method for commodity title clustering, comprising the following steps:

[0008] Step S1, obtaining commodity title data of a transaction website by using a crawler;

[0009] Step S2, processing the commodity title data to obtain two core segments, one is a core content segment and the other is a core content supplement segment, and merging the two segments to obtain a normalization word of the commodity title;

[0010] Step S3, converting the obtained commodity title normalization word into a semantic vector;

[0011] Step S4, performing cos distance calculation on the semantic vector to obtain the similarity index between commodities, and using the result to complete the clustering of commodities and the search recommendation of similar commodities.

[0012] As a further scheme of the present application, in step S1, the implementation of obtaining commodity title data of a transaction website by using a crawler comprises the following:

[0013] Step A1, determining a target website: first, determine the transaction website to be crawled, understand the page structure and data format of the target website, and determine the data type and data field to be crawled;

[0014] Step A2, analyzing the page structure: use the browser developer tool to analyze the page structure and data format of the target website, determine the label and attribute of the data to be crawled, and determine the label and attribute of the data to be crawled;

[0015] Step A3, writing a crawler program: write a crawler program by using Python, obtain the HTML code of the target webpage by sending an HTTP request, parse the HTML code, extract the data to be crawled, and save it to a local file or a database;

[0016] Step A4, timing update: since the transaction website's commodity information is updated frequently, it is necessary to run the crawler program regularly to obtain the latest commodity information.

[0017] As a further scheme of the present application, in step S2, the step of obtaining the core content segment and the core content supplement segment comprises:

[0018] Step S21, segmenting the commodity title;

[0019] Step S22, performing word frequency statistics on all the segmentation results of the commodity title, and marking the stop words in the high-frequency words, wherein the stop words include but are not limited to one, two or three of "de", "he" and "yu";

[0020] Step S23, obtaining the word frequency sorting of all the segmentation results in step S23, and retaining the highest frequency non-stop word in the commodity title as the core content segment;

[0021] Step S24, taking the last non-stop word in the segmentation result as the core content supplement segment;

[0022] Step S25, if the core content segment and the core content supplement segment are the same word, selecting the second highest frequency word as the core content segment;

[0023] Step S26, merging words, merging the core content segment and the core content supplement segment in the form of string splicing to form a new string, and taking the new string as the commodity title normalization word.

[0024] As a further scheme of the present application, in step S3, the commodity title normalization word obtained above is converted into a semantic vector, and the conversion process is as follows:

[0025] Step B1, defining a semantic model: selecting a Word2Vec model as the defined semantic model;

[0026] Step B2, loading a pre-trained model: training a Word2Vec model and loading the model file;

[0027] Step B3, obtaining the vector representation of the normalized word: for each commodity title normalization word, obtaining its corresponding vector representation from the Word2Vec model, which is an N-dimensional real vector;

[0028] Step B4, normalizing the vector representation of the commodity title normalization word: dividing each vector representation by its own module length, so that the module length of each vector representation is 1.

[0029] As a further scheme of the present application, the implementation method of step S4 comprises the following steps:

[0030] Step C1, calculate the similarity between commodities: for each pair of commodities, obtain the semantic vector of the commodity title, calculate the cos distance between the two semantic vectors as the similarity index, the formula of the cos distance between the two semantic vectors is:

[0031]

[0032] In the formula: is the cos distance of vector and vector , and respectively the first commodity and the second commodity whose similarity needs to be calculated semantic vector of the commodity title, and respectively the norm of the commodity title vector and ;

[0033] Step C2, construct a similarity matrix: store the similarity between all commodities using a similarity matrix, the dimension of the similarity matrix is N*N, N is the number of commodities, the similarity between the ith commodity and the jth commodity is stored in the ith row and the jth column or the jth row and the ith column of the matrix;

[0034] Step C3, clustering analysis: taking the similarity matrix as input, using K-Means clustering algorithm to cluster the commodities.

[0035] As a further scheme of the application, step C3 can also realize the search recommendation of similar commodities, taking the similarity matrix obtained by the commodity title of the enterprise competitor as input, sorting the similarity in the similarity matrix, selecting the commodities ranked within the threshold setting range as the search recommendation content, calculating the matching degree score between the commodity title of the competitor and the enterprise according to the matching degree between the commodity title of the competitor and the enterprise, sorting the matching degree score, and recommending the commodities with higher matching degree score to the enterprise.

[0036] An analysis system for commodity title clustering is used to realize the above-mentioned method for commodity title clustering, and the software includes a crawler module, a processing module, a semantic vector module, a similarity calculation module, a clustering module and a recommendation module, wherein:

[0037] The crawler module is used to obtain commodity title data from a transaction website and pass the data to the processing module;

[0038] The processing module is used to process the crawled commodity title data, obtain core content segments and core content supplement segments, construct commodity title normalized words, and pass the commodity title normalized words to the semantic vector module;

[0039] The semantic vector module is used for converting the commodity title normalized words into semantic vectors, which are transmitted to the similarity calculation module;

[0040] The similarity calculation module is used for calculating the cos distance between the semantic vectors as a similarity index, and returning the similarity index to the clustering module;

[0041] The clustering module is used for clustering the commodities according to the similarity index;

[0042] The recommendation module is used for recommending similar commodities according to the similarity.

[0043] Further, the system hardware includes a CPU, a memory bar, a storage element and a GPU, and the CPU, the memory bar, the storage element and the GPU are connected with each other through the Internet technology, wherein:

[0044] The CPU is used for processing data from at least one component of the analysis system for commodity title clustering;

[0045] The memory bar is used for supporting reading and processing of data;

[0046] The storage element is used for storing commodity title data and intermediate data obtained by the crawler;

[0047] The GPU is used for providing graphic calculation and display for the related calculation of the semantic vectors, and providing intermediate storage for the data of the image calculation.

[0048] The method and the system for commodity title clustering have the following technical effects and advantages:

[0049] The present application is through the competitive product analysis on the online commodity transaction information data, obtains the similarity of the commodity title and carries out the cluster analysis to the similarity of the commodity title, and the similar commodities are aggregated together, so that the relationship between the competitive products is obtained, through the word segmentation and the word frequency statistics and other processing of the commodity title, a more accurate commodity title normalized word containing the core content segment and the core content supplement segment is obtained, so that the accuracy of the clustering and the search recommendation is improved, through the conversion of the commodity title into the semantic vector and the calculation of the cos distance, the similarity index between the commodities is obtained, so that the clustering and the search recommendation of the commodities are realized, and the clustering mode is more accurate compared with the traditional attribute-based, label-based and other modes, and different commodities can be more accurately distinguished. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The present application is through the competitive product analysis on the online commodity transaction information data, obtains the similarity of the commodity title and carries out the cluster analysis to the similarity of the commodity title, and the similar commodities are aggregated together, so that the relationship between the competitive products is obtained, through the word segmentation and the word frequency statistics and other processing of the commodity title, a more accurate commodity title normalized word containing the core content segment and the core content supplement segment is obtained, so that the accuracy of the clustering and the search recommendation is improved, through the conversion of the commodity title into the semantic vector and the calculation of the cos distance, the similarity index between the commodities is obtained, so that the clustering and the search recommendation of the commodities are realized, and the clustering mode is more accurate compared with the traditional attribute-based, label-based and other modes, and different commodities can be more accurately distinguished.

[0051] Figure 2 The present application is through the competitive product analysis on the online commodity transaction information data, obtains the similarity of the commodity title and carries out the cluster analysis to the similarity of the commodity title, and the similar commodities are aggregated together, so that the relationship between the competitive products is obtained, through the word segmentation and the word frequency statistics and other processing of the commodity title, a more accurate commodity title normalized word containing the core content segment and the core content supplement segment is obtained, so that the accuracy of the clustering and the search recommendation is improved, through the conversion of the commodity title into the semantic vector and the calculation of the cos distance, the similarity index between the commodities is obtained, so that the clustering and the search recommendation of the commodities are realized, and the clustering mode is more accurate compared with the traditional attribute-based, label-based and other modes, and different commodities can be more accurately distinguished. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] Example 1

[0054] The present invention provides a method for clustering product titles. By combining the attribute information of the bridge itself, the natural environment information of the bridge, and the operating environment information of the bridge, the operating status of the bridge is determined in multiple dimensions. In addition, a graded early warning of the bridge health is made according to the operating status of the bridge itself, thereby ensuring the safety monitoring of the bridge and facilitating the repair and maintenance of the bridge in the future.

[0055] Figure 1 A flowchart of the method for clustering product titles of the present invention is provided, which includes the following steps:

[0056] Step S1, using a crawler to obtain product title data from a trading website;

[0057] Step S2: Process the product title data to obtain two core segments, one of which is the core content segment and the other is the core content supplement segment. The two segments are merged to obtain the normalized words of the product title.

[0058] Step S3: converting the normalized words in the product titles obtained above into semantic vectors;

[0059] In step S4, the semantic vectors are subjected to cosine distance calculation to obtain similarity indicators between products, and the results are used to cluster products and perform search and recommendation of similar products.

[0060] Specifically, the detailed process of each step of the present invention is as follows:

[0061] Step S1:

[0062] The present invention utilizes a crawler to obtain commodity title data from a transaction website.

[0063] It should be noted that, in step S1, the implementation of obtaining product title data from the trading website using crawler technology includes the following:

[0064] Step A1, determine the target website: First, determine the transaction website to be crawled, understand the page structure and data format of the target website, and determine the data type and data fields to be crawled;

[0065] Step A2, analyze the webpage structure: use browser developer tools to analyze the page structure and data format of the target website, determine the label and attribute of the data to be crawled;

[0066] Step A3, write a crawler program: use Python to write a crawler program, get the HTML code of the target webpage by sending HTTP requests, parse the HTML code, extract the data to be crawled, and save it to a local file or database;

[0067] Step A4, update regularly: since the transaction website's product information updates frequently, the crawler program needs to be run regularly to get the latest product information.

[0068] This module can help to obtain a large amount of product data, providing a basis for subsequent data processing. By setting reasonable crawling rules and parameters, the required product information can be quickly and efficiently obtained from the transaction website.

[0069] Step S2:

[0070] According to the title data of the product, the two core segments are obtained, one is the core content segment, and the other is the core content supplement segment, and the two segments are combined to obtain the normalized word of the product title.

[0071] It should be noted that in step S2, the steps of obtaining the core content segment and the core content supplement segment include:

[0072] Step S21, segmenting the product title;

[0073] Step S22, word frequency statistics are performed on all product title segmentation results, and among the high-frequency words, stop words are marked, including but not limited to one, two or three of "of", "and" and "with";

[0074] Step S23, obtain the word frequency sorting of all segmentation results in step S23, and keep the highest frequency non-stop word in the product title as the core content segment;

[0075] Step S24, the last non-stop word in the segmentation result is taken as the core content supplement segment;

[0076] Step S25, if the core content segment and the core content supplement segment are the same word, the second highest frequency word is selected as the core content segment;

[0077] Step S26, merge words, merge the core content segment and the core content supplement segment in the form of string concatenation to form a new string, and take this new string as the normalized word of the product title.

[0078] By counting all the word segmentation results, high-frequency words are found to provide a basis for subsequent acquisition of core content segments and core content supplement segments, and by marking stop words, interference of stop words on subsequent processing can be avoided; by word frequency sorting of the word segmentation results of all commodity titles, the highest frequency non-stop word is selected as the core content segment, and the last non-stop word in the word segmentation result is selected as the core content supplement segment, so that the core information of the commodity title is extracted, the most representative key information in the commodity title can be extracted, and a reliable basis is provided for subsequent commodity clustering and recommendation.

[0079] Step S3:

[0080] According to the normalized words of the commodity title obtained above, the semantic vector is converted.

[0081] Specifically, in step S3, the normalized words of the commodity title obtained above are converted into semantic vectors, and the conversion process is as follows:

[0082] Step B1, define a semantic model: select a Word2Vec model as the defined semantic model;

[0083] Step B2, load the pre-trained model: train a Word2Vec model and load the model file;

[0084] Step B3, obtain the vector representation of the normalized word: for each normalized word of the commodity title, obtain its corresponding vector representation from the Word2Vec model, which is an N-dimensional real vector;

[0085] Step B4, normalize the vector representation of the normalized word of the commodity title: divide each vector representation by its own module length, so that the module length of each vector representation is 1.

[0086] It should be noted that the normalized words of the commodity title are converted into semantic vectors, so as to realize the similarity calculation between the commodity titles, and the semantic vector has the advantage of converting text information into numerical form, which is convenient for calculation and processing.

[0087] Step S4:

[0088] In step S4, the semantic vector is calculated by the cos distance, and the similarity index between the commodities is obtained, and the clustering of the commodities and the search recommendation of similar commodities are completed by using the result.

[0089] It should be noted that the implementation method of step S4 includes the following steps:

[0090] Step C1, calculating the similarity between commodities: for each pair of commodities, obtain the semantic vector of the commodity title, calculate the cos distance between the two semantic vectors as the similarity index, the formula of the cos distance between the two semantic vectors is:

[0091]

[0092] In the formula: is the cos distance of the vector and the vector , and are the commodity title semantic vectors of the first commodity and the second commodity respectively which need to calculate the similarity, and are the norms of the commodity title vectors and ;

[0093] Step C2, constructing a similarity matrix: store the similarity between all commodities using a similarity matrix, the dimension of the similarity matrix is N x N, N is the number of commodities, the similarity between the ith commodity and the jth commodity is stored in the ith row and the jth column or the jth row and the ith column of the matrix;

[0094] Step C3, clustering analysis: taking the similarity matrix as input, using K-Means clustering algorithm to cluster the commodities.

[0095] Further, step C3 can also realize the search and recommendation of similar commodities, taking the similarity matrix obtained from the commodity titles of the enterprise's competitors as input, sorting the similarity in the similarity matrix, selecting the commodities ranked within the threshold setting range as the search recommendation content, calculating the matching degree score between the commodity titles of the competitors and the enterprise according to the matching degree between the commodity titles of the competitors and the enterprise, sorting the matching degree score, and recommending the commodities with higher matching degree score to the enterprise.

[0096] By calculating the cos distance between the commodity titles, the similarity index between the commodities is obtained, which provides a basis for the clustering and recommendation of the commodities, through similarity calculation, similar commodities are clustered together, improving the accuracy and precision of the recommendation, similar commodities can be divided into the same category, users can narrow the search range by selecting the category, thereby improving the search efficiency of the commodities, the commodities in the same category can be displayed together, which is more intuitive to compare the differences between different commodities, thereby promoting the sales of the commodities and improving the competitiveness of the enterprise.

[0097] Embodiment 2

[0098] The difference between embodiment 2 and embodiment 1 of the present application is that the embodiment is to introduce an analysis system for commodity title clustering.

[0099] Figure 2 The structural schematic diagram of the analysis system for commodity title clustering of the present application is given, which includes software including a crawler module, a processing module, a semantic vector module, a similarity calculation module, a clustering module and a recommendation module, wherein:

[0100] The crawler module is used to obtain commodity title data from a transaction website and pass the data to the processing module;

[0101] The processing module is used to process the crawled commodity title data, obtain core content segments and core content supplement segments, construct commodity title normalized words, and pass the commodity title normalized words to the semantic vector module;

[0102] The semantic vector module is used to convert the commodity title normalized words into semantic vectors and pass them to the similarity calculation module;

[0103] The similarity calculation module is used to calculate the cos distance between the semantic vectors as a similarity index and return the similarity index to the clustering module;

[0104] The clustering module is used to cluster commodities according to the similarity index;

[0105] The recommendation module is used to recommend similar commodities according to the similarity.

[0106] Further, the above system hardware includes a CPU, a memory bar, a storage element and a GPU, and the CPU, the memory bar, the storage element and the GPU are connected to each other through Internet technology, wherein:

[0107] The CPU is used to process data from at least one component of the analysis system for commodity title clustering;

[0108] The memory bar is used to support data reading and processing;

[0109] The storage element is used to store the commodity title data obtained by the crawler and intermediate data;

[0110] The GPU is used to provide graphic calculation and display for related calculation of the semantic vector and provide intermediate storage for data of image calculation.

[0111] In summary, the present application can achieve the following beneficial effects:

[0112] More accurate normalization of commodity title: By processing the commodity title through word segmentation and word frequency statistics, a more accurate commodity title normalized word containing core content segments and core content supplement segments is obtained, thereby improving the accuracy of clustering and search recommendation.

[0113] More refined commodity clustering: By converting commodity titles into semantic vectors and calculating the cosine distance, the similarity index between commodities can be obtained, thereby realizing the clustering of commodities. This clustering method is more refined than traditional attribute-based, label-based, etc. It can more accurately distinguish different commodities.

[0114] More intelligent commodity search recommendation: Through the keyword-based recommendation algorithm, more intelligent commodity search recommendation is realized, which improves user experience. By matching the user's search keywords with the keywords in the commodity title, the most suitable commodity for the user's needs is recommended, and the search hit rate is improved.

[0115] Automatic data collection and processing: By using the crawler technology to automatically obtain the commodity title data of the transaction website, and through processing to obtain the normalized word and semantic vector, the manual intervention is reduced, and the data processing efficiency is improved.

[0116] In summary, this method for commodity title clustering can improve the accuracy and intelligence of commodity clustering and search recommendation, and realize automatic data collection and processing, which has practical application value.

[0117] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0118] Finally: The above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for product title clustering, characterized by, Comprising the following steps: Step S1, using crawler to obtain the transaction website commodity title data; Step S2, according to the commodity title data, processing, obtaining two core segments, one is the core content segment, the other is the core content supplement segment, merging the two segments to obtain the normalized word of the commodity title; The steps of obtaining the core content segment and the core content supplement segment include: Step S21, segmenting the commodity title; Step S22, word frequency statistics is carried out on all commodity title segmentation results, and the stop words in the high-frequency words are marked, wherein the stop words include but are not limited to one, two or three of "of", "and" and "with"; Step S23, obtaining the word frequency sorting of all segmentation results in step S22, and retaining the highest frequency non-stop word in the commodity title as the core content segment; Step S24, taking the last non-stop word in the segmentation result as the core content supplement segment; Step S25, if the core content segment and the core content supplement segment are the same word, the second highest frequency word is selected as the core content segment; Step S26, merging words, merging the core content segment and the core content supplement segment in the form of string splicing to form a new string, and taking the new string as the normalized word of the commodity title; Step S3, converting the obtained commodity title normalized word into a semantic vector; Step S4, calculating the cosine distance of the semantic vector to obtain the similarity index between the commodities, and using the result to complete the clustering and search recommendation of similar commodities.

2. The method for product title clustering of claim 1, wherein: In step S1, the implementation of obtaining the commodity title data of the transaction website by using the crawler technology includes The following: Step A1, determining the target website: first, determine the transaction website to be crawled, understand the page structure and data format of the target website, determine the data type and data field to be crawled; Step A2, analyze the page structure: use the browser developer tool to analyze the page structure and data format of the target website, determine the label and attribute of the data to be crawled; Step A3, write the crawler program: use Python to write the crawler program, send HTTP request to obtain the HTML code of the target webpage, parse the HTML code, extract the data to be crawled, and save it to the local file or database; Step A4, update regularly: since the transaction website commodity information updates frequently, the crawler program needs to be run regularly to obtain the latest commodity information.

3. The method for product title clustering as claimed in claim 1 wherein: Step S3, according to the above obtained commodity title normalized word, convert it into a semantic vector, the conversion process is as follows: Step B1, define semantic model: select Word2Vec model as the defined semantic model; Step B2, load pre-trained model: train a Word2Vec model and load the model file; Step B3, obtain the vector representation of the normalized word: for each commodity title normalized word, obtain its corresponding vector representation from the Word2Vec model, which is an N-dimensional real vector; Step B4, normalize the vector representation of the normalized title of the commodity: divide each vector representation by its own module length, so that the module length of each vector representation is 1.

4. The method for product title clustering of claim 1, wherein, The implementation method of step S4 includes the following steps: Step C1, calculate the similarity between commodities: for each pair of commodities, obtain the semantic vector of the commodity title, calculate the cos distance between the two semantic vectors as the similarity index, and the formula of the cos distance between the two semantic vectors is: In the formula: is a vector and a vector cos distance of and respectively are the product title semantic vectors of the first product and the second product which need to calculate the similarity, and respectively are the norms of the product title vectors and . Step C2, construct a similarity matrix: store the similarity between all commodities using a similarity matrix, the dimension of the similarity matrix is N*N, N is the number of commodities, and the similarity between the i-th commodity and the j-th commodity is stored in the i-th row and the j-th column or the j-th row and the i-th column of the matrix; Step C3, clustering analysis: taking the similarity matrix as input, the K-Means clustering algorithm is used for clustering of commodities.

5. The method for product title clustering of claim 4, wherein, Step C3 can also be used to realize the search recommendation of similar commodities, taking the similarity matrix obtained by the title of the commodity of the competitor of the enterprise as input, sorting the similarity in the similarity matrix, selecting the commodities ranked within the threshold setting range as the search recommendation content, calculating the matching degree score between the title of the commodity of the competitor and the enterprise according to the matching degree between the title of the commodity of the competitor and the enterprise, sorting the matching degree score, and recommending the commodities with higher matching degree score to the enterprise.

6. An analysis system for product title clustering for implementing a method for product title clustering according to any one of claims 1 to 5, characterized in that The software includes a crawler module, a processing module, a semantic vector module, a similarity calculation module, a clustering module, and a recommendation module, wherein: The crawler module is used to obtain commodity title data from a transaction website and pass the data to the processing module; The processing module is used to process the crawled commodity title data, obtain core content segments and core content supplement segments, construct normalized commodity title words, and pass the normalized commodity title words to the semantic vector module; The semantic vector module is used to convert the normalized commodity title words into semantic vectors and pass them to the similarity calculation module; The similarity calculation module is used to calculate the cos distance between the semantic vectors as the similarity index and return the similarity index to the clustering module; The clustering module is used to cluster the commodities according to the similarity index; The recommendation module is used to recommend similar commodities according to the similarity.

7. The analysis system for product title clustering of claim 6, wherein, The hardware includes CPU, memory bar, storage, and GPU, and the CPU, memory bar, storage, and GPU are connected to each other through Internet technology, wherein: The CPU is used to process data from at least one component of the analysis system for commodity title clustering; The memory bar is used to support data reading and processing; The storage is used to store the crawler-obtained commodity title data and intermediate data; The GPU is used for graphic calculation and display for related calculation of semantic vectors, and provides intermediate storage for image calculation data.

Images