Attribute word mining methods and related products

Abstract

In the field of artificial intelligence technology, this invention specifically relates to an attribute word mining method and related products. The method includes: acquiring candidate words related to the attribute parameters of the object being mined; clustering the candidate words to obtain clusters composed of the candidate words; determining a ranking score for each cluster based on the consistency of the topic distribution of the candidate words within the cluster, wherein the ranking score is positively correlated with the consistency of the topic distribution of the candidate words within the cluster; ranking the clusters according to their ranking scores; and filtering the attribute words of the object being mined based on the ranking results. This application can reduce the difficulty of attribute word mining and improve its accuracy and efficiency.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to an attribute word mining method, an attribute word mining device, a computer-readable medium, an electronic device, and a computer program product. Background Technology

[0002] Attribute terms generally refer to words used to describe the attribute parameters of a product. Product attribute parameters may include, for example, material, specifications, grade, color, style, size, sizing, season, etc. For example, attribute terms indicating the material attribute of clothing may include mink fur, fox fur, plush, etc., while attribute terms indicating the price attribute may include cheap, cost-effective, low price, etc.

[0003] With the development of internet technology, the basic attribute words of products often have a large number of variations or derivative words due to differences in users' language habits or the emergence of new expressions during the long-term and extensive online dissemination process. This brings great challenges to attribute word mining. Summary of the Invention

[0004] The purpose of this application is to provide an attribute word mining method, attribute word mining device, computer-readable medium, electronic device, and computer program product, which at least to some extent overcomes the technical problems of high difficulty and poor accuracy in attribute word mining in related technologies.

[0005] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0006] According to one aspect of the embodiments of this application, an attribute word mining method is provided, the method comprising:

[0007] Retrieve candidate words related to the attribute parameters of the mining object;

[0008] The candidate words are clustered to obtain clusters composed of the candidate words;

[0009] The ranking score of the cluster is determined based on the consistency of the topic distribution of candidate words in the cluster, and the ranking score is positively correlated with the consistency of the topic distribution of candidate words in the cluster.

[0010] The clusters are sorted according to their ranking scores, and the attribute words of the mining objects are filtered according to the ranking results.

[0011] According to one aspect of the embodiments of this application, an attribute word mining apparatus is provided, the apparatus comprising:

[0012] The acquisition module is configured to acquire candidate words related to the attribute parameters of the mining object;

[0013] The clustering module is configured to perform clustering processing on the candidate words to obtain clusters composed of the candidate words;

[0014] The scoring module is configured to determine the ranking score of the cluster based on the consistency of the topic distribution of candidate words in the cluster, wherein the ranking score is positively correlated with the consistency of the topic distribution of candidate words in the cluster.

[0015] The filtering module is configured to sort the clusters according to their ranking scores and filter the attribute words of the mining objects based on the ranking results.

[0016] According to one aspect of the embodiments of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the attribute word mining method as described in the above technical solutions.

[0017] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the attribute word mining method as described above by executing the executable instructions.

[0018] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the attribute word mining method as described in the above technical solutions.

[0019] In the technical solution provided in this application embodiment, by clustering candidate words and sorting the clusters obtained from the clustering according to the consistency of topic distribution, the attribute words of the mining object can be screened according to the sorting results. Based on word clustering, the semantic relevance between candidate words can be fully utilized to mine the set of words belonging to the same cluster. Then, based on the consistency of topic distribution, clusters with high semantic clustering and consistent topic distribution are mined. Therefore, the attribute words of the mining object can be accurately and efficiently screened, which can significantly reduce the difficulty of attribute word mining and improve the accuracy and efficiency of attribute word mining. It is especially suitable for attribute word mining of emerging words or compound words with a large number of derivative variations and complex and diverse forms.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0022] Figure 1 An exemplary system architecture block diagram applying the technical solution of this application is shown.

[0023] Figure 2 A flowchart illustrating the steps of an attribute word mining method according to one embodiment of this application is shown.

[0024] Figure 3 A flowchart illustrating the steps for obtaining candidate words based on text corpus in one embodiment of this application is shown.

[0025] Figure 4 A schematic diagram of a corpus is shown in one embodiment of this application.

[0026] Figure 5 This paper illustrates a list of high-quality phrases obtained by phrase mining a text corpus in one embodiment of this application.

[0027] Figure 6 A list of product attributes of an e-commerce website is shown in one embodiment of this application.

[0028] Figure 7 This illustrates a cluster formed after clustering candidate words in one embodiment of this application.

[0029] Figure 8 A flowchart illustrating the steps for determining cluster ranking scores in one embodiment of this application is shown.

[0030] Figure 9 A flowchart illustrating the steps for determining semantic consistency parameters in one embodiment of this application is shown.

[0031] Figure 10 A flowchart illustrating the steps for determining topic distribution parameters in one embodiment of this application is shown.

[0032] Figure 11 A statistical diagram illustrating the probability distribution of topics in one embodiment of this application is shown.

[0033] Figure 12 This illustration shows a cluster sequence obtained by sorting the clusters in one embodiment of this application.

[0034] Figure 13This illustration shows a module block diagram of attribute word mining and further applications in an application scenario according to an embodiment of this application.

[0035] Figure 14 A structural block diagram of the attribute word mining device provided in an embodiment of this application is shown.

[0036] Figure 15 A computer system architecture block diagram suitable for implementing the embodiments of this application is shown. Detailed Implementation

[0037] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0038] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0039] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0041] In the specific implementation of this application, data related to product descriptions or product reviews published by users on online trading platforms are involved. When the various embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0042] Figure 1An exemplary system architecture block diagram illustrating the application of the technical solution of this application is shown schematically.

[0043] like Figure 1 As shown, the system architecture 100 may include terminal devices 110, a network 120, and a server 130. Terminal devices 110 may include various electronic devices such as smartphones, tablets, laptops, desktop computers, smart speakers, smart wearable devices, and smart in-vehicle devices. Server 130 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Network 120 may be a communication medium of various connection types capable of providing a communication link between terminal devices 110 and server 130, such as a wired or wireless communication link.

[0044] Depending on the implementation requirements, the system architecture in this application embodiment can have any number of terminal devices, networks, and servers. For example, server 130 can be a server group composed of multiple server devices. In addition, the technical solutions provided in this application embodiment can be applied to terminal device 110, or to server 130, or can be implemented jointly by terminal device 110 and server 130. This application does not impose any special limitations on this.

[0045] For example, a client application for an online trading platform can be installed on terminal device 110. The online trading platform can include e-commerce websites or applications, as well as social networking tools, instant messaging tools, video streaming platforms, live streaming applications, and so on, all with online trading capabilities. Users can access the online trading platform through terminal device 110 to engage in commodity trading activities. During the commodity trading process, various textual content related to the attribute parameters of the goods is involved. For example, sellers can publish product titles and detailed product descriptions on the online trading platform, while buyers can post reviews of the goods after completing the transaction. Server 130 can collect and summarize the various textual content on the online trading platform to form a corpus, and then mine attribute words representing the attribute parameters of the goods from the corpus.

[0046] In some embodiments of this application, server 130 can utilize cloud computing technology to obtain network computing resources for attribute word mining in an on-demand and easily scalable manner.

[0047] Cloud computing is a computing model that distributes computing tasks across a large pool of computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" appear infinitely scalable, readily available, on-demand, and expandable, with payment based on usage.

[0048] As a provider of fundamental cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) is established. Various types of virtual resources are deployed within the resource pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices. Cloud computing is a product of the convergence and development of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0049] In some embodiments of this application, a machine learning model based on artificial intelligence technology can be installed on the terminal device 110 or the server 130, thereby using the machine learning model to automatically perform natural language processing on the collected text data.

[0050] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0051] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0052] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0053] The following detailed description of the attribute word mining method, attribute word mining device, computer-readable medium, electronic device, and computer program product provided in this application, in conjunction with specific embodiments, provides a detailed explanation of these technical solutions.

[0054] Figure 2This illustration shows a flowchart of the attribute word mining method in one embodiment of this application. The attribute word mining method can be performed by… Figure 1 The terminal device or server shown is used for execution; this application embodiment uses the attribute word mining method executed by the server as an example for illustration. For example... Figure 2 As shown, the attribute word mining method can mainly include the following steps S210 to S240.

[0055] Step S210: Obtain candidate words related to the attribute parameters of the mining object;

[0056] Step S220: Perform clustering on the candidate words to obtain clusters composed of the candidate words;

[0057] Step S230: Determine the ranking score of the cluster based on the consistency of the topic distribution of candidate words in the cluster. The ranking score is positively correlated with the consistency of the topic distribution of candidate words in the cluster.

[0058] Step S240: Sort the clusters according to their ranking scores, and filter the attribute words of the mining objects based on the ranking results.

[0059] In the attribute word mining method provided in this application embodiment, candidate words are clustered, and the resulting clusters are sorted according to the consistency of topic distribution. The attribute words of the target object can then be selected based on the sorting results. Based on word clustering, the semantic relevance between candidate words can be fully utilized to mine sets of words belonging to the same cluster. Then, based on the consistency of topic distribution, clusters with high semantic aggregation and consistent topic distribution are mined. Therefore, the attribute words of the target object can be accurately and efficiently selected, significantly reducing the difficulty of attribute word mining and improving its accuracy and efficiency. This method is particularly suitable for mining attribute words of emerging or compound words with numerous derivative variations and complex forms.

[0060] The following sections provide a detailed explanation of each step in the attribute word mining method provided in this application.

[0061] In step S210, candidate words related to the attribute parameters of the mining object are obtained.

[0062] In one embodiment of this application, the mining object can be a business object with one or more attribute parameters, such as a product category in an online trading platform, whose corresponding attribute parameters may include the product's material, specifications, price, grade, color, style, etc.

[0063] During data communication with terminal devices, the server can collect communication data and analyze the data content to obtain candidate words related to the attribute parameters of the target object. For example, when the target object is a product category, it can obtain the products under that product category and extract candidate words from the relevant product information. Candidate words can include single words or compound words consisting of at least two words.

[0064] In one embodiment of this application, candidate words may include words extracted from text corpora and separately distributed words obtained through word acquisition. The separately distributed words are "head" words that are frequently used in the attribute descriptions of the target object but have a limited number of categories, while the words extracted from the text corpus are generally "mid-to-long-tail" words that are less frequently used in the attribute descriptions of the target object but have a larger number of categories. By combining these two types of word acquisition methods, the accuracy and comprehensiveness of candidate words can be balanced, improving the mining effect of attribute words.

[0065] Figure 3 A flowchart illustrating the steps for obtaining candidate words based on text corpus in one embodiment of this application is shown. Figure 3 As shown, based on the above embodiments, obtaining candidate words related to the attribute parameters of the mining object in step S210 may include the following steps S310 to S320.

[0066] Step S310: Collect text data from the target data to obtain a corpus composed of text data.

[0067] The text corpus consists of texts of varying lengths composed of a number of words. In this embodiment, the full set of words can be extracted from the text corpus, and then candidate words related to the attribute parameters of the mining object can be obtained through word filtering.

[0068] In one embodiment of this application, the method for collecting text corpus of the target object may include the following steps S311 to S312.

[0069] In step S311, the online trading platform related to the mining object is obtained.

[0070] In one embodiment of this application, the target of data mining can be a product category, that is, a category of products. For example, it could be product categories such as clothing, mobile phones, and computers.

[0071] Online trading platforms can include e-commerce websites or e-commerce applications, as well as online social networking tools, instant messaging tools, video playback platforms, live streaming applications, and so on, that have online trading functions.

[0072] In step S312, web data is crawled from the online trading platform to obtain text corpus related to the target of the mining.

[0073] This application embodiment can utilize web crawlers to crawl web data and obtain text corpora related to the target object from online trading platforms.

[0074] A web crawler (also known as a web spider, web robot, or web crawler) is a program or script that automatically retrieves information from the World Wide Web according to certain rules.

[0075] Based on their system architecture and implementation technology, web crawlers can be broadly classified into the following types: general-purpose web crawlers, focused web crawlers, incremental web crawlers, and deep web crawlers.

[0076] General-purpose web crawlers, also known as scalable web crawlers, expand their crawling scope from a few seed URLs to the entire web, primarily collecting data for portal sites, search engines, and large web service providers. These web crawlers cover a vast range and number of pages, requiring high crawling speed and storage space, while having relatively lower requirements on the order in which pages are crawled. Furthermore, due to the large number of pages to be refreshed, they typically employ parallel processing, but this results in a relatively long time between page refreshes.

[0077] A focused web crawler, also known as a topical web crawler, selectively crawls pages related to a predefined topic. Compared to general web crawlers, focused web crawlers only crawl pages relevant to the topic, significantly saving hardware and network resources. The fewer pages saved are also updated more frequently, and they can effectively meet the information needs of specific groups in specific fields. Compared to general web crawlers, focused web crawlers add link evaluation and content evaluation modules. The key to the focused crawler's crawling strategy is evaluating the importance of page content and links; different methods calculate different importance levels, resulting in different link access orders.

[0078] An incremental web crawler is a crawler that incrementally updates downloaded web pages and only crawls newly generated or changed pages. It can, to a certain extent, ensure that the crawled pages are as recent as possible. Compared to web crawlers that periodically crawl and refresh pages, incremental crawlers only crawl newly generated or updated pages when needed, without re-downloading unchanged pages. This effectively reduces the amount of data downloaded, updates crawled pages promptly, and reduces time and space consumption, but it increases the complexity of the crawling algorithm and the difficulty of implementation. The architecture of an incremental web crawler includes a crawling module, a sorting module, an updating module, a local page set, a set of URLs to be crawled, and a local page URL set. Incremental crawlers have two goals: to keep the pages stored in the local page set up-to-date and to improve the quality of the pages in the local page set. To achieve the first goal, incremental crawlers need to update the content of local pages by revisiting web pages. Common methods include: (1) uniform update method: the crawler visits all web pages at the same frequency, regardless of the frequency of page changes; (2) individual update method: the crawler revisits each page according to the frequency of individual page changes; (3) classification-based update method: the crawler divides web pages into two categories based on their frequency of change: a subset of web pages that are updated faster and a subset of web pages that are updated slower, and then visits these two categories of web pages at different frequencies.

[0079] Web pages can be categorized by their structure into surface web pages and deep web pages (also known as invisible web pages or hidden web pages). Surface web pages are those that can be indexed by traditional search engines, primarily consisting of static pages accessible via hyperlinks. Deep web pages, on the other hand, contain most of their content that cannot be accessed through static links; they are hidden behind search forms and can only be accessed by submitting specific keywords. For example, pages whose content is only visible after user registration are considered deep web pages. A deep web crawler is a tool used to crawl the content of deep web pages. Its architecture includes six basic functional modules (crawler controller, parser, form analyzer, form processor, response analyzer, and LVS controller) and two internal data structures (URL list and LVS table). The LVS (Label Value Set) represents a collection of tags / values ​​used to represent the data source for filling in forms.

[0080] The embodiments of this application can be implemented by combining the above-mentioned crawling techniques.

[0081] The text corpus may include at least one of the following: product title text, product description text, or product review text from an online trading platform.

[0082] The product title text is a descriptive text added by the seller to represent the product name. It is typically displayed at the top of the product list and product details page. To highlight product features, sellers usually add various attribute descriptions to the product title text. Furthermore, product title text is generally short and contains few redundant words, making it easy to extract highly accurate attribute terms.

[0083] Product description text is added by the seller to provide a comprehensive description of the product, and it is generally displayed in the lower middle part of the product details page. Since the product title has a limited length, sellers add specific product attribute descriptions to the product description text. Product description text is characterized by its unlimited length and high frequency of redundant words. Extracting attribute words from the product description text can improve the comprehensiveness of attribute word mining.

[0084] Product reviews are descriptive texts added by buyers after a transaction. Buyers typically include product descriptions and user experiences from a consumer perspective, along with other product attributes. Extracting attribute words from these reviews can further improve the comprehensiveness of attribute word mining.

[0085] The collected text data can be compiled into a corpus for subsequent extraction of candidate words and as a basic database for semantic analysis.

[0086] Figure 4 A schematic diagram of a corpus from one embodiment of this application is shown. Figure 4 As shown, this corpus mainly consists of two parts: corpus source and corpus content. The corpus source indicates that the text corpus was collected from sources such as product title text, product description text, or product review text, for example, "product title" or "merchant review" as shown in the figure. The corpus content is the specific text content collected, such as the descriptive text of clothing and underwear shown in the figure.

[0087] Step S320: Extract candidate words from the text corpus that are related to the attribute parameters of the object being mined.

[0088] A text corpus consists of a number of words. By performing natural language processing on the text corpus, candidate words related to the attribute parameters of the object being mined can be extracted from it.

[0089] In one embodiment of this application, the method for extracting candidate words from a text corpus may include the following steps S321 to S322.

[0090] In step S321, the text corpus is segmented to obtain the text segments that make up the text corpus.

[0091] Word segmentation is the process of breaking down long texts such as sentences, paragraphs, and articles into data structures based on words, facilitating subsequent processing and analysis. Word segmentation methods can be broadly categorized into three types: dictionary-based matching, statistical methods, and deep learning-based methods.

[0092] Dictionary-based word segmentation offers advantages such as speed and low cost. The basic idea is to segment and adjust the Chinese text to be segmented according to certain rules, then match it with words in a dictionary. If a match is successful, the text is segmented according to the dictionary word; if a match fails, adjustments or reselection are made, and this process is repeated. Representative methods include forward maximum matching, backward maximum matching, and bidirectional matching.

[0093] Statistical word segmentation methods have the advantage of strong adaptability. Commonly used statistical word segmentation methods include algorithms such as HMM, CRF, SVM, and deep learning. For example, Stanford and HanLP word segmentation tools are based on the CRF algorithm. Taking CRF as an example, its basic idea is to annotate and train Chinese characters, taking into account not only the frequency of word occurrences but also the context, and it has good learning capabilities. Therefore, it has good performance in identifying ambiguous words and out-of-vocabulary words.

[0094] Deep learning-based word segmentation methods have the advantages of high accuracy and strong adaptability. For example, a word segmenter can be implemented using bidirectional LSTM+CRF, achieving a character accuracy of up to 97.5%. Deep learning-based word segmentation methods are essentially sequence labeling, thus possessing strong versatility; they can be used for tasks such as named entity recognition.

[0095] In one embodiment of this application, a combination of machine learning algorithms and a dictionary is used to improve word segmentation accuracy and domain adaptability. In this embodiment, phrase mining is first performed on the text corpus to obtain high-quality phrases; then, word segmentation is performed on the text corpus based on a dictionary containing these high-quality phrases to obtain the word segments that make up the text corpus.

[0096] The purpose of phrase mining is to extract high-quality phrases from a large amount of text corpora, which is one of the basic tasks in the field of NLP. Phrase mining mainly solves the problem of insufficient professional dictionaries in professional fields (such as medicine, technology, etc.) and reduces the cost of manual collation. High-quality phrases can include evaluation dimensions such as frequency, consistency, information content, and integrity. The frequency dimension means that the higher the frequency of a phrase appears, the higher the probability that it is a high-quality phrase; the consistency dimension is used to judge whether the word collocations in the phrase are reasonable; the information content dimension refers to that high-quality phrases need to express certain themes and concepts. For example, for the two phrases "machine learning" and "this article", in the corpus of machine learning, the former has higher quality. The integrity dimension means that high-quality phrases need to be a complete semantic unit. For example, "support vector machine" has higher semantic integrity compared with "vector machine".

[0097] The methods for phrase mining can include unsupervised learning and supervised learning.

[0098] The phrase mining method based on unsupervised learning can include the following multiple steps:

[0099] (1) Screening candidate phrases: Cut the article using n-gram (n can take any value from 1 to 6), and screen according to the set frequency threshold, retaining the phrases with a frequency higher than the frequency threshold.

[0100] (2) Calculating statistical features: Statistical features can include at least one of tf-idf, textRank, PMI (point mutual information), or left and right neighbor entropy. Tf-idf or textRank can filter out words with high frequency but low importance in the text corpus, such as pronouns, adverbs, prepositions, auxiliary words, etc. PMI is used to judge which segmentation method is more reasonable by measuring the consistency of phrases and comparing n-gram phrases with the same n value, mainly for dealing with the situation of word segmentation out-of-bounds. For example, "cinema" is more consistent than "of the movie". The left and right neighbor entropy describes the richness of the left and right collocations of a phrase. A good phrase should have very rich left and right collocations. If the richness of the left and right collocations of a phrase is low, it means the quality of the phrase is low. For example, the right side of "Aristo" always appears with a "Der" character, and its right entropy is not rich, and generally its phrase quality is considered low.

[0101] (3) Scoring the quality of phrases, combining the results of the above feature statistics, evaluating the quality of phrases and obtaining a score, and taking the top k phrases with the highest scores as the output.

[0102] The phrase mining method based on supervised learning adds the processes of sample annotation, classifier learning, and corpus cutting on the basis of unsupervised learning.

[0103] (1) Sample labeling: Manual or remote supervision is used. Remote supervision generally uses online knowledge bases. If a phrase is in the lexicon, it is considered a high-quality phrase.

[0104] (2) Classifier learning: The classifier can use various models such as decision tree, Bayesian, random forest, SVM, etc. The feature vector used in the classifier can be the statistical features in the above unsupervised learning methods (such as the values ​​calculated by IF-IDF, PMI, and left and right neighbor entropy).

[0105] (3) Corpus segmentation: The process of corpus segmentation is the process of filtering phrases, filtering high-quality phrases from candidate phrases and proceeding to the next iteration.

[0106] In one embodiment of this application, the AutoPhrase open-source tool can be used to perform phrase mining on text corpora to obtain high-quality phrases containing emerging words or compound words.

[0107] Figure 5 This illustration shows a list of high-quality phrases obtained by phrase mining a text corpus in one embodiment of this application. For example... Figure 5 As shown, by scoring and ranking compound words according to their quality, a number of phrases that rank highly can be marked as high-quality phrases.

[0108] Adding high-quality phrases to the dictionary allows the Hanlp word segmentation tool to be used in conjunction with the dictionary to segment the text corpus, thereby obtaining the text segments that make up the text corpus.

[0109] In step S322, irrelevant words in the text segmentation are filtered out to obtain candidate words related to the attribute parameters of the mining object.

[0110] In one embodiment of this application, irrelevant words include at least one of stop words, product words, or brand words.

[0111] Stop words are words that are automatically filtered out before or after processing natural language data (or text) in information retrieval to save storage space and improve search efficiency. These stop words are manually entered and not automatically generated, forming a stop word list. Generally speaking, stop words can be divided into two categories. One category consists of function words found in human language. These function words are extremely common and, compared to other words, have little actual meaning, such as "the," "is," "at," "which," and "on" in English. The other category includes lexical words, such as "want." These words are widely used, but search engines cannot guarantee truly relevant search results for them, making it difficult to narrow down the search and reducing search efficiency. Therefore, these words are usually removed from the question to improve search performance.

[0112] Product terms and brand terms are words used to represent the category name or brand name of a product, such as "mobile phone" and corresponding mobile phone brands, "clothing" and corresponding clothing brands, and so on. Product terms and brand terms can be obtained through online data collection.

[0113] In addition to extracting candidate words from text corpora, embodiments of this application can also collect word corpora of the mining object to obtain candidate words related to the attribute parameters of the mining object.

[0114] Methods for collecting word corpora of the target object may include: acquiring online trading platforms related to the target object; and crawling web data from the online trading platforms to obtain word corpora related to the target object. The word corpus may include attribute words of product categories from the online trading platforms.

[0115] Online trading platforms can include e-commerce websites or e-commerce applications, as well as online social networking tools, instant messaging tools, video playback platforms, live streaming applications, and so on, that have online trading functions.

[0116] Figure 6 This document illustrates a product attribute list for an e-commerce website in one embodiment of this application. This list primarily corresponds to various product attribute parameters included under the clothing category. For example, the "women's clothing" category includes attribute words such as dresses, middle-aged and elderly women's clothing, plus-size women's clothing, cheongsams / Tang suits, skirts, short jackets, and T-shirts. Additionally, the "Advanced Options" section provides various attribute parameters such as size, color, and material for selection. By crawling data from this list using a web crawler, precise product attribute words can be obtained.

[0117] In step S220, the candidate words are clustered to obtain clusters composed of the candidate words.

[0118] Clustering is the process of dividing a dataset into different classes or clusters based on a specific criterion (such as distance), maximizing the similarity of data objects within the same cluster and maximizing the differences between data objects in different clusters. In other words, after clustering, data of the same class should be grouped together as much as possible, while data of different classes should be separated as much as possible.

[0119] This application embodiment uses the K-Means clustering algorithm to cluster candidate words. Since the K-Means clustering algorithm only operates on a given complete dataset and does not require any special training data, it is an unsupervised machine learning technique.

[0120] The most common implementation of the K-Means clustering algorithm is the iterative refinement heuristic. For example, in this embodiment, given the number of partitions k, an initial partition is created, and k objects are randomly selected from the dataset, each initially representing a cluster centroid. For other objects, their distances to each cluster centroid are calculated, and they are assigned to the nearest cluster. An iterative relocation technique is used to attempt to improve the partitions by moving objects between partitions. The relocation technique involves recalculating the cluster average when a new object joins a cluster or an existing object leaves a cluster, and then redistributing the objects. This process is repeated until the objects in each cluster no longer change.

[0121] Figure 7 This illustration shows a cluster formed after clustering candidate words in one embodiment of this application. For example, cluster slot_345 includes multiple candidate words such as red maple, autumn, withering, late autumn, branches, and "one leaf tells of autumn," which are grouped into the same cluster because they have a certain semantic connection with autumn. Similarly, cluster slot_297 includes candidate words such as flying fish, shark, mermaid, marine life, deep sea, and starfish, which are grouped into the same cluster because they have a certain semantic connection with the ocean.

[0122] In step S230, the ranking score of the cluster is determined based on the consistency of the topic distribution of candidate words in the cluster. The ranking score is positively correlated with the consistency of the topic distribution of candidate words in the cluster.

[0123] Each cluster can determine one or more semantic topics based on the semantic information of all candidate words within it. For example, cluster slot_345, mentioned above, can be identified as having the semantic topic "autumn," while cluster slot_297 can be identified as having the semantic topic "ocean." By analyzing the consistency of topic distribution among clusters, the ranking score of each cluster can be determined. The higher the consistency of topic distribution within a cluster, the higher the ranking score; conversely, the lower the consistency of topic distribution within a cluster, the lower the ranking score.

[0124] Figure 8 A flowchart illustrating the steps for determining cluster ranking scores in one embodiment of this application is shown. Figure 8 As shown, based on the above embodiments, the method for determining the ranking score of a cluster based on the consistency of the topic distribution of candidate words in the cluster in step S230 may include the following steps S810 to S830.

[0125] Step S810: Perform semantic topic analysis on the clusters to obtain the topic distribution consistency parameters of candidate words in the clusters.

[0126] The topic distribution consistency parameter is used to represent the degree of consistency in the topic distribution of candidate words in a cluster.

[0127] In one embodiment of this application, the method for performing semantic topic analysis on clusters may include the following steps S811 to S813.

[0128] In step S811, the semantic consistency parameter of the cluster is determined based on the context information of the candidate words in the cluster in the corpus. The semantic consistency parameter is used to represent the degree of semantic aggregation of the candidate words in the cluster.

[0129] Figure 9 A flowchart illustrating the steps for determining semantic consistency parameters in one embodiment of this application is shown. Figure 9 As shown, based on the above embodiments, the step S811 of determining the semantic consistency parameter of the cluster based on the context information of the candidate words in the cluster in the corpus may include the following steps S910 and S930.

[0130] Step S910: Obtain the distribution matrix of the cluster in the corpus. The distribution matrix is ​​used to represent the frequency of occurrence of candidate words in the cluster in each text corpus of the corpus.

[0131] For example, in one embodiment of this application, there are four clusters S1-S4, wherein cluster S1 includes candidate words: affordable, cheap, cost-effective; cluster S2 includes candidate words: pretty, beautiful, gorgeous; cluster S3 includes candidate words: mink, cowhide, artificial leather; and cluster S4 includes candidate words: plush, white duck down, grey duck down.

[0132] The corpus includes five text sets, namely:

[0133] 1. This mink coat is very beautiful, I love it, and it's also affordable. It looks so beautiful on me!

[0134] 2. Everyone says my white duck down wool coat, which I wear every day, is beautiful!

[0135] 3. There's a piece of clothing that's really outstanding. You'll definitely feel something special when you wear it. It's made of high-quality cowhide. I'll take you to see it next time!

[0136] 4. Oh dear weather, it's getting cold, I wonder when I'll be able to wear my pretty fluffy dress again.

[0137] 5. This faux leather coat is a best-selling item and has received high praise. Come and take a look!

[0138] By statistically analyzing the distribution frequency of each cluster in each text corpus, the following distribution matrix V(S) can be determined.

[0139]

[0140] Step S920: Extract multiple semantic consistency vectors of clusters from the distribution matrix. The semantic consistency vectors are used to represent the distribution frequency of candidate words of all clusters in the text corpus that contain the same cluster.

[0141] Taking cluster S2 as an example, multiple semantic consistency vectors extracted from the distribution matrix can form the following semantic consistency vector set C(S2).

[0142]

[0143] Since no candidate words of cluster S2 appear in the third and fifth corpus texts, their corresponding semantic consistency vectors C3 and C5 are represented as 0 vectors.

[0144] Step S930: Perform similarity calculation on multiple semantic consistency vectors to obtain the semantic consistency parameters of the cluster.

[0145] In this application embodiment, the semantic consistency parameter h(S) of the cluster can be calculated according to the following formula. i ):

[0146]

[0147] Among them, C a and C b CosSim() represents the similarity between any two semantically consistent vectors, expressed as cosine distance.

[0148] In step S812, the topic distribution parameters of the clusters are determined based on the topic distribution information of the candidate words in the clusters in the corpus. The topic distribution parameters are used to represent the degree of topic distribution of the candidate words in the clusters.

[0149] Figure 10 A flowchart illustrating the steps for determining topic distribution parameters in one embodiment of this application is shown. Figure 10 As shown, based on the above embodiments, step S812, which determines the topic distribution parameters of the cluster based on the topic distribution information of candidate words in the cluster in the corpus, may include the following steps S1010 to S1030.

[0150] Step S1010: Calculate the topic distribution probability of each semantic topic in the topic set in the candidate word set. The topic set includes multiple semantic topics extracted from the candidate word set.

[0151] All candidate words can be collectively formed into a candidate word set. From this set, multiple semantic topics with the highest distribution probabilities can be extracted, and these semantic topics collectively form a topic set. In this embodiment, each cluster obtained by clustering candidate words can correspond to one or more semantic topics; for example, a cluster can contain multiple semantic topics, and multiple clusters can also correspond to the same semantic topic.

[0152] Figure 11 A statistical diagram illustrating the probability distribution of topics in one embodiment of this application is shown. For example... Figure 11 As shown, K semantic topics were extracted from the corpus, namely topic-1, topic-2, ..., topic-K. The probability distribution of semantic topic-1 in the candidate word set is 0.4543, that of semantic topic-2 is 0.3765, that of semantic topic-3 is 0.3276, ..., that of semantic topic-K is 0.06786.

[0153] Step S1020: Calculate the probability distribution of candidate words in the topic set.

[0154] like Figure 11As shown, for each semantic topic in the topic set, the distribution probability of candidate words that have semantic association with it can be determined. For example, for semantic topic-1, the distribution probability of candidate word "coat" can be determined to be 0.0532, the distribution probability of candidate word "shirt" can be determined to be 0.0352, the distribution probability of candidate word "baby" can be determined to be 0.0003, and so on.

[0155] Based on the distribution probability of candidate words associated with topic sets, we can statistically analyze the word distribution probability of each candidate word in a cluster under each semantic topic in the topic set.

[0156] like Figure 11 As shown, cluster S i The candidate words include: baby, mothers, baby boy. For cluster S i The candidate word "little baby" can be used to calculate its probability distribution under various semantic topics. For example, the probability distribution of this candidate word under semantic topic-1 is 0.0003, under semantic topic-2 it is 0.0005, under semantic topic-3 it is 0, and so on, and under semantic topic-K it is 0.0012.

[0157] Step S1030: Determine the topic distribution parameters of the cluster based on the word distribution probability and topic distribution probability.

[0158] In one embodiment of this application, the topic distribution parameter logp(Si|d) of the cluster can be determined according to the following formula:

[0159]

[0160] Among them, S i S represents a cluster, d represents a candidate word, and L represents the cluster S. i The number of candidate words in w l Let t be the l-th candidate word in the cluster; K is the number of semantic topics in the topic set, and t k p(w) is the k-th semantic topic in the topic set; l |t k p(t) represents the word distribution probability of the candidate words determined in step S1020 within the topic set. k |d) represents the topic distribution probability of the semantic topic determined in step S1010 in the candidate word set.

[0161] In step S813, the semantic consistency parameter and the topic distribution parameter are fused to obtain the topic distribution consistency parameter of the candidate words in the cluster.

[0162] In one embodiment of this application, the product of the semantic consistency parameter and the topic distribution parameter can be used as the topic distribution consistency parameter of candidate words in the cluster.

[0163] Step S820: Count the frequency of candidate words of the cluster in the corpus to obtain the word frequency parameters of the cluster.

[0164] By statistically analyzing the frequency of candidate words in a cluster within a corpus, the word frequency of each candidate word in the cluster can be determined. The word frequency parameters of the cluster are then obtained by summing the word frequencies of each candidate word.

[0165] Step S830: Weight and fuse the topic distribution consistency parameter and the word frequency parameter to obtain the ranking score of the cluster.

[0166] In this application embodiment, the ranking score W(S) of the cluster can be determined according to the following formula. i ):

[0167] W(S i ) = log f(S i )+α·log h(S i )·logp(S i |d)

[0168] Here, α is an adjustment parameter used to adjust the importance of the topic distribution consistency parameter in cluster ranking.

[0169] In step S240, the clusters are sorted according to their ranking scores, and the attribute words of the mining objects are filtered according to the ranking results.

[0170] Figure 12 This illustration shows a sequence of clusters obtained by sorting the clusters in one embodiment of this application. For example... Figure 12 As shown, in the sorted cluster sequence, the top-ranked clusters have a high degree of consistency in topic distribution, and the candidate words in each cluster are words that have a certain semantic association with clothing and underwear.

[0171] In this embodiment, the top N clusters can be selected, and candidate words from these N clusters can be used as attribute words for the mining object "clothing and underwear". Alternatively, this embodiment can further filter the candidate words in the N clusters to obtain more accurate attribute words. For example, attribute words can be filtered from the N clusters using a pre-trained machine learning model, or the N clusters can be manually filtered and labeled.

[0172] Figure 13 This illustration shows a module block diagram of attribute word mining and further applications in an application scenario according to an embodiment of this application.

[0173] like Figure 13 As shown, in one application scenario, this embodiment of the application, based on product title text from e-commerce websites or user review text from social networks, utilizes a combination of methods such as contextual information, new word mining tools, and LDA topic models to effectively mine attribute words, significantly reducing manual costs and thus better serving related projects. For the attribute word mining needs of a specific category, the main mining process is as follows:

[0174] 1) For this category, use the attribute word list of the e-commerce website to directly crawl a large number of "head" attribute words.

[0175] 2) For the text corpus 1301 of this category, we performed word mining based on the AutoPhrase open-source tool to obtain new words / compound words.

[0176] 3) Add the new words / compound words obtained in step 2) to the custom word list, and use the Hanlp word segmentation tool to segment the text corpus of this category to obtain the corresponding candidate words 1302.

[0177] 4) Filter the candidate words 1302 in step 3) by stopping words, product words (collected online), and brand words (collected online).

[0178] 5) Cluster the filtered candidate words 1302 using the clustering model 1303 based on the k-means clustering method to obtain K clusters 1304.

[0179] 6) The candidate words in each cluster 1304 are combined with the contextual information of the text corpus in the category and the topic information of the cluster based on the LDA topic model to form the ranking model 1305 of the cluster.

[0180] 7) The top N clusters after sorting are manually labeled to obtain the attribute word set (i.e. cluster) of different categories as the new attribute word 1306 mined.

[0181] The attribute words obtained through mining can be applied to various specific application areas such as AI copywriting, AI product understanding, and advertising attribute feature collection.

[0182] For example, embodiments of this application can be applied to AI-powered intelligent writing by mining attribute words. By utilizing different product attribute words, different descriptive texts can be retrieved, thereby generating suitable advertising copy or product copy.

[0183] This application embodiment can be applied to AI product understanding by mining attribute words. That is, for product copywriting on the Internet, the mined attribute words can be used to better understand the different attributes of the product such as "price", "color", "condition" and so on, thereby better understanding customers and improving the ability of attribute words to represent product characteristics.

[0184] This application embodiment can apply attribute word mining to the collection of advertising attribute features. By providing attribute words of different products or texts as feature dimensions for collection, the click-through rate of advertisements can be predicted. This can accurately mine the mapping relationship between attribute features and advertising click-through rate, improve the predictive ability of attribute words, and thus improve the prediction effect of click-through rate.

[0185] AI writing, product understanding, and advertising understanding are current hot topics in artificial intelligence. Among these, attribute words are a crucial element, making attribute word mining essential for AI writing and understanding. Furthermore, for many e-commerce products, attribute words influence a product's search ranking and click-through rate, and can even largely determine their performance.

[0186] This application provides a function to clean and mine the purest attribute words from a corpus, with particularly useful results for mining mid- to long-tail attribute words, which is also a challenge in attribute word mining. Furthermore, this application greatly reduces manual labor and lowers costs. More importantly, the attribute words mined in this application are all clustered, enabling the rapid construction of attribute word systems for specific categories, bringing significant practical value to various fields such as AI writing and understanding.

[0187] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0188] The following describes an embodiment of the apparatus of this application, which can be used to execute the attribute word mining method in the above embodiments of this application.

[0189] Figure 14 A schematic diagram illustrating the structure of the attribute word mining apparatus provided in an embodiment of this application is shown. Figure 14 As shown, the attribute word mining device 1400 mainly includes:

[0190] The acquisition module 1410 is configured to acquire candidate words related to the attribute parameters of the mining object;

[0191] Clustering module 1420 is configured to perform clustering processing on the candidate words to obtain clusters composed of the candidate words;

[0192] The scoring module 1430 is configured to determine the ranking score of the cluster based on the consistency of the topic distribution of candidate words in the cluster, wherein the ranking score is positively correlated with the consistency of the topic distribution of candidate words in the cluster.

[0193] The filtering module 1440 is configured to sort the clusters according to their ranking scores and filter the attribute words of the mining objects according to the ranking results.

[0194] In one embodiment of this application, the scoring module 1430 may further include:

[0195] The topic analysis module 1431 is configured to perform semantic topic analysis on the cluster to obtain topic distribution consistency parameters of candidate words in the cluster;

[0196] The word frequency statistics module 1432 is configured to count the frequency of occurrence of candidate words of the cluster in the corpus, and obtain the word frequency parameters of the cluster.

[0197] The parameter fusion module 1433 is configured to perform weighted fusion of the topic distribution consistency parameter and the word frequency parameter to obtain the ranking score of the cluster.

[0198] In one embodiment of this application, based on the above embodiments, the topic analysis module 1431 may further include:

[0199] The semantic consistency determination module is configured to determine the semantic consistency parameter of the cluster based on the context information of the candidate words in the cluster in the corpus. The semantic consistency parameter is used to represent the degree of semantic aggregation of the candidate words in the cluster.

[0200] The topic distribution determination module is configured to determine the topic distribution parameters of the cluster based on the topic distribution information of candidate words in the candidate word set. The topic distribution parameters are used to represent the degree of topic distribution of candidate words in the cluster.

[0201] The distribution consistency determination module is configured to perform parameter fusion on the semantic consistency parameter and the topic distribution parameter to obtain the topic distribution consistency parameter of the candidate words in the cluster.

[0202] In one embodiment of this application, based on the above embodiments, the semantic consistency determination module can be further configured to: obtain the distribution matrix of the cluster in the corpus, the distribution matrix being used to represent the frequency of occurrence of candidate words in the cluster in each text corpus of the corpus; extract multiple semantic consistency vectors of the cluster from the distribution matrix, the semantic consistency vectors being used to represent the distribution frequency of candidate words of all clusters in the text corpus where the same cluster appears; and perform similarity calculation on the multiple semantic consistency vectors to obtain the semantic consistency parameters of the cluster.

[0203] In one embodiment of this application, based on the above embodiments, the topic distribution determination module can be further configured to: count the topic distribution probability of each semantic topic in the topic set in the corpus, wherein the topic set includes multiple semantic topics extracted from the candidate word set; count the word distribution probability of the candidate words of the cluster in the topic set; and determine the topic distribution parameters of the cluster based on the word distribution probability and the topic distribution probability.

[0204] In one embodiment of this application, based on the above embodiments, the acquisition module 1410 may further include:

[0205] The text acquisition module 1411 is configured to acquire text data of the mining object to obtain a corpus composed of text data.

[0206] The word extraction module 1412 is configured to extract candidate words related to the attribute parameters of the mining object from the text corpus.

[0207] In one embodiment of this application, based on the above embodiments, the text acquisition module 1411 may be further configured to: acquire online trading platforms related to the mining object; crawl online data from the online trading platforms to obtain text corpus related to the mining object.

[0208] In one embodiment of this application, based on the above embodiments, the text corpus includes at least one of the product title text, product description text, or product review text of the online trading platform.

[0209] In one embodiment of this application, based on the above embodiments, the word extraction module 1412 may be further configured to: perform word segmentation processing on the text corpus to obtain text segments that make up the text corpus; filter out irrelevant words in the text segments to obtain candidate words related to the attribute parameters of the mining object.

[0210] In one embodiment of this application, based on the above embodiments, the word extraction module 1412 may be further configured to: perform phrase mining on the text corpus to obtain high-quality phrases in the text corpus; and perform word segmentation on the text corpus according to a dictionary containing the high-quality phrases to obtain text segments that make up the text corpus.

[0211] In one embodiment of this application, based on the above embodiments, the irrelevant words include at least one of stop words, product words, or brand words.

[0212] In one embodiment of this application, based on the above embodiments, the acquisition module 1410 may further include:

[0213] The word acquisition module 1413 is configured to acquire word corpus of the mining object and obtain candidate words related to the attribute parameters of the mining object.

[0214] In one embodiment of this application, based on the above embodiments, the word acquisition module 1413 may be further configured to: acquire online trading platforms related to the mining object; crawl online data from the online trading platforms to obtain word corpus related to the mining object.

[0215] In one embodiment of this application, based on the above embodiments, the word corpus includes attribute words of product categories in the online trading platform.

[0216] The specific details of the attribute word mining device provided in the various embodiments of this application have been described in detail in the corresponding method embodiments, and will not be repeated here.

[0217] Figure 15 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0218] It should be noted that, Figure 15 The computer system 1500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0219] like Figure 15As shown, the computer system 1500 includes a central processing unit (CPU) 1501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1502 or programs loaded from storage section 1508 into random access memory (RAM). The RAM 1503 also stores various programs and data required for system operation. The CPU 1501, ROM 1502, and RAM 1503 are interconnected via a bus 1504. An input / output interface 1505 (I / O interface) is also connected to the bus 1504.

[0220] The following components are connected to the input / output interface 1505: an input section 1506 including a keyboard, mouse, etc.; an output section 1507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1508 including a hard disk, etc.; and a communication section 1509 including a network interface card such as a local area network card, modem, etc. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the input / output interface 1505 as needed. A removable medium 1511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1510 as needed so that computer programs read from it can be installed into the storage section 1508 as needed.

[0221] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1509, and / or installed from removable medium 1511. When the computer program is executed by central processing unit 1501, it performs various functions defined in the system of this application.

[0222] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0223] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0224] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0225] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0226] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0227] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for mining attribute words, characterized in that, include: Retrieve candidate words related to the attribute parameters of the mining object; The candidate words are clustered to obtain clusters composed of the candidate words; Obtain the distribution matrix of the cluster in the corpus, the distribution matrix being used to represent the frequency of occurrence of candidate words in the cluster in each text corpus of the corpus; Multiple semantic consistency vectors of the clusters are extracted from the distribution matrix. The semantic consistency vectors are used to represent the distribution frequency of candidate words of all clusters in the text corpus where the same cluster appears. A similarity operation is performed on the multiple semantic consistency vectors to obtain the semantic consistency parameter of the cluster. The semantic consistency parameter is used to represent the degree of semantic aggregation of candidate words in the cluster. Based on the topic distribution information of candidate words in the cluster within the candidate word set, the topic distribution parameters of the cluster are determined, and the topic distribution parameters are used to represent the degree of topic distribution of candidate words in the cluster. The semantic consistency parameter and the topic distribution parameter are fused to obtain the topic distribution consistency parameter of candidate words in the cluster. The frequency of candidate words in the cluster is counted in the corpus to obtain the word frequency parameter of the cluster. The topic distribution consistency parameter and the word frequency parameter are weighted and fused to obtain the ranking score of the cluster. The ranking score is positively correlated with the topic distribution consistency of candidate words in the cluster. The clusters are sorted according to their ranking scores, and the attribute words of the mining objects are filtered according to the ranking results.

2. The attribute word mining method according to claim 1, characterized in that, Based on the topic distribution information of candidate words in the cluster in the corpus, the topic distribution parameters of the cluster are determined, including: The topic distribution probability of each semantic topic in the topic set is calculated in the corpus, and the topic set includes multiple semantic topics extracted from the candidate word set. Statistically calculate the probability distribution of candidate words in the topic set for the clusters. The topic distribution parameters of the cluster are determined based on the word distribution probability and the topic distribution probability.

3. The attribute word mining method according to claim 1 or 2, characterized in that, Retrieve candidate terms related to the attribute parameters of the object being mined, including: Textual data is collected from the aforementioned objects to obtain a corpus composed of textual data; Extract candidate words related to the attribute parameters of the mining object from the text corpus.

4. The attribute word mining method according to claim 3, characterized in that, Text corpus collection of the target data includes: Obtain the online trading platforms related to the object being mined; Web data is crawled from the online trading platform to obtain text corpus related to the target of the mining.

5. The attribute word mining method according to claim 4, characterized in that, The text corpus includes at least one of the following: product title text, product description text, or product review text from the online trading platform.

6. The attribute word mining method according to claim 3, characterized in that, Extracting candidate words related to the attribute parameters of the mining object from the text corpus, including: The text corpus is segmented to obtain the text segments that make up the text corpus; Irrelevant words are filtered out from the text segmentation to obtain candidate words that are related to the attribute parameters of the mining object.

7. The attribute word mining method according to claim 6, characterized in that, The text corpus is segmented into words, including: Phrase mining is performed on the text corpus to obtain high-quality phrases from the text corpus; The text corpus is segmented according to a dictionary containing the high-quality phrases to obtain the text segments that make up the text corpus.

8. The attribute word mining method according to claim 1 or 2, characterized in that, Retrieve candidate terms related to the attribute parameters of the object being mined, including: A corpus of words is collected from the target object to obtain candidate words related to the attribute parameters of the target object.

9. The attribute word mining method according to claim 8, characterized in that, The textual data collection for the target audience includes: Obtain the online trading platforms related to the object being mined; Web data is crawled from the online trading platform to obtain a corpus of words related to the target of the mining.

10. An attribute word mining device, characterized in that, include: The acquisition module is configured to acquire candidate words related to the attribute parameters of the mining object; The clustering module is configured to perform clustering processing on the candidate words to obtain clusters composed of the candidate words; The scoring module is configured to obtain the distribution matrix of the cluster in the corpus, the distribution matrix being used to represent the frequency of occurrence of candidate words in the cluster in each text corpus of the corpus; Multiple semantic consistency vectors of the clusters are extracted from the distribution matrix. These semantic consistency vectors represent the frequency distribution of candidate words in all clusters within the text corpus containing the same cluster. Similarity calculations are performed on these multiple semantic consistency vectors to obtain semantic consistency parameters for the clusters. These semantic consistency parameters represent the degree of semantic aggregation of candidate words within the clusters. Based on the topic distribution information of candidate words in the clusters within the candidate word set, topic distribution parameters of the clusters are determined. These topic distribution parameters represent the degree of topic distribution of candidate words within the clusters. The semantic consistency parameters and topic distribution parameters are fused to obtain topic distribution consistency parameters for the candidate words within the clusters. The frequency of occurrence of candidate words in the clusters within the corpus is statistically analyzed to obtain the word frequency parameters of the clusters. The topic distribution consistency parameter and the word frequency parameter are weighted and fused to obtain the ranking score of the cluster. The ranking score is positively correlated with the topic distribution consistency of candidate words in the cluster. The filtering module is configured to sort the clusters according to their ranking scores and filter the attribute words of the mining objects based on the ranking results.

11. A computer-readable medium, characterized in that, The computer-readable medium stores a computer program that, when executed by a processor, implements the attribute word mining method according to any one of claims 1 to 9.

12. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to cause the electronic device to perform the attribute word mining method according to any one of claims 1 to 9 by executing the executable instructions.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the attribute word mining method according to any one of claims 1 to 9.

Images