Method, device and computing equipment for determining commodity attribute words
Through automated methods, product attribute words are determined based on comment text segmentation and classification, which solves the problems of resource waste and limited vocabulary caused by manual determination in the existing technology, and realizes efficient and personalized attribute words acquisition, which is suitable for Large-scale data processing.
Patent Information
- Application Number
- CN201910100736.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-01-31
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2039-01-31
AI Technical Summary
The existing product attribute words are mostly determined manually, resulting in limited magnitude of the vocabulary and consume a lot of human resources, and lack of efficient automated methods to obtain and process emotional tendencies in product reviews.
By obtaining product review text, segmenting and classification, first- and second-level attribute words are determined based on the frequency and similarity of word appearance, and relevant second-level attribute words are obtained through importance sorting to form a richer and more personalized set of attribute words. .
It improves the degree of personalization and accuracy of product attribute words, reduces human resources investment, is suitable for large-scale data processing, and significantly improves the internal correlation and difference of attribute words.
Smart Images

Figure CN111507789B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of network information processing, and in particular to a method, device and computing equipment for determining commodity attribute words. Background Art
[0002] With the rapid development of Internet technology and e-commerce, users are increasingly inclined to online shopping. Various e-commerce platforms generate a large number of user comments on products every day. Sentiment analysis of these comment texts can not only assist merchants in decision-making and online marketing, assist platforms in product recommendations, but also help public opinion analysis and help users make purchase decisions. Since comment texts can usually reflect product attribute characteristics, comment texts can be studied from the perspective of product attribute characteristics to determine emotional tendencies. Obviously, the acquisition and processing of product attribute words is an important technical basis for mining and utilizing product review data. The accuracy and personalization of product attribute words will strictly affect the sentiment analysis results of comment texts based on product attribute characteristics.
[0003] In existing methods, commodity attribute words are mostly determined manually, which results in a limited attribute word library and consumes a lot of human resources.
[0004] Therefore, a more advanced product attribute word solution needs to be provided. Summary of the invention
[0005] To this end, embodiments of the present invention provide a method, apparatus, and computing device for determining commodity attribute words, in an effort to solve or at least alleviate at least one of the above problems.
[0006] According to one aspect of an embodiment of the present invention, a method for determining attribute words of a product is provided, comprising: obtaining at least one review text of the product; segmenting the at least one review text to obtain multiple clauses and multiple words included in the multiple clauses; dividing at least part of the multiple clauses into multiple categories; determining the first-level attribute words of the product from the words included in the clauses under each category based on the frequency of occurrence of the words; determining candidate second-level attribute words related to the first-level attribute words from the multiple words included in the multiple clauses based at least on the similarity between the words and the first-level attribute words; and calculating the relevance importance of the candidate second-level attribute words, and determining the second-level attribute words related to the first-level attribute words from the candidate second-level attribute words based on the relevance importance.
[0007] According to another aspect of an embodiment of the present invention, a method for determining attribute words of products under a predetermined category is provided, comprising: obtaining at least one review text of all products under the predetermined category; segmenting at least one review text to obtain multiple clauses and multiple words included in the multiple clauses; dividing at least one clause with the highest occurrence frequency among the multiple clauses into multiple categories; determining the first-level attribute words of the products under the predetermined category from the words included in the clauses under each category based on the occurrence frequency of the words; determining candidate second-level attribute words related to the first-level attribute words from the multiple words included in the multiple clauses, at least based on the similarity between the words and the first-level attribute words; calculating the relevance importance of the candidate second-level attribute words, and determining the second-level attribute words related to the first-level attribute words from the candidate second-level attribute words based on the relevance importance; judging whether there are other first-level attribute words among the second-level attribute words related to the first-level attribute words, and the relative importance of the nodes corresponding to the other first-level attribute words is less than that of the nodes corresponding to the first-level attribute words; if so, taking the other first-level attribute words and the second-level attribute words related to the other first-level attribute words as the second-level attribute words related to the first-level attribute words.
[0008] According to another aspect of an embodiment of the present invention, a device for determining commodity attribute words is provided, comprising: a comment acquisition module, adapted to acquire at least one comment text of a commodity; a text segmentation module, adapted to segment at least one comment text to obtain multiple clauses and multiple words included in the multiple clauses; a clause classification module, adapted to classify at least part of the multiple clauses into multiple categories; a first-level attribute word determination module, adapted to determine the first-level attribute words of the commodity from the words included in the clauses under each category based on the frequency of occurrence of the words; and a second-level attribute word determination module, adapted to determine candidate second-level attribute words related to the first-level attribute words from the multiple words included in the multiple clauses based at least on the similarity between the words and the first-level attribute words; and further adapted to calculate the relevance importance of the candidate second-level attribute words, and determine the second-level attribute words related to the first-level attribute words from the candidate second-level attribute words based on the relevance importance.
[0009] According to another aspect of an embodiment of the present invention, a device for determining attribute words of products under a predetermined category is provided, comprising: a comment acquisition module, adapted to acquire at least one comment text of the products under the predetermined category; a text segmentation module, adapted to segment at least one comment text to obtain multiple clauses and multiple words included in the multiple clauses; a clause classification module, adapted to classify at least one clause with the highest occurrence frequency among the multiple clauses into multiple categories; a primary attribute word determination module, adapted to determine the primary attribute words of the products under the predetermined category based on the occurrence frequency of the words included in the clauses under each category; and a secondary attribute word determination module , suitable for determining candidate second-level attribute words related to the first-level attribute words from multiple words included in multiple clauses, at least based on the similarity between the words and the first-level attribute words; calculating the relevant importance of the candidate second-level attribute words, and determining the second-level attribute words related to the first-level attribute words from the candidate second-level attribute words based on the relevant importance; judging whether there are other first-level attribute words in the second-level attribute words related to the first-level attribute words, and the relative importance of the corresponding nodes of the other first-level attribute words is less than that of the corresponding nodes of the first-level attribute words; if so, taking the other first-level attribute words and the second-level attribute words related to the other first-level attribute words as the second-level attribute words related to the first-level attribute words.
[0010] According to another aspect of an embodiment of the present invention, a comment sentiment analysis system is provided, comprising: a device for determining product attribute words according to an embodiment of the present invention, suitable for determining the attribute words of a product based on at least one comment text of the product; and a processing front end, suitable for obtaining, for each attribute word of the product, the sentiment tendency of each comment text for the attribute.
[0011] According to another aspect of an embodiment of the present invention, a computing device is provided, comprising: one or more processors; and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods according to the embodiments of the present invention.
[0012] According to another aspect of an embodiment of the present invention, a computer-readable storage medium storing one or more programs is provided, wherein the one or more programs include instructions, and when the instructions are executed by a computing device, the computing device performs any of the methods according to the embodiments of the present invention.
[0013] The commodity attribute word set obtained by the commodity attribute word determination scheme according to the embodiment of the present invention has a higher internal correlation and is more different from other (classes) of commodities. At the same time, it has good practicality for large-scale data and can greatly save manpower. First, by clustering the words, the proportion of common words in the first-level attribute words is reduced, and the personalization degree of the obtained first-level attribute words is greatly improved (the attribute words of a certain (class) of commodities have fewer repetitions with the attribute words of other (classes) of commodities, that is, the attribute words of the (class) of commodities have a high degree of personalization). Secondly, by determining the first candidate second-level attribute word and the second candidate second-level attribute word, and converting the similarity matching problem of the words into the problem of sorting important nodes in the network, more, richer and more reasonable second-level attribute words can be obtained, and it also has better practicality for large-scale data. Furthermore, by merging the same or similar first-level attribute words and second-level attribute words, the accuracy of the obtained attribute words is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] To achieve the above and related purposes, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings, which indicate various ways in which the principles disclosed herein can be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The above and other purposes, features and advantages of the present disclosure will become more apparent by reading the following detailed description in conjunction with the accompanying drawings. Throughout the present disclosure, the same reference numerals generally refer to the same parts or elements.
[0015] Figure 1 shows an architecture diagram of a commodity attribute word determination system 100 according to an embodiment of the present invention;
[0016] Figure 2 A hierarchical structure diagram of commodity attribute words according to an embodiment of the present invention is shown;
[0017] Figure 3 A schematic diagram of a computing device 300 according to one embodiment of the present invention is shown;
[0018] Figure 4 A flowchart of a method 400 for determining a commodity attribute word according to an embodiment of the present invention is shown;
[0019] Figure 5 A schematic diagram showing a method of determining candidate secondary attribute words according to an embodiment of the present invention is shown;
[0020] Figure 6 A schematic diagram of a word network according to an embodiment of the present invention is shown;
[0021] Figure 7 A structural block diagram of a device 700 for determining a commodity attribute word according to an embodiment of the present invention is shown;
[0022] Figure 8 A flowchart showing a method 800 for determining a commodity attribute word under a predetermined category according to an embodiment of the present invention; and
[0023] Fig. 9 A structural block diagram of a device 900 for determining commodity attribute words under a predetermined category according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0024] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0025] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0026] Figure 1 FIG. 1 is a schematic diagram of a commodity attribute word determination system 100 according to an embodiment of the present invention. Figure 1 As shown, the commodity attribute word determination system 100 includes a processing front end 110 and a commodity attribute word determination device 700. The processing front end 110 and the commodity attribute word determination device 700 can communicate via one or more networks such as a local area network (LAN) or a wide area network (WAN) such as the Internet.
[0027] It should be noted that the present invention does not limit the specific form of the commodity, and the commodity can be a product, a service, or a combination of products and services, etc. Preferably, the commodity can be a certain category of commodities, that is, all commodities under a predetermined category, such as all commodities under the category of facial masks. The commodity attribute word determination system 100 determines the attribute words of the category of commodities. Of course, the commodity can also be a certain commodity, and the commodity attribute word determination system 100 determines the attribute words of the commodity.
[0028] The attribute words of a product are used to describe the properties and characteristics of the product. In the embodiment of the present invention, the attribute words of a product may include a primary attribute word and secondary attribute words related to the primary attribute word. For example, a primary attribute word of a facial mask product is efficacy, and the secondary attribute words related to the primary attribute word efficacy are whitening, moisturizing, anti-acne, anti-wrinkle, etc.
[0029] Figure 2 FIG. 1 shows a hierarchical structure diagram of commodity attribute words according to an embodiment of the present invention. Figure 2 As shown in the figure, the attribute word set of a product can be expressed as: Among them, F j represents the jth first-level attribute word, and the value range of j is [1, K]. Each first-level attribute word can include several related second-level attribute words, so the first-level attribute word can be represented as F j ={F j1 ,F j2 ,...,F jm}.
[0030] The processing front end 110 is any requesting party that needs to obtain the attribute words of the product. For example, in one embodiment, the processing front end 110 can be part of a review sentiment analysis system. The review sentiment analysis system needs to analyze the review text of the product to determine its sentiment tendency. Therefore, the processing front end 110 can send the product information for which the attribute words need to be obtained to the product attribute word determination device 700, so that the product attribute word determination device 700 obtains the review text of the product and determines the attribute words of the product therefrom. Optionally, the processing front end 110 can also send the review text of the product to the product attribute word determination device 700 for processing to obtain the attribute words of the product. In other words, the comment sentiment analysis system may include a commodity attribute word determination device 700 and a processing front end 110. After the commodity attribute word determination device 700 determines the commodity attribute words from the commodity comment text, the processing front end 110 may obtain the sentiment tendency (positive, negative or neutral) of the commodity attribute corresponding to the attribute word reflected in each comment text by at least using a pre-built sentiment word library (possibly also a degree adverb word library, a negative word word library, a conjunction word library, etc.) for each attribute word of the commodity. Any sentiment tendency analysis method in the art may be used here to obtain the sentiment tendency of the comment text for the commodity attribute, and the present invention does not limit this.
[0031] The processing front end 110 may also be part of a user public opinion analysis system. The user public opinion analysis system needs to analyze the comment text of the product to determine the market feedback of the product, the user's shopping intention and demand. Therefore, the processing front end 110 can send the product information or the comment text of the product for which the attribute words need to be obtained to the product attribute word determination device 700 for processing to obtain the attribute words of the product.
[0032] The processing front end 110 can also be part of a commodity trend prediction system. The commodity trend prediction system needs to analyze the comment text of the commodity to determine the popular trend of the commodity attributes. Therefore, the processing front end 110 can send the commodity information or the comment text of the commodity that needs to obtain the attribute words to the commodity attribute word determination device 700 for processing to obtain the attribute words of the commodity. Then, the processing front end 110 can obtain the popular trend of the commodity attributes by comparing different versions of the commodity attribute words obtained at different times, so as to provide merchants with effective suggestions for new product development for the commodity. The present invention is not limited to the specific form of the processing front end 110. The commodity attribute word determination device 700 can receive the request of the processing front end 110 in various ways. For example, the commodity attribute word determination device 700 can provide an application program interface (API) with a predetermined format definition to facilitate the processing front end 110 to organize the request to obtain the commodity attribute words according to the definition and send it to the commodity attribute word determination device 700.
[0033] The commodity attribute word determination device 700 may receive the request, analyze the commodity review text, and determine the primary attribute word of the commodity and the secondary attribute words related to the primary attribute word from the commodity review text.
[0034] The specific structures of the various devices and units mentioned above and the corresponding processing methods will be described below in conjunction with the accompanying drawings.
[0035] According to an embodiment of the present invention, each device and unit in the above-mentioned commodity attribute word determination system 100 can be implemented by a computing device 300 as described below.
[0036] Figure 3 FIG. 3 is a schematic diagram of a computing device 300 according to an embodiment of the present invention. Figure 3 As shown, in a basic configuration 302, computing device 300 typically includes a system memory 306 and one or more processors 304. A memory bus 308 may be used for communication between processor 304 and system memory 306.
[0037] Depending on the desired configuration, the processor 304 can be any type of processor, including but not limited to: a microprocessor (μP), a microcontroller (μC), a digital information processor (DSP), or any combination thereof. The processor 304 can include one or more levels of cache such as a first-level cache 310 and a second-level cache 312, a processor core 314, and registers 316. An example processor core 314 can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 318 can be used with the processor 304, or in some implementations, the memory controller 318 can be an internal part of the processor 304.
[0038] Depending on the desired configuration, system memory 306 may be any type of memory, including but not limited to: volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.), or any combination thereof. System memory 306 may include an operating system 320, one or more applications 322, and program data 324. In some implementations, application 322 may be arranged to execute instructions on the operating system by one or more processors 304 using program data 324.
[0039] The computing device 300 may also include an interface bus 340 that facilitates communication from various interface devices (e.g., output devices 342, peripheral interfaces 344, and communication devices 346) to the basic configuration 302 via the bus / interface controller 330. Example output devices 342 include a graphics processing unit 348 and an audio processing unit 350. They can be configured to facilitate communication with various external devices such as a display or speakers via one or more A / V ports 352. Example peripheral interfaces 344 may include a serial interface controller 354 and a parallel interface controller 356, which may be configured to facilitate communication with external devices such as input devices (e.g., keyboards, mice, pens, voice input devices, touch input devices) or other peripherals (e.g., printers, scanners, etc.) via one or more I / O ports 358. Example communication devices 346 may include a network controller 360, which may be arranged to facilitate communication with one or more other computing devices 362 via a network communication link via one or more communication ports 364.
[0040] A network communication link can be an example of a communication medium. Communication media can generally be embodied as computer-readable instructions, data structures, program modules in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any information delivery medium. A "modulated data signal" can be a signal in which one or more of its data sets or its changes can be carried out in a manner that encodes information in the signal. As non-limiting examples, communication media can include wired media such as a wired network or a dedicated line network, and various wireless media such as sound, radio frequency (RF), microwave, infrared (IR) or other wireless media. The term computer-readable medium used herein can include both storage media and communication media.
[0041] The computing device 300 can be implemented as a server, such as a database server, an application server, a WEB server, etc., or as a personal computer including a desktop computer and a notebook computer. Of course, the computing device 300 can also be implemented as a part of a small-sized portable (or mobile) electronic device.
[0042] In an embodiment according to the present invention, the computing device 300 is implemented as a commodity attribute word determination device 700, and is configured to execute a commodity attribute word determination method according to an embodiment of the present invention. The application 322 of the computing device 300 includes a plurality of program instructions for executing a commodity attribute word determination method according to an embodiment of the present invention, and the program data 324 may also store configuration information of the commodity attribute word determination device 700, etc.
[0043] Figure 4 FIG. 4 is a flowchart of a method 400 for determining a product attribute word according to an embodiment of the present invention. The method 400 is executed in the above-mentioned device 700 for determining a product attribute word. Figure 4 As shown, the method 400 for determining a product attribute word begins with step S410 .
[0044] In step S410, at least one comment text of the product is obtained. As described above, if it is necessary to determine the attribute words of a predetermined category of products, the comment texts of all products under the predetermined category are obtained. If it is necessary to determine the attribute words of a specific product, only the comment text for the product is obtained. In various embodiments, various data acquisition methods, such as web crawlers, etc., can be used to obtain comment texts, and the present invention does not limit this.
[0045] Since users' expressions on the Internet are usually more casual and diverse, there may be situations such as numbers, letters and symbols mixed in Chinese characters, incomplete sentences and paragraphs, or a large number of repeated phrases and short sentences. Therefore, according to one embodiment of the present invention, it is necessary to perform data cleaning on at least one of the obtained comment texts. Any data cleaning method in the art can be used to clean the comment text, and the present invention does not limit the specific data cleaning method.
[0046] Then, in step S420, the acquired at least one comment text is segmented to obtain a plurality of clauses and a plurality of words included in the plurality of clauses.
[0047] First, each review text can be segmented into clauses, i.e., divided into clauses, to obtain multiple clauses. In one embodiment, the clauses can be segmented according to separators such as punctuation marks and blank characters. For example, the review text "goods are of good quality and cheap" can be segmented into two clauses, "goods are of good quality" and "cheap".
[0048] Then, each clause is segmented, i.e., word segmentation, to obtain a plurality of words. Any word segmentation method in the art can be used for word segmentation, for example, a word segmentation method based on a vocabulary, a word segmentation method based on a statistical model (N-gram language model), a word segmentation method based on sequence annotation, and the like. The present invention does not limit the specific word segmentation method. The following is an example: "Today's weather is very good" can be segmented to obtain the four words "today", "weather", "very" and "good".
[0049] According to an embodiment of the present invention, in order to facilitate the subsequent rapid acquisition of words obtained from clause segmentation, the correspondence between each clause and the words obtained from the clause segmentation may also be stored here.
[0050] After multiple clauses are obtained through segmentation, at least some of the multiple clauses may be divided into multiple categories in step S430.
[0051] In one embodiment, the multiple clauses can be directly divided into categories. Preferably, in another embodiment, some of the multiple clauses can be selected for category division. For example, the frequency of occurrence of each clause in the multiple clauses can be counted, and at least one clause with the highest frequency of occurrence can be selected for classification. In other words, the clauses are sorted from large to small according to the frequency of occurrence, and some of the clauses in the front order (usually the clauses in the top 50% of the order) can be selected, and the rest of the clauses can be discarded. Here, the frequency of occurrence usually refers to the number of occurrences.
[0052] According to an embodiment of the present invention, a feature vector of at least part of the clauses can be generated, so that at least part of the clauses can be divided into multiple categories based on the feature vectors of the clauses using a clustering algorithm. Any clustering method in the art can be used to cluster the clauses, such as a K-means clustering algorithm, a DBSCAN clustering algorithm (Density-Based Spatial Clustering of Applications with Noise), etc. The present invention does not limit the specific clustering algorithm.
[0053] Among them, word vectors can be generated for multiple words included in the multiple clauses obtained by segmentation, that is, word vectors are generated for all words obtained by segmentation from the comment text. In various embodiments, a word embedding model can be used to generate word vectors for words. The toolkit word2vec provided by Google and other companies provides a variety of word embedding models. The present invention is not limited to a specific word embedding model, and all word embedding methods that can generate word vectors for words are within the protection scope of the present invention.
[0054] After obtaining the word vectors of the words, feature vectors are generated for at least part of the above clauses based on the word vectors of these words. Specifically, for each selected clause, the word vectors of the words obtained from the clause segmentation can be found from the word vectors of all previously generated words. In one embodiment, all words can be hash-encoded so that they can be quickly searched. Then, the feature vector of the clause is generated based on the word vectors of the words obtained from the clause segmentation. In one embodiment, the feature vector of the clause is generated by taking the average of the word vectors of the words obtained from the clause segmentation, that is, the feature vector of the clause can be calculated by the following formula:
[0055]
[0056] Among them, v SR The feature vector representing the clause, w i represents the i-th word obtained from the clause segmentation, v wi represents the word vector of the i-th word obtained from the clause segmentation, SR represents the word vector set of all words obtained from the clause segmentation, and |SR| represents the cardinality of the word vector set SR. For example, if N words are obtained from a clause segmentation, the value range of i is [1, N], and |SR| = N.
[0057] After dividing at least part of the selected clauses into multiple categories, in step S440, the first-level attribute words of the product are determined from the words included in the clauses under each category based on the frequency of occurrence of the words. Specifically, for each category, all the words included in the clauses under the category can be determined, and then the frequency of occurrence of each word included in the clauses under the category is counted, and at least one word with the highest frequency of occurrence is selected as the first-level attribute word of the product. In other words, these words are sorted from large to small according to the frequency of occurrence, and the first predetermined number of words in the front order are selected, and the remaining clauses are discarded. The value range of the first predetermined number is usually [60,100]. In this way, these first-level attribute words determined from the clauses under each category together form the first-level attribute words of the product.
[0058] According to an embodiment of the present invention, it is also necessary to perform data cleaning on the determined primary attribute words to remove some erroneous words, stop words, repeated words, etc. Any data cleaning method in the art can be used to perform data cleaning, and the present invention does not limit the specific data cleaning method.
[0059] Subsequently, in step S450, candidate secondary attribute words related to the first-level attribute words are determined from the multiple words included in the aforementioned multiple clauses (that is, all the words obtained from the segmentation of the comment text in step S420), at least based on the similarity between the words and the first-level attribute words. In one embodiment, for each first-level attribute word, the similarity between the first-level attribute word and each word included in the aforementioned multiple clauses can be calculated, and at least one word with the highest similarity can be selected as a candidate secondary attribute word related to the first-level attribute word. In other words, these words are sorted from large to small according to the similarity, and the second predetermined number of words in the front order are selected, and the remaining words are discarded. The value of the second predetermined number is usually 30. The similarity can usually be calculated using any similarity algorithm in the field, such as Euclidean distance, cosine similarity, etc. The present invention does not limit the specific similarity algorithm.
[0060] Preferably, if Figure 5 As shown, in another embodiment, the candidate secondary attribute words may include a first candidate secondary attribute word and a second candidate secondary attribute word. For each first-level attribute word, based on the similarity between each word included in the aforementioned multiple clauses and the first-level attribute word, a first candidate secondary attribute word similar to the first-level attribute word can be determined from the multiple words included in the aforementioned multiple clauses. Then, for each first candidate secondary attribute word, based on the similarity between each word included in the aforementioned multiple clauses and the first candidate secondary attribute word, a second candidate secondary attribute word similar to the first candidate secondary attribute word is determined. The specific similarity calculation and first / second candidate secondary attribute word determination methods have been described in detail above and will not be repeated here.
[0061] Then, in step S460, the relevance importance of the candidate secondary attribute words is calculated, and the secondary attribute words related to the primary attribute words are determined from the candidate secondary attribute words based on the relevance importance.
[0062] A word network may be constructed for each first-level attribute word with the first-level attribute word and candidate second-level attribute words related to the first-level attribute word as nodes, and second-level attribute words related to the first-level attribute word may be determined from the candidate second-level attribute words based on the constructed word network.
[0063] Specifically, in the word network, an edge with direction and weight is constructed between the node corresponding to the first-level attribute word and the node corresponding to the first candidate second-level attribute word similar to the first-level attribute word, the edge uses the similarity between the first-level attribute word and the first candidate second-level attribute word as weight, and the direction is from the first-level attribute word to the first candidate second-level attribute word. At the same time, an edge with direction and weight is constructed between the node corresponding to the first candidate second-level attribute word and the node corresponding to the second candidate second-level attribute word similar to the first candidate second-level attribute word, the edge uses the similarity between the first candidate second-level attribute word and the second candidate second-level attribute word as weight, and the direction is from the first candidate second-level attribute word to the second candidate second-level attribute word.
[0064] Figure 6 A schematic diagram of a word network according to an embodiment of the present invention is shown. The node identified by number 1 represents a first-level attribute word, the node identified by number 2 represents a first candidate second-level attribute word, and the node identified by number 3 represents a second candidate second-level attribute word. It should be noted that the role represented by the node is not unique, and the first-level attribute word may also be a second candidate second-level attribute word similar to a first candidate second-level attribute word, and the first candidate second-level attribute word may also be a second candidate second-level attribute word similar to other first candidate second-level attribute words, and so on.
[0065] After constructing a word network for the first-level attribute words, for the word network of each first-level attribute word, an important node ranking algorithm can be used to calculate the relative importance of each node in the word network, and finally the first candidate second-level attribute word and / or the second candidate second-level attribute word whose relative importance of the corresponding node is greater than a predetermined value is selected as the second-level attribute word related to the first-level attribute word. Among them, any important node ranking algorithm in the art can be used to calculate the relative importance of the node, such as PageRank algorithm, in-and-out importance algorithm, betweenness centrality algorithm, or SimRank algorithm, etc.
[0066] In addition, considering that there may be redundancy in the obtained attribute words (for example, a first-level attribute word is the second-level attribute word of other first-level attribute words), according to one embodiment of the present invention, after determining the second-level attribute words related to the first-level attribute words from the candidate second-level attribute words based on the relevant importance, the same or similar attribute words can be merged.
[0067] Specifically, for each first-level attribute word, determine whether there are other first-level attribute words in the second-level attribute words determined to be related to the first-level attribute word. If there are no other first-level attribute words, no processing is performed. If there are other first-level attribute words, continue to determine whether the relative importance of the node corresponding to the other first-level attribute word in the word network corresponding to the first-level attribute word is less than that of the node corresponding to the first-level attribute word. If less, the other first-level attribute word is no longer regarded as a first-level attribute word, but as a second-level attribute word related to the first-level attribute word. At the same time, the second-level attribute words related to the other first-level attribute words are also regarded as second-level attribute words related to the first-level attribute word.
[0068] If the relative importance of the other first-level attribute word in the word network corresponding to the first-level attribute word is not less than the node corresponding to the first-level attribute word, the first-level attribute word is no longer regarded as a first-level attribute word, but as a second-level attribute word related to the other first-level attribute word. At the same time, the second-level attribute words related to the first-level attribute word are also regarded as second-level attribute words related to the other first-level attribute word.
[0069] In this way, the final first-level attribute words of the product and the second-level attribute words related to the first-level attribute words are obtained.
[0070] In summary, the commodity attribute word set obtained by the commodity attribute word determination scheme according to the embodiment of the present invention has a higher internal correlation and is more different from other (classes) of commodities. At the same time, it has good practicality for large-scale data and can greatly save manpower. In the commodity attribute word determination scheme according to the embodiment of the present invention, firstly, by clustering the words, the proportion of common words in the first-level attribute words is reduced, and the personalization degree of the obtained first-level attribute words is greatly improved (the attribute words of a certain (class) commodity have less repetition with the attribute words of other (classes) of commodities, that is, the attribute words of the (class) commodity have a high degree of personalization). Secondly, by determining the first candidate second-level attribute word and the second candidate second-level attribute word, and converting the similarity matching problem of the words into the problem of sorting important nodes in the network, more richer and more reasonable second-level attribute words can be obtained, and it also has better practicality for large-scale data. Finally, by merging the same or similar first-level attribute words and second-level attribute words, the accuracy of the obtained attribute words is greatly improved.
[0071] Figure 7FIG. 7 is a block diagram showing a structure of a device 700 for determining a commodity attribute word according to an embodiment of the present invention. It should be noted that Figure 7 The device 700 for determining product attribute words is described in a functional module-based manner.
[0072] like Figure 7 As shown, the device 700 for determining product attribute words may include a comment acquisition module 710 , a text segmentation module 720 , a clause classification module 730 , a primary attribute word determination module 740 and a secondary attribute word determination module 750 .
[0073] The review acquisition module 710 is adapted to acquire at least one review text of a product. The text segmentation module 720 is connected to the review acquisition module 710 and is adapted to segment the at least one review text to obtain multiple clauses and multiple words included in the multiple clauses.
[0074] The clause classification module 730 is connected to the text segmentation module 720 and is suitable for dividing at least part of the multiple clauses obtained by the text segmentation module 720 into multiple categories. The primary attribute word determination module 740 is connected to the text segmentation module 720 and the clause classification module 730 and is suitable for determining the primary attribute words of the commodity from the words included in the clauses under each category based on the frequency of occurrence of the words. The secondary attribute word determination module 750 is connected to the text segmentation module 720 and the primary attribute word determination module 740 and is suitable for determining the candidate secondary attribute words related to the primary attribute words from the multiple words included in the multiple clauses obtained by the text segmentation module 720, at least based on the similarity between the words and the primary attribute words determined by the primary attribute word determination module 740. The secondary attribute word determination module 750 is also suitable for calculating the relevance importance of these candidate secondary attribute words and determining the secondary attribute words related to the primary attribute words from these candidate secondary attribute words based on the relevance importance.
[0075] For detailed processing logic and functions of each module in the commodity attribute word determination device 700, please refer to the above combined Figure 1-Figure 6 The description of the method 400 for determining the commodity attribute words will not be repeated here.
[0076] Figure 8 The flowchart of a method 800 for determining a commodity attribute word under a predetermined category according to an embodiment of the present invention is shown. The method 800 is suitable for being executed in a device 900 for determining a commodity attribute word under a predetermined category to be described below. It should be noted that the method 800 for determining a commodity attribute word under a predetermined category can replace the method 400 for determining a commodity attribute word, and can achieve a technical effect similar to that of the method 400 for determining a commodity attribute word.
[0077] like Figure 8As shown, the method 800 for determining the commodity attribute words under the predetermined category begins with step S810. In step S810, at least one review text of all commodities under the predetermined category is obtained. Then, in step S820, the obtained at least one review text is segmented to obtain multiple clauses and multiple words included in the multiple clauses.
[0078] Then, in step S830, at least one of the clauses with the highest frequency of occurrence is divided into multiple categories. In step S840, the first-level attribute words of the products under the predetermined category are determined from the words included in the clauses under each category based on the frequency of occurrence of the words.
[0079] After the first-level attribute word is determined, in step S850, candidate second-level attribute words related to the first-level attribute word are determined from the multiple words included in the aforementioned multiple clauses, based at least on the similarity between the words and the first-level attribute word. In step S860, the relevance importance of the candidate second-level attribute words is calculated, and the second-level attribute words related to the first-level attribute word are determined from the candidate second-level attribute words based on the relevance importance.
[0080] Finally, in step S870, it is determined whether there are other first-level attribute words in the second-level attribute words related to the first-level attribute word, and the relative importance of the node corresponding to the other first-level attribute word is less than that of the node corresponding to the first-level attribute word. If there are, in step S880, the other first-level attribute word and the second-level attribute words related to the other first-level attribute word are used as the second-level attribute words related to the first-level attribute word.
[0081] For detailed processing logic and implementation process of each step in the method 800 for determining the commodity attribute words under the predetermined category, please refer to the above combined Figure 1-Figure 6 The description of the method 400 for determining the commodity attribute words will not be repeated here.
[0082] Fig. 9 FIG. 9 is a block diagram showing a structure of a device 900 for determining a commodity attribute word under a predetermined category according to an embodiment of the present invention. It should be noted that Fig. 9 The device 900 for determining product attribute words under a predetermined category is described in a functional module division manner. Moreover, the device 900 for determining product attribute words under a predetermined category can replace the device 700 for determining product attribute words and can achieve a similar technical effect as the device 700 for determining product attribute words.
[0083] like Fig. 9 As shown, the device 900 for determining product attribute words under a predetermined category may include a comment acquisition module 910 , a text segmentation module 920 , a clause classification module 930 , a first-level attribute word determination module 940 and a second-level attribute word determination module 950 .
[0084] The review acquisition module 910 is adapted to acquire at least one review text of all commodities under a predetermined category. The text segmentation module 920 is connected to the review acquisition module 910 and adapted to segment the at least one review text to obtain multiple clauses and multiple words included in the multiple clauses.
[0085] The clause classification module 930 is connected to the text segmentation module 920 and is adapted to classify at least one clause with the highest occurrence frequency among the multiple clauses obtained by the text segmentation module 920 into multiple categories.
[0086] The primary attribute word determination module 940 is connected to the text segmentation module 920 and the clause classification module 930, and is adapted to determine the primary attribute words of the products under the predetermined category based on the frequency of occurrence of the words from the words included in the clauses under each category.
[0087] The secondary attribute word determination module 950 is connected to the text segmentation module 920 and the primary attribute word determination module 940, and is adapted to determine candidate secondary attribute words related to the primary attribute words from the multiple words included in the multiple clauses obtained by the text segmentation module 920, based at least on the similarity between the words and the primary attribute words determined by the primary attribute word determination module 940. The secondary attribute word determination module 950 is also adapted to calculate the relevance importance of these candidate secondary attribute words, and determine the secondary attribute words related to the primary attribute words from these candidate secondary attribute words based on the relevance importance.
[0088] The secondary attribute word determination module 950 is also adapted to determine whether there are other primary attribute words in the secondary attribute words related to the primary attribute word, and the relative importance of the node corresponding to the other primary attribute word is less than that of the node corresponding to the primary attribute word. If there are, the other primary attribute words and the secondary attribute words related to the other primary attribute words are taken as the secondary attribute words related to the primary attribute word.
[0089] For detailed processing logic and functions of each module in the device 900 for determining the commodity attribute words under the predetermined category, please refer to the above combined Figure 1-Figure 6 The description of the method 400 for determining the commodity attribute words will not be repeated here.
[0090] It should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.
[0091] Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in the devices described in the embodiment, or alternatively may be located in one or more devices different from the devices in the examples. The modules in the foregoing examples may be combined into one module or may be divided into multiple submodules.
[0092] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.
[0093] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.
[0094] In addition, some of the embodiments are described herein as methods or combinations of method elements that can be implemented by a processor of a computer system or by other devices that perform the functions. Therefore, a processor with necessary instructions for implementing the method or method elements forms a device for implementing the method or method elements. In addition, the elements described herein of the device embodiments are examples of devices for implementing the functions performed by the elements for the purpose of implementing the invention.
[0095] As used herein, unless otherwise specified, the use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved, and is not intended to imply that the objects so described must have a given order in time, space, order, or in any other manner.
[0096] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.
Claims
1. A method for determining a commodity attribute word, the method comprising the steps of: Obtain at least one review text of the product; Segmenting the at least one comment text to obtain a plurality of clauses and a plurality of words included in the plurality of clauses; dividing at least some of the plurality of clauses into a plurality of categories; Determine the primary attribute words of the product based on the occurrence frequency of the words from the words included in the clauses under each category; Determining, from a plurality of words included in the plurality of clauses, candidate secondary attribute words related to the primary attribute words based at least on similarities between the words and the primary attribute words; as well as calculating the relevance importance of the candidate secondary attribute words, and determining secondary attribute words related to the primary attribute words from the candidate secondary attribute words based on the relevance importance, The step of calculating the relevant importance of the candidate secondary attribute words includes: constructing a word network for the first-level attribute word with the first-level attribute word and candidate second-level attribute words related to the first-level attribute word as nodes; and In the word network, an important node ranking algorithm is used to calculate the relative importance of each node in the word network.
2. The method of claim 1, wherein: After the step of obtaining a plurality of clauses and a plurality of words included in the plurality of clauses, the method further comprises the steps of: Generating word vectors for a plurality of words included in the plurality of clauses; Based on the word vector, a feature vector is generated for the at least part of the clause.
3. The method of claim 2, wherein: The step of dividing at least some of the plurality of clauses into a plurality of categories comprises: Based on the feature vector, the at least part of the clauses are divided into a plurality of categories using a clustering algorithm.
4. The method of claim 2, wherein: The step of determining the primary attribute words of the product based on the occurrence frequency of the words from the words included in the clauses under each category includes: For each category, the occurrence frequencies of the words included in the clauses under the category are counted, and at least one word with the highest occurrence frequency is selected as the primary attribute word of the product.
5. The method of claim 1, wherein: The step of determining, from among the multiple words included in the multiple clauses, candidate secondary attribute words related to the primary attribute words based on the similarity between the words and the primary attribute words comprises: Based on the similarity between the word and the first-level attribute word, determining a first candidate second-level attribute word similar to the first-level attribute word; Based on the similarity between the word and the first candidate secondary attribute word, a second candidate secondary attribute word similar to the first candidate secondary attribute word is determined.
6. The method of claim 1, wherein: In the word network, there is an edge with a weight of similarity between the first-level attribute word and the first candidate second-level attribute word corresponding node, and there is an edge with a weight of similarity between the first candidate second-level attribute word and the second candidate second-level attribute word corresponding node similar to the first candidate second-level attribute word. Then the step of determining the secondary attribute words related to the primary attribute words from the candidate secondary attribute words based on the related importance includes: A first candidate secondary attribute word and / or a second candidate secondary attribute word whose relative importance of the corresponding node is greater than a predetermined value is selected as the secondary attribute word.
7. The method of claim 6, wherein: The important node ranking algorithm includes the Page Rank algorithm, the In-Degree Importance algorithm, the Betweenness Centrality algorithm, or the SimRank algorithm.
8. The method of claim 1, wherein: After the step of determining the secondary attribute words related to the primary attribute words from the candidate secondary attribute words based on the related importance, the method further comprises the steps of: Determine whether there are other first-level attribute words in the second-level attribute words related to the first-level attribute word, and the relative importance of the nodes corresponding to the other first-level attribute words is less than that of the nodes corresponding to the first-level attribute word; If so, the other first-level attribute words and the second-level attribute words related to the other first-level attribute words are taken as the second-level attribute words related to the first-level attribute words.
9. The method of claim 8, wherein: The method further comprises the steps of: If there are other first-level attribute words among the second-level attribute words related to the first-level attribute word, and the relative importance of the nodes corresponding to the other first-level attribute words is greater than that of the nodes corresponding to the first-level attribute words, then the first-level attribute word and the second-level attribute words related to the first-level attribute words are taken as the second-level attribute words related to the other first-level attribute words.
10. The method of claim 1, wherein: The step of dividing at least some of the plurality of clauses into a plurality of categories comprises: Counting the occurrence frequency of each clause in the multiple clauses; At least one clause with the highest frequency of occurrence is selected for partitioning.
11. The method of claim 1, wherein: Before the step of segmenting the at least one comment text, the method further comprises the steps of: Performing data cleaning on the at least one comment text; as well as After the step of determining the primary attribute words of the commodity based on the occurrence frequency of the words included in the clauses under each category, the method further includes the steps of: Perform data cleaning on the determined first-level attribute words.
12. The method of claim 2, wherein: The step of generating word vectors for the multiple words included in the multiple clauses includes: A word embedding model is used to generate word vectors for the multiple words included in the multiple clauses.
13. The method according to any one of claims 1 to 12, wherein: The products include all products under the predetermined category.
14. A method for determining a commodity attribute word under a predetermined category, the method comprising the steps of: Obtain at least one review text for all products under the predetermined category; Segmenting the at least one comment text to obtain a plurality of clauses and a plurality of words included in the plurality of clauses; dividing at least one clause with the highest frequency of occurrence among the plurality of clauses into a plurality of categories; Determining the primary attribute words of the products under the predetermined category based on the occurrence frequency of the words from the words included in the clauses under each category; Determining, from a plurality of words included in the plurality of clauses, candidate secondary attribute words related to the primary attribute words based at least on similarities between the words and the primary attribute words; Calculating the relevance importance of the candidate secondary attribute words, and determining secondary attribute words related to the primary attribute words from the candidate secondary attribute words based on the relevance importance; Determine whether there are other first-level attribute words in the second-level attribute words related to the first-level attribute word, and the relative importance of the nodes corresponding to the other first-level attribute words is less than that of the nodes corresponding to the first-level attribute word; If it exists, the other first-level attribute words and the second-level attribute words related to the other first-level attribute words are taken as the second-level attribute words related to the first-level attribute words. The step of calculating the relevant importance of the candidate secondary attribute words includes: constructing a word network for the first-level attribute word with the first-level attribute word and candidate second-level attribute words related to the first-level attribute word as nodes; and In the word network, an important node ranking algorithm is used to calculate the relative importance of each node in the word network.
15. A device for determining a commodity attribute word, comprising: A review acquisition module, adapted to acquire at least one review text of a product; A text segmentation module, adapted to segment the at least one comment text to obtain a plurality of clauses and a plurality of words included in the plurality of clauses; a clause classification module, adapted to classify at least some of the plurality of clauses into a plurality of categories; A primary attribute word determination module, adapted to determine the primary attribute words of the commodity from the words included in the clauses under each category based on the occurrence frequency of the words; as well as a secondary attribute word determination module adapted to determine, from a plurality of words included in the plurality of clauses, candidate secondary attribute words related to the primary attribute words based at least on similarities between the words and the primary attribute words; It is also suitable for calculating the relevance importance of the candidate secondary attribute words, and determining the secondary attribute words related to the primary attribute words from the candidate secondary attribute words based on the relevance importance, wherein the calculating the relevance importance of the candidate secondary attribute words includes: constructing a word network for the first-level attribute word with the first-level attribute word and candidate second-level attribute words related to the first-level attribute word as nodes; and In the word network, an important node ranking algorithm is used to calculate the relative importance of each node in the word network.
16. A device for determining a commodity attribute word under a predetermined category, comprising: A review acquisition module, adapted to acquire at least one review text of all products under the predetermined category; A text segmentation module, adapted to segment the at least one comment text to obtain a plurality of clauses and a plurality of words included in the plurality of clauses; a clause classification module, adapted to classify at least one clause with the highest occurrence frequency among the plurality of clauses into a plurality of categories; A primary attribute word determination module, adapted to determine the primary attribute words of the products under the predetermined category from the words included in the clauses under each category based on the occurrence frequency of the words; as well as Secondary attribute word determination module, suitable for Determining, from a plurality of words included in the plurality of clauses, candidate secondary attribute words related to the primary attribute words based at least on similarities between the words and the primary attribute words; Calculating the relevance importance of the candidate secondary attribute words, and determining secondary attribute words related to the primary attribute words from the candidate secondary attribute words based on the relevance importance; Determine whether there are other first-level attribute words in the second-level attribute words related to the first-level attribute word, and the relative importance of the nodes corresponding to the other first-level attribute words is less than that of the nodes corresponding to the first-level attribute word; If so, the other first-level attribute words and the second-level attribute words related to the other first-level attribute words are used as the second-level attribute words related to the first-level attribute words, wherein the calculating the related importance of the candidate second-level attribute words includes: constructing a word network for the first-level attribute word with the first-level attribute word and candidate second-level attribute words related to the first-level attribute word as nodes; and In the word network, an important node ranking algorithm is used to calculate the relative importance of each node in the word network.
17. A comment sentiment analysis system, comprising: The device for determining a commodity attribute word according to claim 15, adapted to determine the commodity attribute word according to at least one review text of the commodity; as well as The processing front end is adapted to obtain, for each attribute word of the product, the sentiment tendency of each review text with respect to the attribute.
18. A computing device comprising: one or more processors; and Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising instructions for executing any of the methods according to claims 1-14.
19. A computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which when executed by a computing device, cause the computing device to perform any of the methods according to claims 1-14.
Citation Information
Patent Citations
Method and device for extracting product attribute information
CN101968788A
Commodity review property word extracting method based on noumenon lexicon
CN104715049A
Commodity property characteristic word clustering method
CN105243129A
Public praise analysis method and apparatus for product review data
CN106021433A
Entity attribute information extraction method and device based on syntactic dependency
CN107957991A
