A method and apparatus for analyzing comments of an auxiliary tool

Abstract

The application relates to a review analysis method and device of an auxiliary tool, and belongs to the technical field of computer processing, which solves the problems that the existing method cannot highlight aspect information in samples, the aspect information of the samples is not considered, the model is increased in recombination error when fitting different sentences with different emphases, and the learning effect is affected, and the like. The method comprises the following steps: aspect extraction is performed on evaluation sentences of the auxiliary tool by using an aspect extraction model to obtain evaluation sentences of the auxiliary tool classified according to aspects, wherein the aspect extraction on the evaluation sentences of the auxiliary tool to obtain the evaluation sentences of the auxiliary tool classified according to aspects further comprises: calculating a probability distribution of the evaluation sentences belonging to each aspect according to aspect-related keywords; sentiment classification and abstract extraction are performed on the evaluation sentences classified according to aspects by using a BERT model to obtain aspect-level summaries of different aspects of the auxiliary tool. Through aspect extraction, the manual labeling cost is saved, and the aspects related to the keyword content can be output.

Description

Technical Field

[0001] This invention relates to the field of computer processing technology, and in particular to a method and apparatus for comment analysis of an auxiliary tool. Background Technology

[0002] In traditional open-source project development, tasks such as code submission, inspection, testing, integration, and deployment all require manual completion. In recent years, an increasing number of open-source projects have used auxiliary tools to automate these tasks. For example, continuous integration tools like Travis can help projects automatically merge code, compile, and run tests. Static analysis tools like Codacy check for errors and ensure code conformity to coding standards. Continuous delivery and continuous deployment tools enable reliable releases in shorter cycles. Research and practice show that auxiliary tools can significantly accelerate project development, improve open-source software quality, and reduce the number of defects. However, existing auxiliary tool app stores and official websites lack user reviews and ratings. Therefore, project managers and developers cannot understand the advantages and disadvantages of various tools and their actual usage evaluations, making it difficult to make effective choices among similar tools. This hinders efficient open-source project development and the widespread application of auxiliary tools.

[0003] There are numerous questions and answers related to auxiliary tools on software Q&A communities, with reviews scattered throughout these answers. Summarizing these reviews can address the lack of feedback on existing auxiliary tools and help developers quickly obtain valuable and concise information about them. Furthermore, auxiliary tool reviews often cover various aspects, such as usability, performance, and functionality. Summarizing these different aspects of each tool can highlight its advantages and disadvantages, helping developers make informed choices. Therefore, a method is needed to summarize auxiliary tool reviews from software Q&A communities, focusing on different aspects.

[0004] In the aspect extraction stage, existing methods for software review aspect extraction mainly include three approaches: rule-based, supervised learning, and unsupervised learning. Rule-based and supervised learning methods require significant manual effort to summarize rules, keywords, or labeled datasets, and are difficult to apply across different scenarios, thus having considerable limitations. Unsupervised learning methods struggle to constrain aspects and often fail to capture software quality attributes typically considered in software reviews, making them impractical. Therefore, this paper proposes a weakly supervised learning approach, using aspect-related keywords as supervised data to complete the aspect extraction task. This saves on manual annotation costs and outputs aspects relevant to the keyword content, facilitating subsequent analysis and summarization.

[0005] However, existing weakly supervised aspect extraction methods have the following shortcomings:

[0006] 1. Existing methods extract keywords from labeled data and extract keywords for each aspect based on manually labeled sample sentences, which still requires manual labeling costs;

[0007] 2. Existing methods fit different samples by weighting keywords with fixed weights. However, different sentences describing the same aspect may have different focuses. For example, a discussion of the performance of assistive tools may include sentences discussing time efficiency and sentences discussing resource utilization efficiency. For sentences discussing time efficiency, time-related keywords (such as response time) are more important; for sentences discussing resource utilization efficiency, resource-related keywords (such as memory usage) are more important. Existing methods use the same aspect embedding vector to fit samples with different focuses, without considering aspect-specific information for the samples. This may lead to increased reconstruction errors when fitting sentences with different focuses, affecting learning performance.

[0008] 3. Similarly, existing methods ignore the role of keywords when calculating sentence representations, failing to highlight aspect information in the sample. For example, in the auxiliary tool discussion sentence "It took almost 1 hour for Travis to respond, when I ran it last night," the second half mainly describes the time context and is unrelated to the evaluation aspect. Such irrelevant information increases the error in subsequent sentence reconstruction processes and reduces the model's learning ability.

[0009] In the review summary stage, existing software review summarization methods often employ general approaches such as sentiment dictionaries and machine learning for sentiment analysis or summarization. Related research also indicates that the semantics of words in software engineering-related contexts differ significantly from those in general contexts, leading to unsatisfactory sentiment analysis results using general methods. This problem also exists in the context of auxiliary tool discussions. For example, in the context of auxiliary tool evaluations, "Travis" refers to the continuous integration tool TravisCI, while in general contexts, "Travis" generally refers to a person's name. Directly using word vectors or other language models from general contexts can lead to errors in the representation of technical terms, thus affecting subsequent classification and clustering tasks. Summary of the Invention

[0010] Based on the above analysis, the embodiments of the present invention aim to provide a comment analysis method and apparatus for auxiliary tools, in order to solve the problems of existing methods ignoring the role of keywords, failing to highlight aspect information in samples, and causing the model to increase recombination error and affect learning effect when fitting sentences with different focuses if aspect information of samples is not considered.

[0011] On one hand, embodiments of the present invention provide a method for analyzing comments on an auxiliary tool, comprising: using an aspect extraction model to extract aspects from the evaluation sentences of the auxiliary tool to obtain evaluation sentences of the auxiliary tool classified by aspect, wherein extracting aspects from the evaluation sentences of the auxiliary tool to obtain evaluation sentences of the auxiliary tool classified by aspect includes: calculating the probability distribution of the evaluation sentences belonging to each aspect based on aspect-related keywords; and using a BERT model to perform sentiment classification and summary extraction on the evaluation sentences classified by aspect to obtain aspect-level summaries of different aspects of the auxiliary tool.

[0012] The beneficial effects of the above technical solution are as follows: This application calculates the probability distribution of the evaluation sentence belonging to each aspect based on the relevant keywords, which saves the cost of manual annotation and can output aspects related to the keyword content, which is convenient for subsequent analysis and summarization.

[0013] Further improvements to the above method include calculating the probability distribution of the evaluation sentence belonging to each aspect based on aspect-related keywords, which further comprises: calculating a sentence vector using word vectors based on aspect information attention; calculating an aspect embedding vector by weighted summation of each keyword vector in the aspect based on keyword attention; and weighted summation of all aspect embedding vectors to obtain a sentence reconstruction vector. The aspect extraction model is trained by reducing the inner product between the sentence vector and the sentence reconstruction vector, and the aspect extraction model is used to predict the probability distribution of the evaluation sentence belonging to each aspect.

[0014] A further improvement to the above method, calculating sentence vectors based on aspect information attention using word vectors, further includes: obtaining word vectors by multiplying the word set and the word vector matrix, and obtaining keyword vectors by multiplying the aspect keyword set and the word vector matrix, wherein the word set is generated based on the evaluation sentences of the auxiliary tool; calculating word attention weights based on the similarity between the word vectors and the keyword vectors using the aspect information attention, wherein the similarity between the word vectors and the keyword vectors is obtained by multiplying the word vectors, the attention parameter matrix, and the average sum of the keyword vectors; and obtaining the sentence vector by weighted summation of the word vectors and the word attention weights.

[0015] A further improvement to the above method, the weighted summation of aspect embedding vectors based on keyword attention for each keyword vector in the aspect further includes: extracting m keyword sets from the evaluation sentences of K aspects based on TF-IDF, where K and m are both positive integers; calculating the attention weight of the keywords based on the similarity between the keyword vectors and the sentence vectors, where the similarity between the keyword vectors and the sentence vectors is obtained by multiplying the keyword vectors, the keyword attention parameter matrix, and the current sentence vector; and calculating the aspect embedding vector by weighted summation of each keyword vector in the aspect based on the attention weight of the keywords.

[0016] Based on a further improvement of the above method, the linear layer in the aspect extraction model predicts the probability distribution of the evaluation sentence belonging to each aspect; the aspect extraction model calculates the sentence reconstruction vector based on the probability distribution of the evaluation sentence belonging to each aspect and the corresponding aspect embedding vector.

[0017] Based on further improvements to the above method, the evaluation aspects of the auxiliary tool's sentences include compatibility, learnability, functionality, performance, reliability, and ease of use.

[0018] Further improvements to the above method, using the BERT model for sentiment analysis and summary extraction to obtain aspect-level summaries of different aspects of the auxiliary tool, further include: classifying evaluation sentences categorized by aspect into evaluation sentences with sentiment polarity based on the BERT model, wherein the sentiment polarity includes positive, negative, or neutral; and extracting summary sentences from evaluation sentences with sentiment polarity of positive or negative sentiment as aspect-level summaries of the auxiliary tool.

[0019] On the other hand, embodiments of the present invention provide a comment analysis device for an auxiliary tool, comprising: an aspect classification module, used to extract aspects from the evaluation sentences of the auxiliary tool using an aspect extraction model to obtain evaluation sentences of the auxiliary tool classified by aspect, wherein the aspect classification module is further used to calculate the probability distribution of the evaluation sentences belonging to each aspect based on aspect-related keywords; a sentiment analysis module, used to perform sentiment classification on the evaluation sentences classified by aspect using a BERT model; and a summary extraction module, used to extract summaries from the sentiment-classified evaluation sentences to obtain aspect-level summaries of different aspects of the auxiliary tool.

[0020] Based on further improvements to the above-mentioned device, the aspect classification module further includes: a sentence vector calculation module, an aspect embedding vector calculation module, and a sentence reconstruction module. The sentence vector calculation module is used to calculate a sentence vector using word vectors based on aspect information attention. The aspect embedding vector calculation module is used to calculate an aspect embedding vector by weighted summation of each keyword vector in the aspect based on keyword attention. The sentence reconstruction module is used to weighted summation of all aspect embedding vectors to obtain a sentence reconstruction vector. The aspect extraction model is trained by reducing the inner product between the sentence vector and the sentence reconstruction vector, and the aspect extraction model is used to predict the probability distribution of the evaluation sentence belonging to each aspect.

[0021] Based on a further improvement of the above method, the sentence vector calculation module further includes: a word vector submodule, a keyword vector submodule, and a word attention submodule. The sentence vector submodule is configured to obtain the word vectors by multiplying a word set by a word vector matrix, wherein the word set is generated based on the evaluation sentences of the auxiliary tool; the keyword vector submodule is configured to obtain keyword vectors by multiplying an aspect keyword set by the word vector matrix; the word attention submodule is configured to calculate word attention weights based on the aspect information attention using the similarity between the word vectors and the keyword vectors, wherein the similarity between the word vectors and the keyword vectors is obtained by multiplying the word vectors, the attention parameter matrix, and the summation average of the keyword vectors; and the sentence vector submodule is configured to obtain the sentence vectors by weighted summation of the word vectors and the word attention weights.

[0022] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0023] 1. This invention proposes a method for extracting aspects from user reviews of open-source development aids. This method extracts keywords from descriptive texts in the software engineering field, further reducing reliance on manual annotation; it also improves the model by incorporating aspect attention and keyword attention mechanisms. This significantly improves the accuracy of aspect extraction for user reviews of aids. 2. This invention proposes a method for summarizing user reviews of open-source development aids. Based on BERT pre-training on text within the context of aids, the trained language model can better represent the semantics of professional terms related to aids. Based on more accurate semantic information, sentiment analysis and summary summarization are performed, thereby improving the effectiveness of user review summarization. 3. This invention can generate aspect-level summaries for different aids, helping users understand the characteristics of different tools to make tool selections. It can also summarize user reviews of aids to assist developers in improving their tools.

[0024] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0025] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0026] Figure 1 This is a diagram illustrating the summary of user comments for auxiliary tools.

[0027] Figure 2 A flowchart of a comment analysis method for an auxiliary tool according to an embodiment of the present invention;

[0028] Figure 3 To extract the structural diagram of the model according to an embodiment of the present invention;

[0029] Figure 4 This refers to aspect information in the discussion sentences according to embodiments of the present invention;

[0030] Figure 5 This is a schematic diagram of BERT pre-training according to an embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of the emotion classification process according to an embodiment of the present invention;

[0032] Figure 7 This is a schematic diagram of a summarization method according to an embodiment of the present invention;

[0033] Figure 8 A block diagram of a comment analysis device for an auxiliary tool according to an embodiment of the present invention. Detailed Implementation

[0034] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0035] One specific embodiment of the present invention discloses a method for comment analysis of auxiliary tools. (See reference) Figure 1The method for analyzing comments on auxiliary tools includes: in step S102, using an aspect extraction model to extract aspects from the evaluation sentences of the auxiliary tools to obtain evaluation sentences categorized by aspect, wherein extracting aspects from the evaluation sentences of the auxiliary tools to obtain evaluation sentences categorized by aspect further includes: calculating the probability distribution of the evaluation sentences belonging to each aspect based on aspect-related keywords; obtaining evaluation sentences categorized by aspect based on the probability distribution of the evaluation sentences belonging to each aspect; in step S104, using a BERT model to perform sentiment classification and summary extraction on the evaluation sentences categorized by aspect to obtain aspect-level summaries of different aspects of the auxiliary tools.

[0036] Compared with existing technologies, the comment analysis method of the auxiliary tool provided in this embodiment calculates the probability distribution of the evaluation sentence belonging to each aspect based on the aspect-related keywords, which saves the cost of manual annotation and can output aspects related to the keyword content, which is convenient for subsequent analysis and summarization.

[0037] In the following text, refer to Figure 1 The various steps of the comment analysis method for auxiliary tools according to embodiments of the present invention will be described in detail.

[0038] In step S102, the aspect extraction model is used to extract aspects from the evaluation sentences of the auxiliary tool to obtain evaluation sentences categorized by aspect. This step further includes: calculating the probability distribution of the evaluation sentence belonging to each aspect based on aspect-related keywords. Specifically, calculating the probability distribution of the evaluation sentence belonging to each aspect based on aspect-related keywords further includes: calculating a sentence vector using word vectors based on aspect information attention; calculating an aspect embedding vector by weighted summation of each keyword vector in the aspect based on keyword attention; and weighted summation of all aspect embedding vectors to obtain a sentence reconstruction vector. The aspect extraction model is trained by reducing the inner product between the sentence vector and the sentence reconstruction vector, and the aspect extraction model is used to predict the probability distribution of the evaluation sentence belonging to each aspect. Specifically, refer to... Figure 3 The aspect extraction model is a neural network model, comprising a first neural network sub-model for calculating sentence vectors, a second neural network sub-model for calculating aspect embedding vectors, and a third neural network sub-model for calculating sentence reconstruction vectors. The first neural network sub-model takes a sentence as input and outputs a sentence vector; the second neural network sub-model takes keywords as input and outputs aspect embedding vectors; and the third neural network sub-model takes both the sentence vector and aspect embedding vectors as input and outputs the probability distribution evaluating the sentence's belonging to each aspect and the sentence reconstruction vector.

[0039] The sentence vector calculation based on aspect-based attention using word vectors further includes: obtaining word vectors through the product of the word set and the word vector matrix, and obtaining keyword vectors through the product of the keyword set of the aspect and the word vector matrix. The word set is generated based on the evaluation sentences of the auxiliary tool. Specifically, the aspects of the evaluation sentences of the auxiliary tool include compatibility, learnability, functionality, performance, reliability, and usability. For example, the main quality attribute of usability includes six sub-attributes: understandability, learnability, operability, error protection, user interface aesthetics, and accessibility, each with detailed definitions. The keyword set is generated according to Table 1 below. For example, the aspect "usability" includes the keyword sets "understandability," "learnability," "operability," "error protection," "user interface aesthetics," and "accessibility." For example, the words in the sentence are mapped to the vector space using the word vector matrix E; that is, each word in sentence 1 (Travis tests, merges and builds your code automatically.) and sentence 2 (It took almost 1 hour for Travis to respond when I ran it last night.) is converted into a word vector using the word vector matrix E. Travis→e Travis = (0.1, -0.2, ..., 0.5), which is a 1*200 dimensional vector. The word vector matrix E is pre-trained on a large amount of text and is not closely related to the model in this paper. Aspect-based attention calculates word attention weights using the similarity between word vectors and keyword vectors. The similarity between word vectors and keyword vectors is obtained by multiplying the sum of the word vectors, the attention parameter matrix, and the keyword vectors. The sentence vector is obtained by weighted summation of word vectors and word attention weights.

[0040] The aspect embedding vector is calculated by weighted summation of each keyword vector in the aspect based on keyword attention. This process further includes: extracting m keyword sets from the evaluation sentences of K aspects based on TF-IDF, where K and m are both positive integers; calculating keyword attention weights based on the similarity between keyword vectors and sentence vectors, where the similarity between keyword vectors and sentence vectors is obtained by multiplying the keyword vector, the keyword attention parameter matrix, and the current sentence vector; and calculating the aspect embedding vector by weighted summation of each keyword vector in the aspect based on the keyword attention weights.

[0041] The linear layer in the aspect extraction model predicts the probability distribution of the evaluation sentence belonging to each aspect; the aspect extraction model calculates the sentence reconstruction vector based on the probability distribution of the evaluation sentence belonging to each aspect and the corresponding aspect embedding vector.

[0042] In step S104, the BERT model is used to perform sentiment classification and summary extraction on the aspect-based evaluation sentences to obtain aspect-level summaries of different aspects of the auxiliary tool. Specifically, the sentiment analysis and summary extraction using the BERT model to obtain aspect-level summaries of different aspects of the auxiliary tool further includes: classifying the aspect-based evaluation sentences into evaluation sentences with sentiment polarity based on the BERT model, where sentiment polarity includes positive, negative, or neutral; and extracting summary sentences from evaluation sentences with positive and negative sentiment polarities as aspect-level summaries of the auxiliary tool. Sentences with neutral sentiment polarity are usually objective statements and do not require processing (i.e., no summary sentence extraction is needed).

[0043] Another specific embodiment of the present invention discloses a comment analysis device for an auxiliary tool. (See reference) Figure 8 The comment analysis device for the auxiliary tool includes: an aspect classification module 802, a sentiment analysis module 804, and a summary extraction module 806. The aspect classification module 802 uses an aspect extraction model to extract aspects from the evaluation sentences of the auxiliary tool to obtain evaluation sentences categorized by aspect. The aspect classification module further calculates the probability distribution of each evaluation sentence belonging to each aspect based on aspect-related keywords. The sentiment analysis module 804 uses a BERT model to perform sentiment classification on the aspect-categorized evaluation sentences. The summary extraction module 806 extracts summaries from the sentiment-categorized evaluation sentences to obtain aspect-level summaries of different aspects of the auxiliary tool.

[0044] The aspect classification module further includes: a sentence vector calculation module, an aspect embedding vector calculation module, and a sentence reconstruction module. The sentence vector calculation module is used to calculate sentence vectors based on aspect information attention using word vectors. Specifically, the sentence vector calculation module further includes: a word vector submodule, a keyword vector submodule, a word attention submodule, and a sentence vector submodule. The word vector submodule is used to obtain word vectors by multiplying a word set by a word vector matrix, where the word set is generated based on the evaluation sentences from the auxiliary tool. The keyword vector submodule is used to obtain keyword vectors by multiplying an aspect keyword set by a word vector matrix. The word attention submodule is used to calculate word attention weights based on aspect information attention using the similarity between word vectors and keyword vectors, where the similarity between word vectors and keyword vectors is obtained by multiplying the sum of the word vectors, the attention parameter matrix, and the keyword vectors. The sentence vector submodule is used to obtain sentence vectors by weighted summation of word vectors and word attention weights.

[0045] The aspect embedding vector calculation module is used to calculate the aspect embedding vector by weighted summation of each keyword vector in the aspect based on keyword attention.

[0046] The sentence reconstruction module is used to perform a weighted summation of all aspect embedding vectors to obtain a sentence reconstruction vector. The aspect extraction model is trained by reducing the inner product between the sentence vector and the sentence reconstruction vector, and the aspect extraction model is used to predict the probability distribution of the evaluation sentence belonging to each aspect.

[0047] This invention proposes a method for analyzing user reviews of auxiliary tools, specifically a method for summarizing aspects of user reviews of open-source development auxiliary tools.

[0048] like Figure 2 As shown, the aspect-level summary method for user reviews of open-source development aids mainly consists of two stages: aspect extraction and review summary. In the aspect extraction stage, for each sentence discussing the aids, the aspect being addressed is first identified, such as ease of learning or compatibility. Then, in the review summary stage, sentiment analysis and summary extraction methods are used to perform aspect-level summaries on multiple tools, comparing their performance and strengths and weaknesses in the same aspect.

[0049] In the aspect extraction stage, existing methods for software review aspect extraction mainly include three approaches: rule-based, supervised learning, and unsupervised learning. Unsupervised learning methods struggle to constrain aspects, often failing to capture software quality attributes and other commonly considered aspects in software reviews, making them impractical. Therefore, this paper proposes a weakly supervised learning approach, using aspect-related keywords as supervised data to complete the aspect extraction task. This saves on manual annotation costs and outputs aspects relevant to the keyword content, facilitating subsequent analysis and summarization.

[0050] In the review summary stage, existing software review summarization methods often employ general approaches such as sentiment dictionaries and machine learning for sentiment analysis or summarization. Related research also indicates that the semantics of words in software engineering-related contexts differ significantly from those in general contexts, leading to unsatisfactory sentiment analysis results using general methods. This problem also exists in the context of auxiliary tool discussions. For example, in the context of auxiliary tool evaluations, "Travis" refers to the continuous integration tool TravisCI, while in general contexts, "Travis" generally refers to a person's name. Directly using word vectors or other language models from general contexts can lead to errors in the representation of technical terms, thus affecting subsequent classification and clustering tasks.

[0051] The aspect extraction method in the auxiliary tool review analysis method according to embodiments of the present invention makes the following three improvements: First, the model extracts keywords describing each aspect based on aspect description sentences in ISO-25010 software quality model technical documents and related papers on software review aspect classification, thereby further eliminating the cost of manual annotation; the model proposes keyword attention weights, calculates the keyword attention weights for each aspect based on the input sentence, and captures keyword information that is more important to the input sample; simultaneously, the model adopts an attention mechanism that considers aspect information, giving greater weight to words in the input sentence that are more relevant to the aspect, highlighting the aspect information in the sample. Through the above improvements, the effectiveness of the weakly supervised aspect extraction method is improved and the application cost is reduced.

[0052] The summary method in the auxiliary tool comment analysis method according to the present invention is improved as follows: Based on a large amount of auxiliary tool discussion text, the method pre-trains on the auxiliary tool context text based on the bidirectional encoding representation from Transformers (BERT), which enables the trained language model to better represent the semantics of auxiliary tool-related professional terms, and performs sentiment analysis and summary based on more accurate semantic information, thereby improving the effect of user comment summary.

[0053] The following text will refer to Figures 2 to 7 Taking a specific example, the comment analysis method of the auxiliary tool according to the embodiment of the present invention will be described in detail.

[0054] This invention proposes an aspect-level summary method for user evaluations of source development assistance tools, mainly divided into two stages: aspect extraction and comment summary. In the aspect extraction stage, for discussion sentences about assistance tools, the aspect addressed by the sentence is first identified, such as ease of learning and compatibility. Then, in the comment summary stage, sentiment analysis and abstract extraction methods are used to perform aspect-level summaries on multiple tools, comparing the performance and advantages / disadvantages of each tool in the same aspect.

[0055] Aspect extraction stage

[0056] The aspect extraction process involves extracting and identifying the aspects addressed in comments from discussions about assistive tools. For example, it uses regular expressions to find sentences mentioning the name of an assistive tool, thus collecting these discussion sentences. This project employs weakly supervised learning, using aspect-related keywords as supervisory data. The model calculates the probability distribution of input sentences belonging to each aspect based on the keyword information, thereby completing the aspect extraction task. This saves on manual annotation costs and outputs aspects relevant to the keyword content, facilitating subsequent analysis and summarization.

[0057] To identify aspects of user comments from sentences discussed by auxiliary tools, this invention first obtains and defines the aspects of the evaluation auxiliary tool. Then, keywords representing the meaning of each aspect are extracted for subsequent weakly supervised auxiliary tool comment aspect extraction. This invention summarizes seven aspects, and each aspect and its description are shown in Table 1.

[0058] Table 1. Assistive Tools: Discussion and Explanation of Sentence Aspects

[0059]

[0060]

[0061] For example, compatibility: Environment, Version, OS; ease of learning: Documentation, Learn, Understand; functionality: Code, Build, Management; performance: Time, CPU, Memory; reliability: Bug, Recover, Error; usability: Useful, Easy, Simple; other (i.e., keywords without comments): Travis, Jenkins, TeamCity (i.e., auxiliary tool names).

[0062] Existing weakly supervised aspect extraction models automatically extract the top m words with the highest scores from labeled datasets based on TF-IDF (term frequency–inverse document frequency) as keywords for each aspect. However, these keyword extraction methods still require a certain amount of manpower to annotate specific datasets.

[0063] Unlike aspect extraction tasks in general domains, aspect extraction from software product discussion texts can leverage a large number of existing technical documents or academic papers in the software engineering field. These documents formally define aspects used to evaluate software, such as compatibility and usability. Keywords for these aspects are extracted from these documents, ensuring that the extracted keywords can express the meaning of the aspects to a certain extent. For technical documents, the aspect classification scheme adopted in this invention references the ISO-25010 software quality model. The ISO-25010 technical documents detail eight main quality attributes for measuring software quality (functionality, performance, compatibility, usability, reliability, security, maintainability, and portability), and provide multiple sub-attributes and definitions under each main quality attribute. For example, the main quality attribute of usability includes six sub-attributes: understandability, learnability, operability, error protection, user interface aesthetics, and accessibility, each with detailed definitions. The quality attributes in the aforementioned technical documents largely overlap with the aspects evaluated by the auxiliary tools defined in this project. Therefore, the attribute definition sentences in the technical documents can represent the meaning of the corresponding aspects, used to extract keywords for each aspect, thus eliminating the need for manual annotation. Similarly, academic papers also define aspects used to evaluate software quality, which can be used to extract keywords from each aspect by using the definition sentences in the paper to represent the meaning of the corresponding aspect.

[0064] Based on the above background, this invention extracts keywords representing each aspect from descriptive sentences rather than labeled data, thereby eliminating the need for manual data labeling and fully utilizing information from reliable sources such as technical documents and academic papers to extract higher-quality and more universal keywords. This project collects descriptive sentences from various aspects through the following steps:

[0065] The ISO-25010 document provides comprehensive software quality attributes, each with a detailed definition. This invention adds quality attribute definitions to the descriptive sentences of the corresponding aspects based on the following rules: 1. For performance, reliability, and compatibility, the definition sentences of the corresponding quality attributes and all their sub-attributes from the ISO technical document are added to the descriptive sentences of the corresponding aspects. 2. For learnability & understandability and usability, this invention adds the definition sentences of the understandability and learnability sub-attributes from the ISO technical document to the descriptive sentences of learnability & understandability, and adds the definition sentences of all other sub-attributes under usability (operability, error protection, user interface aesthetics, accessibility) from the ISO technical document to the descriptive sentences of usability. 3. Since functionality involves the capabilities provided by auxiliary tools, the GitHub Marketplace categorizes auxiliary tools according to their functionality. This invention crawls the functional category definition sentences of each auxiliary tool from the GitHub Marketplace as the descriptive sentences for functionality.

[0066] After successfully defining the aspects of the assistive tool and extracting the keywords of each aspect, this invention proposes a weakly supervised aspect extraction method for assistive tool reviews. The basic idea of ​​the weakly supervised aspect extraction model is to predict the probability of each review sentence belonging to an aspect based on the input text and aspect keywords, thereby identifying the aspect that the sentence addresses.

[0067] To enable effective prediction without labeled data, existing models initialize aspect vectors using keywords and then learn the relationship between text and aspects from unlabeled text data. To this end, existing models first predict the input sentence, then linearly combine the aspect vectors based on the predicted values ​​to obtain the reconstructed sentence vector. The model then backpropagates to update its parameters by continuously reducing the difference between the sentence vector and the reconstructed vector, thereby optimizing the model's aspect prediction results for the sentence.

[0068] The model structure graph-assisted tool proposed in this invention provides a method for extracting weakly supervised aspects of the comments. Figure 3 As shown, the gray areas represent areas for model improvement. For an input sentence, the model identifies aspects and completes training based on the following steps: 1) Sentence vector calculation: For each input sentence, the model represents each word in the sentence using pre-trained word vectors, and calculates the sentence representation vector z based on aspect information attention. s2) Aspect vector calculation: Based on the input keywords for each aspect and the current sample, aspect embedding vectors applicable to the current sample are calculated using keyword attention. 3) Sentence reconstruction: The model predicts the aspect distribution of the current sample, obtaining the probability that the sentence belongs to each aspect. Using the probability of each aspect, a weighted sum of all aspect embedding vectors is performed to obtain the reconstructed sentence vector r. s The model reduces the sentence vector z. s The sentence reconstruction vector r obtained by summing the aspect embedding vectors s The inner product between them is used to train the model.

[0069] During the sentence vector calculation phase, the model calculates a corresponding sentence vector z for each input sentence. s Sentence vectors are used to represent the semantic information of sentences. Existing models already use attention mechanisms when calculating sentence vectors, but the attention weights are calculated based on the semantic similarity between words and the overall sentence. Then, the word vectors are weighted and summed based on the attention weights to obtain the current sentence vector. This method of sentence vector calculation fails to highlight the aspect information in the sample sentence, increasing the error in subsequent sentence reconstruction. In sentences discussing auxiliary tools, there are often both discussions or evaluations of a particular aspect of the auxiliary tool and information unrelated to that aspect. The following... Figure 4 Taking the sentence as an example, "took up", "hour" and "response" are related to auxiliary tools and are key information for subsequent classification and sentence reconstruction. However, "when I ran it last night" is a conditional modifier and is not related to auxiliary tools. This irrelevant information in the sentence will increase the error in the subsequent sentence reconstruction process and reduce the model's learning ability.

[0070] In the aspect-level summarization method for auxiliary tool reviews, this invention proposes an aspect-based attention mechanism to replace the attention mechanism in the original model. This aspect-based attention mechanism calculates the attention weight of each word in a sentence based on the similarity between words and aspect information, where aspect information is the average of keyword vectors. Therefore, words more relevant to aspects receive higher attention weights, thus highlighting the aspect information in the sample sentence.

[0071] Calculate the sentence vector z s The specific three steps are as follows:

[0072] Convert words into word vector representations: For each word W in the input sentence, the model... i Multiplying this by the word vector matrix E transforms it into the word vector form of this word. For example, Sentence 1: Travis tests, merges and builds yourcode automatically. Sentence 2: It took almost 1hour for Travis to respond when Iran it last night.W i Let E be a one-hot (0-1) representation of each word in the sentence, where only the dimension representing that word is 1, and all other elements are 0. The word vector matrix E∈R V×d Where V is the length of the text library and d is the dimension of the word vectors. The word vector matrix E is pre-trained and initialized on Stack Overflow discussion texts using the Word2Vec model. A d-dimensional vector that can implicitly represent W i The semantic information of this word.

[0073] Using the word vector matrix E, convert each word in sentences 1 and 2 above into a word vector, for example: Travis → e Travis = (0.1, -0.2, ..., 0.5), which is a 1*200 dimensional vector. Note that this word vector matrix E is pre-trained on a large amount of text. It is not closely related to the model in this paper. Using the word vector matrix E (the same as above), the keywords of each aspect are transformed into keyword vectors, for example: Code → e Code = (0.01, -0.25, ..., -0.5), which is a 1*200 dimensional vector. Therefore, for each input sentence (W1, W2, ..., W...), the model... n The input sentence is multiplied by the word vector matrix E to transform it into word vector form. Used for subsequent calculations.

[0074] Similarly, for aspect A i Keyword set Multiply by the word vector matrix E to convert it into word vector form.

[0075] Calculate attention weights based on aspect information: Reference Figure 3 For the input sentence Calculate the attention weights (a1, a2, ..., a) for each word. n This process assigns greater weight to words that are more similar to sentences and aspects, so that the sentence representation vector z can be calculated by weighted summation of the word vectors based on these weights in the next step. s This highlights the more crucial aspects of the current input sentence. Each word W... i Attention weight a i The calculation formula is as follows:

[0076]

[0077]

[0078]

[0079] Where n is the number of words in the sentence, y s This is the summation and average of the keyword vectors, used to represent more critical aspect information in the aspect extraction task. The keywords are obtained from the keyword extraction step in the previous section and transformed into keyword vectors through the word vector matrix. K is the number of aspects, and m is the number of keywords in each aspect. Matrix M is the attention parameter matrix, which is randomly initialized by the Xavier method of the torch package in Python before model training begins. In practice, pre-trained parameters can also be read according to the actual task, or other initialization methods can be manually specified. The dimension of this matrix is ​​(d*d), where d is the dimension of the word vectors. i The intermediate result obtained from Formula 2 represents the similarity between word vectors and keyword vectors in the sentence, and is used in Formula 1 to calculate attention weights. i For the word W i The corresponding attention weight indicates that the current word better reflects aspect information, while a smaller weight indicates that the current word is less relevant to aspect information. The model can be trained so that the attention parameter matrix M can capture aspect-related words, i.e., aspect-related word vectors. The magnitude of the result multiplied by M is larger. (Using word vectors) The result of multiplying by M, and then multiplying by the summation average of the keyword vectors, can calculate the similarity between the current word and aspect information, giving more attention weight to more relevant words.

[0080] Based on attention weight weighted average: obtain the attention weight (a1, a2, ..., a) for each word. n Then, the model obtains the sentence representation vector z by weighted averaging using the following formula. s .

[0081]

[0082] In the aspect vector calculation stage, the model calculates each sentence vector z. s Calculate the aspect vectors (T1,…,T) of K aspects. KAn aspect embedding vector (AEM) is used to represent the semantic information of a particular aspect. Existing weakly supervised models obtain an aspect embedding vector by weighting and summing keywords with fixed weights, and then use the same AEM to fit different samples. However, different sentences describing the same aspect may have different focuses. For example, a discussion of the performance of assistive tools may include sentences discussing time efficiency and sentences discussing resource utilization efficiency. For sentences discussing time efficiency, time-related keywords (such as response time) are more important; for sentences discussing resource utilization efficiency, resource-related keywords (such as memory usage) are more important. Existing models use the same AEM to fit samples with different focuses, failing to consider aspect information specific to the sample. This makes it impossible to capture the keyword information that is more important to the input sample, which may lead to increased reconstruction error when fitting sentences with different focuses, affecting the learning effect.

[0083] In the aspect-level summarization method for auxiliary tool reviews, this invention uses a keyword attention mechanism to calculate aspect vectors. The keyword attention mechanism uses keyword vectors... and sentence vector z s The similarity is used to calculate the attention weight of each keyword in the aspect. Then, based on the keyword weights, the vectors of each keyword in the aspect are weighted and summed to obtain the aspect vector. In this way, for different input samples, the keyword information that is more important to the input sample is captured.

[0084] Sentence vector computation aims to generate a 1*200 dimensional vector that represents the information of the entire sentence. To improve aspect extraction, we need to highlight aspect-related information within this vector.

[0085] For sentence 1 above (and the same applies to sentence 2):

[0086] First, calculate the aspect information vector y according to Formula 3. s y s This is the summation and average of the keyword vectors. That is: (sum of all keyword vectors related to compatibility + sum of all keyword vectors related to learnability + ... + sum of all keyword vectors related to other aspects) / total number of keywords (see the main text for the specific formula). We assume this y... s = (0.05, -0.1, ..., 0.2).

[0087] Next, according to Formula 2, calculate the relationship between each word in the sentence and y. s similarity d i Assume the results are as shown in Tables 2 and 3 below:

[0088] Table 2

[0089]

[0090] Table 3

[0091]

[0092] Note d i The magnitude of the weights is uncontrollable and cannot be used as a probability distribution. We transform it into a probability distribution using Formula 3.2 (the weights are between 0 and 1, and the sum of all weights is 1), which is the attention weight 'a' for each word in the current sentence. i Assume the results are as shown in Tables 4 and 5 below:

[0093] Table 4

[0094]

[0095] Table 5

[0096]

[0097] These two tables show that aspect-based attention weights assign lower weights to words unrelated to aspects and higher weights to relevant words, thereby minimizing noise in sentences. For example, the fact that `and` is not 1 here is because not all words in sentence 1 are listed.

[0098] Finally, based on tool formula 4, the attention weight a for each word is calculated. i Given the word vectors of each word, calculate the current sentence representation vector z. s .

[0099] Sentence 1: e Travis *a Travis +e Test *a Test +......=z s1 = (0.1, 0.2, ...)

[0100] Sentence 2: z s2

[0101] Regarding aspect A i Calculate this aspect vector T i The specific three steps are as follows:

[0102] Keyword Extraction and Representation: As described in the keyword extraction section above, this invention extracts m keywords from sentences describing K aspects based on TF-IDF. Then, similar to the word vectorization process described earlier, for aspect A... i Keyword set Multiply by the word vector matrix E to convert it into word vector form.

[0103] Calculating Keyword Attention Weights: Keyword attention weight calculation for input aspect A i Keyword vectors and the current sample sentence representation vector z s Calculate the attention weight for each word. Keywords that are more relevant to the current sample sentence are given greater weight, so that the aspect embedding vector T can be calculated by weighted summation of the keywords in the next step. i .

[0104] Each keyword attention weights The calculation formula is as follows:

[0105]

[0106]

[0107] Where m is the number of keywords in aspect i, and z s This is the sentence representation vector of the current input sentence. Matrix M K_ATT This is the keyword attention parameter matrix, which is randomly initialized by the Xavier method of the torch package in Python before model training begins. In practice, pre-trained parameters can also be read according to the actual task, or other initialization methods can be manually specified. The matrix has dimensions (d*d), where d is the dimension of the word vectors. The intermediate result obtained from Formula 6 reflects the keyword vector. With sentence vector z s The similarity is used in Formula 5 to calculate the current keyword. Attention weights. Keywords Corresponding attention weights. Keyword attention weights. Based on keyword vectors With the current sample sentence vector z s inner product calculation, and z s The closer the keywords are, the greater the attention weight they receive. This allows keywords that are more relevant to the current sample sentence to receive greater weight, highlighting more relevant keyword information when calculating the aspect vector in the subsequent weighted average, thus helping the model capture keyword information that is more important to the input sample.

[0108] Based on attention weighted average: This yields the attention weight for each keyword in the input sample for the i-th aspect. Then, the model obtains the aspect embedding vector T by weighted averaging using the following formula. i .

[0109]

[0110] Aspect vector calculation aims to generate a 1*200 dimensional vector that represents information about the current aspect (such as compatibility information). To improve aspect extraction, we need to calculate a set of aspect vectors applicable to sentence 1 and a set of aspect vectors applicable to sentence 2, that is, to capture the keyword information that is more important to the input sample for different input samples in the original text.

[0111] For sentence 1 (and the same applies to sentence 2):

[0112] Based on tool formula 6, iterate through each keyword of each aspect and calculate the current sentence representation vector z. s With keyword vectors similarity Assuming the result is as follows

[0113] Table 6

[0114]

[0115] Table 7

[0116]

[0117] Note The magnitude of the weights is uncontrollable and cannot be used as a probability distribution. We transform it into a probability distribution using Formula 5 (the weights are between 0 and 1, and the sum of all weights is 1). That is, for the current sentence, the attention weight of each keyword in the current aspect. Assume the results are as shown in Tables 8 and 9 below:

[0118] Table 8

[0119]

[0120] Table 9

[0121]

[0122] These two tables show that keyword attention weights allocate greater weight to keywords more relevant to the sentence, thus capturing the most important keyword information for the current input sample. For example, in terms of performance, since sentence 2 complains about the slowness, "time" receives a greater weight.

[0123] Finally, based on tool formula 7, the attention weight is determined for each keyword. Given the word vectors of each keyword, calculate a set of aspect vectors corresponding to the current sentence.

[0124] Function:

[0125] After obtaining the sentence representation vector z of the input sample sentence s and a set of aspect vectors (T1,…,T) for this sample. K Following this, the aspect-level summarization method for auxiliary tool reviews first predicts the aspect distribution p for each sample sentence, where p = (p1, p2, ..., p...). k ), p i Let be the probability that the current sentence belongs to the i-th aspect. Then, using the probability value as weight, the recombined vector r corresponding to the current sample sentence is calculated by weighting the aspect vectors. s The specific calculation of sentence reconstruction can be divided into the following two steps:

[0126] Predicting the aspect distribution of a sentence: The model processes the sentence representation vector z through a linear layer. s The aspect distribution p is used for prediction, where p = (p1, p2, ..., p...). k ), p i The probability that the current sentence belongs to the i-th aspect is calculated using the following formula:

[0127] p = softmax(Wz) s +b) Formula 8

[0128] Where W represents the linear layer weights, to accelerate learning efficiency, this invention uses the sum of keyword vectors from each aspect to initialize a column of parameters in W corresponding to that aspect. In practical use, parameters can also be randomly initialized according to the actual task, or other initialization methods can be manually specified. The dimension of W is (K*d), where d is the word vector dimension and K is the number of aspects. s Let be the sentence representation vector, and b be the linear layer bias value, which is initialized to 0. The model will adjust the linear layer bias value automatically during subsequent training. The Softmax function can convert the input vector values ​​into a probability distribution where each dimension sums to 1. The model predicts the probability value of the model belonging to each aspect through aspect distribution, and this probability distribution can be used to calculate the subsequent sentence reconstruction vector.

[0129] Calculate the sentence reconstruction vector: The model is based on the aspect distribution p and the aspect embedding vector T = (T1, T2, ..., T...). k The aspect-based sentence reconstruction vector r is calculated using the following formula. s .

[0130]

[0131] After obtaining the sentence representation vector z of the input sample sentence s and a set of aspect vectors (T1,…,T) for this sample.K After that, we first predict the aspect of the current sentence, and then calculate the sentence reconstruction vector for subsequent training.

[0132] For example, in Formula 8, we assume that the first column of W (representing compatibility) is initialized to (1, -1, ...), and b is 0. Then, the probability P is calculated roughly as follows:

[0133] Softmax((0.1,0.2,…)*(1,-1,…)+0) → the probability of belonging to compatibility.

[0134] Assume that sentence 1, calculated as described above, belongs to the probability distribution of each aspect as shown in Table 10 below:

[0135] Table 10

[0136]

[0137] According to Formula 9, calculate the sentence reconstruction vector. For example, for sentence 1:

[0138] r s1 = 0.02 * compatibility vector + 0.01 * learnability vector + ...

[0139] The model trains itself by reducing the inner product between the sentence vectors and the recombined vectors formed by aspects. To further accelerate training, the model incorporates negative samples, similar to word vector training. Negative samples are sentences randomly selected from the current batch (excluding the current sentence S) for comparison with the current sentence. The number of negative samples can be used as a hyperparameter; in practical applications, the optimal number of negative samples should be set based on evaluation results. Note that the number of negative samples should be less than the batch size.

[0140] The objective function of the model requires the recombined vector r of sentence S. s With sentence vector z s Try to get as close as possible, and recombine vector r. s Sentence vector n of negative samples i As far as possible. The specific objective function formula is as follows:

[0141]

[0142] Where D is the current text library, m is the number of negative samples, and θ is the set of trainable parameters.

[0143] The model reduces r s1 and z s1The difference allows the model to learn more suitable parameters for each layer. For example, for sentence 1, which is basically about functionality, the sentence vector can be understood as pointing in a direction similar to the function vector.

[0144] However, during the classification, the probability of functionality is 0.33, while reliability and other factors account for 0.3 and 0.28 respectively. Therefore, the calculated r... s1 It will lean more towards the direction of reliability and other aspects. Therefore, r s1 and z s1 There is a significant error, which can be calculated using Formula 10 and then updated using backpropagation. W are these parameters.

[0145] Since the aspect-level summarization method for auxiliary tool reviews can be trained without labeled data, the number of discussion sentences will continuously increase as users ask and answer questions in real-world scenarios. For newly appearing sentences, the model can also be incrementally trained on the existing model with new data as the data increases, thereby updating the model's parameters and ensuring the model's generalization ability in real-world scenarios.

[0146] Comments Summary Phase

[0147] Based on the method for extracting comments on assistive tools, discussion sentences for each tool can be categorized by aspect. Many of these discussion sentences are objective statements that do not contain subjective user evaluations. These objective sentences often provide objective descriptions and introductions of the usage details of the assistive tools, such as "The Travis test log is divided into three parts." These objective sentences do not directly reflect the characteristics and advantages / disadvantages of the tools, and are of little help in selecting similar assistive tools.

[0148] Therefore, it is necessary to filter out sentiment-laden evaluation sentences through sentiment analysis, and then summarize user reviews of different sentiment polarities to summarize users' positive and negative evaluations of different tools in different aspects. Based on the above idea, the auxiliary tool review summarization method proposed in this invention, after extracting aspects from the evaluation sentences of each auxiliary tool in the previous step, obtains aspect-level summaries of different aspects of each tool through sentiment analysis and summary extraction.

[0149] First, the proposed method pre-trains BERT on text within the context of assistive tools. This allows the trained language model to better represent the semantics of assistive tool-related terminology, enabling downstream tasks such as classification and clustering to perform classification or clustering based on more accurate semantic information. A schematic diagram of BERT pre-training is shown below (e.g.,...). Figure 5 (As shown).

[0150] refer to Figure 5For auxiliary tools discussing sentences, the sentence is first segmented into words. Then, based on the Masked Language Model, a word in the sentence is randomly replaced with a mask "[Mask]". Subsequently, BERT will predict the word at the mask position to train the language model.

[0151] Next, the masked sequence is passed through an embedding layer to obtain the embedding vector E for each symbol (word). Token Input the BERT model. The BERT model consists of multiple encoding layers of a Transformer model. Based on a self-attention mechanism, the Transformer calculates the attention weights between each symbol (word) in the input sentence and other symbols (words) in the sentence. It then combines this weighted calculation with information from other symbols (words) in the sentence to calculate the weighted output T of the encoding layer for the current symbol. Token Therefore, T Token It contains information about the sentence's context.

[0152] Finally, based on T Token By comparing the similarity with all words in the vocabulary, the probability distribution of the masked words is predicted, which is equivalent to predicting the current word based on the sentence context. The output probability distribution is then compared with the true values ​​(i.e., the probability of the masked word is 1, and the probability of other words is 0) using cross-entropy loss calculation. This cross-entropy loss is then used to update the relevant parameters of the model through negative feedback, forming the model's learning process.

[0153] Pre-training BERT on discussion texts about assistive tools allows the model to improve its accuracy in predicting individual words within the context of such discussions, demonstrating that the trained language model better represents the semantics of assistive tool-related terminology. Firstly, the model can represent terms more accurately based on their context. For example, given the input sequence ["Travis", "enablees", "automate", "test"], where "Travis" is masked, the pre-trained model can more accurately predict "Travis" based on the contextual information of "enables", "automate", and "test". This indicates that the model has learned the connection between "Travis" and contextual information such as "enables", "automate", and "test", meaning it can learn the unique contextual information of terms like assistive tool names and better represent them.

[0154] Secondly, because similar terms in assistive tool discussion texts share similar contexts, the model can learn similar representations (embedding vectors and encoding layer outputs) for similar terms. For example, in the context of assistive tool discussions, assistive tools such as "Travis" and "TeamCity" often appear alongside "enables," "automate," and "test," sharing a similar context. Therefore, the model learns very similar representation vectors for "Travis" and "TeamCity." However, in general contexts, "Travis" often refers to a person's name and would not have a high degree of similarity to "TeamCity."

[0155] After pre-training, this invention uses a BERT-based sentiment analysis method to identify user reviews with sentiment polarity. Then, a BERT-based summary extraction method is used to statistically analyze the sentiment polarity of user reviews on the same aspect of the auxiliary tool and extract summary sentences as aspect-level summaries of the tool.

[0156] In the sentiment analysis phase, the model, based on the language model pre-trained in the previous step and suitable for the context of auxiliary tools, combines linear layers to form a sentiment classifier. The classifier is then trained on a manually labeled auxiliary tool evaluation dataset with sentiment polarity annotations. After fine-tuning, the BERT-based sentiment classifier can be directly used to classify the sentiment polarity of auxiliary tool discussion sentences. A diagram illustrating the fine-tuning and classification process is provided (e.g.,...). Figure 6 (As shown).

[0157] refer to Figure 6 BERT discusses the sentence for each auxiliary tool, and after serialization, adds the "[CLS]" symbol to the sequence. The "[CLS]" symbol is then used by the BERT model to calculate the encoding layer output T. [CLS] By combining the semantic information of other symbols (words) in the sentence, the semantic information of the current sentence is implicitly included, and it can be used as the sentence embedding vector of the current sentence for further classification.

[0158] Next, a three-class classification of sentiment is performed on the sentence embedding vectors using a linear layer and a softmax layer to determine whether the user's sentiment polarity in the current auxiliary tool evaluation sentence is positive, negative, or neutral. Cross-entropy loss is calculated based on the sentiment polarity-labeled data, and the linear layer classifier is trained through backpropagation to improve the sentiment classification performance.

[0159] After sentiment analysis, this invention can obtain user review sentences with sentiment polarity for each tool and aspect. In the summarization stage, for auxiliary tool i, extractive summarization is performed on the review sentences with positive and negative sentiment for each aspect.

[0160] In the summarization stage, the model first calculates the sentence embedding vector for each input sentence based on the BERT model, and then obtains clusters in the sentence set using the K-means method, extracting the cluster centers as the summary sentences. Building upon the method in that paper, to better represent the semantics of assistive tool-related technical terms, this invention also implements a summarization extraction method for assistive tool reviews based on BERT pre-trained on assistive tool discussion texts. A schematic diagram of the method is shown below (e.g., ...). Figure 7 (As shown).

[0161] First, for user reviews of auxiliary tool i (a), sentences expressing positive and negative sentiments are summarized based on the sentiment analysis results from the previous step. For each auxiliary tool review sentence S... i The language model obtained through the previous pre-training step, suitable for the context of auxiliary tools, yields the corresponding sentence embedding vectors. This sentence embedding vector implicitly contains the semantic information of the current sentence.

[0162] Subsequently, for user reviews of the same sentiment polarity in aspect a of auxiliary tool i, the K-means clustering algorithm was used to cluster the sentences. Clustering is a common method for review summarization, used to extract more diverse summary sentences from different clusters. The K-means clustering algorithm randomly selects k samples as initial cluster centers (c1, c2, ..., c...). k The number of clusters, k, is manually specified, and the iteration is then performed based on the following steps:

[0163] For each sentence in the sentence set Based on formula Calculate its Euclidean distance to the k cluster centers and classify the sentence into the nearest cluster.

[0164] For each cluster C j Based on the formula Update cluster C j The center.

[0165] Clustering algorithms often require manual specification of the appropriate number of clusters. To improve clustering performance based on actual data, this project uses the Calinski-Harabaz (CH) coefficient to evaluate the quality of clustering results obtained with different numbers of clusters for each group of sentences. The group with the best evaluation quality is selected to dynamically determine the number of clusters based on the actual data. The Calinski-Harabaz (CH) coefficient calculates the difference between within-group and between-group covariance. For good clustering results, smaller covariance within a category is better, while larger covariance between categories is better.

[0166] Finally, based on the clustering results, each sentence in the cluster is traversed, the Euclidean distance between the sentence embedding vector and the current cluster center is calculated, and the sentence closest to the cluster center in the cluster is selected as the summary sentence of the current cluster. The summary sentences of all clusters in the sentence set are the extractive summary of aspect a of the current auxiliary tool i.

[0167] In summary, this invention achieves aspect-level summarization of user reviews from different tools through two stages: aspect extraction and comment summary.

[0168] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0169] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for analyzing comments on an auxiliary tool, characterized in that, include: An aspect extraction model is used to extract aspects from the evaluation sentences of the auxiliary tool to obtain evaluation sentences categorized by aspect. This process includes: calculating the probability distribution of each evaluation sentence belonging to each aspect based on aspect-related keywords; and... The BERT model is used to perform sentiment classification and summary extraction on aspect-based evaluation sentences to obtain aspect-level summaries of different aspects of the auxiliary tool. The calculation of the probability distribution of the evaluation sentence belonging to each aspect based on aspect-related keywords further includes: calculating a sentence vector using word vectors based on aspect information attention; calculating an aspect embedding vector by weighted summation of each keyword vector in the aspect based on keyword attention; obtaining a sentence reconstruction vector by weighted summation of all aspect embedding vectors, wherein the aspect extraction model is trained by reducing the inner product between the sentence vector and the sentence reconstruction vector; and predicting the probability distribution of the evaluation sentence belonging to each aspect using the aspect extraction model.

2. The comment analysis method for auxiliary tools according to claim 1, characterized in that, Aspect-based attention, utilizing word vectors to calculate sentence vectors, further includes: The word vectors are obtained by multiplying the word set and the word vector matrix, and the keyword vectors are obtained by multiplying the keyword set of aspects and the word vector matrix, wherein the word set is generated based on the evaluation sentences of the auxiliary tool; Based on the aforementioned information, attention calculates word attention weights using the similarity between the word vectors and the keyword vectors, wherein the similarity between the word vectors and keyword vectors is obtained by multiplying the word vectors, the attention parameter matrix, and the summed average of the keyword vectors; and The sentence vector is obtained by weighted summation of the word vectors and the word attention weights.

3. The comment analysis method for auxiliary tools according to claim 2, characterized in that, The aspect embedding vector is calculated by weighted summation of each keyword vector in the aspect based on keyword attention, and further includes: Based on TF-IDF, extract m keyword sets from evaluation sentences of K aspects, where K and m are both positive integers; Attention weights for keywords are calculated based on the similarity between the keyword vector and the sentence vector, wherein the similarity between the keyword vector and the sentence vector is obtained by multiplying the keyword vector, the keyword attention parameter matrix, and the current sentence vector; and The aspect embedding vector is calculated by weighting and summing the aspect vectors of each keyword vector based on the attention weights of the keywords.

4. The comment analysis method for auxiliary tools according to claim 1, characterized in that, The linear layer in the model is used to predict the probability distribution of the evaluation sentence belonging to each aspect; The aspect extraction model calculates the sentence reconstruction vector based on the probability distribution of the evaluation sentence belonging to each aspect and the corresponding aspect embedding vector.

5. The comment analysis method for auxiliary tools according to claim 1, characterized in that, The evaluation criteria for the auxiliary tools include compatibility, ease of learning, functionality, performance, reliability, and ease of use.

6. The comment analysis method for auxiliary tools according to claim 1, characterized in that, Using the BERT model for sentiment analysis and summary extraction to obtain aspect-level summaries of different aspects of the auxiliary tools further includes: Based on the BERT model, evaluation sentences categorized by aspect are classified into evaluation sentences with sentiment polarity, wherein the sentiment polarity includes positive, negative, or neutral; and Extract summary sentences from evaluative sentences with positive and negative emotional polarities, and use them as aspect-level summaries for the aforementioned auxiliary tool.

7. A comment analysis device for an auxiliary tool, characterized in that, include: An aspect classification module is used to extract aspects from the evaluation sentences of the auxiliary tool using an aspect extraction model to obtain evaluation sentences of the auxiliary tool classified by aspect. The aspect classification module is further used to calculate the probability distribution of the evaluation sentences belonging to each aspect based on aspect-related keywords. The sentiment analysis module is used to perform sentiment classification on aspect-based evaluation sentences using the BERT model; and The summary extraction module is used to extract summaries from the evaluation sentences of sentiment classification to obtain aspect-level summaries of different aspects of the auxiliary tool; The aspect classification module further includes: a sentence vector calculation module, an aspect embedding vector calculation module, and a sentence reconstruction module. The sentence vector calculation module is used to calculate sentence vectors using word vectors based on aspect information attention. The aspect embedding vector calculation module is used to calculate aspect embedding vectors by weighted summation of each keyword vector in the aspect based on keyword attention. The sentence reconstruction module is used to obtain a sentence reconstruction vector by weighted summation of all aspect embedding vectors, wherein the aspect extraction model is trained by reducing the inner product between the sentence vectors and the sentence reconstruction vectors, and the aspect extraction model is used to predict the probability distribution of the evaluation sentence belonging to each aspect.

8. The comment analysis device for auxiliary tools according to claim 7, characterized in that, The sentence vector calculation module further includes: a word vector submodule, a keyword vector submodule, a word attention submodule, and a sentence vector submodule, wherein... The word vector submodule is used to obtain the word vectors by multiplying the word set and the word vector matrix, wherein the word set is generated based on the evaluation sentences of the auxiliary tool; The keyword vector submodule is used to obtain keyword vectors by multiplying the keyword set of an aspect with the word vector matrix; The word attention submodule is used to calculate word attention weights based on the similarity between the word vectors and the keyword vectors, using the aspect information attention. The similarity between the word vectors and keyword vectors is obtained by multiplying the word vectors, the attention parameter matrix, and the summation average of the keyword vectors. The sentence vector submodule is used to obtain the sentence vector by weighted summation of the word vector and the word attention weight.

Images