User evaluation triple recognition method and system based on large model and storage medium

Abstract

The invention provides a user evaluation triple recognition method and system based on a large model and a storage medium, and the method comprises the steps: carrying out the quality screening of collected comment data through employing a first large language model, carrying out the meaning group segmentation of screened high-quality comments through employing a second large language model, and carrying out the meaning group segmentation of the screened high-quality comments; performing sentiment analysis and evaluation word extraction on each meaning group fragment, and summarizing all evaluation words in combination with semantic information to obtain evaluation dimensions; performing preliminary clustering on the evaluation dimensions by using a phrase clustering model, and performing cleaning and naming normalization on a preliminary clustering result; carrying out secondary clustering and normalization on the redefined evaluation dimension; re-classifying the outlier dimensions removed in the two clustering processes, and incorporating the re-classified outlier dimensions into secondary clustering and normalized evaluation dimensions; finally, the evaluation dimensions and the corresponding core evaluation words and emotional tendencies are jointly stored as a comment triple; according to the method, the efficiency and accuracy of triple recognition evaluation can be improved, and meanwhile, the dependence of manual intervention is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and more specifically, to a method, system and storage medium for identifying user evaluation triples based on a large model. Background Art

[0002] In the current information age, user reviews have become an important data source for product and service quality evaluation. These reviews contain rich user feedback information, which is of great value for enterprises to conduct market analysis, improve product quality and increase customer satisfaction. In user review analysis, evaluation triples, as an effective representation method, usually include three main elements: evaluation dimension, evaluation word and sentiment tendency. The evaluation dimension is used to define the theme or aspect of the review, the evaluation word specifically describes the evaluation of the dimension, and the sentiment tendency represents the emotional attitude of the review.

[0003] Currently, the identification of evaluation triples mainly relies on the following two solutions:

[0004] 1) Manual dimension code table: This solution relies on manually compiled evaluation dimensions and evaluation word lists. It manually collects and organizes evaluation words and uses regular expressions and other rules for matching, thereby labeling the review data with corresponding dimension labels. The dimension framework of this solution can be adjusted and updated at any time as needed, which is highly flexible, but it requires a lot of manpower to collect evaluation words and write matching logic, which is inefficient. As the number of evaluation dimensions increases, the maintenance and update costs will also increase significantly. In addition, due to the limitations of manual coding, the evaluation dimensions may not be comprehensive enough, resulting in low accuracy and coverage of the recognition results.

[0005] 2) Intelligent triplet model: This solution is usually based on deep learning technology, especially pre-trained models such as BERT. It trains the model by annotating a large amount of training data, thereby realizing automatic recognition of evaluation triples. Although the model obtained through training can automatically identify evaluation triples, it reduces human intervention and can provide higher recognition accuracy than manual methods. However, a large amount of annotated data is required to train the model, and the data annotation cost is high and the development cycle is long. In addition, once the evaluation dimension framework is determined, it is difficult to modify and expand it, and its flexibility is relatively low.

[0006] In addition, user comments are very diverse in expression, and new content emerges every once in a while. The existing manual dimensional code table is very prone to mismatching and recalling irrelevant data. Although the intelligent triplet model can improve recognition accuracy, it lacks generalization for new user expressions. Summary of the invention

[0007] In order to overcome the defects of the above-mentioned prior art that manual recognition is incomplete and inefficient, and that intelligent model recognition has high cost, poor flexibility and low accuracy, the present invention provides a user evaluation triplet recognition method, system and storage medium based on a large model. Based on a large language model, fully automated evaluation triplet recognition can be achieved, reducing dependence on manual labor, improving recognition efficiency and accuracy, and enhancing the flexibility and adaptability of dimensional framework modification.

[0008] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0009] A method for identifying user evaluation triples based on a large model comprises the following steps:

[0010] S1: Collect user comment data from multiple platforms on the Internet and build a comment dataset;

[0011] S2: setting a quality screening condition for user comment data, and using the pre-trained first language model to screen the comment data set to obtain a high-quality comment data set;

[0012] S3: using the pre-trained second largest language model to segment the high-quality comment data set into semantic groups, and segmenting each high-quality user comment data into a plurality of semantic group segments;

[0013] S4: using the pre-trained second language model to perform sentiment analysis on each of the phrase segments to obtain the sentiment tendency of each phrase segment; and extracting core evaluation words from each phrase segment, and summarizing the extracted core evaluation words in combination with the semantic information of the corresponding phrase segment to extract specific evaluation dimensions;

[0014] S5: Use the pre-trained phrase clustering model to perform preliminary clustering on all the extracted evaluation dimensions according to similarity to obtain preliminary clustering results;

[0015] S6: using the pre-trained second language model to clean the preliminary clustering results, remove outlier dimensions that do not meet the preset conditions, redefine the evaluation dimensions of the cleaned preliminary clustering results, normalize the names of all evaluation dimensions, and obtain the redefined preliminary clustering results;

[0016] S7: repeating steps S5 to S6, performing secondary clustering and naming normalization on the redefined preliminary clustering results, and obtaining the redefined secondary clustering results;

[0017] S8: Reclassify the outlier dimensions removed in the two clustering processes and incorporate them into the redefined secondary clustering results to construct the final evaluation dimension framework;

[0018] The core evaluation words and sentiment tendency of each idea group segment are matched with the final evaluation dimension framework to obtain the evaluation triplet of each idea group segment, thereby completing the identification of the user evaluation triplet.

[0019] Preferably, in step S2, the quality screening conditions of the user comment data are set to include: any one of comment length, grammatical completeness, comment paragraph density and richness of the included objects.

[0020] Preferably, in step S3, the idea group segment is specifically a segment expressing a specific evaluation dimension in the high-quality user review data, and each idea group segment includes at least one evaluation word;

[0021] The semantic structure analysis function of the second largest language model is used to segment the meaning groups.

[0022] Preferably, in step S4, the sentiment tendency includes: any one of positive, neutral and negative;

[0023] The core evaluation words include: evaluation attribute words and evaluation description words;

[0024] The evaluation attribute words are used to reflect the main evaluation dimensions of the review; the evaluation description words are used to describe the characteristics or status of the evaluation dimensions;

[0025] The sentiment classification function of the second largest language model is used to perform sentiment analysis; the vocabulary extraction function of the second largest language model is used to extract core evaluation words; the semantic understanding function of the second largest language model is used to obtain the semantic information of each of the meaning groups, and to refine the evaluation dimensions.

[0026] Preferably, in step S6, the outlier dimensions that do not meet the preset conditions include: evaluation dimensions that cannot match all core evaluation words, and evaluation dimensions that do not meet preset definition specifications.

[0027] Preferably, the first large language model has a smaller number of parameters than the second large language model.

[0028] Preferably, the first large language model is a quantized mini version of the SocialGPT large language model;

[0029] The second largest language model is specifically the SocialGPT large language model;

[0030] The phrase clustering model is specifically a BGE-M3 semantic vector model.

[0031] Preferably, the parameter amount of the first largest language model is specifically 0.2B; the parameter amount of the second largest language model is specifically 32B.

[0032] The present invention also provides a user evaluation triplet recognition system based on a large model, which uses the above-mentioned user evaluation triplet recognition method based on a large model, including:

[0033] Comment collection unit: used to collect user comment data from multiple platforms on the Internet and build a comment dataset;

[0034] A comment screening unit: used to set the quality screening conditions of the user comment data, and use the pre-trained first language model to screen the comment data set to obtain a high-quality comment data set;

[0035] A comment segmentation unit is used to segment the high-quality comment data set into semantic groups using the pre-trained second language model, and to segment each high-quality user comment data into a plurality of semantic group segments;

[0036] Sentiment analysis and dimension extraction unit: used to perform sentiment analysis on each of the phrase segments using the pre-trained second language model to obtain the sentiment tendency of each phrase segment; and extract core evaluation words from each phrase segment, and summarize the extracted core evaluation words in combination with the semantic information of the corresponding phrase segment to extract specific evaluation dimensions;

[0037] Clustering unit: used to use the pre-trained phrase clustering model to perform preliminary clustering on all extracted evaluation dimensions according to similarity and obtain preliminary clustering results;

[0038] A naming normalization unit is used to clean the preliminary clustering results using a pre-trained second language model, remove outlier dimensions that do not meet preset conditions, redefine the evaluation dimensions of the cleaned preliminary clustering results, normalize the names of all evaluation dimensions, and obtain the redefined preliminary clustering results;

[0039] Secondary normalization unit: used to repeatedly execute the clustering unit and the naming normalization unit, perform secondary clustering and naming normalization on the redefined preliminary clustering results, and obtain the redefined secondary clustering results;

[0040] Evaluation triplet identification unit: used to reclassify the outlier dimensions removed in the two clustering processes and incorporate them into the redefined secondary clustering results to construct the final evaluation dimension framework;

[0041] The core evaluation words and sentiment tendency of each idea group segment are matched with the final evaluation dimension framework to obtain the evaluation triplet of each idea group segment, thereby completing the identification of the user evaluation triplet.

[0042] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are implemented.

[0043] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0044] The present invention provides a user evaluation triple identification method, system and storage medium based on a large model. The scheme of the present invention adopts a three-stage process design, namely data screening, dimension generation and dimension normalization. After collecting the comment data, the first large language model is used to perform data quality screening; then the second large language model is used to segment the screened data into phrases, sentiment analysis is performed on each phrase segment, core evaluation words are extracted from each phrase segment, and the extracted evaluation words are summarized in combination with the semantic information of the phrase segment to extract specific evaluation dimensions; then the phrase clustering model is used to perform preliminary clustering on the evaluation dimensions, and the preliminary clustering results are cleaned; after cleaning, the evaluation dimensions are redefined, and then the redefined evaluation dimensions are secondary clustered and normalized; the outlier dimensions eliminated in the two clustering processes are reclassified and incorporated into the evaluation dimensions after secondary clustering and normalization; finally, the evaluation dimensions and their corresponding core evaluation words and sentiment tendencies are jointly saved as comment triples;

[0045] The present invention has the following advantages:

[0046] 1) Significantly improve development efficiency: The present invention can fully automatically complete the recognition of evaluation triples, including dimension generation, evaluation word extraction and sentiment analysis, reducing the manual intervention in traditional methods, making it more efficient to extract valuable evaluation information from a large number of user comments; through a fully automated evaluation triple recognition solution, the present invention replaces the time-consuming manual operation in the traditional method with automatic processing, greatly shortening the time from comment data to evaluation triple generation; specifically, the present invention can shorten the triple recognition cycle from the traditional several months to a few days, greatly improving the processing speed;

[0047] 2) Reduce labor costs: The automated process of the present invention significantly reduces the reliance on manual intervention and reduces labor costs; through the automatic processing and dimensional design of large language models, enterprises do not need a large amount of human resources for data annotation or code table collection, thus saving manpower and economic costs;

[0048] 3) Improve recognition accuracy: The present invention combines two large language models in the data processing flow, one for data screening and preprocessing, which has the advantage of high computational efficiency; the other is responsible for in-depth semantic analysis and dimension generation, ensuring high-precision recognition results; this combination improves processing efficiency while maintaining a high level of model performance; using the powerful semantic understanding and analysis capabilities of the large language model, the present invention can accurately extract evaluation dimensions and evaluation words, improve the accuracy and reliability of recognition results, and thus make review analysis more accurate;

[0049] 4) The dimension normalization process of the present invention includes two rounds of processing, namely, preliminary clustering and secondary clustering, to ensure the accuracy and consistency of the evaluation dimensions; the preliminary clustering is used to merge similar dimensions, while the secondary clustering further optimizes the dimension framework, removes redundant dimensions and ensures a clear definition of the final dimensions; this dual processing strategy improves the accuracy of dimension normalization and the reliability of the system;

[0050] 5) The present invention also introduces a reclassification mechanism for outlier dimensions, analyzes and reclassifies the outlier dimensions generated in the two-round clustering process, ensures the matching of each dimension with the evaluation word, and classifies them into the optimized dimension system; this mechanism solves the problem of inconsistent dimensions and improves the comprehensiveness and accuracy of the dimension framework;

[0051] 6) Enhanced dimension flexibility: The dynamic dimension generation mechanism of the present invention enables the evaluation dimensions to be quickly adjusted according to changes in actual data and application scenarios, overcoming the problem that the traditional triple model has poor flexibility and cannot be updated in a timely manner; at the same time, due to the adaptability and flexibility of the large language model, the present invention can process data from different fields, including but not limited to different industries and different framework requirements; this enables the technical solution of the present invention to produce excellent results in a wide range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of a method for identifying user evaluation triples based on a large model provided in Example 1.

[0053] Figure 2 This is a framework diagram of a large-model-based user evaluation triplet identification method provided in Example 2.

[0054] Figure 3 This is a structural diagram of a user evaluation triplet recognition system based on a large model provided in Example 3. DETAILED DESCRIPTION

[0055] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0056] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0057] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0058] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0059] Example 1

[0060] like Figure 1 As shown, this embodiment provides a method for identifying user evaluation triples based on a large model, comprising the following steps:

[0061] S1: Collect user comment data from multiple platforms on the Internet and build a comment dataset;

[0062] S2: setting a quality screening condition for user comment data, and using the pre-trained first language model to screen the comment data set to obtain a high-quality comment data set;

[0063] S3: using the pre-trained second largest language model to segment the high-quality comment data set into semantic groups, and segmenting each high-quality user comment data into a plurality of semantic group segments;

[0064] S4: using the pre-trained second language model to perform sentiment analysis on each of the phrase segments to obtain the sentiment tendency of each phrase segment; and extracting core evaluation words from each phrase segment, and summarizing the extracted core evaluation words in combination with the semantic information of the corresponding phrase segment to extract specific evaluation dimensions;

[0065] S5: Use the pre-trained phrase clustering model to perform preliminary clustering on all the extracted evaluation dimensions according to similarity to obtain preliminary clustering results;

[0066] S6: using the pre-trained second language model to clean the preliminary clustering results, remove outlier dimensions that do not meet the preset conditions, redefine the evaluation dimensions of the cleaned preliminary clustering results, normalize the names of all evaluation dimensions, and obtain the redefined preliminary clustering results;

[0067] S7: repeating steps S5 to S6, performing secondary clustering and naming normalization on the redefined preliminary clustering results, and obtaining the redefined secondary clustering results;

[0068] S8: Reclassify the outlier dimensions removed in the two clustering processes and incorporate them into the redefined secondary clustering results to construct the final evaluation dimension framework;

[0069] The core evaluation words and sentiment tendency of each idea group segment are matched with the final evaluation dimension framework to obtain the evaluation triplet of each idea group segment, thereby completing the identification of the user evaluation triplet.

[0070] In the specific implementation process, the main purpose of this method is to achieve fully automated evaluation triple recognition by combining two large language models, reduce dependence on manual work, improve recognition results, and provide flexible dimensional framework design capabilities;

[0071] This method includes three main stages: data screening, dimension generation and dimension normalization; the process of each stage will be described in detail below;

[0072] 1) Data collection and screening:

[0073] Data collection: First, crawl user comment data from multiple platforms on the Internet in compliance with regulations through crawler technology to build a comment data set. In this embodiment, the user comment data are all comment data of specific industries or brands;

[0074] Data quality assessment: In the data screening stage, the quality of the review data must first be assessed to ensure the accuracy of subsequent processing and reduce subsequent resource consumption. Use a small-parameter LLM model (such as a fine-tuning model with 0.2B parameters) to set data quality assessment criteria based on review density, object richness, and other conditions. The small-parameter LLM model is used as a data quality screening tool in this step, which can effectively filter out high-quality review data and avoid the negative impact of low-quality data on the final effect.

[0075] 2) Dimension generation:

[0076] Segmentation of concept segments: Segment the filtered review data into concept segments; concept segments are segments with independent meanings in reviews, and each concept segment is required to be concentrated on one evaluation dimension. Use a large number of parameterized LLM models (such as the 32B fine-tuning model) to analyze the semantic structure of the reviews and segment the reviews into concept segments with clear semantics. This process can accurately identify and divide the main evaluation core of the reviews.

[0077] Determine the sentiment of the comments: Perform sentiment analysis on each phrase segment to determine whether its sentiment tendency is positive, negative, or neutral. Sentiment analysis is one of the key steps in evaluating triple recognition. Through the sentiment classification capability of the large language model, accurate sentiment labels can be assigned to each phrase segment.

[0078] Extract evaluation words: Extract core evaluation words from each phrase segment; evaluation words are divided into attribute words and description words, where attribute words reflect the main evaluation dimension of the review (such as "comfort"), and description words describe the characteristics or status of the evaluation dimension (such as "very good"); through the vocabulary extraction function of the LLM model with a large number of parameters, relevant evaluation words can be accurately extracted from the phrase segment and classified;

[0079] Extracting evaluation dimensions: Combine the semantic information of the meaning group fragments to summarize the extracted evaluation words and extract specific evaluation dimensions. The dimension extraction process uses the semantic understanding ability of the large language model to map the evaluation words into a structured dimension framework to form a hierarchical evaluation dimension.

[0080] 3) Dimension normalization:

[0081] Evaluation dimension clustering: Perform preliminary clustering on the extracted evaluation dimensions, merge the dimensions using the phrase clustering model, and set a loose threshold in the preliminary clustering process to preliminarily classify similar dimensions into one category and associate the evaluation words with these clustering results; this process aims to reduce redundant dimensions and improve the aggregation effect of dimensions;

[0082] Cleaning of clustering results: After the initial clustering, the clustering results need to be cleaned to remove outlier dimensions that do not meet the specifications; this includes dimensions that obviously do not match the evaluation words or do not meet the definition specifications; the cleaning process helps to improve the accuracy and consistency of the dimension framework;

[0083] Redefining evaluation dimensions: Based on the cleaned clustering results, a unified dimension label is redefined for each class; for example, "wearing experience-comfort-softness", "product fabric-fabric experience-softness", and dimensions with the same semantics in this class are merged into a unified "wearing experience-comfort-softness" dimension; the redefinition process ensures the consistency and accuracy of the dimension labels;

[0084] Secondary clustering and normalization of evaluation dimensions: Perform secondary clustering and normalization on the redefined dimensions to ensure that the final dimensions are more accurate. In this step, set a more stringent and accurate threshold for the second round of clustering. After the secondary clustering, perform secondary cleaning and secondary redefinition to further optimize the accuracy and consistency of the dimensions.

[0085] Reclassification of outlier dimensions: reclassify the outlier dimensions removed during the two rounds of cleaning and include them in the evaluation dimensions after the second normalization; exclude the dimensions that cannot be classified to ensure that the final dimension framework has a high degree of integrity and consistency;

[0086] This method can reduce manual intervention and improve efficiency through automated model training and data processing processes. At the same time, this method can also improve recognition effects: by utilizing the powerful semantic understanding capabilities of the large language model, the accuracy and recall rate of evaluation triple recognition are improved. In addition, this method can also enhance dimensional flexibility and realize automatic generation and dynamic adjustment of dimensional frameworks to meet the needs of different fields and application scenarios.

[0087] Example 2

[0088] This embodiment provides a method for identifying user evaluation triples based on a large model, comprising the following steps:

[0089] S1: Collect user comment data from multiple platforms on the Internet and build a comment dataset;

[0090] S2: setting a quality screening condition for user comment data, and using the pre-trained first language model to screen the comment data set to obtain a high-quality comment data set;

[0091] S3: using the pre-trained second largest language model to segment the high-quality comment data set into semantic groups, and segmenting each high-quality user comment data into a plurality of semantic group segments;

[0092] S4: using the pre-trained second language model to perform sentiment analysis on each of the phrase segments to obtain the sentiment tendency of each phrase segment; and extracting core evaluation words from each phrase segment, and summarizing the extracted core evaluation words in combination with the semantic information of the corresponding phrase segment to extract specific evaluation dimensions;

[0093] S5: Use the pre-trained phrase clustering model to perform preliminary clustering on all the extracted evaluation dimensions according to similarity to obtain preliminary clustering results;

[0094] S6: using the pre-trained second language model to clean the preliminary clustering results, remove outlier dimensions that do not meet the preset conditions, redefine the evaluation dimensions of the cleaned preliminary clustering results, normalize the names of all evaluation dimensions, and obtain the redefined preliminary clustering results;

[0095] S7: repeating steps S5 to S6, performing secondary clustering and naming normalization on the redefined preliminary clustering results, and obtaining the redefined secondary clustering results;

[0096] S8: Reclassify the outlier dimensions removed in the two clustering processes and incorporate them into the redefined secondary clustering results to construct the final evaluation dimension framework;

[0097] Match the core evaluation words and sentiment tendency of each idea group segment with the final evaluation dimension framework, obtain the evaluation triplet of each idea group segment, and complete the identification of the user evaluation triplet;

[0098] In step S2, the quality screening conditions of the user comment data are set to include: any one of comment length, grammatical completeness, comment paragraph density and included object richness;

[0099] In step S3, the idea group segment is specifically a segment expressing a specific evaluation dimension in the high-quality user review data, and each idea group segment includes at least one evaluation word;

[0100] Using the semantic structure analysis function of the second largest language model to segment the meaning groups;

[0101] In step S4, the sentiment tendency includes: any one of positive, neutral and negative;

[0102] The core evaluation words include: evaluation attribute words and evaluation description words;

[0103] The evaluation attribute words are used to reflect the main evaluation dimensions of the review; the evaluation description words are used to describe the characteristics or status of the evaluation dimensions;

[0104] Using the sentiment classification function of the second largest language model to perform sentiment analysis; using the vocabulary extraction function of the second largest language model to extract core evaluation words; using the semantic understanding function of the second largest language model to obtain semantic information of each of the meaning group segments, and to refine the evaluation dimensions;

[0105] In step S6, the outlier dimensions that do not meet the preset conditions include: evaluation dimensions that cannot match all core evaluation words, and evaluation dimensions that do not meet the preset definition specifications;

[0106] The parameter amount of the first largest language model is smaller than that of the second largest language model; in this embodiment, the first largest language model is specifically a quantized mini version of the SocialGPT large language model, and the parameter amount is specifically 0.2B;

[0107] The second largest language model is specifically the SocialGPT large language model, and the number of parameters is specifically 32B;

[0108] The phrase clustering model is specifically a BGE-M3 semantic vector model.

[0109] In the specific implementation process, Figure 2 As shown, this method includes three main stages: data screening, dimension generation and dimension normalization; the process of each stage will be described in detail below;

[0110] 1) Data collection and screening:

[0111] Data collection: First, crawl user comment data from multiple social media and e-commerce platforms on the Internet in compliance with regulations through crawler technology to build a comment data set. In this embodiment, the user comment data are all comment data about footwear and clothing products;

[0112] Data quality assessment: In the data screening stage, the comment data needs to be evaluated and screened first to ensure the accuracy of subsequent processing and reduce subsequent resource consumption; use the LLM model with a small number of parameters (the model used in this embodiment is a mini version obtained by quantizing the self-developed SocialGPT, with a parameter of 0.2B), and set the data quality assessment standard according to the density of comments, object richness and other conditions; the LLM model with a small number of parameters is used as a data quality screening tool in this step, which can effectively filter out high-quality comment data and avoid the negative impact of low-quality data on the final effect; as shown in Table 1, Table 1 is an example of the original comment data and the filtered data, where " / " means discarding the corresponding comment data;

[0113] Table 1 Examples of original review data and filtered data

[0114] Raw data example Example of filtered data The comfort of these shoes is very good and the price is very reasonable. The comfort of these shoes is very good and the price is very reasonable. good / The shoes have a stylish design but the soles are very hard and not comfortable enough. The shoes have a stylish design but the soles are very hard and not comfortable enough. Someone recommended / Not worth the price. Not worth the price. The pants have a good fit and are very comfortable to wear. The pants have a good fit and are very comfortable to wear.

[0115] As can be seen from Table 1, through data screening, some low-quality comments, such as those with ambiguity or insufficient information, were removed, ensuring that the data quality for subsequent processing was high;

[0116] 2) Dimension generation:

[0117] Segmentation of concept segments: segment the filtered comment data into concept segments; concept segments are segments with independent meanings in comments, and each concept segment is required to be concentrated on one evaluation dimension; use the LLM model with a large number of parameters (in this embodiment, the self-developed SocialGPT model is fine-tuned on a specific tag recognition task, with a parameter size of 32B) to analyze the semantic structure of the comments and segment the comments into concept segments with clear semantics. This process can accurately identify and divide the main evaluation core of the comments;

[0118] Determine the sentiment of the comments: Perform sentiment analysis on each phrase segment to determine whether its sentiment tendency is positive, negative, or neutral. Sentiment analysis is one of the key steps in evaluating triple recognition. Through the sentiment classification capability of the large language model, accurate sentiment labels can be assigned to each phrase segment.

[0119] Extract evaluation words: Extract core evaluation words from each phrase segment; evaluation words are divided into attribute words and description words, where attribute words reflect the main evaluation dimensions of the review (such as "comfort" and "style design"), and description words describe the characteristics or status of the evaluation dimensions (such as "very good", "very", and "fashionable"); through the vocabulary extraction function of the LLM model with a large number of parameters, relevant evaluation words can be accurately extracted from the phrase segment and classified;

[0120] Extracting evaluation dimensions: Combine the semantic information of the meaning group fragments, summarize the extracted evaluation words, and extract specific evaluation dimensions; the dimension extraction process uses the semantic understanding ability of the large language model to map the evaluation words into a structured dimension framework to form a hierarchical evaluation dimension. Table 2 shows an example of data after the dimension generation stage.

[0121] Table 2 Data example after dimension generation phase

[0122] Sense Group Fragment Emotional tendency Core evaluation words Evaluation Dimensions Very good comfort front Comfort level #Very good Product experience-user experience-comfort The price is also very reasonable front Price is very reasonable Price / Promotion-Price-Price Satisfaction Shoes design fashion front Design#Fashion Product Design-Appearance Design-Style Design The sole is very hard and not comfortable enough Negative Comfortable #soles are very hard User experience - comfort - not comfortable enough Not worth the price Negative Price #Not worth it Price-Value Good fit of pants front Version#Good Pattern design Very comfortable to wear front Comfortable #wear Wearing experience-wearing comfort-comfort

[0123] As can be seen from Table 2, in the dimension generation stage, the deep semantic analysis and meaning group segmentation of the large-parameter LLM model can effectively extract the information of different dimensions in the comments. At the same time, the classification of evaluation words and the refinement of dimensions enable the core information of each comment to be accurately identified and summarized.

[0124] 3) Dimension normalization:

[0125] Evaluation dimension clustering: The extracted evaluation dimensions are preliminarily clustered, and the dimensions are merged using a phrase clustering model (the BGE-M3 semantic vector model fine-tuned with comment phrases is used in this embodiment), and a loose threshold is set in the preliminary clustering process to ensure a wide range of clustering. This step is used to preliminarily classify similar dimensions into one category and associate evaluation words with these clustering results; this process aims to reduce redundant dimensions and improve the aggregation effect of dimensions; Table 3 shows an example of preliminary clustering results;

[0126] Table 3. Examples of preliminary clustering results

[0127]

[0128]

[0129] Clustering result cleaning: After the initial clustering, the clustering results need to be cleaned. Each clustering result needs to be analyzed to find out and remove outlier dimensions that do not meet the specifications or that obviously do not match the evaluation words. The cleaning process helps to improve the accuracy and consistency of the dimension framework. Table 4 shows an example of the initial clustering results after cleaning.

[0130] Table 4 Example of preliminary clustering results after cleaning

[0131]

[0132] Redefining evaluation dimensions: Based on the cleaned clustering results, a unified dimension label is redefined for each class; for example, "wearing experience-comfort-softness", "product fabric-fabric experience-softness", and dimensions with the same semantics in this class are merged into a unified "wearing experience-comfort-softness" dimension; the redefinition process ensures the consistency and accuracy of the dimension labels;

[0133] Secondary clustering and normalization of evaluation dimensions: The redefined dimensions are subjected to secondary clustering and normalization to ensure that the final dimensions are more accurate. In this step, a more stringent and accurate threshold is set for the second round of clustering. After the secondary clustering, secondary cleaning and secondary redefinition are performed to further optimize the accuracy and consistency of the dimensions. Table 5 shows an example of evaluation dimensions after secondary clustering and normalization.

[0134] Table 5 Examples of evaluation dimensions after secondary clustering and normalization

[0135]

[0136] Reclassification of outlier dimensions: reclassify the outlier dimensions removed in the two rounds of cleaning and include them in the evaluation dimensions after the second normalization; exclude the dimensions that cannot be classified to ensure that all evaluation words can be accurately matched to the final dimension system and that the final dimension framework has a high degree of integrity and consistency; Table 6 shows an example of the reclassification results of outlier dimensions;

[0137] Cluster ID Outlier Dimension Evaluation words Normalized dimension name 1 2 Price-Value Price #Not worth it Price / Promotion-Price-Price Satisfaction 3 Pattern design Version#Good Product Design-Appearance Design-Style Design

[0138] Finally, the evaluation triples of each phrase segment are obtained. Table 7 shows an example of the evaluation triples of the phrase segment finally obtained.

[0139] Table 7 Examples of evaluation triples of the final obtained semantic segment

[0140] Sense Group Fragment Emotional tendency Evaluation words Final evaluation dimension Very good comfort front Comfort level #Very good Product experience-wearing experience-comfort The price is also very reasonable front Price is very reasonable Price / Promotion-Price-Price Satisfaction Shoes design fashion front Design#Fashion Product Design-Appearance Design-Style Design The sole is very hard and not comfortable enough Negative Comfortable #soles are very hard Product experience-wearing experience-comfort Not worth the price Negative Price #Not worth it Price / Promotion-Price-Price Satisfaction Good fit of pants front Version#Good Product Design-Appearance Design-Style Design Very comfortable to wear front Comfortable #wear Product experience-wearing experience-comfort

[0141] As can be seen from Table 7, the dimension normalization stage optimizes the classification of evaluation dimensions through preliminary and secondary clustering, making the dimension framework more accurate and consistent; at the same time, the cleaning and redefinition steps ensure the uniformity of each dimension label and the effective matching of evaluation words;

[0142] It is worth mentioning that the first language model, the second language model and the phrase clustering model used in this method do not rely on a specific model. Various commonly used open source and closed source large models can achieve the expected goals and effects, and this embodiment does not make any specific restrictions here.

[0143] This method can reduce manual intervention and improve efficiency through automated model training and data processing processes. At the same time, this method can also improve recognition effects: by utilizing the powerful semantic understanding capabilities of the large language model, the accuracy and recall rate of evaluation triple recognition are improved. In addition, this method can also enhance dimensional flexibility and realize automatic generation and dynamic adjustment of dimensional frameworks to meet the needs of different fields and application scenarios.

[0144] Example 3

[0145] like Figure 3 As shown, this embodiment provides a user evaluation triple identification system based on a large model, and applies a user evaluation triple identification method based on a large model described in Embodiment 1 or 2, including:

[0146] Comment collection unit 301: used to collect user comment data from multiple platforms on the Internet and construct a comment data set;

[0147] The comment screening unit 302 is used to set the quality screening conditions of the user comment data, and use the pre-trained first language model to screen the comment data set to obtain a high-quality comment data set;

[0148] A comment segmentation unit 303 is used to segment the high-quality comment data set into semantic groups using the pre-trained second language model, and to segment each high-quality user comment data into a plurality of semantic group segments;

[0149] Sentiment analysis and dimension extraction unit 304: used to perform sentiment analysis on each of the phrase segments using the pre-trained second language model to obtain the sentiment tendency of each phrase segment; and extract core evaluation words from each phrase segment, and summarize the extracted core evaluation words in combination with the semantic information of the corresponding phrase segment to extract specific evaluation dimensions;

[0150] Clustering unit 305: used to perform preliminary clustering of all extracted evaluation dimensions according to similarity using a pre-trained phrase clustering model to obtain preliminary clustering results;

[0151] The naming normalization unit 306 is used to clean the preliminary clustering results by using the pre-trained second language model, remove the outlier dimensions that do not meet the preset conditions, redefine the evaluation dimensions of the cleaned preliminary clustering results, normalize the names of all evaluation dimensions, and obtain the redefined preliminary clustering results;

[0152] A secondary normalization unit 307 is used to repeatedly execute the clustering unit 305 and the naming normalization unit 306, perform secondary clustering and naming normalization on the redefined preliminary clustering results, and obtain the redefined secondary clustering results;

[0153] Evaluation triplet identification unit 308: used to reclassify the outlier dimensions removed in the two clustering processes and incorporate them into the redefined secondary clustering results to construct the final evaluation dimension framework;

[0154] The core evaluation words and sentiment tendency of each idea group segment are matched with the final evaluation dimension framework to obtain the evaluation triplet of each idea group segment, thereby completing the identification of the user evaluation triplet.

[0155] In the specific implementation process, first, the comment collection unit 301 collects data from multiple platforms on the Internet;

[0156] After collecting the comment data, the comment screening unit 302 first uses the first language model to screen the data quality;

[0157] Then, the comment segmentation unit 303 uses the second largest language model to segment the filtered data into phrases, and the sentiment analysis and dimension extraction unit 304 performs sentiment analysis on each phrase segment, extracts core evaluation words from each phrase segment, and summarizes the extracted evaluation words in combination with the semantic information of the phrase segment to extract specific evaluation dimensions;

[0158] Then the clustering unit 305 performs preliminary clustering of the evaluation dimensions using the phrase clustering model; the naming normalization unit 306 cleans the preliminary clustering results; and after cleaning, the evaluation dimensions are redefined;

[0159] Then, the secondary normalization unit 307 performs secondary clustering and normalization on the redefined evaluation dimensions;

[0160] Finally, the evaluation triplet identification unit 308 reclassifies the outlier dimensions removed in the two clustering processes and incorporates them into the evaluation dimensions after secondary clustering and normalization; the final evaluation dimensions and their corresponding core evaluation words and sentiment tendencies are saved together as comment triplet;

[0161] This system can reduce manual intervention and improve efficiency through automated model training and data processing processes. At the same time, this system can also improve recognition effects: by utilizing the powerful semantic understanding capabilities of large language models, the accuracy and recall rate of evaluation triple recognition are improved. In addition, this system can also enhance dimensional flexibility and realize automatic generation and dynamic adjustment of dimensional frameworks to meet the needs of different fields and application scenarios.

[0162] The same or similar reference numerals correspond to the same or similar components;

[0163] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0164] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for identifying user evaluation triples based on a large model, characterized in that: The following steps are involved: S1: Collect user comment data from multiple platforms on the Internet and build a comment dataset; S2: setting a quality screening condition for user comment data, and using the pre-trained first language model to screen the comment data set to obtain a high-quality comment data set; S3: using the pre-trained second largest language model to segment the high-quality comment data set into semantic groups, and segmenting each high-quality user comment data into a plurality of semantic group segments; S4: using the pre-trained second language model to perform sentiment analysis on each of the phrase segments to obtain the sentiment tendency of each phrase segment; And extract the core evaluation words from each phrase segment, and summarize the extracted core evaluation words in combination with the semantic information of the corresponding phrase segment to extract specific evaluation dimensions; S5: Use the pre-trained phrase clustering model to perform preliminary clustering on all the extracted evaluation dimensions according to similarity to obtain preliminary clustering results; S6: using the pre-trained second language model to clean the preliminary clustering results, remove outlier dimensions that do not meet the preset conditions, redefine the evaluation dimensions of the cleaned preliminary clustering results, normalize the names of all evaluation dimensions, and obtain the redefined preliminary clustering results; S7: repeating steps S5 to S6, performing secondary clustering and naming normalization on the redefined preliminary clustering results, and obtaining the redefined secondary clustering results; S8: Reclassify the outlier dimensions removed in the two clustering processes and incorporate them into the redefined secondary clustering results to construct the final evaluation dimension framework; The core evaluation words and sentiment tendency of each idea group segment are matched with the final evaluation dimension framework to obtain the evaluation triplet of each idea group segment, thereby completing the identification of the user evaluation triplet.

2. According to the large model-based user evaluation triple identification method of claim 1, it is characterized in that: In the step S2, the quality screening conditions of the user comment data are set to include: any one of the comment length, grammatical completeness, comment paragraph density and the richness of the objects included.

3. The method for identifying user evaluation triples based on a large model according to claim 1, characterized in that: In step S3, the idea group segment is specifically a segment expressing a specific evaluation dimension in the high-quality user review data, and each idea group segment includes at least one evaluation word; The semantic structure analysis function of the second largest language model is used to segment the meaning groups.

4. The method for identifying user evaluation triples based on a large model according to claim 1, characterized in that: In step S4, the sentiment tendency includes: any one of positive, neutral and negative; The core evaluation words include: evaluation attribute words and evaluation description words; The evaluation attribute words are used to reflect the main evaluation dimensions of the review; the evaluation description words are used to describe the characteristics or status of the evaluation dimensions; The sentiment classification function of the second largest language model is used to perform sentiment analysis; the vocabulary extraction function of the second largest language model is used to extract core evaluation words; the semantic understanding function of the second largest language model is used to obtain the semantic information of each of the meaning groups, and to refine the evaluation dimensions.

5. The method for identifying user evaluation triples based on a large model according to claim 1, characterized in that: In step S6, the outlier dimensions that do not meet the preset conditions include: evaluation dimensions that cannot match all core evaluation words, and evaluation dimensions that do not meet the preset definition specifications.

6. A method for identifying user evaluation triples based on a large model according to any one of claims 1 to 5, characterized in that: The first largest language model has a smaller number of parameters than the second largest language model.

7. The method for identifying user evaluation triples based on a large model according to claim 6, characterized in that: The first language model is specifically a quantized mini version of the SocialGPT large language model; The second largest language model is specifically the SocialGPT large language model; The phrase clustering model is specifically a BGE-M3 semantic vector model.

8. The method for identifying user evaluation triples based on a large model according to claim 7, characterized in that: The parameter amount of the first largest language model is specifically 0.2B; the parameter amount of the second largest language model is specifically 32B.

9. A user evaluation triplet recognition system based on a large model, applying a user evaluation triplet recognition method based on a large model as claimed in any one of claims 1 to 8, characterized in that: include: Comment collection unit: used to collect user comment data from multiple platforms on the Internet and build a comment dataset; A comment screening unit: used to set the quality screening conditions of the user comment data, and use the pre-trained first language model to screen the comment data set to obtain a high-quality comment data set; A comment segmentation unit is used to segment the high-quality comment data set into semantic groups using the pre-trained second language model, and to segment each high-quality user comment data into a plurality of semantic group segments; Sentiment analysis and dimension extraction unit: used for performing sentiment analysis on each of the phrase segments using a pre-trained second language model to obtain the sentiment tendency of each phrase segment; And extract the core evaluation words from each phrase segment, and summarize the extracted core evaluation words in combination with the semantic information of the corresponding phrase segment to extract specific evaluation dimensions; Clustering unit: used to use the pre-trained phrase clustering model to perform preliminary clustering on all extracted evaluation dimensions according to similarity and obtain preliminary clustering results; A naming normalization unit is used to clean the preliminary clustering results using a pre-trained second language model, remove outlier dimensions that do not meet preset conditions, redefine the evaluation dimensions of the cleaned preliminary clustering results, normalize the names of all evaluation dimensions, and obtain the redefined preliminary clustering results; Secondary normalization unit: used to repeatedly execute the clustering unit and the naming normalization unit, perform secondary clustering and naming normalization on the redefined preliminary clustering results, and obtain the redefined secondary clustering results; Evaluation triplet identification unit: used to reclassify the outlier dimensions removed in the two clustering processes and incorporate them into the redefined secondary clustering results to construct the final evaluation dimension framework; The core evaluation words and sentiment tendency of each idea group segment are matched with the final evaluation dimension framework to obtain the evaluation triplet of each idea group segment, thereby completing the identification of the user evaluation triplet.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 8 are implemented.

Images