Method for generating public opinion report based on search enhancement generation

Abstract

The opinion report generation method based on search enhancement generation relates to the technical field of artificial intelligence, and aims to solve the problem of poor timeliness of existing methods in generating opinion reports. The present application can be applied to analyze the public opinion of recent hot topics in social media platforms. A large amount of news and comment data is collected through a crawler or an API provided by each website. A large language model is used to summarize the collected news and detect the position of the comments. Then, a large language model is used to generate an opinion report. The opinion report generated by the present application has strong timeliness.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for generating public opinion reports based on retrieval-enhanced generation. Background Art

[0002] With the rapid development of social media, some major emergencies can quickly spread and spark discussion. Public opinion reports are text reports focused on analyzing public sentiment. By analyzing and assessing public sentiment within the public opinion field, they examine the causes of public opinion and provide actionable recommendations. With the help of public opinion reports, relevant departments can promptly understand public opinion on specific topics and events, identify potential crises, and more effectively formulate response strategies. Previously, the writing of public opinion reports required a significant amount of manual work. For example, writers would need to retrieve and organize relevant media reports, extract information from them, manually analyze data and write text descriptions, and manually analyze user comments and summarize opinions. All of these steps consume significant time, impacting the timeliness of public opinion reports. Summary of the Invention

[0003] The purpose of the present invention is to propose a method for generating public opinion reports based on retrieval enhancement to address the problem of poor timeliness of public opinion reports generated by existing methods.

[0004] The technical solution adopted by the present invention to solve the above technical problems is:

[0005] The method for generating a public opinion report based on retrieval enhancement generation comprises the following steps:

[0006] Step 1: Get relevant news and comments based on the topic words entered by the user;

[0007] Step 2: Get the news summary and perform stance detection on the comments to obtain the stance of the comments;

[0008] Step 3: Input the summary and comments’ stance into the large language model to generate a public opinion report.

[0009] Furthermore, the specific steps of step one are:

[0010] Step 1: Get the topic word entered by the user, use the topic word entered by the user as the search word to search for news, then sort the search results by focus, and finally use requests to crawl the news after focus sorting;

[0011] Step 1 and 2: Select social media comments as the data source, use the topic words entered by the user as search words to search and crawl the topic words;

[0012] Step 13: Based on the topic words obtained in step 12, crawl the text and links under each topic word one by one. If the corresponding topic word is not retrieved in step 12, use the topic word entered by the user as the search word and crawl the text and links of the search results;

[0013] Step 14: Crawl the user comments and user locations under each link. During the crawling process, if the number of comments obtained exceeds the set threshold, the crawling will stop.

[0014] Furthermore, the specific steps of step 2 are:

[0015] Step 21: Based on the news crawled in step 11, use the large language model to generate a summary of each news item;

[0016] Step 22: Use regular expressions to remove special symbols from the text and user comments;

[0017] Step 2 and 3: Filter comments based on their similarity to the main text, and remove comments with too low similarity to the main text;

[0018] Step 24: For the comments retained after filtering in steps 2 and 3, use the large language model to detect the stance of each user comment.

[0019] Furthermore, the specific steps of steps two and three are:

[0020] Step 231: Convert the text and user comments into vector representation V(p i ) and V(c ij ), where p i represents the i-th text, c ij represents the jth user comment under the i-th text;

[0021] Step 232: Get the text vector V(p i ) and user review vector V(c ij )'s cosine similarity CosSim(p i ,c ij ), CosSim(p i ,c ij ) is expressed as:

[0022]

[0023] Step 233: Arrange the user comments in descending order based on the cosine similarity obtained in step 232, and filter out comments with too low similarity to the text based on the set threshold.

[0024] Furthermore, the specific steps of step 24 are:

[0025] Step 241: Use the stance detection results to label the user comments. Then, construct an example pool S based on the user comments, text, and labels. The example pool S is expressed as:

[0026] S={s1,s2,...,s N}

[0027] s i =(t i ,c i ,l i )

[0028] Where i = 1, 2...N, t i represents the comment text of the i-th comment, c i Indicates the main text of the Weibo post. i Indicates the label, s i represents a triple consisting of a microblog comment, a microblog text, and a tag, and N represents the total number of annotated comments;

[0029] Step 242: Randomly extract examples from the example pool S and use the extracted examples to construct model prompts that include context and few-shot examples. Context refers to providing the body of the comment in the prompt of the large language model, and few-shot examples refers to randomly extracting several triplets from the example pool S.

[0030] Step 243: Based on the large language model and using the model prompts built in step 242, perform stance detection on each comment.

[0031] Furthermore, the specific steps of step three are:

[0032] Step 31: Input the news summary into the large language model to generate an event summary;

[0033] Step 32: Utilize the stance detection results of all user comments and the user locations corresponding to the user comments, and use the large language model to generate a text description of the user's provincial distribution;

[0034] Step 3: Based on the news summary, use the large language model to obtain the main opinions of the media and the news supporting them, that is, the media discussion topics in public opinion;

[0035] Step 3 and 4: Based on the user comments and their stance detection results, extract positive and negative opinions as user comments in public opinion;

[0036] Step 35: Input the event overview, the text description of the user's provincial distribution, the user's provincial distribution map, and public opinion into the large language model to obtain the public opinion report output by the large language model;

[0037] Step 36: Based on the text description of the user's provincial distribution, use pyecharts to generate a user provincial distribution map, and then display the user provincial distribution map in the public opinion report to obtain the final public opinion report.

[0038] Furthermore, the specific steps of step 31 are:

[0039] Step 3: Number the abstracts of each news item to obtain the abstract set R. The abstract set R is expressed as:

[0040] R={r1,r2,...,r M}

[0041] Where M represents the total number of news summaries, r i represents the summary of the i-th news, i = 1, 2...M;

[0042] Step 312: Based on the summary set R, use the large language model to generate an event summary with citations, where the citations are the news corresponding to the news summary.

[0043] Furthermore, the specific steps of step 33 are:

[0044] Step 331: Based on the summary set R obtained in step 311, use the embedding model to convert each news summary in the summary set R into an embedded representation;

[0045] Step 332: Based on the embedded representation obtained in step 331, use the K-Means algorithm and combine it with the silhouette coefficient to determine the optimal number of clusters k within a given range. optimal , and finally cluster the news summaries into k optimal news summary clusters;

[0046] Step 333: Based on the news summary clusters obtained in step 332, prompt the large language model to extract the opinion of the news summary in each cluster;

[0047] Step 334: Convert the opinions extracted in step 331 into vector representations, then calculate the cosine similarity between the news summaries within the cluster and the opinion vector representations, and finally use the news summaries with cosine similarity higher than the threshold as the basis for supporting the opinion.

[0048] Furthermore, the specific steps of steps three and four are:

[0049] Step 341: Divide user comments into three groups: support, opposition, and neutral based on the results of stance detection;

[0050] Step 342: Use the embedding model to convert the comments in the support and opposition sets into vector representations. Then, use the HDBSCAN clustering algorithm to cluster the comments in these two comment sets to obtain several comment clusters;

[0051] Step 343: Based on the comment clusters obtained in step 342, the large language model is prompted to extract the opinions of the user comments in each cluster. If the comment cluster belongs to the support comment set, the extracted opinions are positive; if the comment cluster belongs to the opposition comment set, the extracted opinions are negative;

[0052] Step 343: Use the embedding model to convert the opinions extracted by the large language model into vector representations, calculate the cosine similarity between user comments and opinions within the cluster, and use comments with similarity above the threshold as evidence to support the opinion.

[0053] Furthermore, the embedding model is text-embedding-3-small or bge-large-zh.

[0054] The beneficial effects of the present invention are:

[0055] This application can be used to analyze public opinion on recent hot topics on social media platforms. It collects large amounts of news and comment data through crawlers or APIs provided by various websites. It then uses a large language model to summarize the collected news and perform stance analysis on the comments. The large language model is then used to generate public opinion reports. The public opinion reports generated by this application are highly timely. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a diagram of a public opinion report with citations;

[0057] Figure 2 This is the overall framework diagram of this application. DETAILED DESCRIPTION

[0058] It should be noted that, unless there is any conflict, the various embodiments disclosed in this application can be combined with each other.

[0059] Specific embodiment 1: The method for generating a public opinion report based on search enhancement generation described in this embodiment includes the following steps:

[0060] Step 1: Get relevant news and comments based on the topic words entered by the user;

[0061] Step 2: Get the news summary and perform stance detection on the comments to obtain the stance of the comments;

[0062] Step 3: Input the summary and comments’ stance into the large language model to generate a public opinion report.

[0063] The overall structure of this application is as follows Figure 2 shown.

[0064] The specific steps of step one are:

[0065] Step 1: Use the topic words entered by the user as search words to search for news in Baidu News. Sort the search results by focus. Use requests to crawl the first five pages of news. When implementing, you can store them in a list.

[0066] Step 1 and 2: Select Sina Weibo as the data source for social media comments, use the topic words entered by the user as search terms, and crawl the topic words on the first page of Weibo topic search pages;

[0067] Step 13: crawl the first two pages of microblogs and links for each topic word one by one according to the topic words obtained in step 12. If the corresponding topic word is not retrieved in step 12, use the topic word entered by the user as the search word to directly crawl the first two pages of microblogs and links for the search results;

[0068] Step 14: Crawl the Weibo user comments and user locations under each Weibo link in the results of step 13. During the crawling process, if the number of comments obtained exceeds the set threshold, stop obtaining.

[0069] The specific steps of step 2 are:

[0070] Step 21: Use the large language model to generate a summary of each news item in the list;

[0071] Step 22: Use regular expressions to remove special symbols from Weibo text and Weibo user comments;

[0072] Step 2 and 3: Filter the comments based on the similarity between the Weibo user comments and the Weibo text;

[0073] Step 24: For the comments retained after filtering in Steps 2 and 3, use the large language model to detect the stance of each Weibo user comment, classifying the comments into three categories: support, opposition, and neutral. The target of stance detection is the topic word entered by the user.

[0074] The specific steps for steps two and three are:

[0075] Step 231: Define the body of the crawled i-th microblog as p i , the jth comment under this microblog is c ij , use a small embedding model (such as text-embedding-3-small) to convert the Weibo text and Weibo comments into vector representation V(p i and V(c ij );

[0076] Step 232: Get the Weibo text vector V(p i ) and Weibo user comment vector V(c ij )'s cosine similarity CosSim ( p i ,c ij ), CosSim ( p i ,c ij ) is expressed as:

[0077]

[0078] Step 233: Sort all comments in descending order based on the cosine similarity between the comments and the Weibo text to which they belong, save them into a list, and truncate the list based on the set threshold to filter out comments with too low semantic similarity to the Weibo text.

[0079] The specific steps of step 24 are:

[0080] Step 241: Use the stance detection results to label the Weibo user comments. Then, construct an example pool S based on the Weibo user comments, Weibo text, and tags. The example pool S is expressed as:

[0081] S= { s1,s2,...,s N }

[0082] s i =(t i ,c i ,l i )

[0083] Where i = 1, 2...N, t i represents the comment text of the i-th comment, c i Indicates the main text of the Weibo post. i Indicates the label, s i represents a triple consisting of a Weibo comment, a Weibo text, and a tag, and N represents the total number of annotated comments;

[0084] Step 242: Randomly extract examples from the example pool S and use the extracted examples to construct model prompts that include context and few-shot examples. Context refers to providing the Weibo body content of the Weibo comment to which the Weibo comment belongs in the prompt of the large language model. Few-shot examples refer to randomly extracting several triplets from the example pool S to provide examples for stance detection in the large language model.

[0085] Step 243: Based on the large language model and using the model prompts built in step 242, perform stance detection on each comment.

[0086] The specific steps of step three are:

[0087] Step 31: Input the news summary into the large language model to generate an event summary;

[0088] Step 32: Utilize the stance detection results of all Weibo user comments and the location of the user corresponding to the Weibo user comments, and use the large language model to generate a text description of the user's provincial distribution;

[0089] Step 3: Based on the news summary, use the large language model to obtain the main opinions of the media and the news supporting them, that is, the media discussion topics in public opinion;

[0090] Step 3 and 4: Based on the Weibo user comments and their stance detection results, extract positive and negative opinions as user comments in public opinion;

[0091] Step 35: Use the first three parts of the report as input, namely the event overview, data analysis, and public opinion, to prompt LLM to generate the event summary and suggestions part of the report. At this point, a public opinion report consisting of four parts is formed, namely the event overview, data analysis, public opinion, and event summary and suggestions.

[0092] The specific steps of step 31 are:

[0093] Step 3: Number the summary of each news item. The summary of the i-th news item is denoted as r. i , use these summaries to build a summary set R;

[0094] R={r1,r2,...,r M}

[0095] Where M represents the total number of news summaries, r i represents the summary of the i-th news, i = 1, 2...M;

[0096] Step 3.1.2: Based on the abstract set, use LLM to generate an event summary with citations, where the citations are the news corresponding to the abstracts;

[0097] That is, for the paragraph S generated by LLM, N}, where s i Represents a sentence, requiring LLM to be used for each s after the sentence i Provide a reference set C i ={c i1 ,c i2 ,...,c iK}, c ij represents the jth citation of the i-th sentence. ij ∈R and C iCan be an empty set;

[0098] The specific steps of step 32 are:

[0099] Step 321: Fill in the values ​​of the stance detection results distribution, regional distribution of comments, etc. into the pre-written data description template;

[0100] Step 322: Inject the results obtained in the previous step into the LLM prompt, prompting LLM to generate the data analysis part of the public opinion report and analyze the existing data.

[0101] Furthermore, the specific steps of step 33 are:

[0102] Step 331: Based on the summary set R obtained in step 311, use the embedding model to convert each news summary in the summary set R into an embedded representation;

[0103] Step 332: Based on the embedding representation obtained in step 331, use the K-Means algorithm and combine it with the silhouette coefficient to determine the optimal number of clusters k within a given range (e.g., an integer between 2 and 10). optimal , and finally cluster the news summaries into k optimal news summary clusters;

[0104] Step 333: Based on the news summary clusters obtained in step 332, prompt the large language model to extract the opinion of the news summary in each cluster;

[0105] Step 334: Convert the opinions extracted in step 331 into vector representations, then calculate the cosine similarity between the news summaries within the cluster and the opinion vector representations, and finally use the news summaries with cosine similarity higher than the threshold as the basis for the opinions.

[0106] You can also use relevance to score on a 5-point scale, requiring the large language model to output an integer between 1 and 5;

[0107] A score of 1 indicates "very irrelevant," meaning the news summary has little to no relevance to the opinion, or the content of the summary completely deviates from the main theme of the opinion and fails to provide any basis for or against the opinion. A score of 2 indicates "not very relevant," meaning the summary is somewhat relevant to the opinion, but the relevance is low. The summary may touch upon certain aspects of the opinion, but it does not effectively support or oppose the opinion. A score of 3 indicates "moderately relevant," meaning the summary is somewhat relevant to the opinion, and the content of the summary can partially support or oppose the opinion, but the overall relevance is average. A score of 4 indicates "highly relevant," meaning the summary is highly relevant to the opinion, and the content of the summary can well support or oppose the opinion, providing relatively detailed and specific reasons or evidence. A score of 5 indicates "very relevant," meaning the content of the summary is completely relevant to the opinion, and the content of the summary clearly and in detail supports or opposes the opinion, providing sufficient reasons and evidence.

[0108] The specific steps of steps three and four are:

[0109] Step 341: Divide the obtained comments into three groups: support, opposition, and neutral according to the results of stance detection;

[0110] Step 342: The set with a neutral stance detection result is not used to extract opinions. An embedding model (such as text-embedding-3-small or bge-large-zh) is used to convert the comments in the support and opposition comment sets into vector representations. The HDBSCAN clustering algorithm is then used to cluster the comments in these two comment sets to obtain several comment clusters.

[0111] Step 342: Based on the comment clusters obtained in step 342, LLM is prompted to extract the opinions of social media users on the input topic within each cluster. If the comment cluster belongs to the support comment set, the extracted opinions are positive; if the comment cluster belongs to the opposition comment set, the extracted opinions are negative;

[0112] Step 343: Use the embedding model to convert the opinions extracted by LLM into vector representations, calculate the cosine similarity between the comments and opinions within the cluster, and take the comments with similarity higher than the threshold as the basis of the opinion.

[0113] This application can be used to analyze the public opinion of recent hot topics on social media platforms (such as Sina Weibo, Zhihu), collect a large amount of news and comment data through crawlers or APIs provided by various websites, and use a large language model to summarize the collected news and perform position detection on the comments (support, opposition, neutrality). Subsequently, the instruction understanding and generation capabilities of the large language model are used to generate a readable public opinion report part by part, and provide a basis for the content of each part of the report generated. The framework proposed by this method can be deployed without GPU resources by calling the large language model API, avoiding the huge computing power and data overhead brought by model training.

[0114] This application first uses requests to crawl data from news and social media platforms based on input topic terms, obtaining relevant news, social media user comments, and user-related information. This data is then analyzed using a large language model to obtain summaries of the relevant news and categorize comments on the topic into three categories: support, opposition, and neutrality. Based on this data analysis, the large language model uses the acquired news and comment data through prompt engineering to generate a complete public opinion report, providing a basis for each generated content component.

[0115] Most of the existing systems that automatically generate public opinion reports through LLM combine retrieval enhancement generation technology to improve the accuracy of the generated public opinion report content, but do not provide references to the report content generated by LLM, making it difficult for users to trust and verify the report content generated by LLM without any supporting evidence. In this application, this application gradually generates the content of each part of the public opinion report, rather than using the retrieved information to generate a complete public opinion report at one time. This method allows the content related to the specific part of the report to be injected into its context in a targeted manner during the process of prompting LLM to generate the public opinion report, reducing the impact of irrelevant information on the content generated by the model. In addition, in the process of generating each part of the content, we guide LLM to provide references for the content it generates through the means of prompt engineering. In the part of extracting opinions, the model is also required to provide the basis for the opinions. This improves the quality of automatically generated public opinion reports in two aspects: (1) Under such a prompt strategy, it helps LLM generate text that faithfully follows the provided information; (2) The content generated by LLM and its basis can be explicitly displayed in the generated report content, and users can quickly verify the accuracy of the report content. Figure 1 As shown, users can click on the hyperlink to view the original news and verify the accuracy of the generated report content.

[0116] It should be noted that the specific embodiments are merely explanations and illustrations of the technical solutions of the present invention and cannot be used to limit the scope of protection. Any minor changes made based on the claims and description of the present invention should still fall within the scope of protection of the present invention.

Claims

1. A method for generating public opinion reports based on retrieval enhancement generation, characterized in that The method comprises the following steps: Step 1: Get relevant news and comments based on the topic words entered by the user; Step 2: Get the news summary and perform stance detection on the comments to obtain the stance of the comments; Step 3: Input the summary and comments’ stance into the large language model to generate a public opinion report; The specific steps of step three are: Step 31: Input the news summary into the large language model to generate an event summary; Step 32: Utilize the stance detection results of all user comments and the user locations corresponding to the user comments, and use the large language model to generate a text description of the user's provincial distribution; Step 3: Based on the news summary, use the large language model to obtain the main opinions of the media and the news supporting them, that is, the media discussion topics in public opinion; Step 3 and 4: Based on the user comments and their stance detection results, extract positive and negative opinions as user comments in public opinion; Step 35: Input the event overview, the text description of the user's provincial distribution, and public opinion into the large language model to obtain the public opinion report output by the large language model; Step 36: Based on the text description of the user's provincial distribution, use pyecharts to generate a user provincial distribution map, and then display the user provincial distribution map in the public opinion report to obtain the final public opinion report.

2. The method for generating a public opinion report based on retrieval enhancement generation according to claim 1 is characterized in that The specific steps of step one are: Step 1: Get the topic word entered by the user, use the topic word entered by the user as the search word to search for news, then sort the search results by focus, and finally use requests to crawl the news after focus sorting; Step 1 and 2: Select social media comments as the data source, use the topic words entered by the user as search words to search and crawl the topic words; Step 13: Based on the topic words obtained in step 12, crawl the text and links under each topic word one by one. If the corresponding topic word is not retrieved in step 12, use the topic word entered by the user as the search word and crawl the text and links of the search results; Step 14: Crawl the user comments and user locations under each link. During the crawling process, if the number of comments obtained exceeds the set threshold, the crawling will stop.

3. The method for generating a public opinion report based on retrieval enhancement generation according to claim 2 is characterized in that The specific steps of step 2 are: Step 21: Based on the news crawled in step 11, use the large language model to generate a summary of each news item; Step 22: Use regular expressions to remove special symbols from the text and user comments; Step 2 and 3: Filter comments based on their similarity to the main text, and remove comments with too low similarity to the main text; Step 24: For the comments retained after filtering in steps 2 and 3, use the large language model to detect the stance of each user comment.

4. The method for generating a public opinion report based on retrieval enhancement generation according to claim 3 is characterized in that The specific steps of steps two and three are: Step 231: Convert the text and user comments into vector representations and ,in, Indicates the A text, Indicates the The following text User reviews; Step 232: Get the text vector and user review vectors Cosine similarity of , Expressed as: ； Step 233: Arrange the user comments in descending order based on the cosine similarity obtained in step 232, and filter out comments with too low similarity to the text based on the set threshold.

5. The method for generating a public opinion report based on retrieval enhancement generation according to claim 4 is characterized in that The specific steps of step 24 are: Step 241: Use the stance detection results to label user comments, and then build an example pool based on user comments, text, and tags , sample pool Expressed as: in, =1,2... , Indicates the Comment text of comments, Indicates the main text of the Weibo post. Indicates a label, Represents a triple consisting of a Weibo comment, Weibo text, and tags. Indicates the total number of annotated comments; Step 242: In the sample pool Randomly extract examples from the pool and use the extracted examples to build model prompts containing context and few-shot examples. The context means providing the body content of the comment in the prompt of the large language model, and the few-shot examples refer to the examples from the sample pool. Randomly select several triplets from ; Step 243: Based on the large language model and using the model prompts built in step 242, perform stance detection on each comment.

6. The method for generating a public opinion report based on retrieval enhancement generation according to claim 5 is characterized in that The specific steps of step three are: Step 3: Number the abstracts of each news item to get the abstract set , Abstracts Expressed as: in, Indicates the total number of news summaries, Indicates the A summary of the news, =1,2... ; Step 3-12: Based on the summary set ,Use a large language model to generate an event summary with citations, where the citations are the news corresponding to the news summary.

7. The method for generating a public opinion report based on retrieval enhancement generation according to claim 6 is characterized in that The specific steps of step three are: Step 331: Based on the summary set obtained in step 311 , using the embedding model to transform the summary set Each news summary in is converted into an embedding representation; Step 332: Based on the embedded representation obtained in step 331, use the K-Means algorithm and the silhouette coefficient to determine the optimal number of clusters within a given range. , and finally cluster the news summaries into news summary clusters; Step 333: Based on the news summary clusters obtained in step 332, prompt the large language model to extract the opinion of the news summary in each cluster; Step 334: Convert the opinions extracted in step 331 into vector representations, then calculate the cosine similarity between the news summaries within the cluster and the opinion vector representations, and finally use the news summaries with cosine similarity higher than the threshold as the basis for supporting the opinion.

8. The method for generating a public opinion report based on retrieval enhancement generation according to claim 7 is characterized in that The specific steps of steps three and four are: Step 341: Divide user comments into three groups: support, opposition, and neutral based on the results of stance detection; Step 342: Use the embedding model to convert the comments in the support and opposition sets into vector representations. Then, use the HDBSCAN clustering algorithm to cluster the comments in these two comment sets to obtain several comment clusters; Step 343: Based on the comment clusters obtained in step 342, the large language model is prompted to extract the opinions of the user comments in each cluster. If the comment cluster belongs to the support comment set, the extracted opinions are positive; if the comment cluster belongs to the opposition comment set, the extracted opinions are negative; Step 343: Use the embedding model to convert the opinions extracted by the large language model into vector representations, calculate the cosine similarity between user comments and opinions within the cluster, and use comments with similarity above the threshold as evidence to support the opinion.

9. The method for generating a public opinion report based on retrieval enhancement generation according to claim 8 is characterized in that The embedding model is text-embedding-3-small or bge-large-zh.

Images