Social media analysis report generation method and analysis report generation device

Abstract

The invention provides a social media analysis report generation method and an analysis report generation device, which can be applied to the technical field of data processing, and the method comprises the following steps: collecting webpage collection data of a target webpage from the target webpage related to a data collection instruction by using an intelligent agent; based on the document structure information, performing semantic analysis on the webpage screenshot by utilizing a preset visual model to obtain target semantic information corresponding to each piece of user comment data; according to the document structure information, the user comment data and the target semantic information, an information summarization page is generated, the information summarization page comprises a plurality of social comment data arranged according to a document structure, and the social comment data comprises user comment data published by a user at any time point and the corresponding target semantic information; and based on the comment attributes, performing data analysis on the plurality of social comment data to obtain an analysis report.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method and apparatus for generating a social media analysis report. Background Technology

[0002] With the development of the Internet, the evaluation information posted by travelers on social media platforms has grown rapidly, covering aspects such as service experience, facility quality, and ticketing policies. It is characterized by its large volume, rapid updates, and multimodal nature, and has become an important data source for optimizing travel services and monitoring public opinion.

[0003] However, this information is scattered across web pages with varying structures, often presented in a mix of text, images, PDFs, etc. Traditional web crawlers rely on manually written parsing rules, which are difficult to handle dynamic rendering and frequent redesigns. They generally suffer from poor adaptability, long data acquisition time, high maintenance costs, and weak semantic understanding capabilities. Data obtained in this way is unlikely to form highly feasible analysis reports. Summary of the Invention

[0004] In view of this, this application provides a method for generating a social media analysis report and an apparatus for generating such a report.

[0005] One aspect of this application provides a method for generating a social media analysis report, comprising: responding to a data collection instruction corresponding to a data collection requirement, using an intelligent agent to collect webpage collection data from a target webpage related to the data collection instruction, wherein the webpage collection data includes a screenshot of the target webpage, document structure information, and user comment data; based on the document structure information, performing semantic parsing on the webpage screenshot using a preset visual model to obtain target semantic information corresponding to each user comment data, wherein the target semantic information represents comment attributes unrelated to the comment content of the user comment data and not included in the document structure information and the user comment data; generating an information summary page based on the document structure information, the user comment data, and the target semantic information, wherein the information summary page includes multiple social comment data arranged according to the document structure, the social comment data including user comment data posted by a user at any time and the corresponding target semantic information; and performing data analysis on the multiple social comment data based on the comment attributes to obtain an analysis report.

[0006] Another aspect of this application provides an apparatus for generating a social media analysis report, comprising: a collection module, configured to, in response to a data collection instruction corresponding to a data collection requirement, use an intelligent agent to collect webpage collection data from a target webpage related to the data collection instruction, wherein the webpage collection data includes a screenshot of the target webpage, document structure information, and user comment data; a obtaining module, configured to, based on the document structure information, perform semantic parsing on the webpage screenshot using a preset visual model to obtain target semantic information corresponding to each user comment data, wherein the target semantic information represents comment attributes unrelated to the comment content of the user comment data and not included in the document structure information and the user comment data; a generation module, configured to, based on the document structure information, the user comment data, and the target semantic information, generate an information summary page, wherein the information summary page includes multiple social comment data arranged according to the document structure, the social comment data including user comment data posted by a user at any point in time and the corresponding target semantic information; and an analysis module, configured to, based on the comment attributes, perform data analysis on the multiple social comment data to obtain an analysis report.

[0007] Another aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described above.

[0008] Another aspect of this application provides a computer-readable storage medium storing computer-executable instructions that, when executed, are used to implement the method described above.

[0009] Another aspect of this application provides a computer program product comprising computer-executable instructions which, when executed, are used to implement the method described above.

[0010] According to embodiments of this application, an intelligent agent is used to collect webpage data from target webpages associated with data collection instructions. A preset visual model is used to perform semantic parsing on webpage screenshots to obtain target semantic information corresponding to each user comment. Based on document structure information, user comment data, and target semantic information, an information summary page is generated. Data analysis is then performed on multiple social media comment data sets to obtain an analysis report. Because an intelligent agent and a preset visual model are used to collect social media data, and the intelligent agent, based on its own semantic understanding and autonomous operation capabilities, combined with the semantic parsing of the preset visual model, can obtain multimodal social media comment data, its collection robustness and maintainability are good. The analysis report obtained based on this multimodal social media comment data can provide staff with more valuable information. Attached Figure Description

[0011] The above and other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0012] Figure 1 The illustration schematically shows an exemplary system architecture for generating social media analysis reports and an apparatus for generating analysis reports, which can be applied according to embodiments of this application;

[0013] Figure 2 A flowchart illustrating a method for generating a social media analytics report according to an embodiment of this application is shown schematically.

[0014] Figure 3 A flowchart illustrating a method for generating target comment data according to an embodiment of this application is shown schematically;

[0015] Figure 4 A flowchart illustrating a method for generating target comment data according to an embodiment of this application is shown schematically;

[0016] Figure 5 The illustration shows a schematic diagram of a webpage display page according to an embodiment of this application;

[0017] Figure 6 A block diagram illustrating a social media analytics report generation apparatus according to an embodiment of this application is shown schematically; and

[0018] Figure 7 A block diagram of an electronic device suitable for implementing the methods described above, according to an embodiment of this application, is illustrated schematically. Detailed Implementation

[0019] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0020] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0021] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0022] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0023] In the embodiments of this application, the collection, updating, analysis, processing, use, transmission, provision, disclosure, and storage of data (e.g., including but not limited to user personal information) comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures have been taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security.

[0024] In the embodiments of this application, authorization or consent from the webpage platform is obtained before obtaining or collecting user personal information from the target webpage, and the user's personal information is obscured during the data processing to ensure that the personal information is not leaked.

[0025] Embodiments of this application provide a method and apparatus for generating a social media analysis report, which can be applied to the field of data processing technology. The method includes: using an intelligent agent to collect webpage collection data from a target webpage related to a data collection instruction; performing semantic parsing on webpage screenshots based on document structure information using a preset visual model to obtain target semantic information corresponding to each user comment data; generating an information summary page based on the document structure information, user comment data, and target semantic information, wherein the information summary page includes multiple social comment data arranged according to the document structure, and the social comment data includes user comment data posted by a user at any time and the corresponding target semantic information; and performing data analysis on the multiple social comment data based on comment attributes to obtain an analysis report.

[0026] Figure 1 An exemplary system architecture 100, illustrating an embodiment of the present application, is shown, in which a method and apparatus for generating social media analytics reports can be applied. It should be noted that... Figure 1The examples shown are merely examples of system architectures that can be applied to the embodiments of this application, in order to help those skilled in the art understand the technical content of this application, but do not mean that the embodiments of this application cannot be used in other devices, systems, environments or scenarios.

[0027] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social media platform software, etc. (for example only).

[0029] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0030] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0031] It should be noted that the method for generating social media analysis reports provided in this application embodiment can generally be executed by server 105. Correspondingly, the apparatus for generating social media analysis reports provided in this application embodiment can generally be located in server 105. The method for generating social media analysis reports provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the apparatus for generating social media analysis reports provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0032] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0033] Figure 2 A flowchart illustrating a method for generating a social media analytics report according to an embodiment of this application is shown schematically.

[0034] like Figure 2 As shown, the method for generating this social media analysis report includes operations S201 to S204.

[0035] In operation S201, in response to the data collection command corresponding to the data collection requirement, the intelligent agent collects web page collection data from the target web page related to the data collection command. The web page collection data includes web page screenshots, document structure information and user comment data of the target web page.

[0036] In operation S202, based on document structure information, a preset visual model is used to perform semantic parsing on the webpage screenshot to obtain target semantic information corresponding to each user comment data. The target semantic information represents comment attributes that are not included in the document structure information and user comment data and are unrelated to the comment content of the user comment data.

[0037] In operation S203, an information summary page is generated based on document structure information, user comment data, and target semantic information. The information summary page includes multiple social comment data arranged according to the document structure. The social comment data includes user comment data posted by users at any point in time and the corresponding target semantic information.

[0038] In operation S204, data analysis is performed on multiple social comment data based on comment attributes to obtain an analysis report.

[0039] According to an embodiment of this application, data collection requirements can characterize the types of information that staff need to collect, such as "collecting comments about a certain high-speed train on a certain platform in the past two weeks". Based on this data collection requirement, corresponding data collection instructions can be generated.

[0040] According to embodiments of this application, the intelligent agent has a controller embedded with a preset language model and can collect data from the Internet through an external tool interface. This external tool interface has functions for controlling the browser, taking screenshots, and making network requests. Based on the external tool interface and the controller, autonomous decision-making and task completion can be achieved.

[0041] According to embodiments of this application, the information summary page can be a webpage that summarizes all the information obtained in this collection, or it can be a regular Word, Excel, or PDF file. Document structure information can refer to the HTML document structure of the webpage, also known as the DOM tree (Document Object Model tree), which is the way a browser parses an HTML document into a tree-like representation.

[0042] According to embodiments of this application, the preset visual model may refer to a Computer Vision Large Model, such as the Qwen-VL model.

[0043] According to an embodiment of this application, after generating a corresponding data collection instruction based on data collection requirements, the intelligent agent can respond to the data collection instruction and open the corresponding target webpage. The opened target webpage can be screenshotted through an external tool interface to obtain a webpage screenshot. The text information of user comments can be directly obtained to obtain user comment data, and at the same time, the document structure information of the target webpage can be obtained.

[0044] According to an embodiment of this application, based on the document structure information, a preset visual model is used to perform semantic parsing on the webpage screenshot to obtain target semantic information corresponding to each user comment data. This target semantic information mainly includes comment attributes that are not included in the document structure information and the user comment data and are unrelated to the comment content. Comment attributes can refer to some attributes of the user who posted the comment data, such as username, membership level, or image information uploaded by the user. It should be noted that, in order to ensure user privacy, in this embodiment, the username is represented by a number, letter, or a combination of both, such as using 001 to represent user 1.

[0045] According to embodiments of this application, user comment data and target semantic information are correlated and summarized based on document structure information to form an information summary page containing all comments. This summary page displays at least one social comment posted by each user. Based on comment attributes, such as username, data analysis is performed on multiple social comment data sets to obtain an analysis report. This analysis report can be specifically configured according to actual needs; for example, it could reflect changes in the number of user complaints about train delays, or it could be a summary of user suggestions for train optimization and improvement.

[0046] According to embodiments of this application, an intelligent agent is used to collect webpage data from target webpages associated with data collection instructions. A preset visual model is used to perform semantic parsing on webpage screenshots to obtain target semantic information corresponding to each user comment. Based on document structure information, user comment data, and target semantic information, an information summary page is generated. Data analysis is then performed on multiple social media comment data sets to obtain an analysis report. Because an intelligent agent and a preset visual model are used to collect social media data, and the intelligent agent, based on its own semantic understanding and autonomous operation capabilities, combined with the semantic parsing of the preset visual model, can obtain multimodal social media comment data, its collection robustness and maintainability are good. The analysis report obtained based on this multimodal social media comment data can provide staff with more valuable information.

[0047] According to an embodiment of this application, based on document structure information, a preset visual model is used to perform semantic analysis on a webpage screenshot to obtain target semantic information not included in the document structure information and user comment data. This includes: aligning the webpage screenshot and document structure information with spatial coordinates to obtain position mapping information, wherein the position mapping information represents the position of webpage elements in the document structure information in the webpage screenshot; and based on the position mapping information, using a preset visual model to perform semantic analysis on the webpage screenshot to obtain target semantic information.

[0048] According to an embodiment of this application, the webpage screenshot and document structure information are aligned in spatial coordinates to construct a "visual-structure mapping table", that is, position mapping information, which records the position rectangle (boundingbox) of each HTML element in the image.

[0049] According to an embodiment of this application, after obtaining the location mapping information, a preset visual model is invoked to perform semantic parsing on the webpage screenshot to identify the target semantic information in the webpage screenshot, such as: the star rating icon corresponding to the star mark, the certified user mark next to the user's avatar, and the screenshot content in the comment section being information such as train delay notification.

[0050] According to embodiments of this application, by aligning webpage screenshots and document structure information with spatial coordinates, position mapping information is obtained. Based on the position mapping information, a preset visual model is invoked to perform semantic analysis on the webpage screenshots, obtaining target semantic information not included in the document structure information and user comment data. This can improve the completeness of information in user comments, thereby improving the accuracy of the analysis report.

[0051] According to an embodiment of this application, data analysis is performed on multiple social comment data based on comment attributes to obtain an analysis report, including: filtering and removing abnormal comments from multiple social comment data based on comment attributes to obtain multiple target comment data, wherein abnormal comments indicate that the similarity between multiple user comment data published by a user within a preset time period meets a similarity threshold; and analyzing the target comment data using a preset language model based on different prompt word templates to generate comment analysis reports corresponding to different scenarios.

[0052] According to the embodiments of this application, in multiple social comment data posted by users, some users engage in "spam" behavior, that is, they repeatedly post the same or highly similar comments. Such behavior can lead to inaccurate analysis reports. Alternatively, some users may post multiple comments that are abnormal, such as malicious evaluations. Such information can also lead to inaccurate analysis reports. Therefore, this embodiment filters and removes abnormal evaluations from multiple social comment data based on the user's comment attributes to obtain multiple target comment data.

[0053] According to embodiments of this application, by setting different prompt word templates, a preset language model analyzes multiple target comment data obtained through filtering based on the prompt word templates, generating comment analysis reports corresponding to different scenarios. The prompt word templates can be specifically set according to the needs of staff, such as "statistics on user complaints about 'security check queues' in the second quarter" or "generating a train service quality analysis report for the second quarter," etc. Based on the aforementioned prompt word templates, comment analysis reports for corresponding scenarios can be generated.

[0054] According to embodiments of this application, by filtering out abnormal comments in social comment data, the impact of low-quality comments such as duplicates, spam, and irrelevant comments on the accuracy of subsequent analysis can be reduced, thereby improving the credibility of target comment data during data analysis.

[0055] Figure 3 A flowchart illustrating a method for generating target comment data according to an embodiment of this application is shown schematically.

[0056] According to an embodiment of this application, the comment attribute includes user random number information.

[0057] According to embodiments of this application, multiple social comment data are filtered and removed for abnormal evaluations based on comment attributes to obtain multiple target comment data. This includes: filtering and removing multiple social comment data for abnormal evaluations based on comment attributes using Jaccard similarity to obtain multiple first evaluation data; marking multiple first evaluation data for abnormal evaluations based on cosine similarity to obtain multiple second evaluation data with abnormal labels; and filtering the multiple first evaluation data for each user's random number information based on the second evaluation data with abnormal labels and target attribute information to obtain multiple target comment data, wherein the target attribute information represents the attribute information of the comments posted by the user.

[0058] According to embodiments of this application, the user's random identification information is the username represented by numbers and letters as described above. The attribute information of a user's posted comment may refer to the sentiment of the comment and the interaction between the user and others.

[0059] According to an embodiment of this application, see Figure 3 In the screening of abnormal evaluations, the first screening is performed using Jaccard similarity, which yields multiple first evaluation data. Then, cosine similarity is used to calculate whether any of the multiple first evaluation data corresponding to the user's random ID is abnormal, and any abnormalities are marked, thus obtaining multiple second evaluation data with abnormal labels.

[0060] According to an embodiment of this application, for each user, multiple first evaluation data with abnormal tags and target attribute information are filtered based on the user's random number information to obtain multiple target comment data. This process mainly involves removing the first evaluation data containing abnormal comments.

[0061] According to embodiments of this application, by using Jaccard similarity, cosine similarity, and comment filtering methods based on target attribute information, abnormal comments can be accurately removed, thereby improving the accuracy and reliability of comment data during data analysis.

[0062] According to an embodiment of this application, based on comment attributes, multiple social comment data are filtered and removed for abnormal evaluations using Jaccard similarity to obtain multiple first evaluation data. This includes: for each user's random ID information, calculating the Jaccard similarity information between each pair of social comment data corresponding to the user's random ID information; and deleting social comment data related to the user's random ID information from the multiple social comment data when the number of Jaccard similarity information that meets the similarity threshold among the multiple Jaccard similarity information is a target number, thereby obtaining multiple first evaluation data.

[0063] According to an embodiment of this application, see Figure 3 In the process of filtering Jaccard similarity, for each user (i.e., user random number information), the Jaccard similarity information between any two social comment data of that user can be calculated by formula (1):

[0064] (1)

[0065] in, This represents the Jaccard similarity information between the minimum hash signature of social comment data A and the minimum hash signature of social comment data B.

[0066] According to the embodiments of this application, before calculating the Jaccard similarity information, the social comment data can be cleaned, format standardized and outlier handled. After that, the content of the social comment data is uniformly segmented and normalized, and the time field is converted into a standard UTC timestamp. The anonymous user identifier is uniformly marked as the user's random number information.

[0067] According to the embodiments of this application, for each calculated Jaccard similarity information, if it meets the similarity threshold, it means that the two social comment data corresponding to the Jaccard similarity information are highly similar. If the target number of Jaccard similarity information of the user meets the similarity threshold, it means that the user has abnormal behavior (such as spamming or flooding the screen), that is, multiple social comment data of the user are abnormal. At this time, the user's comments can be deleted from all social comment data.

[0068] According to embodiments of this application, the number of targets and the similarity threshold can be specifically set according to actual needs, for example, the number of targets is 2 and the similarity threshold is 0.85.

[0069] According to embodiments of this application, the collected social comment data is filtered using Jaccard similarity, which can improve the credibility of the data required for data analysis and thus improve the accuracy of the data analysis results.

[0070] According to an embodiment of this application, multiple first evaluation data are labeled as having abnormal evaluations based on cosine similarity to obtain multiple second evaluation data with abnormal labels. This includes: using a pre-trained language model to perform semantic vector encoding on the multiple first evaluation data to obtain multiple semantic encoding features; for each user's random number information, calculating the cosine similarity between every two semantic encoding features related to the user's random number information; calculating semantic consistency data based on the multiple cosine similarities; and, if the semantic consistency data meets a consistency threshold, labeling the first evaluation data related to the user's random number information to obtain multiple second evaluation data with abnormal labels.

[0071] According to embodiments of this application, the specific model of the pre-trained language model is set according to the actual situation, such as the Qwen series of text-embedding models.

[0072] According to an embodiment of this application, a pre-trained language model is used to encode the semantic vector of each first evaluation data to obtain the corresponding high-dimensional sentence embedding, i.e., semantic encoding features.

[0073] According to an embodiment of this application, see Figure 3 For multiple comments posted by the same user, the cosine similarity between every two semantic encoding features is calculated to capture potential semantic consistency. Based on multiple cosine similarities, semantic consistency data is calculated using the semantic consistency scoring function of formula (2):

[0074]

[0075] in, Represents n semantic coding features of user (i.e., user random ID information) u. and These represent the i-th and j-th semantic coding features of the user, respectively. This represents semantically consistent data, where n is greater than or equal to 3.

[0076] According to an embodiment of this application, if the semantic consistency data meets a consistency threshold θ (which can also be combined with a comment quantity threshold such as whether the user's comment count is greater than 3), first evaluation data related to the user's random ID information is marked, along with multiple second evaluation data with abnormal labels. The consistency threshold θ can be adaptively adjusted based on the distribution of all semantic coding features, for example, it can be set to θ=0.9. If the semantic consistency data does not meet the consistency threshold θ, it can be identified as target comment data.

[0077] According to embodiments of this application, by calculating semantic consistency based on cosine similarity, the potential semantic consistency between multiple comments posted by a user can be captured, thereby determining whether the user has abnormal behavior. Thus, data for data analysis is generated based on the second evaluation data marked by semantic consistency, reducing the impact of abnormal data on data analysis and improving the accuracy of data analysis.

[0078] According to an embodiment of this application, the target attribute information includes at least one of the following: the sentiment classification label of the first evaluation data, the comment interaction data, the device address that published the first evaluation data, and the comment publication time.

[0079] According to an embodiment of this application, based on second evaluation data with anomaly labels and target attribute information, multiple first evaluation data of user random number information are filtered to obtain multiple target comment data, including: for each user random number information, generating attribute evaluation results based on the target attribute information of the multiple first evaluation data, wherein the attribute evaluation results include at least one of sentiment polarity results, interaction sparsity results, address clustering results, and posting frequency results; calculating anomaly tendency score based on sentiment polarity results, interaction sparsity results, address clustering results, and posting frequency results; and when the anomaly tendency score meets the scoring threshold and at least one second evaluation data of user random number information has anomaly labels, removing the evaluation data corresponding to user random number information from the multiple first evaluation data to obtain multiple target comment data.

[0080] According to embodiments of this application, sentiment classification tags may include positive or negative evaluations, which can be determined through a preset language model. Comment interaction data includes the number of social behaviors such as comments, likes, and shares, and the device address may refer to device fingerprint consistency (based on browser characteristics such as User-Agent and screen resolution).

[0081] According to embodiments of this application, the target attribute information of each first evaluation data is analyzed using a preset language model to generate attribute evaluation results corresponding to different types. These results can be represented numerically. A weighted sum is calculated of sentiment polarity results, interaction sparsity results, address clustering results, and posting frequency results to obtain an abnormal tendency score. In another embodiment, this abnormal tendency score can be calculated using a preset language model.

[0082] According to an embodiment of this application, see Figure 3 If the abnormal tendency score meets the scoring threshold (e.g., 0.9), and one of the user's second evaluation data has an abnormal label, then it can be determined that the user has abnormal behavior. Therefore, the user's evaluation data can be removed from multiple first evaluation data sets to obtain multiple target comment data sets. In this embodiment, removal can refer to treating these as samples of different risk levels, such as high-risk, medium-risk, and low-risk samples. High-risk samples can be directly deleted or blocked, medium-risk samples can have their weight reduced, and low-risk samples can be manually reviewed. Finally, the samples that have been downgraded and manually reviewed are combined with the target comment data to form a target dataset, which can be used for subsequent data analysis.

[0083] According to embodiments of this application, by combining different target attribute information of a user with anomaly tags determined based on cosine similarity, it is possible to accurately identify whether a user has abnormal behavior. Thus, the target comment data obtained by deleting the user's evaluation data can provide a highly credible data foundation for data analysis, thereby improving the accuracy of data analysis.

[0084] According to an embodiment of this application, in response to a data acquisition instruction corresponding to a data acquisition requirement, an intelligent agent is used to collect web page acquisition data from a target web page related to the data acquisition instruction. This includes: processing the input data acquisition requirement information using a preset language model to obtain multiple task elements, and generating a data acquisition instruction based on the multiple task elements; in response to the data acquisition instruction, calling the intelligent agent to load the target page corresponding to the data acquisition instruction through a target browser, and obtaining document structure information and user comment data; and taking a screenshot of the target page to obtain a web page screenshot.

[0085] According to an embodiment of this application, upon receiving data collection request information input by a user (such as "collect screenshots of negative reviews of a certain high-speed rail on a certain social media platform in the past two weeks"), a preset language model performs semantic understanding, and extracts structured task elements through prompting engineering, including target platform: a certain platform, time range: the past two weeks, topic keywords: a certain high-speed rail, sentiment type: negative, and content format: comments containing images.

[0086] According to an embodiment of this application, standardized data acquisition instructions are generated based on the above-mentioned task elements, and an intelligent agent with multimodal perception and action planning capabilities is scheduled to perform data acquisition.

[0087] According to an embodiment of this application, the intelligent agent loads the target page through the target browser (such as a headless browser like Playwright), synchronously obtains document structure information, user comment data, network request logs (XHR / Fetch), and takes a screenshot of the current window to obtain a webpage screenshot.

[0088] According to an embodiment of this application, invoking an intelligent agent to load a target page corresponding to a data collection instruction through a target browser includes: the intelligent agent scrolling the target page based on a human scrolling behavior pattern, wherein the human scrolling behavior pattern includes scrolling speed, pause frequency, and mouse trajectory jitter; in the case of scrolling the target page, the intelligent agent simulates clicking a page loading control to load the page, and determines whether to load a new target page by checking whether the user comment data on the page has changed.

[0089] According to embodiments of this application, during the loading of a target page, the intelligent agent incorporates a human scrolling behavior model, enabling it to perform simulated user scrolling operations based on human scrolling behavior patterns. This model is trained on a large amount of anonymized real user operation data, which includes: scrolling speed distribution (slow start → constant speed → deceleration and stop), pause frequency (pausing for 0.5–1.5 seconds every 2–3 screens scrolled), and slight mouse trajectory jitter (simulating hand tremors). The intelligent agent calls a preset language model to generate a parameter sequence conforming to this distribution, driving the target browser to scroll at the rhythm of a real user, significantly reducing the risk of being identified as an automated script.

[0090] According to an embodiment of this application, during the loading process described above, if pagination exists, the intelligent agent identifies the page number control and simulates a click. If the page number is not visible (e.g., infinite scrolling), the agent determines whether a new page has been loaded by monitoring changes in the user IDs of comments in the network request log, and decides whether to continue scrolling down based on changes in page height.

[0091] According to embodiments of this application, for interactive controls such as buttons, filter options, and expand more, the intelligent agent can combine a preset visual model with the control labels and document structure information obtained from the webpage screenshot to double-verify the control elements. For example, recognizing the "Load More" button not only relies on its document structure information (innerText="Load More"), but also confirms through the preset visual model that its visual position is at the bottom of the comment list and has a clickable style.

[0092] According to an embodiment of this application, during the process of semantic parsing of webpage screenshots by a preset visual model, the webpage screenshots and document structure information can be jointly input into the preset visual model to perform joint reasoning.

[0093] In one specific embodiment, prompts can be given, such as: "Please analyze the content of each comment in the screenshot, extract the username, posting time, text comment, sentiment, and identify whether the attached image contains voucher information such as tickets or delay notices." Based on this, the pre-defined visual model can output structured JSON.

[0094]

[0095] At the same time, this preset visual model will also remove some text content that can be directly obtained from user comment data, and will also need to identify the comment attributes of the person who posted the comment. In this way, the final target semantic information can be obtained.

[0096] According to embodiments of this application, loading the target page using human scrolling behavior patterns reduces the risk of being identified as an automated script, thereby improving data collection efficiency and effectively reducing the time cost required for data analysis. Simultaneously, end-to-end semantic extraction is achieved through joint visual-language modeling, significantly improving the parsing accuracy of unstructured content such as image comments and screenshots.

[0097] According to embodiments of this application, the method of using an intelligent agent to collect web page collection data from a target web page associated with a data collection instruction further includes: switching the collection path to load the target page when loading the target page based on the collection path fails; and / or logging into the target page using an acquired access token before loading the target page; and / or reloading the target page based on a behavior avoidance strategy when the page prompts verification information during the loading of the target page, wherein the behavior avoidance strategy includes at least one of a delayed login strategy, a proxy pool switching strategy, and a browser state switching strategy.

[0098] According to embodiments of this application, when data collection fails on a certain collection path, the intelligent agent can invoke a preset language model to analyze the cause of the failure (based on error codes, page feedback, and response time) and generate alternative solutions. For example, the collection path can be switched, specifically from collecting data from a platform's search page to collecting data from the platform's topic page. This method has the ability to continuously adapt to changes in webpage structure, significantly reducing the frequency of manual intervention.

[0099] According to embodiments of this application, for platforms or web pages requiring login, the intelligent agent obtains an access token through legitimate authorization and maintains the session state in memory. If login fails, a preset language model can be invoked to analyze error messages (such as "account abnormal" or "frequent operations") and generate recovery strategies: "reduce the request frequency to 3 times per minute" or "switch account pools".

[0100] According to embodiments of this application, when verification information, such as a slider CAPTCHA, is detected, the agent does not directly attempt to crack it, but instead initiates a behavior avoidance strategy, including:

[0101] (1) Delayed login strategy: Invoke the preset language model to generate a reasonable delay (such as "retry after waiting 120 seconds");

[0102] (2) Switching proxy pool strategy: Switch to the backup IP proxy pool (based on legitimate cloud services);

[0103] (3) Browser state switching strategy: simulate the behavior pattern of users closing the browser and then accessing it again.

[0104] It should be noted that this embodiment does not crack the verification information, but only avoids detection by simulating legitimate user behavior.

[0105] According to the embodiments of this application, by switching the collection path and behavior avoidance strategy, the required data can be obtained in a timely manner. Moreover, by simulating human behavior in data collection without cracking verification information, the efficiency of data collection can be improved while complying with relevant regulations.

[0106] Figure 4 A flowchart illustrating a method for generating target comment data according to an embodiment of this application is shown schematically.

[0107] According to embodiments of this application, such as Figure 4 As shown, based on different prompt word templates, a preset language model is used to analyze the target comment data and generate comment analysis reports corresponding to different scenarios, including operations S401 to S402:

[0108] In operation S401, based on the input classification prompt words, a preset language model is used to classify and label different target comment data, resulting in multiple target comment data with classification labels.

[0109] In operation S402, based on different scenario keywords, the target comment data in the database is analyzed using a preset language model to generate comment analysis reports corresponding to different scenarios. The analysis reports include comment analysis reports.

[0110] According to embodiments of this application, the classification prompt words can be specifically set according to actual needs. For example, they can be emotion prompt words. Based on this emotion prompt word model, three categories of labels can be output in the format of "positive," "neutral," or "negative," ensuring consistent and parsable results. Examples of classification prompt words are as follows:

[0111] "You are a professional public opinion analysis assistant. Please determine the sentiment trend of the following passenger comments. Output only one of 'positive,' 'neutral,' or 'negative,' without explanation. User input: [Target comment data] Model output: 'Positive'"

[0112] This model incorporates inference optimization strategies such as low temperature (temperature=0.1) and short generation length (max_new_tokens=10) to improve response speed and output stability. For complex semantic expressions (such as "The flight attendant was nice, but unfortunately the train was three hours late"), the model accurately identifies the overall sentiment as "negative" based on contextual understanding, effectively avoiding misjudgments caused by local positive words.

[0113] The category labels of the sentiment analysis results are written into structured records using the "emotion" field, and the system supports the generation of visual charts such as sentiment distribution pie charts and time trend line charts, making it easier for managers to grasp public opinion dynamics.

[0114] According to an embodiment of this application, in order to support the efficient operation of the Retrieval-Augmented Generation (RAG) mechanism, the target comment data generated in the preceding stage is first classified by scenario, organized in a structured manner, and processed into semantic vectors to construct a scenario-enhanced knowledge base for report generation.

[0115] Specifically, the system writes the core information from Table 1 into a structured database and simultaneously vectorizes and stores it in a vector database.

[0116] Table 1

[0117] Sentiment statistics Sentiment distribution and negative sentiment trend curves across different platforms / lines / time periods Used in sections such as "Overall Public Opinion Overview" and "Trend Comparison". Topic clustering results Topic ID, Keyword Set, c-TF-IDF Weights, Representative Comments, Summary Text Supporting sections such as "Analysis of Key Issues" and "Focus on Hot Topics" Preset keyword time series data Daily / weekly frequency variation curves for keywords such as "charging", "toilet", and "seat". Used to generate service element attention evolution analysis Historical reports and industry knowledge Historical analysis report excerpts, policy documents, technical standards, and news report summaries Provide background information and suggestions for improvement.

[0118] According to embodiments of this application, the comment analysis reports for different scenarios include: periodic comprehensive reports (monthly / quarterly / annual), special analyses of emergencies (such as large-scale delays on a certain route), in-depth diagnostics of service modules (such as "passenger feedback analysis of the air conditioning system"), and industry technology dynamic reports (such as "current status of intelligent ticketing development"). The retrieval strategies, data combination methods, and prompt templates are dynamically adapted to different scenarios.

[0119] When a user initiates a report request (e.g., with the prompt template "Generate Passenger Service Quality Analysis Report for the Second Quarter of 2025"), the system executes the following process:

[0120] (1) Analyze the task intent and dimensional conditions: extract parameters such as time range (2025), analysis object (the entire line), and output granularity (comprehensive);

[0121] (2) Construct semantic query vectors: Encode the task description into vectors, combine them with rule filtering conditions (such as time ∈ [2025-04, 2025-06]), and perform mixed retrieval in the vector database;

[0122] (3) Hierarchical retrieval of key contexts:

[0123] Retrieve the statistical results of sentiment distribution within this period (for use in "Public Opinion Overview");

[0124] Extract the top 5 most frequently asked topics and their representative comments (for the "Main Issues" section);

[0125] Identify anomalous topic clusters with a month-on-month growth exceeding 20% ​​(for "risk warning" purposes);

[0126] Match trend comparison data from historical reports for the same period (for "longitudinal analysis");

[0127] Retrieve relevant regulations or standard provisions as the basis for recommendations.

[0128] All search results are logically sorted by chapter to form a structured context input.

[0129] (4) Chapter-by-chapter prompts and controllable generation

[0130] A chapter-by-chapter progressive generation strategy is adopted to avoid content loss due to a single generation of long text. Each chapter uses an independent prompt template, which is combined with the retrieved context information for constraint generation.

[0131] For example, when generating the "Improvement Suggestions" section, the prompt template is: ["Please generate three specific and feasible improvement suggestions based on the following background information." The model's search for the topic "security check queues" showed a 32% month-on-month increase in complaints, with keywords including "few lanes," "no guidance," and "peak-hour congestion." Related comments mentioned "only two security checkpoints open during morning peak hours," and "no crowd control signage." Standard clauses were also provided. Model output requirements: Each suggestion should not exceed 50 characters, focus on operational aspects, and use formal language. Model output example: "It is recommended to dynamically increase the number of security checkpoints during morning peak hours and set up temporary guidance signage to alleviate passenger flow pressure."]

[0132] This prompt template ensures that the generated content is based on evidence, logically sound, and stylistically consistent, significantly enhancing the professionalism and credibility of the report.

[0133] According to embodiments of this application, the method for generating social media analytics reports realizes an intelligent report output system centered on scenarios, supported by data, bridged by retrieval, and aimed at generation. The retrieval-enhanced generation mechanism is not simply "splitting data and calling models," but rather deeply integrates business logic, ensuring that the generated report possesses factual accuracy, scenario adaptability, and management guidance value by inputting the correct contextual data at the right time and with the correct structure, truly achieving a leap from "data piling up" to "decision support."

[0134] According to embodiments of this application, based on different prompt word templates, target comment data is analyzed using a preset language model to generate comment analysis reports corresponding to different scenarios. The analysis report also includes: clustering the target comment data in the database based on classification tags to obtain clustering results; determining target keywords based on the clustering results; and generating a discussion popularity change graph based on the target keywords and the clustering results of different target keywords in different time periods. The analysis report also includes the discussion popularity change graph.

[0135] According to an embodiment of this application, based on emotion recognition, a semantic representation model is used to encode target comment data into semantic vectors to capture the potential association between semantically similar expressions such as "train delayed by two hours" and "high-speed rail is seriously delayed".

[0136] According to the embodiments of this application, unsupervised clustering is performed using clustering algorithms such as BERTopic: comment clusters are automatically identified through dimensionality reduction and density clustering, and high-frequency target keywords (such as "security check queue" and "long time") of each cluster are extracted to form preliminary topic tags.

[0137] To enhance the business orientation of the analysis, a pre-set keyword guidance mechanism is introduced, with a pre-set set of service-related keywords (such as ['backrest', 'seat', 'charging', 'toilet', 'air conditioning', 'luggage rack']). On the one hand, the frequency of these keywords and their time trends are statistically analyzed to generate daily / weekly discussion popularity charts; on the other hand, for each keyword, a subset of relevant comments is selected, and clustering is performed again within it to uncover the specific causes of problems. For example, in comments containing "backrest", sub-topics such as "backrest cannot be adjusted", "excessive tilt", and "uncomfortable" can be identified, forming a fine-grained service problem profile.

[0138] According to embodiments of this application, a preset language model can also be invoked to generate natural language summaries for each clustered topic. The input prompt is: "Please summarize the core content of this passenger evaluation topic based on the following keywords and representative comments, within 50 characters." An example output is: "Passengers generally report that the number of security checkpoints is insufficient, resulting in excessively long queuing times during peak hours." This enables the transformation from abstract keywords to readable business insights.

[0139] According to embodiments of this application, all the above analysis results can be integrated into a structured topic set, including topic ID, keywords, summary, sentiment distribution, list of related comments, and time distribution information.

[0140] Figure 5 The illustration shows a schematic diagram of a webpage display page according to an embodiment of this application.

[0141] According to embodiments of this application, the generated text content and charts (such as sentiment trend charts, topic word clouds, and keyword time-series curves) can also be automatically integrated into a richly illustrated PDF or Word report, enabling one-click export. All charts are automatically generated by the system using libraries such as Matplotlib and ECharts based on the latest data, ensuring consistency between the text and the charts.

[0142] According to embodiments of this application, the analysis report and charts mentioned above can be displayed through web pages, software, and other applications, such as... Figure 5 As shown, this allows administrators to view key indicators such as public opinion dynamics, preset keyword trends, and regional heat maps in real time, and can directly trigger report generation tasks. The application adopts a front-end and back-end separation architecture, supports multi-role permission management, and meets the needs of users at different levels such as operations, management, and decision-making.

[0143] Figure 6 A block diagram of a social media analytics report generation apparatus according to an embodiment of this application is shown schematically.

[0144] like Figure 6 As shown, the social media analysis report generation device 600 includes a collection module 610, an acquisition module 620, a generation module 630, and an analysis module 640.

[0145] The acquisition module 610 is used to respond to the data acquisition command corresponding to the data acquisition requirement, and use an intelligent agent to collect web page acquisition data from the target web page related to the data acquisition command. The web page acquisition data includes web page screenshots, document structure information and user comment data of the target web page.

[0146] The module 620 is used to perform semantic parsing on webpage screenshots based on document structure information and using a preset visual model to obtain target semantic information corresponding to each user comment data. The target semantic information represents comment attributes that are not included in the document structure information and user comment data and are unrelated to the comment content of the user comment data.

[0147] The generation module 630 is used to generate an information summary page based on document structure information, user comment data, and target semantic information. The information summary page includes multiple social comment data arranged according to the document structure. The social comment data includes user comment data posted by users at any point in time and the corresponding target semantic information.

[0148] The analysis module 640 is used to perform data analysis on multiple social comment data based on comment attributes and generate an analysis report.

[0149] According to embodiments of this application, an intelligent agent is used to collect webpage data from target webpages associated with data collection instructions. A preset visual model is used to perform semantic parsing on webpage screenshots to obtain target semantic information corresponding to each user comment. Based on document structure information, user comment data, and target semantic information, an information summary page is generated. Data analysis is then performed on multiple social media comment data sets to obtain an analysis report. Because an intelligent agent and a preset visual model are used to collect social media data, and the intelligent agent, based on its own semantic understanding and autonomous operation capabilities, combined with the semantic parsing of the preset visual model, can obtain multimodal social media comment data, its collection robustness and maintainability are good. The analysis report obtained based on this multimodal social media comment data can provide staff with more valuable information.

[0150] Any one or more of the modules, submodules, units, and subunits according to the embodiments of this application, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to the embodiments of this application can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to the embodiments of this application can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or implemented by hardware or firmware in any other reasonable manner by integrating or packaging circuits, or implemented in any one of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, one or more of the modules, submodules, units, and subunits according to the embodiments of this application can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0151] For example, any multiple of the acquisition module 610, obtaining module 620, generating module 630, and analysis module 640 can be combined into one module / unit / subunit, or any one of these modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functionality of one or more of these modules / units / subunits can be combined with at least part of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of this application, at least one of the acquisition module 610, obtaining module 620, generating module 630, and analysis module 640 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 610, the acquisition module 620, the generation module 630, and the analysis module 640 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0152] It should be noted that the social media analysis report generation device part in the embodiments of this application corresponds to the social media analysis report generation method part in the embodiments of this application. The description of the social media analysis report generation device part is specifically referred to in the social media analysis report generation method part, and will not be repeated here.

[0153] Figure 7 A block diagram of an electronic device suitable for implementing the methods described above, according to an embodiment of this application, is illustrated schematically. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0154] like Figure 7 As shown, an electronic device 700 according to an embodiment of this application includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0155] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0156] According to embodiments of this application, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0157] According to embodiments of this application, the method flow according to embodiments of this application can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by processor 701, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0158] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0159] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0160] For example, according to embodiments of this application, a computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0161] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this application. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the methods provided in the embodiments of this application.

[0162] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc. described above can be implemented by computer program modules.

[0163] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0164] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0165] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations are not explicitly described in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments of this application can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of this application.

[0166] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. This application does not depart from its scope, and those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A method for generating a social media analytics report, comprising: In response to a data collection command corresponding to a data collection requirement, an intelligent agent is used to collect web page collection data from a target web page related to the data collection command. The web page collection data includes screenshots of the target web page, document structure information, and user comment data. Based on the document structure information, the webpage screenshot is semantically parsed using a preset visual model to obtain target semantic information corresponding to each user comment data. The target semantic information represents comment attributes that are not included in the document structure information and the user comment data and are unrelated to the comment content of the user comment data. Based on the document structure information, the user comment data, and the target semantic information, an information summary page is generated. The information summary page includes multiple social comment data arranged according to the document structure. The social comment data includes user comment data posted by users at any point in time and the corresponding target semantic information. Based on the aforementioned comment attributes, data analysis is performed on multiple social comment data sets to obtain an analysis report.

2. The method according to claim 1, wherein, Based on the document structure information, a preset visual model is used to perform semantic parsing on the webpage screenshot to obtain target semantic information corresponding to each user comment data, including: Spatial coordinate alignment is performed on the webpage screenshot and document structure information to obtain position mapping information, wherein the position mapping information represents the position of the webpage element in the document structure information in the webpage screenshot; Based on the location mapping information, the webpage screenshot is semantically parsed using the preset visual model to obtain the target semantic information.

3. The method according to claim 1, wherein, Based on the aforementioned comment attributes, data analysis is performed on multiple sets of social comment data to obtain an analysis report, including: Based on the comment attributes, abnormal evaluations are filtered and removed from multiple social comment data to obtain multiple target comment data. The abnormal evaluations indicate that the similarity between multiple user comment data published by a user within a preset time period meets a similarity threshold. Based on different prompt word templates, the target comment data is analyzed using a preset language model to generate comment analysis reports corresponding to different scenarios.

4. The method according to claim 3, wherein, The comment attributes include user random ID information; Specifically, based on the comment attributes, abnormal comments are filtered and removed from multiple social comment data sets to obtain multiple target comment data sets, including: Based on the aforementioned comment attributes, multiple social comment data are filtered and removed for abnormal evaluations using Jaccard similarity to obtain multiple first evaluation data. Based on cosine similarity, multiple first evaluation data are labeled as abnormal, resulting in multiple second evaluation data with abnormal labels; For each user's random ID information, based on the second evaluation data with the abnormal label and the target attribute information, multiple first evaluation data of the user's random ID information are filtered to obtain multiple target comment data, wherein the target attribute information represents the attribute information of the comment published by the user.

5. The method according to claim 4, wherein, Based on the aforementioned comment attributes, abnormal evaluations are filtered and removed from multiple social comment data using Jaccard similarity, resulting in multiple first evaluation data, including: For each user's random ID information, calculate the Jaccard similarity information between each two social comment data corresponding to the user's random ID information; If the number of Jaccard similarity information that meets the similarity threshold among the multiple Jaccard similarity information is the target number, social comment data related to the user's random number information is deleted from the multiple social comment data to obtain multiple first evaluation data.

6. The method according to claim 4, wherein, Based on cosine similarity, multiple first evaluation data are labeled as having anomalies, resulting in multiple second evaluation data with anomaly labels, including: Using a pre-trained language model, semantic vectors are encoded on multiple first evaluation data sets to obtain multiple semantic encoded features; For each user random number information, calculate the cosine similarity between every two semantic coding features among multiple semantic coding features related to the user random number information; Calculate semantic consistency data based on multiple cosine similarities; If the semantic consistency data meets the consistency threshold, the first evaluation data related to the user's random number information is marked, and multiple second evaluation data with abnormal labels are also marked.

7. The method according to claim 4, wherein, The target attribute information includes at least one of the following: the sentiment classification label of the first evaluation data, comment interaction data, the device address that published the first evaluation data, and the comment publication time; Specifically, based on the second evaluation data with abnormal tags and target attribute information, multiple first evaluation data of the user random number information are filtered to obtain multiple target comment data, including: For each user's random ID information, an attribute evaluation result is generated based on the target attribute information of multiple first evaluation data, wherein the attribute evaluation result includes at least one of sentiment polarity result, interaction sparsity result, address clustering result, and posting frequency result; Based on the emotional polarity results, interaction sparsity results, address clustering results, and posting frequency results, an abnormal tendency score is calculated; If the abnormal tendency score meets the scoring threshold and at least one second evaluation data of the user's random number information has an abnormal label, the evaluation data corresponding to the user's random number information is removed from the multiple first evaluation data to obtain multiple target comment data.

8. The method according to claim 1, wherein, In response to a data acquisition command corresponding to a data acquisition requirement, an intelligent agent is used to acquire web page acquisition data from a target web page associated with the data acquisition command, including: The input data acquisition requirement information is processed using a preset language model to obtain multiple task elements, and the data acquisition instruction is generated based on the multiple task elements. In response to the data collection command, the intelligent agent is invoked to load the target page corresponding to the data collection command through the target browser, and to obtain the document structure information and user comment data; Take a screenshot of the target page to obtain the webpage screenshot.

9. The method according to claim 8, wherein, Invoking the intelligent agent to load the target page corresponding to the data collection instruction through the target browser includes: The intelligent agent loads the target page by scrolling based on human scrolling behavior patterns, wherein the human scrolling behavior patterns include scrolling speed, pause frequency, and mouse trajectory jitter. When the target page is being loaded via scrolling, the agent simulates clicking the page loading control to load the page, and determines whether to load a new target page based on whether the user comment data on the page has changed.

10. The method according to claim 8 or 9, further comprising: If loading the target page based on the collection path fails, switch the collection path to load the target page; and / or Before loading the target page, log in to the target page using the obtained access token; and / or If the page prompts for verification information during the loading of the target page, the target page is reloaded based on a behavior avoidance strategy, wherein the behavior avoidance strategy includes at least one of a delayed login strategy, a proxy pool switching strategy, and a browser state switching strategy.

11. The method according to claim 3, wherein, Based on different prompt word templates, the target comment data is analyzed using a preset language model to generate comment analysis reports corresponding to different scenarios, including: Based on the input classification prompts, a pre-defined language model is used to classify and label different target comment data, resulting in multiple target comment data with classification labels; Based on different scenario keywords, the target comment data in the database is analyzed using a preset language model to generate comment analysis reports corresponding to different scenarios, wherein the analysis report includes the comment analysis report.

12. The method of claim 11, further comprising: Clustering is performed on the target comment data in the database based on the classification labels to obtain the clustering results; The target keywords are determined based on the clustering results, and a discussion popularity change chart is generated based on the clustering results of the target keywords and different target keywords in different time periods. The analysis report also includes the discussion popularity change chart.

13. An apparatus for generating a social media analytics report, comprising: The acquisition module is used to respond to a data acquisition command corresponding to the data acquisition requirement, and use an intelligent agent to acquire web page acquisition data of the target web page related to the data acquisition command. The web page acquisition data includes web page screenshots, document structure information and user comment data of the target web page. The module is used to perform semantic parsing on the webpage screenshot based on the document structure information and using a preset visual model to obtain target semantic information corresponding to each user comment data, wherein the target semantic information represents comment attributes that are not included in the document structure information and the user comment data and are unrelated to the comment content of the user comment data; The generation module is used to generate an information summary page based on the document structure information, the user comment data, and the target semantic information. The information summary page includes multiple social comment data arranged according to the document structure. The social comment data includes user comment data posted by users at any time and the corresponding target semantic information. The analysis module is used to perform data analysis on multiple social comment data based on the comment attributes and obtain an analysis report.

Images