A review text analysis method based on a multi-modal knowledge graph

Abstract

The application belongs to the technical field of natural language processing and artificial intelligence, and relates to a comment text analysis method based on a multi-modal knowledge graph. First, an evaluation dimension ontology corresponding to a target field and an associated multi-modal knowledge graph are constructed. For a current comment text, an evaluation dimension is determined according to deep semantics thereof, and relevant target knowledge features associated with the evaluation dimension are dynamically selected from the knowledge graph based on sub-elements corresponding to the evaluation dimension. The selected target knowledge features are dynamically assigned fusion weights based on the context semantics of the comment and are subjected to weighted fusion to generate knowledge-enhanced features. Fine-grained semantic classification results for at least one predefined evaluation dimension are output. The application effectively solves the problem of coarse semantic understanding and analysis granularity and difficulty in realizing dimension-level fine analysis by introducing an evaluation dimension ontology structure and a multi-modal knowledge graph and combining a dynamic knowledge selection and weight fusion mechanism, and realizes precise, reliable and interpretable fine-grained semantic analysis.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and artificial intelligence, and in particular to a method for analyzing comment text based on multimodal knowledge graphs. Background Technology

[0002] With the widespread adoption of the internet, user-generated online reviews have become a crucial source of information for evaluating the reputation of goods, services, and destinations. Automated semantic understanding analysis of this type of text (such as sentiment analysis, attribute evaluation, and dimensional scoring) can provide data support for businesses to understand user needs and improve service quality, and also offer references for consumer decision-making. Therefore, fine-grained semantic understanding of online text has become a key research direction in the field of natural language processing.

[0003] As application scenarios become more sophisticated, simply making coarse-grained judgments on text at the overall level (such as overall positive / negative, compliant / non-compliant) is no longer sufficient to meet actual business needs. The industry urgently needs an analytical method that can automatically extract specific evaluation information about different aspects of the analyzed object from text, integrate additional multimodal information such as images to conduct deep reasoning, and output credible and interpretable analytical conclusions.

[0004] Currently, mainstream aspect-level semantic understanding methods are mainly based on deep learning models, such as LSTM and Transformer neural network architectures that incorporate attention mechanisms. These methods, by designing aspect-feature pair extraction mechanisms, can identify semantic tendencies or attribute features in text that relate to specific aspects to a certain extent. However, existing technologies still have the following significant drawbacks:

[0005] 1. Insufficient Granularity and Structure in Analysis: Existing methods for analyzing review texts typically treat the entire review as a single input, outputting an overall sentiment (positive / negative), lacking a structured organization of analytical dimensions. Taking restaurant review analysis as an example, existing methods can identify the semantic tendency of "the taste is very good, but the price is too high," but they cannot accurately map "taste" and "price" to predefined evaluation dimensions such as "food quality" and "value for money." This makes it difficult for the analysis results to directly support precise decisions on product optimization or service improvement. Evaluation dimensions refer to a classification framework for evaluating products or services from different perspectives. For example, for mobile phone products, evaluation dimensions may include photography, performance, and appearance. Sub-elements refer to the specific sub-elements under each evaluation dimension. For example, sub-elements under the photography dimension may include night scene capabilities and portrait effects.

[0006] Secondly, the insufficient ability to fuse multi-source information exacerbates the aforementioned problems. Most existing technologies rely solely on a single text modality, neglecting the rich semantic information contained in multimodal data such as images and videos. In many scenarios, textual descriptions are vague or ambiguous, making it difficult to accurately define specific analytical elements based solely on text, thus limiting the accuracy of the analysis results.

[0007] In summary, existing technologies suffer from coarse-grained semantic understanding analysis, making it difficult to achieve fine-grained analysis at the dimensional level. Summary of the Invention

[0008] To address the problems of existing technologies, embodiments of the present invention provide a method for analyzing comment text based on a multimodal knowledge graph. The technical solution is as follows:

[0009] S1. Obtain a dataset from the target domain, the dataset including comment text data of multiple evaluation objects, image data associated with the comment text data, and corresponding spatiotemporal information; S2. Construct an evaluation dimension ontology structure corresponding to the target domain. The evaluation dimension ontology structure includes multiple predefined evaluation dimensions for the evaluation object, and each evaluation dimension is further associated with at least one predefined sub-element. S3. Based on the dataset, construct a multimodal knowledge graph associated with the ontology structure of the evaluation dimension; S4. Obtain the current comment text, perform deep semantic encoding on the current comment text, identify at least one evaluation dimension corresponding to its semantics, and generate text semantic features; S5. Based on the identified evaluation dimensions, determine the sub-elements corresponding to the evaluation dimensions, retrieve the associated features in the multimodal knowledge graph with the sub-elements as the core to form a candidate set, calculate the semantic correlation between the text semantic features and each feature in the candidate set, and finally select the target knowledge features based on the semantic correlation. S6. Based on the semantic features of the text, dynamically assign fusion weights to the target knowledge features and perform weighted fusion to generate knowledge-enhanced features; S7. Based on the knowledge enhancement features, output fine-grained semantic classification results for the identified at least one evaluation dimension; The fine-grained semantic classification results include: the confidence scores corresponding to the semantic classification results and the semantic category results; the importance scores of the evaluation elements; and the key evidence indicators supporting the judgment of the semantic classification results.

[0010] Optionally, the construction of the evaluation dimension ontology structure corresponding to the target domain includes: Collect historical datasets from the target domain, and preprocess the historical datasets to obtain preprocessed datasets; Evaluation elements are extracted based on the preprocessed dataset; Cluster analysis of the evaluation elements yields multiple dimensions; Based on the multiple dimensions, the evaluation words corresponding to each dimension are classified to obtain sub-elements, and the evaluation words are associated with the evaluation elements. The multiple dimensions are associated with and stored with the multiple sub-elements to obtain the evaluation dimension ontology structure corresponding to the target domain, and the multiple dimensions correspond one-to-one with the multiple sub-elements.

[0011] Optionally, constructing a multimodal knowledge graph associated with the evaluation dimension ontology structure based on the dataset includes: Based on the evaluation dimension ontology structure, the historical text data in the target domain dataset is structured to construct a static knowledge layer; Based on the evaluation dimension ontology structure, real-time text data in the target domain dataset is extracted to form a dynamic knowledge layer; Based on the evaluation dimension ontology structure, visual semantic features are extracted from the image data in the target domain dataset, and the visual semantic features are cross-modal associated with the corresponding entities in the multimodal knowledge graph to form a visual knowledge layer. The static knowledge layer, the dynamic knowledge layer, and the visual knowledge layer are associated and mapped to form a multimodal knowledge graph that integrates text, images, and spatiotemporal information.

[0012] Optionally, the step of dynamically assigning fusion weights to the target knowledge features based on the text semantic features, and performing weighted fusion to generate knowledge-enhanced features includes: Perform entity recognition on the current comment text and extract the evaluation element entities from the comment text; Based on the evaluation element entities, text description features, visual features, and historical evaluation statistical features are extracted from the corresponding nodes of the multimodal knowledge graph. Based on the contextual semantics of the current target comment text, dynamically calculate the fusion weights of the text description features, the visual features, and the historical evaluation statistical features; The knowledge enhancement features are generated based on the fusion weights of the text description features, the visual features, and the historical evaluation statistics features.

[0013] Optionally, the step of dynamically assigning fusion weights to the target knowledge features based on the text semantic features, and performing weighted fusion to generate knowledge-enhanced features further includes: The image associated with the current comment text is used to extract the visual features of the associated image; Based on cross-modal semantic alignment, the correlation between the visual features and the semantic features of the current comment text is determined; Based on the correlation, fusion weights are assigned to the visual features; The visual features are weighted according to the fusion weights. The weighted visual features are fused with the textual description features and the historical evaluation statistical features to generate the knowledge-enhanced features.

[0014] Optionally, steps S4 to S7 are implemented using an adaptive knowledge fusion neural network model.

[0015] Optionally, the neural network model for adaptive knowledge fusion is an attention-based GRU model for adaptive knowledge fusion.

[0016] Optionally, the adaptive knowledge fusion attention GRU model is obtained through three-stage policy training, which includes: Pre-training was performed on a general sentiment corpus; Make domain-specific fine-tuning to the commentary text in the target domain; Enhanced samples generated based on the multimodal knowledge graph are introduced for knowledge optimization training.

[0017] Optionally, the adaptive knowledge fusion attention GRU model includes a context-aware GRU encoding layer, a dynamic knowledge selector, a cross-modal attention mechanism layer, a decoupling layer, a collaborative reasoning layer, and an interpretable output layer connected in sequence.

[0018] Optionally, the adaptive knowledge fusion attention GRU model includes: The context-aware GRU encoding layer is used to perform deep semantic encoding on the current comment text to obtain its context-aware text semantic features. The dynamic knowledge selector dynamically selects relevant target knowledge features from the multimodal knowledge graph based on the text semantic features. The cross-modal attention mechanism layer: dynamically weights and fuses the text semantic features and the knowledge features through a cross-modal attention mechanism to generate knowledge-enhanced features; The decoupling layer is used to separate and decode the knowledge enhancement features into dimensional semantic features based on predefined evaluation dimensions. The collaborative reasoning layer is used to perform internal calibration and joint optimization of the dimensional semantic features to generate dimensional reasoning signals. The interpretable output layer outputs the fine-grained semantic classification result based on the dimensional inference signal.

[0019] The beneficial effects of the technical solution provided by the embodiments of this invention are as follows: First, this invention obtains a target domain dataset containing comment text and multimodal data, constructs a corresponding evaluation dimension ontology structure, and builds a multimodal knowledge graph associated with the ontology structure based on this dataset. That is, by fusing multi-source data of text and images to construct a domain knowledge graph, it breaks through the limitation of traditional methods that rely solely on a single text, laying a rich knowledge foundation for fine-grained analysis. In the analysis stage, firstly, the comment focus (dimension) is automatically identified through semantic encoding, providing accurate "search terms" for knowledge retrieval; secondly, only a subset of knowledge strongly related to the identified dimension is extracted from the massive multimodal knowledge graph, effectively filtering out irrelevant noise; finally, based on the semantic features of the text, fusion weights are dynamically assigned to different knowledge, thereby realizing a three-level innovative mechanism of accurate dimension identification → targeted knowledge retrieval → dynamic weighted fusion, laying a complete technical foundation for fine-grained and interpretable analysis of comment text. Finally, by targeting knowledge enhancement features, fine-grained semantic classification results for at least one evaluation dimension are output, and each dimension includes a structured report containing the semantic tendency, judgment confidence, element importance, and key decision-making basis of each dimension. This makes the analysis results auditable and trustworthy, allowing users to clearly understand the origin of their decisions. It effectively solves the problem of coarse granularity in existing semantic understanding analysis methods, and achieves accurate, reliable, and interpretable fine-grained semantic analysis, significantly improving its practical value in scenarios such as service quality diagnosis and product optimization. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a method for analyzing comment text based on a multimodal knowledge graph, as shown in an embodiment of the present invention. Figure 2 This is the accuracy curve of the GRU model of this invention; Figure 3 This is a graph showing the loss rate of the GRU model. Figure 4 It is the confusion matrix of the GRU model; Figure 5 This is a line graph showing the negative review rate of A-level scenic spots in southern Xinjiang. Detailed Implementation

[0022] This invention provides a method for analyzing comment text based on a multimodal knowledge graph. (See also...) Figure 1 The method includes: Step S1: Obtain the dataset of the target domain, which includes comment text data of multiple evaluation objects, image data associated with the comment text data, and spatiotemporal information.

[0023] The target field can be tourist attractions, e-commerce products, or local life services, and this embodiment of the invention does not specifically limit it.

[0024] Furthermore, spatiotemporal information includes not only geographical location but also time information, making it a complete concept of "time + space".

[0025] In addition, obtaining the dataset for the target domain includes the following steps: S11. Collect raw multimodal data of the target domain from multiple publicly available Internet data sources using web data collection tools.

[0026] In this invention, the multimodal data refers to image data (such as product display images and user-taken photos) associated with comment text data; and comment text data (such as ratings, time, and location).

[0027] S12. After the original multimodal data is collected, the original multimodal data is cleaned and preprocessed, including removing irrelevant characters and duplicate data, performing text segmentation, and associating and storing images with corresponding comments to form a structured dataset of the target domain.

[0028] The dataset was obtained through data acquisition tools, including web crawlers, data interfaces of social media and OTA platforms, image data acquisition tools, and supporting data cleaning and preprocessing tools.

[0029] It should be noted that the data acquisition tool in the embodiments of the present invention may be one or more, and the embodiments of the present invention do not make specific limitations on it.

[0030] In addition, social media platforms include OTA platforms such as Ctrip, Meituan, and Mafengwo, as well as Weibo and Xiaohongshu.

[0031] It should be noted that this invention enables the automatic and accurate association of comment content with specific geographical locations (such as distinguishing between the Terracotta Warriors in Xi'an and Kanas in Xinjiang Uygur Autonomous Region), laying the foundation for subsequent fine-grained and regionalized analysis.

[0032] Meanwhile, to ensure the geographic accuracy of subsequent analysis, each collected comment text data includes the following core location metadata: Precise location identifier: the official name of the scenic area and its standard geographic location ID (e.g., 'G_CAT0000001' corresponds to 'Kanas Scenic Area'); Administrative level information: four-level administrative codes: country, province, city, and district / county; Spatiotemporal stamp: the comment's publication timestamp (accurate to the second) and the consumption time (e.g., check-in date, visit date); User location clues: the user's self-filled location (e.g., 'a tourist from Xi'an') and anonymized network location tags.

[0033] Step S2: Construct an evaluation dimension ontology structure corresponding to the target domain. The evaluation dimension ontology structure contains multiple predefined evaluation dimensions for the evaluation object, and each evaluation dimension is further associated with at least one sub-element.

[0034] The evaluation dimension ontology structure takes the comment target as the core entity and includes multiple predefined evaluation dimensions and their sub-elements. The comment target is the specific target entity evaluated by the user in the comment text.

[0035] In this embodiment of the invention, the predefined evaluation dimensions are, for example, 'eating, accommodation, transportation, sightseeing, shopping, and entertainment' in the tourism field; and 'quality, price, logistics, service, and appearance' in the e-commerce field.

[0036] Additionally: The construction process of this evaluation dimension ontology structure includes the following steps: S21: Collect historical datasets in the target domain, preprocess the historical datasets, and obtain the preprocessed datasets.

[0037] S22: Extract evaluation elements based on the preprocessed dataset.

[0038] In this process, named entity recognition technology is used to extract entities mentioned in the comments from the preprocessed dataset, and dependency parsing or semantic role labeling is used to identify the evaluation words related to these entities, forming "evaluation object-evaluation word" pairs.

[0039] S23: Cluster analysis is performed on the above-extracted evaluation elements to obtain multiple dimensions.

[0040] Cluster analysis can be either K-means clustering or hierarchical clustering.

[0041] It should be noted that the embodiments of the present invention are illustrated using K-means clustering or hierarchical clustering as examples, and the embodiments of the present invention do not specifically limit this.

[0042] Furthermore, the method for cluster analysis of evaluation factors can be found in relevant technologies, and will not be described in detail in this embodiment of the invention.

[0043] For example, all objects related to "pilaf," "baked buns," and "restaurant" are clustered into one category, named the "eating" dimension. Similarly, "parking lot," "cable car," and "hiking trail" are clustered into the "travel" dimension. The features used for clustering can be word vector similarity, co-occurrence relationships, etc.

[0044] S24: Based on these multiple dimensions, the evaluation words corresponding to each dimension are further classified to obtain sub-elements, and the evaluation words are associated with the evaluation elements.

[0045] The sub-elements are further refined: within each dimension, the associated "evaluation words" are further categorized. For example, under the "eating" dimension, "delicious" and "salty" can be classified as the "taste" sub-element; "expensive" and "affordable" can be classified as the "price" sub-element.

[0046] S25: Associate and store the multiple dimensions with the multiple sub-elements to obtain the evaluation dimension ontology structure corresponding to the target domain, and store the multiple dimensions and the multiple sub-elements in a one-to-one correspondence to obtain the "dimension-sub-element" hierarchical system.

[0047] The “dimension-sub-element” hierarchy is formally stored using an ontology description language (such as OWL) or a simple tree / graph structure, thereby forming a machine-readable and reasonable evaluation dimension ontology structure.

[0048] Furthermore, constructing the evaluation dimension ontology structure indicates that the technical action is to create an ontology. "Ontology," a computer science term, refers to a clearly defined specification describing concepts, attributes, and their interrelationships within a specific domain. This invention emphasizes that this ontology structure is a customized product deeply bound to a specific "target domain." It is not a general natural language ontology, but rather semantically adapted to the specific business characteristics of the target domain (such as catering, electronic products, etc.), thereby endowing the system with the ability to "understand" domain-specific terminology and implicit logic when processing data in that domain.

[0049] It's important to note that the core entity is the review target, and all concepts revolve around the object being evaluated (such as a scenic spot or a product), ensuring the analysis stays focused on the core. The ontology includes multiple predefined evaluation dimensions and their sub-elements: defining the core content and hierarchical structure. Predefined evaluation dimensions are the primary categories of the analysis (e.g., "eating, accommodation, transportation, sightseeing, shopping, and entertainment" in tourism). Sub-elements are further refinements of each dimension (e.g., "eating" can be subdivided into "taste, environment, price, and service"), thus enabling fine-grained analysis.

[0050] Step S3: Based on the dataset, construct a multimodal knowledge graph associated with the ontology structure of the evaluation dimension.

[0051] The construction of a multimodal knowledge graph associated with the ontology structure of this evaluation dimension, based on this dataset, includes the following steps: Step S31: Based on the ontology structure of this evaluation dimension, perform structured processing on the historical text data in the dataset of the target domain to construct a static knowledge layer.

[0052] In this process, entities, attributes, and relationships are extracted from the target domain dataset and used as historical text data to form a structured knowledge layer that represents stable facts within the domain.

[0053] The specific implementation steps are as follows: S311. Entity and Attribute Extraction: Using named entity recognition technology, identify and extract entities (such as attraction names, facilities, and dishes) and their attributes (such as location, price, and opening hours) belonging to predefined evaluation dimensions and their sub-elements from structured or semi-structured data (such as official introductions and encyclopedia entries).

[0054] S312. Relationship Extraction and Construction: Based on semantic rules or relation extraction models, establish relationships between entities (e.g., [Kanas Scenic Area] - [Contains] -> [Fish Viewing Platform]), and relationships between entities and attributes (e.g., [Fish Viewing Platform] - [Altitude] -> "2030 meters").

[0055] S313. Knowledge Fusion and Storage: Disambiguate, align and merge knowledge of the same entity from different sources to form a unified entity profile and store it in a graph structure.

[0056] When constructing the static knowledge layer of a multimodal knowledge graph, the following steps are used to achieve accurate identification and association of geographic location entities: Step 1: Place name entity identification and standardization.

[0057] A pre-trained natural language processing algorithm is used to extract geographic entities from comment text. For the identified entities, a standardized mapping is performed using a 'Chinese Tourist Attraction Standard Name Database'. For example, user-mentioned terms like 'Terracotta Warriors', 'Terracotta Warrior Pit', and 'Qin Shi Huang Terracotta Warriors' are uniformly mapped to the standardized entity 'Terracotta Warriors Museum'.

[0058] Step 2: Multi-level geocoding and hierarchy construction.

[0059] The standardized scenic area entities are parsed using geocoding services (such as Baidu Maps API and Gaode Maps API) to obtain their precise latitude and longitude coordinates, and are automatically associated with their respective city, province, country, and other higher-level geographical entities, thereby constructing a complete hierarchical relationship network of 'country-province-city-scenic area' in the knowledge graph.

[0060] Step 3: Geographic location disambiguation.

[0061] To address the issue of ambiguity surrounding attractions with the same name (such as multiple "West Lakes" across the country), a disambiguation strategy will be implemented: Context disambiguation: Check if the comment body contains higher-level address information (such as 'Hangzhou West Lake' vs 'Fuzhou West Lake').

[0062] User information disambiguation: Refer to the location information in the user's profile.

[0063] Collaborative disambiguation: If the above information is missing, inferences are made based on other related entities mentioned in the comments (such as the co-occurrence of 'West Lake' and 'Leifeng Pagoda').

[0064] Step 4: Cross-modal position association For user-uploaded images, extract the GPS coordinates from their EXIF ​​information (if they exist), or use image recognition models to identify landmarks and buildings in the images and associate them with geographical location entities in the knowledge graph.

[0065] It should be noted that this invention provides a unified multimodal knowledge graph for each domain, integrates data from multiple regions, supports cross-regional correlation analysis, and provides nationwide tourism information services.

[0066] Step S32: Based on the ontology structure of this evaluation dimension, extract real-time text data from the dataset of the target domain to form a dynamic knowledge layer.

[0067] From real-time or near real-time data streams in the target domain (such as the latest comment text), newly emerging and dynamically changing knowledge is extracted to form a knowledge layer representing the evolution of user opinions and emotions. To construct this dynamic knowledge layer, the following steps are used: S321. Real-time extraction of evaluation elements and opinions: Perform syntactic analysis and sentiment calculation on the real-time incoming comment text to dynamically identify the evaluation elements mentioned (such as [the road up the mountain]) and the corresponding opinion words and sentiment tendencies (such as [steep] -> negative).

[0068] S322. Dynamic Relationship Establishment: The extracted viewpoints and emotions are used as dynamic attributes or relationships and associated with the corresponding entity nodes in the static knowledge layer (e.g., associating the viewpoint [steep] with the [road condition] attribute of the entity [uphill road]).

[0069] Step S33: Based on the ontology structure of this evaluation dimension, extract visual semantic features from the image data in the dataset of the target domain, and perform cross-modal association between the visual semantic features and the corresponding entities in the multimodal knowledge graph to form a visual knowledge layer.

[0070] Among these, the process of extracting visual semantic information related to evaluation elements from user-uploaded image data and their descriptive text to form a knowledge layer representing visual content includes: S331. Visual Feature Extraction: Using a pre-trained deep convolutional neural network, extract high-level visual semantic features from images or generate visual concept labels (such as [snow mountain], [lake], [crowded crowd]).

[0071] S332, Visual-Text Alignment: By combining the title, label, or surrounding text description of an image, cross-modal alignment technology is used to establish the association between visual features and text entities / concepts (e.g., the image label [clear lake water] is associated with the [water quality] attribute of the entity [Kanas Lake]).

[0072] Step S34: The static knowledge layer, the dynamic knowledge layer, and the visual knowledge layer are associated and mapped to form a multimodal knowledge graph that integrates text, images, and spatiotemporal information.

[0073] Specifically, the mapping between the dynamic knowledge layer and the visual knowledge layer includes: S341, Cross-layer entity alignment: Identify entities in different knowledge layers that point to the same real-world object (such as the entity [Guanyutai] in the static layer, the "viewing platform" mentioned in the comments in the dynamic layer, and the [viewing platform building] labeled in the image in the visual layer), and associate them with the same graph node.

[0074] S342, Cross-modal relation links: Establish cross-layer relation edges.

[0075] For example, the positive emotion expressed in a dynamic comment "The view from Guanyutai is expansive" (dynamic layer) can be linked to the entity [Guanyutai] (static layer), and simultaneously linked to an image feature that shows the expansive view (visual layer).

[0076] This invention provides a structured semantic framework and a rich background knowledge base for sentiment analysis by pre-constructing an "evaluation dimension ontology" closely integrated with the domain and organizing a multimodal knowledge graph on this basis. This enables the system to automatically and accurately map unstructured comment text to specific evaluation dimensions (such as "eating," "housing," and "transportation") and their sub-elements, thereby outputting independent sentiment judgments for each dimension and element. This completely overcomes the limitation of traditional methods that can only provide overall sentiment polarity and provides atomic analysis units for in-depth insights.

[0077] S4. Obtain the current comment text, perform deep semantic encoding on the current comment text, identify at least one evaluation dimension corresponding to its semantics, and generate text semantic features.

[0078] Step S5: Based on the identified evaluation dimension, determine the sub-elements corresponding to the evaluation dimension, retrieve the associated features in the multimodal knowledge graph with the sub-elements as the core to form a candidate set, calculate the semantic correlation between the text semantic features and each feature in the candidate set, and finally select the target knowledge features based on the semantic correlation.

[0079] The target knowledge feature is a "structured feature package that aggregates relevant textual, visual, and statistical knowledge." This target knowledge feature includes vectorized representations of relevant sub-elements, entities, attributes, and their relationships retrieved from the knowledge graph. Target knowledge features refer to representations that can characterize certain attributes of input data and can be used by machine learning models for prediction or analysis; their specific forms can be, but are not limited to, feature vectors, feature embeddings, tensors, etc. In this embodiment of the invention, the target knowledge feature is simply illustrated using the target knowledge feature vector as an example.

[0080] Specifically, knowledge feature retrieval and selection based on sub-elements includes: S51 Sub-element mapping: Based on the evaluation dimension identified in step S4, query the ontology structure of the evaluation dimension and determine at least one sub-element pre-associated with the evaluation dimension.

[0081] S52 Graph Retrieval and Candidate Set Construction: Using the sub-elements determined in step S51 as core query anchors, perform graph traversal query in the multimodal knowledge graph to retrieve entity nodes, attribute nodes, and relation edges that are directly or indirectly connected to these sub-element nodes through predefined relational paths, thus forming an initial candidate feature set.

[0082] S53 Cross-modal semantic relevance calculation: For each candidate feature in the initial candidate feature set, extract its multimodal feature representation (such as text description vector, visual feature vector, numerical attribute), and calculate its semantic similarity with the text semantic feature vector generated in step S4 to obtain the first relevance score of each candidate feature.

[0083] S54 Structural Importance Fusion and Comprehensive Ranking: Based on the topology of the multimodal knowledge graph, calculate the graph structure importance score of each candidate feature node (e.g., based on the centrality index or its weight in a local subgraph centered on a sub-feature); weight and fuse the first relevance score with the graph structure importance score to generate a comprehensive score, and sort the initial candidate feature set in descending order based on this score.

[0084] S55 Target Feature Subset Generation: Based on a preset filtering strategy (such as selecting the Top-K features or selecting features with a comprehensive score higher than the threshold), features are extracted or selected from the sorted list to form the final target knowledge feature subset.

[0085] S6. Based on the semantic features of the text, dynamically assign fusion weights to the target knowledge features and perform weighted fusion to generate knowledge-enhanced features.

[0086] Specifically, the knowledge enhancement feature can be obtained through steps S61 to S63.

[0087] S61: Extract the evaluation element entities from the current target comment text.

[0088] Among them, the named entity recognition model is used to identify the evaluation element entities in the comment text. For example, from the comment text "Roast duck has crispy skin and tender meat, and is served with cucumber and scallions", the evaluation element entities "roast duck", "cucumber" and "scallions" are identified.

[0089] S62: Based on the evaluation element entity, extract text description features, visual features, and historical evaluation statistical features from the corresponding nodes of the multimodal knowledge graph.

[0090] Among them, extracting text description features, visual features, and historical evaluation statistical features from the corresponding nodes of the multimodal knowledge graph is a semantic extraction based on a pre-trained language model (extended syntax).

[0091] It should be noted that semantic extraction (extended syntax) based on pre-trained language models can refer to existing technologies, and will not be described in detail in the embodiments of this invention.

[0092] For example, the identified evaluation element entities can be linked to entities in a multimodal knowledge graph to establish a relationship between the review text and the knowledge graph. For instance, "roast duck" can be linked to the "roast duck" entity node in the knowledge graph.

[0093] S63: Based on the contextual semantics of the current comment text, dynamically calculate the fusion weight of the text description feature, the fusion weight of the visual feature, and the fusion weight of the historical evaluation statistical feature.

[0094] The phrase "dynamically calculating the fusion weight based on the contextual semantics of the current comment text" refers to a weight allocation method based on an attention mechanism. Its core idea is that the fusion weight is not a pre-set fixed value, but a dynamic value calculated in real-time based on the specific semantic content of the current comment text.

[0095] In addition, the specific weight allocation methods based on attention mechanisms include: S631: Semantic relevance calculation.

[0096] Using the deep semantic encoding features of the current comment text as the query vector, similarity matching calculations are performed with knowledge vectors (Keys) such as text description features, visual features, and historical evaluation statistical features extracted from the knowledge graph (e.g., using algorithms such as dot product and cosine similarity) to obtain the initial relevance score between each feature and the current comment.

[0097] Functional example: When a comment reads "The roast duck has a bright red color, which looks very appetizing," the semantic focus is on "color" and "appearance." The system will calculate that the similarity with visual features (such as the visual vector of a dish image) is significantly higher than the similarity with historical statistical features (such as average ratings).

[0098] S632: Weight normalization and allocation.

[0099] The initial relevance scores obtained above are processed by a normalization function (such as the Softmax function) to transform them into a set of interpretable fusion weights that sum to 1.

[0100] Functional example: Continuing from the previous example, the processed weight allocation might be: visual feature weight: 0.7, text description feature weight: 0.2, historical statistical feature weight: 0.1. This accurately reflects that visual evidence is the most important in the current commentary context.

[0101] This invention, through the aforementioned dynamic calculation mechanism, achieves: Context awareness: Weight allocation is entirely driven by the semantics of the current comment, realizing true "text-specific" weighting. Adaptive focus: Automatically strengthens the contribution of evidence most relevant to the current comment, while weakening the interference of irrelevant or noisy information. Interpretability basis: The generated weights themselves reflect the "emphasis" on which the model makes decisions, providing a basis for the final interpretable output.

[0102] Therefore, the step of dynamically calculating the fusion weights of the text description features, the visual features, and the historical evaluation statistical features based on the contextual semantics of the current comment text represents a fundamental shift from "static weighting" to "dynamic weighting." Regardless of the type of comment, the system can automatically and adaptively allocate the most reasonable fusion weights, ensuring that the knowledge-enhanced features generated after fusion remain highly consistent with the semantic focus of the current comment. This is the key to improving the accuracy of sentiment analysis in this invention.

[0103] S64: Generate the target knowledge feature based on the fusion weight of the text description feature, the fusion weight of the visual feature, and the fusion weight of the historical evaluation statistical feature.

[0104] In one embodiment of the present invention, the weighted fusion calculation is implemented by the following formula: Target knowledge features = (W_text·F_text)⊕(W_visual·F_visual)⊕(W_stat·F_stat).

[0105] Wherein: W_text represents the fusion weight of text description features; W_visual represents the fusion weight of visual features, reflecting the degree of correlation between image information and the semantics of the current text; W_stat represents the fusion weight of historical evaluation statistical features, reflecting the statistical significance of this dimension in historical evaluations; F_text is the text description feature, obtained by deep semantic encoding of the current comment text by a pre-trained language model; F_visual is the visual feature, extracted from images associated with the current evaluation entity in the graph by a visual model; F_stat is the historical evaluation statistical feature, obtained based on historical comment data in the target domain, including information such as evaluation frequency and sentiment polarity distribution; · represents the element-wise multiplication of features and weights, used to weight and modulate each feature; ⊕ represents the feature fusion operation, in this embodiment, the three weighted features are fused into the final target knowledge feature by element-wise addition.

[0106] Furthermore, dynamic weighting enables attention focus at the feature level: A high-weight feature enhancement mechanism: When a feature is highly relevant to the current comment's semantics, it receives a higher fusion weight. For example, for comments primarily based on visual description, visual features have significantly higher weights than text features. The contribution of visual information will be emphasized in the target knowledge features. A low-weight feature suppression mechanism: Features with low semantic relevance to the current comment receive lower weights, effectively suppressing the noise impact of irrelevant features and improving the semantic purity of the target knowledge features.

[0107] Through the above-described weighted fusion mechanism, the present invention achieves the following technical effects: 1) Knowledge enhancement effect: The generated target knowledge features not only contain the semantic information of the original comment text, but also incorporate multimodal external knowledge, forming a richer feature representation.

[0108] 2) Noise suppression effect: By automatically weakening the contribution of irrelevant features through weight allocation, the signal-to-noise ratio of feature representation is improved.

[0109] 3) Context adaptation effect: The generated target knowledge features are fully adapted to the semantic context of the current comment, achieving true dynamic knowledge fusion. The generated target knowledge features will serve as input for the subsequent sentiment analysis module, providing a high-quality feature representation foundation for fine-grained sentiment classification. These features retain the semantic information of the comment itself while incorporating relevant multimodal knowledge evidence, providing ample information support for accurate sentiment judgment.

[0110] If the current comment text is associated with an image, then the visual features of the associated image are extracted first, and the extraction of the visual features of the associated image is carried out according to the following steps: S65: Based on cross-modal semantic alignment, determine the correlation between the visual feature and the semantic features of the current comment text.

[0111] The cosine similarity method is used to calculate the correlation between visual features and text semantic features. The specific calculation steps include: S651. Perform L2 normalization on the text semantic feature F_text and the visual feature F_visual respectively. S652. Calculate the dot product of the normalized vectors to obtain the correlation score S.

[0112] The correlation score S is calculated using the cosine similarity formula.

[0113] Output: Correlation score S, ranging from [-1, 1]. A larger S value indicates a stronger correlation.

[0114] S66: Based on this correlation, assign fusion weights to this visual feature.

[0115] The fusion weights are assigned based on correlation, using the softmax function. The implementation method involves element-wise multiplication of the visual feature vectors with the fusion weights.

[0116] S67: The weighted visual features are fused with the text description features and the historical evaluation statistical features to generate the knowledge-enhanced features.

[0117] In this embodiment, weighted visual features are fused with textual description features and historical evaluation statistics to generate knowledge-enhanced features. Specifically, this includes the following: First, the weighted visual features, textual description features, and historical evaluation statistics are processed for dimensional consistency to ensure they have consistent dimensions in the same feature space. Then, the three types of features are concatenated along their feature dimensions to obtain a concatenated feature containing textual semantic information, visual perception information, and historical statistical information. Next, the concatenated feature is input into a fully connected layer. Through linear transformation and nonlinear activation of the fully connected layer, features of different modalities are deeply fused, and the high-dimensional concatenated feature is mapped to a preset feature dimension. Finally, the fused feature vector is output as a knowledge-enhanced feature. This feature simultaneously contains textual semantics, visual evidence, and historical statistical patterns, and is used for subsequent fine-grained sentiment classification. By dynamically assigning fusion weights to the target knowledge features based on textual semantic features and performing weighted fusion, the generated knowledge-enhanced feature can automatically ignore irrelevant environmental images and focus on the dish itself, weighing evidence: knowing that judging "color" relies on images, and judging "price" requires referring to historical statistics. Denoising and purification: Through a weighted mechanism, important signals are amplified while noise interference is suppressed. The resulting knowledge-enhanced feature is a high-quality information representation that integrates the most relevant visual evidence, textual descriptions, and historical data, providing a solid basis for subsequent accurate judgments.

[0118] Step S7: Based on the knowledge-enhanced features, output fine-grained semantic classification results for the identified at least one evaluation dimension.

[0119] The adaptive knowledge fusion model proposed in this invention is used to implement steps S4 to S7. Its core lies in a dynamic knowledge selector and a cross-modal attention mechanism. This mechanism, like a human expert, proactively and intelligently filters the most relevant information fragments (such as specific historical viewpoints and relevant visual evidence) from a vast multimodal knowledge graph based on the real-time semantic context of the current comment text, and assigns them appropriate fusion weights. This "on-demand, dynamic fusion" mechanism greatly improves the utilization efficiency and synergistic effect of multi-source heterogeneous information, and significantly enhances the model's understanding accuracy and robustness in complex scenarios. In other words, it achieves intelligent knowledge fusion from "static weighting" to "context-aware, dynamic selection."

[0120] The neural network model for adaptive knowledge fusion is the adaptive knowledge fusion attention GRU model.

[0121] In addition, this adaptive knowledge fusion attention GRU model is obtained through three-stage policy training.

[0122] Furthermore, the three-stage strategy training includes the following steps.

[0123] Step S71: Pre-train on a general sentiment corpus.

[0124] For example, massive amounts of publicly available review texts containing both positive and negative tags can be collected, such as IMDb movie reviews and Yelp business reviews. The model is trained on this data, primarily focusing on sentiment classification, learning vocabulary, syntax, and basic sentiment polarity associations to obtain high-quality initial model parameters. This stage is equivalent to the model's "general education."

[0125] Step S72: Perform domain fine-tuning on the target domain comment text.

[0126] Building upon general pre-training, the model is fine-tuned in a supervised manner using review text data from the target domain, adapting it to the specific expression habits and focus of the domain. Taking the tourism sector as an example, a training set is constructed using scenic spot review texts collected from OTA platforms and social media, each with user ratings (which can be mapped to sentiment tags). The model is then trained further on this dataset, familiarizing it with entities specific to the tourism sector (such as "attractions," "cable cars," and "guesthouses"), evaluation dimensions (such as "scenery," "service," and "transportation"), and expression methods, thus transforming it from a "generalist" into a "specialist" in the field of tourism review analysis.

[0127] Step S73: Introduce enhanced samples generated based on the multimodal knowledge graph for knowledge optimization training.

[0128] This stage is crucial for training the model to learn to reason using multimodal knowledge graphs. Based on the constructed multimodal knowledge graph, we automatically construct "knowledge-enhanced samples" to optimize the model's training. Enhanced sample construction: From the knowledge graph, we select an entity (e.g., "Laocun Flavor Restaurant") and its associated multimodal knowledge features (static attributes, historical viewpoint statistics, visual tags). Then, we automatically generate or match a comment text related to this entity from historical data (e.g., "This restaurant's hand-grabbed rice is very special"). This forms a training sample pair: (comment text, related knowledge points). Training objective and process: Using the constructed sample pair as input. During training, the model not only needs to judge sentiment based on the comment text, but its internal mechanisms (especially the dynamic knowledge selector and attention fusion layer) are also guided to learn how to pay attention to and use simultaneously input related knowledge features to assist decision-making. Through training with a large number of such samples, the model is strengthened in its ability to retrieve, filter, and fuse effective information from external knowledge bases based on text content, ultimately achieving a "knowledge-enhanced" reasoning mode.

[0129] It should be noted that through the three-stage training process of "general foundation -> domain focus -> knowledge enhancement" described above, the obtained model not only has a deep understanding of the language of the target domain, but also possesses the associative and reasoning abilities of an expert with an "industry knowledge base," laying a solid foundation for subsequent high-precision and interpretable fine-grained sentiment analysis.

[0130] The adaptive knowledge fusion attention GRU model includes a context-aware GRU encoding layer, a dynamic knowledge selector, a cross-modal attention mechanism layer, a decoupling layer, a collaborative reasoning layer, and an interpretable output layer.

[0131] In addition, the neural network model for adaptive knowledge fusion based on this feature vector includes: The context-aware GRU encoding layer is used to perform deep semantic encoding on the current comment text to obtain its context-aware textual semantic features.

[0132] This dynamic knowledge selector filters target knowledge features that are relevant to the current comment context from the multimodal knowledge graph based on the semantic features of the text.

[0133] The cross-modal attention mechanism layer dynamically weights and fuses the semantic features of the text with the knowledge features to generate knowledge-enhanced features.

[0134] This decoupling layer is used to separate and decode the knowledge enhancement features into sentiment judgment features based on predefined evaluation dimensions.

[0135] This collaborative reasoning layer is used to perform internal calibration and joint optimization of the sentiment judgment feature to generate a sentiment tendency signal.

[0136] The interpretable output layer outputs the fine-grained semantic classification result based on the sentiment tendency signal.

[0137] The fine-grained semantic classification result includes: the semantic classification result of at least one evaluation element and the confidence level of the semantic classification result.

[0138] In addition, the at least one evaluation element also includes an importance score for that evaluation element.

[0139] Furthermore, the fine-grained semantic classification result also includes key evidence indicators that support the judgment of the semantic classification result.

[0140] This invention uses the comment "The hand-grabbed rice at Laocun Flavor Restaurant is amazing, but it's expensive" as an example to illustrate the results obtained from each of the context-aware GRU encoding layer, dynamic knowledge selector, cross-modal attention mechanism layer, decoupling layer, collaborative reasoning layer, and interpretable output layer.

[0141] The context-aware GRU encoding layer understands the entire sentence and outputs semantic vectors encoding positive ('amazing') and negative ('expensive').

[0142] The dynamic knowledge selector selects 'taste'-related features (historical positive review rate, pictures of hand-grabbed rice) and 'price'-related features (historical complaint rate) from the feature package provided by S4.

[0143] The attention fusion layer assigns high weights to 'pictures of people eating with their hands' and 'historical positive reviews', and fuses them with the text semantics.

[0144] The decoupling layer separates the fused features into two independent channels: 'eating-taste' and 'eating-price'.

[0145] The collaborative reasoning module amplifies the 'positive taste' signal and may slightly suppress the strength of the 'negative price' signal due to the 'overall positive' signal.

[0146] Interpretable output layer output: [Taste: Strongly positive, confidence level 0.96, basis: Image A + historical positive reviews]; [Price: Negative, confidence level 0.75, basis: text mentions + historical complaints].

[0147] In addition, the at least one evaluation element also includes an importance score for that evaluation element.

[0148] For example, suppose there is a more complex review: "The hand-grabbed rice is amazing, the meat is so fragrant, and the milk tea is also very rich. But the price is indeed not cheap, 150 yuan per person." Without an importance score, the output will list the sentiment and confidence of three dimensions: "taste (hand-grabbed rice)," "taste (milk tea)," and "price."

[0149] The output with an importance score might be: [Taste - Hand-grabbed Rice]: Strongly positive, confidence level 0.97, importance score: 0.40 [Flavor - Milk Tea]: Positive, Confidence 0.85, Importance Score: 0.20 [Price]: Negative, Confidence 0.90, Importance Score: 0.40 Key evidence: [Description of "hand-grabbed rice is amazing"], [User-uploaded images of "hand-grabbed rice"], [Description of "not cheap"], [Statistics on historical price complaints].

[0150] Specifically, the importance score is generated as follows: When the model performs feature fusion through a cross-modal attention mechanism layer and dimensional separation through a decoupling layer, it generates a series of attention weight distributions for each evaluation element related to the current review. In the output phase, the system backtracks and aggregates all attention weights assigned to the same evaluation element. For example, for the element "taste," it might aggregate the attention weight of the word "amazing" in the review text, the fusion weight of the "historical positive review rate" feature extracted from the knowledge graph, and the visual attention weight of the user-uploaded food image. These weights are averaged (or weighted summed) and normalized to obtain the importance score of the element in the current review analysis. The score ranges from 0 to 1, with a higher score indicating a greater degree of user mention and model attention to the element in this review.

[0151] This invention features a specially designed interpretable output layer at the end of the model. This layer not only outputs the sentiment tendency for each dimension but also simultaneously outputs the confidence level of the judgment, the importance score of the elements, and key decision-making basis. This transforms the analysis results from an unreliable "black box" label into a quantifiable, traceable, and verifiable decision report. Users not only know "what" but also "how certain" and "why," greatly enhancing the credibility and practical value of the analysis results in high-end business decisions (such as service quality diagnosis and product defect tracing). In other words, it achieves an interpretable output that moves from "black box decision-making" to "white box traceability and quantifiable credibility." This embodiment provides a method for generating scenic area diagnostic reports based on fine-grained semantic classification results. This method, based on traditional sentiment analysis, significantly improves the accuracy and actionability of diagnostic reports by introducing interpretable output features.

[0152] This embodiment uses the tourism service sector in southern Xinjiang Uygur Autonomous Region as an application scenario, selecting the Kashgar Old City scenic area as a specific analysis object to demonstrate the complete process of the method of the present invention from data acquisition to fine-grained analysis result output. This embodiment aims to illustrate the feasibility and beneficial effects of the present invention in a specific field, but the scope of protection of the present invention is not limited to this embodiment.

[0153] For 38 scenic spots in southern Xinjiang (including but not limited to Kashgar Old City, Pamir Plateau, Taklamakan Desert, Tianchi Lake in Tianshan Mountains, etc.), multimodal data were collected from the following channels: Review text data: Approximately 100,000 user reviews of scenic spots in southern Xinjiang were collected by crawling travel platforms such as Ctrip, Mafengwo, and Meituan.

[0154] Related image data: Collected visual materials such as scenic spot pictures, food pictures, and accommodation pictures uploaded by users, totaling about 50,000 images.

[0155] Spatiotemporal information: Extract information such as posting time and geographic coordinates from comment metadata.

[0156] First, let's analyze the unique characteristics of tourism in southern Xinjiang: Geographical features: Located in southern Xinjiang Uygur Autonomous Region, it is a multi-ethnic region (mainly Uyghur), with deserts, plateaus, and oases coexisting; Tourism elements: It emphasizes both natural scenery and cultural experiences, highlighting distinctive cuisine, ethnic costumes, and homestay experiences; Tourist focus: Based on data mining of tourism reviews in southern Xinjiang, it was found that tourists most frequently mentioned six aspects: "eating, accommodation, transportation, sightseeing, shopping, and entertainment," and "clothing" (clothing experience) has special significance in southern Xinjiang.

[0157] Based on the above analysis, we construct the ontology structure of tourism evaluation dimensions in southern Xinjiang, and define the core dimensions and sub-elements, as shown below.

[0158] Evaluation Dimension Ontology Structure: "Eating": Sub-elements: ["Taste", "Environment", "Price", "Service"]; "Accommodation": "Sub-elements": ["Location", "Facilities", "Hygiene", "Value for Money"]; "Travel": Sub-elements: ["Transportation Convenience", "Internal Roads", "Parking"]; "Tourism": Sub-elements: ["Natural scenery", "Cultural landscape", "Crowding level", "Facility availability"]; "Purchase": Sub-elements: ["Product Features", "Price Reasonableness", "Shopping Experience"]; “Entertainment”: “Sub-elements”: [“Project richness”, “Uniqueness of experience”, “Safety”].

[0159] In step S3 of the present invention, the data in S1 is organized into a knowledge graph according to the ontology structure in S2: entity extraction: extracting entities such as attractions, services, and attributes from the data; relation construction: establishing semantic relationships between entities; multimodal fusion: integrating text, images, and spatiotemporal information.

[0160] Suppose the current review text reads: "The hand-pulled rice in Kashgar Old City is authentic, but the price is a bit high. The guesthouses are very unique, but rooms are scarce during peak season."

[0161] S4: Semantic Encoding and Dimension Recognition. Deep Semantic Encoding: The system uses a pre-trained model (such as BERT) to encode the comment text into a text semantic feature vector.

[0162] Dimensional Recognition: The system analyzes semantic features and identifies that the comment mainly involves: The food dimension (confidence 0.9) was chosen because it mentioned "hand-grabbed rice", "taste", and "price".

[0163] The accommodation dimension (confidence level 0.85) was selected because it mentioned "homestay", "unique features", and "room shortage".

[0164] S5: Dynamic Knowledge Retrieval (Core Step); Identify sub-elements: Based on the identified "eating" and "living" dimensions, locate their sub-elements.

[0165] Food dimensions → Taste and price.

[0166] Accommodation Dimensions → Features, Tension.

[0167] Candidate set retrieval: Using these sub-elements as the core, retrieve directly related features from the multimodal knowledge graph to form a candidate set.

[0168] For flavor elements: characteristics such as authentic and fresh were found.

[0169] For the price sub-factor: the search results showed characteristics such as relatively expensive, affordable, and cost-effective.

[0170] For the sub-elements of features and tension: the search returned features such as unique features and room tension.

[0171] Calculate semantic relevance: Calculate the similarity between the text semantic feature vector and each feature in the candidate set.

[0172] Target knowledge features: retain highly relevant features and filter out low-relevance features.

[0173] High relevance (retain): Taste - Authentic (correlation 0.92), Price - Relatively expensive (correlation 0.88), Features - Unique (correlation 0.90), Tension - Rooms in short supply (correlation 0.86).

[0174] Low relevance (filtered): Price - affordable (relevance 0.15), Taste - fresh and tender (relevance 0.10).

[0175] S6: Dynamic weighted fusion.

[0176] Weighting: The system assigns fusion weights to the target knowledge features selected by S5 based on the degree of emphasis of the text semantics.

[0177] Taste - Authenticity has a high weight (0.6) because the comments directly affirm "authentic taste".

[0178] Price - Slightly expensive (0.4), because the tone is "slightly expensive" rather than "extremely expensive".

[0179] Unique features - High weight for distinctive features (0.5).

[0180] Tension level - room tension weight (0.5).

[0181] Weighted fusion: This method fuses textual semantic features with weighted target knowledge features to generate knowledge-enhanced features. These features simultaneously incorporate both the original text semantics and external knowledge.

[0182] S7: Output of fine-grained semantic classification results.

[0183] Finally, based on this knowledge-enhanced feature, fine-grained semantic classification results are generated for the identified "eating" and "living" dimensions. The results are presented in a structured manner, including key information such as classification conclusions, confidence levels, element importance, and judgment criteria. The specific output is as follows:

[0184] Analysis results regarding the "eating" dimension: Semantic classification results: positive evaluation for the "taste" sub-element and negative evaluation for the "price" sub-element.

[0185] Overall confidence level: 0.89.

[0186] Sub-element importance scores: The importance score of the "taste" sub-element is 0.90, and the importance score of the "price" sub-element is 0.85.

[0187] Key evidence indicates that the judgment is mainly based on two aspects: Textual basis (60% contribution): The current comment text mentions "authentic taste".

[0188] Knowledge basis (contribution 40%): A strong correlation was found between "hand-grabbed rice" and the attribute "authentic taste" in the multimodal knowledge graph.

[0189] Analysis results regarding the "housing" dimension: Semantic classification results: positive evaluation for the "characteristics" sub-element, and negative evaluation for the "tension" sub-element.

[0190] Overall confidence level: 0.84.

[0191] Sub-element importance scores: The importance score for the "characteristics" sub-element is 0.88, and the importance score for the "tension" sub-element is 0.82.

[0192] Key evidence indicating that the judgment is based on a comprehensive assessment of three aspects: Textual basis (50% contribution): The current comment text mentions that "the homestay is very unique".

[0193] Knowledge Basis 1 (Contribution 30%): It was learned from the multimodal knowledge graph that the occupancy rate of homestays in the area exceeded 95% during the National Day holiday.

[0194] Knowledge Basis 2 (Contribution 20%): The knowledge graph records that "the homestay has unique features" is a common tourist review for this type of accommodation.

[0195] Ultimately, the system completed a fine-grained, interpretable semantic analysis of the comment text using the methods described above.

[0196] Optionally, the positive, neutral, and negative evaluation ratios for each evaluation element are calculated according to the scenic area dimension, and an interpretable diagnostic report containing information for each scenic area is generated, specifically including the following steps: S81. Calculate the positive, neutral, and negative evaluation ratios of each evaluation element according to the characteristics of the scenic area based on the analysis results; S82. Set confidence thresholds and sentiment statistics, and update the historical evaluation statistics and sentiment trend data of the corresponding element nodes in the multimodal dynamic knowledge graph; S83. Generate a report that includes a multi-dimensional emotional distribution heat map of each scenic area, a list of advantageous elements, a list of disadvantageous elements, and a key issue attribution analysis.

[0197] The specific steps for generating a report that includes a multi-dimensional emotional distribution heatmap for each scenic area, a list of advantageous elements, a list of disadvantageous elements, and a key issue attribution analysis include: S91: Evaluation ratio calculated by scenic area dimension (enhanced version).

[0198] Based on traditional statistics, confidence filtering and importance weighting mechanisms are introduced: Confidence filtering: Set a confidence threshold (e.g., 0.8) to only accept results with a confidence level higher than the threshold, thus ensuring statistical quality.

[0199] Importance Weighting: For each evaluation element, a weighted average is calculated based on its importance score. The importance score is calculated as follows:

[0200] Similarity between the overall semantic features of the comment text and the semantic features of the evaluation elements The attention mechanism is used to calculate the attention weight of each element in the comments. Combining the frequency of occurrence and emotional intensity of elements in historical commentary Statistical method: Grouping by both scenic area and evaluation element, calculating the weighted proportions of positive, neutral and negative evaluations, and obtaining the sentiment distribution data of each element in each scenic area.

[0201] Step S92: Update the knowledge graph and set the confidence level.

[0202] Set confidence threshold: Dynamically adjust the confidence threshold based on model performance and application scenario to ensure the quality of data used to generate reports.

[0203] Update the knowledge graph: Feed the reliable results obtained from this analysis into the multimodal dynamic knowledge graph, update the historical evaluation statistics and sentiment trend data of the corresponding element nodes, and record the key evidence.

[0204] Key evidence storage: For each evaluation element, the top N key textual evidence supporting the judgment are retained, stored in order of importance, and provide data support for subsequent interpretability display.

[0205] Step S93: Generate an interpretable diagnostic report.

[0206] Multi-dimensional Emotional Distribution Heat Map: Based on the dimensions of scenic spots and evaluation elements, the emotional distribution of each element is displayed using color coding, intuitively presenting the performance of the scenic spot in each dimension.

[0207] List of Advantages: List the top few factors with the highest positive feedback rates for each scenic spot and present key evidence to support this judgment, so that managers can understand "why it is good".

[0208] List of Weaknesses: List the top few elements with the highest negative review rates for each scenic spot, and display specific negative review text excerpts to accurately pinpoint the problem.

[0209] Key issue attribution analysis: In-depth analysis of key weaknesses, combined with historical trend data and specific evidence, to provide attribution of problems and suggestions for improvement actions.

[0210] This embodiment achieves the following beneficial effects by introducing interpretable output features: Improved statistical accuracy; low-quality judgments are filtered out through confidence levels, and core issues are given greater weight in the statistics through importance weighting, more accurately reflecting the user's focus. Enhanced interpretability; key evidence is directly displayed in the strengths / weaknesses list, enabling managers to understand the basis of their judgments and enhancing the credibility of the results. Improved actionability; importance scores help managers identify the issues that truly need to be prioritized, and concrete evidence provides direct basis for formulating improvement measures. Knowledge graph evolution; key evidence is stored in a knowledge graph, enabling the system to continuously learn and provide richer contextual information for subsequent analysis.

[0211] It should be noted that the above embodiments, using tourist attraction reviews as an example, are merely illustrative to facilitate understanding of the technical solution of the present invention and should not be construed as limiting the present invention. Those skilled in the art will understand that, without departing from the principles of the present invention, this solution can also be applied to other fields involving review text and multimodal data, including but not limited to e-commerce, local life services, and social media content analysis. Any modifications or applications based on the essential technical solution of the present invention should fall within the protection scope of the present invention.

[0212] The three-layer architecture of this invention (evaluation ontology structure, knowledge graph, and fusion model) is ingeniously designed and mutually supportive. The evaluation ontology serves as the blueprint for knowledge graph construction, the knowledge graph is the source for enhanced reasoning by the model, and new knowledge generated by the model during application can in turn update the knowledge graph. This closed-loop design enables the system to continuously learn and evolve. Furthermore, by replacing the evaluation ontology and corresponding domain data, this solution can be quickly migrated to multiple vertical domains such as e-commerce, social media, and local life services for fine-grained reputation analysis, demonstrating excellent domain versatility and commercial application potential.

[0213] In summary, this invention achieves significant breakthroughs in multiple dimensions, including analytical granularity, inference intelligence, result credibility, and system versatility, through methodological and architectural innovation, providing a complete, efficient, and reliable next-generation sentiment analysis solution for comment texts.

[0214] Through the above specific implementation methods, the present invention achieves the following beneficial effects: 1) By introducing an evaluation dimension ontology and a multimodal knowledge graph, a structured framework and rich external knowledge support are provided for fine-grained analysis, enabling the analysis results to move from a holistic approach to a dimensional one; 2) Through dynamic knowledge selection and cross-modal attention fusion, information fusion is transformed from static mixing to context-awareness, enhancing the model's ability to understand complex evaluations; 3) Through sentiment decoupling and interpretable output, the analysis results are transformed from black-box decision-making to white-box attribution, significantly improving the reliability and practical value of the analysis results.

[0215] This solution features flexible domain adaptability, enabling it to quickly adapt to different application scenarios and solving the problem of poor domain adaptability of traditional methods.

[0216] It should be noted that although the current embodiment uses sentiment analysis as an example, the core innovation of this invention lies in its comment text analysis method based on a multimodal knowledge graph—actually a highly versatile technical framework that can be extended to multiple domains. The versatility of this invention is reflected in the fact that its multimodal knowledge graph construction method, adaptive fusion mechanism, and fine-grained analysis framework are all domain-independent; simply changing the domain ontology and training data allows for rapid adaptation to new scenarios. The invention's application scope extends far beyond sentiment analysis; the following specific examples illustrate its versatility:

[0217] 1. Medical field: Symptom-disease association analysis Application scenario: Identifying the correlation between symptoms and diseases from patient complaints and medical records.

[0218] Specific implementation: Construct a medical knowledge graph that includes multimodal entities such as diseases, symptoms, examinations, and drugs.

[0219] Input text: The patient has had a fever for 3 days, cough with yellow sputum, and chest X-ray shows inflammation in the lower lobe of the right lung.

[0220] System output: Evaluation factor: pneumonia (confidence level 95%).

[0221] Key evidence: fever, cough, yellow sputum, and chest X-ray showing inflammation.

[0222] Importance scores: fever (0.8), cough (0.7), chest X-ray (0.9).

[0223] Technological value: It assists doctors in making rapid diagnoses and improves diagnostic and treatment efficiency.

[0224] 2. Financial Risk Control: Risk Event Identification Application scenario: Identifying enterprise risk events from news, announcements, and social media.

[0225] Specific implementation: Construct a financial knowledge graph: entities such as enterprises, executives, related parties, and risk events.

[0226] Input text: The chairman of a company is under investigation, and the stock price has fallen to the daily limit for several consecutive days.

[0227] System output: Evaluation factor: Corporate governance risk (confidence level 92%); Key evidence: The chairman is under investigation, and the stock price has plummeted to its daily limit; Importance score: Chairman under investigation (0.95), stock price hit the daily limit down (0.85); Technological value: Real-time risk warnings to assist investment decisions; 3. Legal field: Case element extraction; Application scenario: Extracting key case elements from legal documents; Specific implementation: Constructing a legal knowledge graph: entities such as crimes, legal provisions, and sentencing factors; Input text: The defendant committed multiple thefts, involving substantial sums, and is a repeat offender; System output: Evaluation criteria: Theft (confidence level 98%); Key evidence: multiple thefts, substantial amounts involved, repeat offender; Importance score: repeat offender (0.9), large amount (0.8), multiple thefts (0.7); Technological value: It assists judges in quickly extracting key information in cases and improves trial efficiency.

[0228] It should be noted that the examples given above are for illustrative purposes only and do not constitute a limitation on the present invention.

[0229] Embodiment 2 of this invention aims to visually demonstrate the performance and effectiveness of the adaptive knowledge fusion attention GRU model of this invention. Experimental data from the Southern Xinjiang scenic area was collected, and the following experiments and analyses were conducted, generating corresponding illustrations: 1. Model training process and convergence analysis The model is trained using this three-stage strategy. For example... Figure 2 The figure shows the accuracy convergence curve of the model during training. The vertical axis represents the sentiment analysis recognition accuracy (Acc), and the horizontal axis represents the number of training iterations (Epoch). The blue curve represents the training set accuracy (trainacc), reflecting the model's fit to the training data, while the orange curve represents the validation set accuracy (valacc), reflecting the model's generalization ability. This curve verifies that the model of this invention can quickly fit effective features during the training phase and exhibits stable generalization performance, solving the technical problems of slow convergence and overfitting that traditional sentiment analysis models often encounter when fusing cross-domain data (such as soil measurement text). The model's accuracy on both the training and validation sets steadily increases with the number of training epochs, eventually stabilizing at a high level. The two curves closely match, indicating that the model has good learning ability and does not exhibit overfitting. Furthermore, as... Figure 3 As shown, both the training loss and validation loss of the model decrease rapidly with each training round and tend to converge to a low level, which further proves the effectiveness of the training process and the convergence of the model.

[0230] 2. Evaluation of model fine-grained classification performance The model was finally evaluated on a separate test set. Figure 4 The confusion matrix of the model on the fine-grained sentiment classification task is shown. Figure 4 The confusion matrix, representing the classification accuracy of the proposed solution for two types of technical features, is used to characterize the classification accuracy of the proposed solution for these two types of technical features. The matrix's horizontal axis represents predicted labels, including both negative and positive technical features, while the vertical axis represents true labels, also including both negative and positive technical features. The values ​​in each cell of the matrix have the following meanings: 10935 represents the number of samples where both the true and predicted labels are negative technical features; 927 represents the number of samples where both the true and predicted labels are negative technical features; 1082 represents the number of samples where both the true and predicted labels are positive technical features; and 11038 represents the number of samples where both the true and predicted labels are positive technical features. The generally low values ​​of the off-diagonal elements in the confusion matrix indicate a low misclassification rate. This result quantitatively demonstrates that the proposed model can accurately perform fine-grained sentiment discrimination.

[0231] 3. Verification of the application value of the model analysis results The deployed system was applied to analyze real review data from multiple A-level scenic spots in southern Xinjiang over a period of time. Figure 4 The analysis results based on the system output of this invention are shown, such as... Figure 5 The chart shown is a line graph illustrating the negative review rate of A-level scenic spots in southern Xinjiang. It displays the changes in the negative review rate of A-level scenic spots in southern Xinjiang across different samples (which may be time series or different scenic spots). Key information: Horizontal axis: Sample number (0-40), which may represent different time points or different scenic spots. Vertical axis: Negative review rate (0.0%-14.0%), i.e., the proportion of tourists giving negative reviews. Blue line: The trend of the negative review rate, showing significant overall fluctuations; Key data: The highest negative review rate is close to 12%, and the lowest is 0%, indicating that the overall negative review rate of A-level scenic spots in southern Xinjiang is well controlled, but some samples have a high risk of negative reviews.

[0232] Application Value: This chart can be used to monitor the service quality of scenic spots, identify periods or areas with high rates of negative reviews, provide data support for tourism management departments, and help improve tourist satisfaction. The chart intuitively reveals the specific shortcomings in the service quality of each scenic spot, providing managers with clear and quantifiable decision-making basis for precise improvements, fully demonstrating the practical value of this invention from data insight to management action.

[0233] Embodiment 3 of this invention verifies the effectiveness of the method and model of this invention. Taking the A-level scenic spots in southern Xinjiang as the review ontology, the evaluation of the model's generalization ability in terms of scene adaptability, data scarcity, and time extrapolation adaptability by constructing a validation set across scenic spot types, data scales, and time intervals specifically includes the following steps: S91. Select data from different types of scenic spots in southern Xinjiang to construct an experimental set and verify the scene adaptability performance of each model among different types of scenic spots. S92. Simulate a data-scarce scenario, fine-tune the model using training samples of different proportions from the target scenic area, and evaluate the generalization ability of the model under large-scale data learning. S93. Divide the dataset into time-series components for training and testing to verify the model's adaptability to changes in tourist evaluation patterns over time.

[0234] The comparative experiment is designed as follows: 1. Experimental setup: Models: Standard GRU, LSTM, and RNN models were constructed for comparison. These baseline models use the same word embeddings but do not include knowledge graph access, dynamic knowledge selection, or cross-modal attention mechanisms.

[0235] Training and testing data: 100,000 comments and images collected from scenic spots in southern Xinjiang were used, and randomly divided into training, validation, and test sets in an 8:1:1 ratio. All models (both the ones in this invention and the baseline) were trained and tested on this same dataset.

[0236] - Evaluation tasks: fine-grained element sentiment classification (determining whether an element is positive or negative) and element identification (identifying which elements are mentioned in the comments).

[0237] 2. Experimental Results: Standard GRU, LSTM, and RNN models were constructed as baseline models on the same test set. All models were trained and tested on the same training, validation, and test sets. The accuracy, recall, and F1 score metrics are shown in Table 1 below. Table 1 Experimental results show that, on the same dataset, the GRU model outperforms the LSTM and RNN models in terms of accuracy, recall, and F1 score on the sentiment classification task.

[0238] To further evaluate the generalization ability of the adaptive knowledge fusion attention GRU model, this invention specifically includes: cross-type adaptability, selecting data from different types of scenic spots in southern Xinjiang, such as natural scenery and historical and cultural sites, to construct an experimental set and verify the model's scene adaptability across different scenic spot types; small-sample robustness, simulating data scarcity scenarios, using training samples of different proportions from the target scenic spot for fine-tuning, and evaluating the model's generalization ability under small-sample learning; and temporal extrapolation adaptability, dividing the dataset according to time order for training and testing, to verify the model's adaptability to changes in tourist evaluation patterns over time.

[0239] In Embodiment 4 of this invention, the trained model is lightweightly packaged and deployed on a local server within the scenic area to perform real-time sentiment analysis on newly added tourist reviews on the platform. A dynamic sentiment distribution baseline is established for each scenic area and each evaluation element based on historical sentiment data. Real-time sentiment scores are monitored statistically, and when the proportion of negative reviews for a certain element fluctuates abnormally beyond the historical baseline threshold within a short period, an early warning mechanism is automatically triggered, sending a warning message to management personnel. Simultaneously, based on fine-grained semantic classification results, an optimization diagnostic report for the scenic area, including shortcomings, degree of impact, actionable improvement suggestions, and priority ranking, is automatically generated. A quantitative evaluation system encompassing multiple dimensions, including tourist satisfaction, scenic area management response efficiency, and online reputation index, is constructed. By continuously monitoring changes in these indicators after applying this method, the effectiveness of scenic area service quality improvement measures is quantitatively evaluated, forming a management closed loop.

[0240] The system uses stream processing to execute steps S1-S7 in real time for new comments, generating an interpretable report within a second-level delay.

[0241] In addition, an early warning mechanism is set up: the system has an early warning decision module that continuously monitors and analyzes the results. Early warning trigger conditions are based on technical rules, such as: - Rule 1 (Sudden Increase Warning): When the confidence level of negative evaluations for key dimensions such as "safety" or "hygiene" is >0.9, and more than 5 such evaluations occur within one consecutive hour, the system automatically pushes a high-level alert to the administrator. - Rule 2 (Trend Warning): When the satisfaction level for a certain dimension decreases by more than 20 percentage points within 24 hours, a trend warning is triggered.

[0242] The real-time monitoring and anomaly alerting of comment sentiment specifically includes the following steps: S101. Establish a dynamic emotional distribution baseline for each scenic spot and each evaluation element based on historical emotional data; S102. Monitor real-time sentiment scores through statistics, and automatically trigger an early warning mechanism when abnormal fluctuations are detected; S103. Based on the fine-grained semantic classification results, automatically generate a scenic area optimization diagnosis report; The scenic area optimization diagnostic report includes: identified shortcomings and their impact; actionable improvement suggestions; and a priority ranking based on improvement costs and expected results.

[0243] Application Example: One day, the system detected a surge of 8 negative reviews regarding "waiting time" in the comments section related to "scenic area shuttle buses" within 30 minutes, all with a confidence level higher than 0.85, triggering a red alert. The administrator reviewed the report and found that many reviews mentioned "vehicle malfunctions causing delays," accompanied by photos of the scene, allowing for a rapid response.

[0244] Based on the structured sentiment analysis results output by this invention, an automated quantitative evaluation system for service quality can be constructed: 1. Data Foundation: Analysis results of all reviews for the scenic area are aggregated on a monthly basis.

[0245] 2. Calculation of core indicators: -Tourist satisfaction index: "∑(Percentage of positive reviews in each dimension × Average confidence level) / Total number of dimensions". This index ranges from 0 to 1, objectively reflecting the overall experience.

[0246] - Management Efficiency Index: Tracks modifiable dimensions such as "service" and "facilities." It calculates the decrease in the percentage of negative feedback this month compared to last month; the greater the decrease, the higher the index, reflecting the effectiveness of management improvements.

[0247] - Online reputation index: Calculated using a normalized formula by combining positive emotional volume, negative emotional volume, and the popularity of comments (number of reposts / likes).

[0248] 3. Report Generation: The system automatically generates a "Scenic Area Service Quality Diagnostic Report" monthly, including trend charts of the above indicators, radar charts analyzing the strengths and weaknesses of each dimension, and representative positive and negative reviews (including key evidence). For example, the report can clearly indicate that "the satisfaction index for 'restaurant prices' decreased by 0.15 this month, mainly due to a surge in complaints about 'a restaurant raising prices'," providing direct evidence for precise management.

[0249] It should be noted that the specific implementation methods of this invention will be described in detail using A-level tourist attractions in southern Xinjiang, China (represented by Kanas Scenic Area and Tianchi Scenic Area of ​​Tianshan Mountains, etc.) as examples. Those skilled in the art will understand that the embodiments described herein are merely for explaining the technical solutions of this invention and are not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0250] This invention fundamentally transforms the analysis of comment text from a "general, black-box, and lagging" approach to a "fine-grained, interpretable, and real-time" approach by constructing an evaluation dimension ontology and a multimodal knowledge graph, combined with a dedicated neural network model for adaptive knowledge fusion. The proposed solution not only outputs sentiment insights for multiple predefined dimensions (such as "eating, accommodation, and transportation") with accompanying confidence levels and key evidence, addressing the problems of coarse analysis and insufficient decision-making basis in traditional methods, but also, after rigorous comparison and generalization verification, significantly outperforms traditional models in both classification accuracy and scenario adaptability. Furthermore, by deploying it as a real-time monitoring and early warning engine, the system can automatically identify negative trends and trigger warnings, achieving a leap from passive analysis to proactive management. Finally, based on continuously outputting structured sentiment data, it can automatically generate quantitative evaluation reports, forming a data-driven closed loop of "analysis-improvement-evaluation," thereby directly transforming technical insights into measurable and traceable service optimization and strategic decision-making capabilities, providing a reliable, intelligent, and actionable complete solution for high-value scenarios such as scenic area management and product optimization.

[0251] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for analyzing comment text based on multimodal knowledge graphs, characterized in that, include: S1. Obtain a dataset from the target domain, the dataset including comment text data of multiple evaluation objects and image data associated with the comment text data; S2. Construct an evaluation dimension ontology structure corresponding to the target domain. The evaluation dimension ontology structure includes multiple predefined evaluation dimensions for the evaluation object, and each evaluation dimension is further associated with at least one sub-element. S3. Based on the dataset, construct a multimodal knowledge graph associated with the ontology structure of the evaluation dimension; S4. Obtain the current comment text, perform deep semantic encoding on the current comment text, identify at least one evaluation dimension corresponding to its semantics, and generate text semantic features; S5. Based on the identified evaluation dimensions, determine the sub-elements corresponding to the evaluation dimensions, retrieve the associated features in the multimodal knowledge graph with the sub-elements as the core to form a candidate set, calculate the semantic correlation between the text semantic features and each feature in the candidate set, and finally select the target knowledge features based on the semantic correlation. S6. Based on the semantic features of the text, dynamically assign fusion weights to the target knowledge features and perform weighted fusion to generate knowledge-enhanced features; S7. Based on the knowledge enhancement features, output fine-grained semantic classification results for the identified at least one evaluation dimension; The fine-grained semantic classification results include: the confidence scores corresponding to the semantic classification results and the semantic category results; the importance scores of the evaluation elements; and the key evidence indicators supporting the judgment of the semantic classification results.

2. The comment text analysis method based on multimodal knowledge graph according to claim 1, characterized in that, The construction of the evaluation dimension ontology structure corresponding to the target domain includes: Collect historical datasets from the target domain, and preprocess the historical datasets to obtain preprocessed datasets; Evaluation elements are extracted based on the preprocessed dataset; Cluster analysis of the evaluation elements yields multiple dimensions; Based on the multiple dimensions, the evaluation words corresponding to each dimension are classified to obtain sub-elements, and the evaluation words are associated with the evaluation elements. The multiple dimensions are associated with and stored with the multiple sub-elements to obtain the evaluation dimension ontology structure corresponding to the target domain, and the multiple dimensions correspond one-to-one with the multiple sub-elements.

3. The comment text analysis method based on multimodal knowledge graph according to claim 1, characterized in that, The construction of a multimodal knowledge graph associated with the evaluation dimension ontology structure based on the dataset includes: Based on the evaluation dimension ontology structure, the historical text data in the target domain dataset is structured to construct a static knowledge layer; Based on the evaluation dimension ontology structure, real-time text data in the target domain dataset is extracted to form a dynamic knowledge layer; Based on the evaluation dimension ontology structure, visual semantic features are extracted from the image data in the target domain dataset, and the visual semantic features are cross-modal associated with the corresponding entities in the multimodal knowledge graph to form a visual knowledge layer. The static knowledge layer, the dynamic knowledge layer, and the visual knowledge layer are associated and mapped to form a multimodal knowledge graph that integrates text, images, and spatiotemporal information.

4. The comment text analysis method based on multimodal knowledge graph according to claim 1, characterized in that, The step of dynamically assigning fusion weights to the target knowledge features based on the text semantic features, and performing weighted fusion to generate knowledge-enhanced features includes: Perform entity recognition on the current comment text and extract the evaluation element entities from the comment text; Based on the evaluation element entities, text description features, visual features, and historical evaluation statistical features are extracted from the corresponding nodes of the multimodal knowledge graph. Based on the contextual semantics of the current target comment text, dynamically calculate the fusion weights of the text description features, the visual features, and the historical evaluation statistical features; The knowledge enhancement features are generated based on the fusion weights of the text description features, the fusion weights of the visual features, and the fusion weights of the historical evaluation statistics features.

5. The comment text analysis method based on multimodal knowledge graph according to claim 1, characterized in that, The step of dynamically assigning fusion weights to the target knowledge features based on the text semantic features, and performing weighted fusion to generate knowledge-enhanced features further includes: The current comment text is associated with an image, and the visual features of the associated image are extracted; Based on cross-modal semantic alignment, the correlation between the visual features and the semantic features of the current comment text is determined; Assign fusion weights to the visual features based on the correlation; The visual features are weighted according to the fusion weights. The weighted visual features are fused with the textual description features and the historical evaluation statistical features to generate the knowledge-enhanced features.

6. The comment text analysis method based on multimodal knowledge graph according to claim 1, characterized in that, Steps S4 to S7 are implemented using an adaptive knowledge fusion neural network model.

7. The comment text analysis method based on multimodal knowledge graph according to claim 6, characterized in that, The neural network model for adaptive knowledge fusion is the attention GRU model for adaptive knowledge fusion.

8. The comment text analysis method based on multimodal knowledge graph according to claim 8, characterized in that, The adaptive knowledge fusion attention GRU model is obtained through three-stage policy training; The three-stage strategy training includes: Pre-training was performed on a general sentiment corpus; Make domain-specific fine-tuning to the commentary text in the target domain; Enhanced samples generated based on the multimodal knowledge graph are introduced for knowledge optimization training.

9. The comment text analysis method based on multimodal knowledge graph according to claim 7, characterized in that, The adaptive knowledge fusion attention GRU model includes a context-aware GRU encoding layer, a dynamic knowledge selector, a cross-modal attention mechanism layer, a decoupling layer, a collaborative reasoning layer, and an interpretable output layer connected in sequence.

10. The comment text analysis method based on multimodal knowledge graph according to claim 10, characterized in that, The adaptive knowledge fusion attention GRU model includes: The context-aware GRU encoding layer is used to perform deep semantic encoding on the current comment text to obtain its context-aware text semantic features. The dynamic knowledge selector dynamically selects relevant target knowledge features from the multimodal knowledge graph based on the text semantic features. The cross-modal attention mechanism layer: dynamically weights and fuses the text semantic features and the knowledge features through a cross-modal attention mechanism to generate knowledge-enhanced features; The decoupling layer is used to separate and decode the knowledge enhancement features into dimensional semantic features based on predefined evaluation dimensions. The collaborative reasoning layer is used to perform internal calibration and joint optimization of the dimensional semantic features to generate dimensional reasoning signals. The interpretable output layer outputs the fine-grained semantic classification result based on the dimensional inference signal.

Images