File travel content semantic analysis method and system

Abstract

The invention belongs to the field of big data processing, and provides a text travel content semantic analysis method and system, and the method comprises the steps: collecting text data and audio and video contents related to a plurality of scenic spots, and classifying the text data and the audio and video contents according to a pre-constructed scenic spot classification library; preprocessing the text data under each scenic spot classification; extracting audio stream and video frame data from the audio and video contents, recording the audio stream into a text by adopting a voice recognition technology, and performing feature extraction on the audio and video contents by utilizing a deep learning model to respectively obtain audio features and video features; based on a semantic template under the scenic spot classification, performing weighted fusion on the audio features, the video features and the text features through an attention mechanism to generate multi-modal semantic features; and performing sentiment classification and subject extraction on the multi-modal semantic features, and outputting semantic tags and analysis results related to the scenic spots.

Description

Technical Field

[0001] The present invention belongs to the field of big data processing, and specifically relates to a method and system for semantic analysis of cultural and tourism content. Background Art

[0002] With the rapid development of the cultural tourism industry, tourists have accumulated a large amount of content data during their travels, including text reviews, picture sharing, and popular audio and video content in recent years (such as short videos, Vlogs, etc.). These data not only reflect tourists' subjective feelings about scenic spot services, cultural activities, transportation facilities, etc., but also contain important information for optimizing scenic spot operations. By performing semantic analysis on these cultural tourism contents, valuable decision-making basis can be provided for scenic spot managers and tourism-related companies, such as evaluating tourist satisfaction, discovering problem points, and optimizing scenic spot services.

[0003] However, existing cultural and tourism content analysis methods mainly focus on the processing of structured data or single text data, such as content classification through keyword extraction, sentiment analysis, etc. These methods show the following shortcomings when facing unstructured data (especially audio and video content).

[0004] The semantic extraction of audio and video data in cultural and tourism content is more complicated than that of text data. Existing methods lack effective unified modeling methods when parsing the semantics of voice information and video images, resulting in insufficient accuracy of semantic analysis. Summary of the invention

[0005] In order to solve the problems in the prior art, the present invention provides a method for semantic analysis of cultural and tourism content, which comprises the following steps:

[0006] Collecting text data and audio and video content related to a plurality of scenic spots, and classifying the text data and audio and video content according to a pre-built scenic spot classification library;

[0007] Preprocessing the text data under each scenic spot classification, the preprocessing includes denoising, word segmentation and semantic feature extraction, and generating semantic templates and keywords of the scenic spot;

[0008] Extract audio stream and video frame data from audio and video content, transcribe the audio stream into text using speech recognition technology, and use deep learning models to extract features from the audio and video content to obtain audio features and video features respectively;

[0009] Based on the semantic templates under the scenic spot classification, the audio features, video features and text features are weightedly fused through an attention mechanism to generate multimodal semantic features;

[0010] The multimodal semantic features are subjected to sentiment classification and topic extraction, and semantic labels and analysis results related to the scenic spots are output.

[0011] Another aspect of the present invention further provides a cultural and tourism content semantic analysis system, the system comprising the following modules:

[0012] A collection module, used for collecting text data and audio and video content related to a plurality of scenic spots, and classifying the text data and audio and video content according to a pre-built scenic spot classification library;

[0013] A preprocessing module is used to preprocess the text data under each scenic spot classification, wherein the preprocessing includes denoising, word segmentation and semantic feature extraction to generate semantic templates and keywords of the scenic spot;

[0014] The extraction module is used to extract audio stream and video frame data from the audio and video content, transcribe the audio stream into text using speech recognition technology, and extract features of the audio and video content using a deep learning model to obtain audio features and video features respectively;

[0015] A fusion module, for weightedly fusing the audio features, video features and text features through an attention mechanism based on the semantic templates under the scenic spot classification to generate multimodal semantic features;

[0016] The analysis module is used to perform sentiment classification and topic extraction on the multimodal semantic features, and output semantic tags and analysis results related to the scenic spots.

[0017] The multimodal semantic analysis method based on scenic spot classification proposed in this invention realizes a comprehensive analysis of the semantics of cultural and tourism content by deeply integrating text, audio and video data, combining dynamic attention mechanism and semantic template guidance. Compared with the existing technology, it has the following beneficial effects:

[0018] By introducing semantic templates under scenic spot classification, the fusion process of multimodal features is guided, which effectively compensates for the ambiguity of semantic expression in audio and video data, making sentiment classification and theme extraction more accurate. The dynamic attention mechanism assigns weights according to the importance of different modal features, focusing on features that match the semantic templates, and further improving the consistency of semantic understanding.

[0019] It integrates the characteristics of text, audio and video modalities, comprehensively captures the information perceived by tourists, and solves the problem that single-modal analysis cannot fully reflect the tourist experience. It realizes the collaborative modeling of cross-modal features, so that the analysis results can simultaneously reflect the comprehensive semantics of language sentiment, voice intonation and visual scenes.

[0020] Accurately distinguish tourists' positive, neutral and negative emotions, and identify the intensity of emotions, providing intuitive sentiment analysis results for scenic spot evaluation. Extract the main themes that tourists are concerned about (such as "landscape", "service", "transportation"), and combine them with sentiment tendency analysis to provide clear improvement directions for scenic spot optimization and service improvement.

[0021] It supports input in multiple modes and can adapt to different types of data (such as short videos, voice comments, etc.), and is applicable in a variety of cultural and tourism scenarios. By expanding the scenic spot classification library and semantic templates, it can quickly adapt to new scenic spots or theme requirements and has good flexibility.

[0022] In summary, the present invention can significantly improve the comprehensiveness and practicality of semantic analysis of cultural and tourism content, provide strong data support for scenic spot management, tourist services and market decision-making, and has broad application value and market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0025] Below, the invention is preferably described in conjunction with the accompanying drawings and specific implementation methods.

[0026] This embodiment solves the above problem through the following steps:

[0027] In one embodiment, reference Figure 1 The present invention provides a method for semantic analysis of cultural tourism content. The method comprehensively extracts and analyzes the semantic features of multimodal content in the field of cultural tourism. By combining scenic spot classification technology and integrating the characteristics of text data, audio content and video data, the method accurately models and analyzes the emotional tendency, topic distribution and implicit semantics in cultural tourism related content. The method specifically includes the following steps:

[0028] Collect text data and audio and video content related to multiple scenic spots, and classify the text data and audio and video content according to a pre-built scenic spot classification library. Specifically:

[0029] Text data related to cultural and tourism attractions are obtained from various online platforms, including but not limited to user comments, question and answer content, guide descriptions, etc. from tourists on social media (such as Weibo), travel review platforms (Mafengwo), e-commerce evaluation platforms (such as Ctrip), and travel guide websites (such as Qyer).

[0030] Obtain audio and video content related to cultural and tourism attractions, including but not limited to Vlogs published on short video platforms (such as Douyin and Kuaishou), short videos of scenic spots uploaded by users, promotional videos on the official websites of scenic spots, and audio records from other platforms (such as podcast content).

[0031] The original data is collected through web crawler technology, open API interface calls, manual uploads and multi-party data cooperation to ensure the diversity and integrity of the collected content.

[0032] The collected text data is cleaned to remove advertising content, irrelevant comments, redundant data and text with incorrect format, and denoised, segmented and normalized. For text data containing multiple languages, machine translation tools are used to uniformly convert it into the target language for subsequent analysis.

[0033] Separate the audio and video streams from the collected audio and video content. For the audio stream, use speech recognition technology (such as ASR, Automatic Speech Recognition) to transcribe the speech content into text. For the video stream, extract key frames, and use image processing technology to identify scene features, landmark buildings and other information in the video; extract and filter additional information such as video titles and description texts.

[0034] A classification library containing multiple attractions is established. Each attraction entry includes the attraction name, geographical location, theme keywords (such as "Great Wall" corresponds to "history", "spectacular", etc.), identifying image features (such as the visual features of landmark buildings) and possibly related tags (such as evaluation keywords commonly used by tourists).

[0035] Design a dynamic update mechanism to support automatic expansion of the scenic spot classification library based on newly added cultural and tourism content, ensuring the real-time and applicability of the classification library.

[0036] The preprocessed text data is matched and classified according to the geographical location, keywords and context information in the scenic spot classification library. The keyword matching method (such as BM25) or machine learning classification algorithm (such as BERT classification model) is used to parse the topic of the text to determine which scenic spot category it belongs to. For fuzzy matching content, context semantic analysis is used to further clarify the classification.

[0037] Utilize the results of audio transcription text, and preliminarily classify the audio content in combination with the keyword matching rules of the scenic spot classification library; for the situation where multiple scenic spots are mentioned in the speech, clarify the attribution of the main scenic spot through context analysis or voice intensity distribution.

[0038] Extract the key frames of the video, match them with the iconic image features in the scenic spot classification library through an image recognition model (such as ResNet or YOLO) to identify the specific scenic spots in the video shooting scene. Combine the video title, description text, and audio transcription content, and use a multimodal fusion classification algorithm (such as a fusion Transformer model) to classify the video content and assign it to the corresponding scenic spot category.

[0039] Integrate the classification results of the text data and audio-visual content to form a multimodal data set with scenic spots as the unit. Each set contains the text, audio, and video content related to the scenic spot, as well as the corresponding preliminary semantic annotation results.

[0040] Save the classification results to the database to provide data support for subsequent semantic analysis.

[0041] Through the above detailed steps, the collected text data and audio-visual content are efficiently classified into the pre-constructed scenic spot classification library, thus laying a data foundation for the multimodal semantic analysis of cultural and tourism content.

[0042] Preprocess the text data under each scenic spot classification. The preprocessing includes denoising, word segmentation, and semantic feature extraction to generate the semantic template and keywords of the scenic spot.

[0043] Specific steps of text data preprocessing:

[0044] Data cleaning: For the data set classified to a specific scenic spot, remove irrelevant information, such as advertising content, duplicate comments, and special symbols (such as "" or HTML tags).

[0045] Exemplarily, the original text: "The Great Wall is absolutely spectacular!!! See the ticket purchase guide at the link: http: / / example.com"

[0046] After cleaning: The text is "The Great Wall is absolutely spectacular".

[0047] Stop word filtering: Remove common stop words that have no semantic contribution, such as "de", "le", "shi", etc.

[0048] Exemplarily, the text "The Great Wall is a very worthy place to go"

[0049] After removing stop words: The text is "The Great Wall is very worthy to go to place".

[0050] Spell checking and correction: Correct spelling errors using language models or dictionaries (such as Hunspell).

[0051] For example, the original text: "The Great Wall is too strong!"

[0052] After correction: The text is "The Great Wall is so magnificent!"

[0053] Sentence segmentation: Divide a long text into multiple independent sentences for subsequent analysis.

[0054] For example, the text: "The Great Wall is very spectacular, and there are many people here. It is recommended to avoid holidays."

[0055] After sentence division: The sentence list is ["The Great Wall is very spectacular", "There are many people here", "It is recommended to avoid holidays"].

[0056] Word segmentation: Use Chinese word segmentation tools (such as Jieba, THULAC) or word segmentation methods combined with semantics to extract vocabulary units.

[0057] For example, the sentence "The Great Wall is magnificent"

[0058] After word segmentation: the vocabulary sequence is [“Great Wall”, “very”, “spectacular”].

[0059] Part-of-speech tagging and named entity recognition: perform part-of-speech tagging on the word segmentation results, and identify place names, scenic spot names, and proper nouns.

[0060] For example, the word segmentation result is ["Great Wall / place name", "very / adverb", "spectacular / adjective"].

[0061] Semantic feature extraction:

[0062] Keyword extraction: Based on algorithms such as TF-IDF or TextRank, high-frequency keywords are extracted from texts under the scenic spot classification.

[0063] For example, for a set of comment texts about the Great Wall: “The Great Wall is magnificent,” “Climbing the Great Wall is a challenge,” and “The Great Wall has a strong sense of history,” the extracted keywords may include “Great Wall,” “magnificent,” “challenge,” and “history.”

[0064] Topic modeling: Use LDA (Latent Dirichlet Allocation) or deep learning models to cluster the text and extract topic words related to the attraction.

[0065] For example, for comments on the Great Wall, the following topics and their keywords may be extracted:

[0066] Theme 1: Landscape: Spectacular, Historical, and Majestic

[0067] Theme 2: Experience: Mountaineering, Challenge, Fatigue

[0068] Sentiment vocabulary extraction: Extract positive or negative sentiment vocabulary based on sentiment analysis models (such as sentiment lexicon or BERT).

[0069] Exemplary positive emotion words: "shocking", "worthy", "magnificent".

[0070] Negative emotion words: "crowded", "tired", "queue".

[0071] Generate semantic templates and keywords

[0072] Semantic template generation: Based on the results of word segmentation and semantic feature extraction, a deep learning model is used to generate semantic templates of scenic spots to represent the context and logical relationships of the semantics related to the scenic spots.

[0073] For example, the semantic template of the Great Wall may be: "The Great Wall is a [spectacular / adjective] [historical building / noun], and climbing the Great Wall is a [challenge / noun], but [worth it / adjective]."

[0074] The template indicates the semantic association between "Great Wall" and "spectacular", "historical building", "challenge", and "worthwhile".

[0075] Attraction keyword generation: Based on the extracted keywords and sentiment words, a keyword list of attractions is generated for reference in multimodal semantic analysis.

[0076] For example, the keywords for the Great Wall may be: "spectacular", "historical", "challenging", "worth it", and "crowded".

[0077] Audio stream and video frame data are extracted from audio and video content, speech recognition technology is used to transcribe the audio stream into text, and a deep learning model is used to extract features of the audio and video content to obtain audio features and video features respectively.

[0078] Audio stream extraction:

[0079] Separate the audio stream from the audio and video files, use multimedia processing tools (such as FFmpeg) to extract the audio part, and generate an audio file in a standard format (such as WAV or MP3).

[0080] Exemplarily, the video file "GreatWall.mp4" is input and the audio stream "GreatWall_audio.wav" is extracted.

[0081] Audio preprocessing: Preprocess the audio data, including noise reduction, segmentation (segmenting the audio into segments according to fixed time windows, such as 5 seconds per segment), volume normalization, and other operations to ensure audio clarity and adaptability.

[0082] Exemplarily, the original audio contains background noise, which is eliminated through a noise reduction algorithm (such as SNR enhancement) to generate a clear audio file.

[0083] Video frame extraction:

[0084] Extract key frames from the video stream at set time intervals (such as extracting 1 frame per second), and save them as image files using image processing tools (such as OpenCV or FFmpeg), ensuring that the frame sequence covers the entire content of the video.

[0085] Exemplarily, the video "GreatWall.mp4" is input, key frames are extracted at 1 frame per second, and the frame sequence is output: ["frame_001.jpg", "frame_002.jpg", ...].

[0086] Key frame screening: Use scene change detection algorithms (such as SSIM similarity calculation or YOLO scene recognition) to screen important frames and remove redundant information frames.

[0087] For example, in a video of the Great Wall, frames containing "city wall" and "distant view" are retained, and continuous static frames are removed.

[0088] Audio stream transcription to text

[0089] Speech recognition: Use speech recognition technology (such as Baidu Speech Recognition API or the open source tool Kaldi) to transcribe the audio stream into text.

[0090] For example, the input audio "GreatWall_audio.wav" is recognized as follows: "The Great Wall is very spectacular. We chose to hike today."

[0091] Optimization of speech-to-text conversion: Speech segmentation: Segment long audio files by sentences or time periods to improve speech recognition accuracy;

[0092] Domain-customized language model: We train exclusive language models based on commonly used terms in the cultural and tourism fields (such as "Great Wall" and "sense of history") to optimize transcription results.

[0093] Deep learning models extract audio and video features

[0094] Audio feature modeling:

[0095] Use pre-trained deep learning models (such as Wav2Vec2.0, OpenAI Whisper) to extract features from audio and generate semantic representation vectors of the audio.

[0096] The extracted features include:

[0097] Semantic features: semantic expression of audio content, such as tourists’ description of their feelings about scenic spots;

[0098] Emotional features: Extract audio emotions such as "excited" and "amazing" through the emotion classification model;

[0099] Audio mode characteristics: such as speaking speed, pitch changes, etc.

[0100] For example, the audio text is input: "The Great Wall is so magnificent, and the view is particularly broad."

[0101] Output audio characteristics:

[0102] Semantic feature vector: [0.85, 0.12, 0.03, …] (high-dimensional feature vector);

[0103] Emotional characteristics: excitement (emotional intensity 80%);

[0104] Audio mode: medium speaking speed, slightly high pitch.

[0105] Video feature extraction:

[0106] Frame-level feature extraction: Use convolutional neural networks (such as ResNet and EfficientNet) to extract features from each frame to capture visual patterns and semantic information in the image.

[0107] For example, for the Great Wall video frame “frame_001.jpg”, the extracted features include “city wall”, “blue sky”, “tourists”, etc.

[0108] Output labels: city wall (95% confidence), blue sky (80% confidence).

[0109] Temporal feature modeling: Use video temporal modeling networks (such as 3D-CNN, I3D, or Transformer architectures) to extract temporal information across frames and identify dynamic semantics in video content.

[0110] For example, for a hiking video of the Great Wall, “moving tourists walking along the wall” is identified.

[0111] Based on the semantic template under the scenic spot classification, the audio features, video features and text features are weightedly fused through an attention mechanism to generate multimodal semantic features.

[0112] In cultural and tourism content, audio and video data contain rich semantic information, but the accuracy of audio and video semantic analysis is often insufficient due to the following reasons:

[0113] The ambiguity and diversity of audio and video data: Audio content may contain ambiguous or incomplete speech expressions, such as a tourist saying "It's really great!" without mentioning the specific attractions. Multiple scenes may alternate in the video, making it difficult to focus on the semantic point of a single frame analysis.

[0114] Single-modal semantic isolation: When audio and video features are analyzed separately, they lack contextual associations and may lead to misunderstandings. For example, the video shows "crowded attractions" but does not directly express emotions, and the audio mentions "beautiful attractions" but does not mention the specific scene.

[0115] Existing models have limitations for multimodal collaboration: relying solely on audio or video features for semantic classification may miss key information, while simple feature splicing makes it difficult to capture the deep semantic associations between modalities.

[0116] To this end, semantic templates under the scenic spot classification are introduced to fuse the features of text, audio and video within a preset range (such as the same user ID, the preset time range, and the same cluster of people). The audio and video features are fused with the text features through the attention mechanism, which effectively improves the accuracy of audio and video semantic analysis.

[0117] Type input includes the following three characteristics:

[0118] Text feature F t :

[0119] Source: Semantic template under the scenic spot classification, indicating the core keywords and themes of the scenic spot.

[0120] For example, for the “Great Wall”, the semantic templates include “spectacular”, “sense of history”, “hiking challenge”, etc.

[0121] Audio Features F a :

[0122] Source: Generate semantic representation of tourists’ speech through speech recognition and feature extraction technology.

[0123] Description: Includes features such as voice emotion and tone changes.

[0124] For example, in the audio “The Great Wall is really shocking!” the emotional feature of “shocking” is extracted.

[0125] Video Features F v :

[0126] Source: Visual semantic features extracted from video frames.

[0127] Description: Contains scene information, object detection results (such as city walls, tourists) and dynamic behavior characteristics.

[0128] For example, the scene features of “city wall” and “blue sky” are extracted from the Great Wall video.

[0129] Key strategies to solve inaccurate audio and video semantic analysis

[0130] Semantic template as baseline semantics:

[0131] The semantic templates under the scenic spot classification provide a clear semantic benchmark for text features. Compared with analyzing audio and video features separately, templates can guide the direction of feature fusion, making the final semantic result more consistent with the context of the scenic spot.

[0132] Attention mechanism dynamic weight allocation:

[0133] In view of the incompleteness or ambiguity of semantic information in audio and video, the attention mechanism is used to calculate the correlation between audio and video features and text templates, dynamically assign weights, and give more weight to the modality that matches the template semantics.

[0134] Inter-modal semantic collaboration modeling:

[0135] Through multimodal fusion methods, the semantic correlation between audio, video and text is captured to solve the problem of information isolation in unimodal analysis.

[0136] Attention Mechanism Weighted Fusion:

[0137] Calculation of similarity between features:

[0138] Text and audio similarity S ta :

[0139]

[0140] Text and video similarity S tv :

[0141]

[0142] Where d represents the vector dimension, F t,i represents the i-th dimension text vector, F a,i represents the i-th dimension speech vector, F v,i Represents the i-th dimension video vector.

[0143] Attention Weight Normalization

[0144] Dynamically assign weights to audio and video features:

[0145]

[0146] Among them, α ta represents the text speech weight, α tv Represents the text video weight.

[0147] The text features, weighted audio features and weighted video features are fused to generate multimodal semantic features:

[0148] Weighted Audio Features:

[0149] F a '=α ta ·F a

[0150] Weighted video features:

[0151] F v '=α tv ·F v

[0152] Among them, F a ' represents the weighted audio feature, F v ' represents weighted video features.

[0153] The text features, weighted audio features and weighted video features are fused to generate multimodal semantic features F m :

[0154] F m =W t ·F t +W a ·F a′ +W v ·F v′

[0155] Where W t , W a , W v is a trainable weight matrix.

[0156] For example, improve the accuracy of semantic analysis of the audio and video of the "Great Wall"

[0157] enter:

[0158] Text features: F t =[0.8,0.7,0.6] represents the semantic template features of “spectacular”, “historical”, and “challenging”.

[0159] Audio characteristics: F a =[0.9,0.4,0.3] represents the sentiment and semantic features of the tourist’s speech “The Great Wall is so spectacular”.

[0160] Video Features: F v =[0.5,0.9,0.2] represents the visual features of the “distant view of the city wall” in the video.

[0161] Text and audio similarity:

[0162] Sta =0.8·0.9+0.7·0.4+0.6·0.3=0.72+0.28+0.18=1.18

[0163] Text and video similarity:

[0164] S tv =0.8·0.5+0.7·0.9+0.6·0.2=0.4+0.63+0.12=1.15

[0165] Audio Weight:

[0166]

[0167] Video weight:

[0168]

[0169] Weighted Audio Features:

[0170] F a '=0.51·[0.9,0.4,0.3]=[0.459,0.204,0.153]

[0171] Weighted video features:

[0172] F v '=0.49·[0.5,0.9,0.2]=[0.245,0.441,0.098]

[0173] Multimodal feature fusion (assuming all weights are 1):

[0174] F m =[0.8,0.7,0.6]+[0.459,0.204,0.153]+[0.245,0.441,0.098]

[0175] F m =[1.504,1.345,0.851]

[0176] The multimodal semantic features [1.504, 1.345, 0.851] express:

[0177] Sense of magnificence: The high weight combination of the text template (0.8) and the audio (0.9) highlights the semantics of the “magnificent” feeling of the Great Wall.

[0178] Sense of history: The video feature (0.9) enhances the “sense of history” and makes up for the lack of scene description in the audio.

[0179] The semantic template guides the distribution of attention weights to solve the problem of audio and video ambiguity. The clear semantic direction provided by the fusion text makes the semantic expression of audio and video more complete. The multimodal semantic features integrate audio, video and text information, significantly improving the comprehensiveness and reliability of semantic analysis.

[0180] In order to obtain the weight parameters in the above model, training can be performed through the following steps:

[0181] Data preparation

[0182] Collect and annotate multimodal data, including text (such as tourist attraction reviews), audio (tourist voices), and video (short videos taken by tourists).

[0183] Preprocess the data, extract text keywords, reduce the noise of audio content, and extract video frames and dynamic features.

[0184] Feature extraction

[0185] Use pre-trained models (such as BERT, Wav2Vec2.0, ResNet) to extract feature vectors of text, audio, and video respectively as model input.

[0186] Attention weight calculation

[0187] Based on the dot product attention mechanism, the similarity between text and audio, text and video is calculated, and weights are dynamically generated to determine the contribution ratio of audio and video to semantic expression.

[0188] Multimodal Fusion

[0189] The audio and video features are weighted by the attention weights and are fused with the text features to generate comprehensive multimodal semantic features.

[0190] Training Goals

[0191] Taking sentiment classification and topic classification as the main tasks, the cross entropy loss function is used to optimize the model, and attention regularization constraints are added to ensure that the modal weights are reasonable.

[0192] Iterative Optimization

[0193] A gradient descent algorithm (such as Adam) is used to train the model on a multimodal dataset, and the weight matrix and fusion strategy of each modality are adjusted until the loss function converges.

[0194] Through the above process, the model can dynamically assign modal weights, generate high-quality multimodal semantic features, and improve the accuracy of audio and video semantic analysis.

[0195] The multimodal semantic features are subjected to sentiment classification and topic extraction, and semantic labels and analysis results related to the scenic spots are output.

[0196] Multimodal semantic features integrate the semantic information of text, audio and video, and contain tourists' comprehensive perception of scenic spots. On this basis, through sentiment classification and topic extraction, these perceptions can be structured into semantic labels and analysis results, providing valuable data support for scenic spot operations.

[0197] Sentiment Classification:

[0198] Sentiment classification aims to identify tourists' emotional tendencies (such as positive, neutral or negative) and their intensity towards attractions, providing a direct basis for the comprehensive evaluation of attractions.

[0199] The multimodal semantic features generated in the previous step are used as the input of the sentiment classification model.

[0200] The feature combines emotional words in text (such as "shocking", "spectacular"), voice intonation in audio (such as excitement), and visual clues in video (such as beautiful scenery).

[0201] A trained sentiment classification model is used to classify the multimodal features. The model determines the overall sentiment tendency of tourists based on the sentiment signals in the semantic features.

[0202] Sentiment tendencies are classified as positive (e.g., “satisfied”), neutral (e.g., “average”), or negative (e.g., “dissatisfied”).

[0203] Generates a sentiment label (e.g., "positive") and sentiment intensity (e.g., "strong sentiment" or "mildly positive"). Sentiment intensity reflects how obvious the sentiment expression is.

[0204] For example, for video and audio content describing the Great Wall, the sentiment classification model identified that the tourist used words such as "shocking" and "spectacular" in the text, the tone of voice showed excitement, the video showed magnificent scenery, the sentiment label was "positive", and the sentiment intensity was "highly positive".

[0205] Topic Extraction:

[0206] Theme extraction aims to identify the main attributes of tourist attractions (such as “landscape”, “history”, and “service”) and their proportion in the features, and provide directional suggestions for the improvement of tourist attractions.

[0207] Based on semantic templates and multimodal features, tourists’ perceptions are divided into different thematic categories, such as “landscape”, “service quality”, “cultural history” or “transportation convenience”.

[0208] The classification process combines keywords in the text, descriptions of services or landscapes in the audio, and scene content in the video.

[0209] Extract core keywords under each theme to highlight tourists’ specific comments on the theme. For example, under the theme of “landscape”, keywords such as “spectacular”, “majestic”, and “broad vision” may be extracted.

[0210] For the same multimodal data, multiple related topics may be extracted and the topic proportions may be given. For example, a tourist review may involve two major topics: "landscape" and "transportation", of which "landscape" has a higher proportion.

[0211] Generate topic tags (such as "landscape", "service") and their relevance scores to measure the importance of the topic and tourists' concerns.

[0212] For example, in the multimodal data of the Great Wall, the topic classification model identified that "landscape" accounted for 70%, "historical culture" accounted for 25%, and "service" accounted for only 5%. Keywords under the "landscape" theme were further extracted, such as "spectacular", "ancient", and "shocking".

[0213] Output semantic labels and analysis results. The semantic labels are composed of sentiment and topic information, which intuitively present the feelings of tourists.

[0214] Sentiment label: positive, neutral, or negative.

[0215] Theme tags: such as "Landscape-Positive", "History and Culture-Positive".

[0216] Analysis results: Generate analysis reports related to scenic spots based on semantic tags:

[0217] Sentiment distribution: Displays the percentage of positive, neutral, and negative sentiments. For example, 90% of tourists expressed positive comments on the Great Wall, 8% were neutral, and 2% were negative.

[0218] Theme distribution: List the main themes and their proportions. For example, "landscape" is the main concern of tourists, followed by "history and culture".

[0219] Keyword extraction: Extract high-frequency words that best reflect tourists’ feelings. For example, the keywords of the Great Wall include “spectacular”, “majestic”, and “shocking”.

[0220] On the other hand, the present invention also provides a cultural and tourism content semantic analysis system, comprising:

[0221] A collection module, used for collecting text data and audio and video content related to a plurality of scenic spots, and classifying the text data and audio and video content according to a pre-built scenic spot classification library;

[0222] A preprocessing module is used to preprocess the text data under each scenic spot classification, wherein the preprocessing includes denoising, word segmentation and semantic feature extraction to generate semantic templates and keywords of the scenic spot;

[0223] The extraction module is used to extract audio stream and video frame data from the audio and video content, transcribe the audio stream into text using speech recognition technology, and extract features of the audio and video content using a deep learning model to obtain audio features and video features respectively;

[0224] A fusion module, for weightedly fusing the audio features, video features and text features through an attention mechanism based on the semantic templates under the scenic spot classification to generate multimodal semantic features;

[0225] The analysis module is used to perform sentiment classification and topic extraction on the multimodal semantic features, and output semantic tags and analysis results related to the scenic spots.

[0226] The prior art mentioned in the above background technology section and specific embodiment section of the present invention can be used as part of the present invention to understand the meaning of some technical features or parameters.

Claims

1. A method for semantic analysis of cultural and tourism content, characterized in that: The method comprises the following steps: Collecting text data and audio and video content related to a plurality of scenic spots, and classifying the text data and audio and video content according to a pre-built scenic spot classification library; Preprocessing the text data under each scenic spot classification, the preprocessing includes denoising, word segmentation and semantic feature extraction, and generating semantic templates and keywords of the scenic spot; Extract audio stream and video frame data from audio and video content, transcribe the audio stream into text using speech recognition technology, and use deep learning models to extract features from the audio and video content to obtain audio features and video features respectively; Based on the semantic templates under the scenic spot classification, the audio features, video features and text features are weightedly fused through an attention mechanism to generate multimodal semantic features; The multimodal semantic features are subjected to sentiment classification and topic extraction, and semantic labels and analysis results related to the scenic spots are output.

2. The method for semantic analysis of cultural and tourism content according to claim 1, characterized in that: The classifying the text data and the audio and video content according to the pre-built scenic spot classification library includes: The preprocessed text data is matched and classified according to the geographical location, keywords and context information in the scenic spot classification library; The audio content is classified by using the result of audio transcription and the keyword matching rules of the scenic spot classification library; Extract video key frames, match them with the identifying image features in the scenic spot classification library through the image recognition model, identify the specific scenic spots in the video shooting scene, and classify them according to the scenic spots.

3. The method for semantic analysis of cultural and tourism content according to claim 1, characterized in that: The semantic template and keywords for generating the scenic spot include: Based on the results of word segmentation and semantic feature extraction, a deep learning model is used to generate a semantic template of the scenic spot, which represents the context and logical relationship of the semantics related to the scenic spot. Based on the extracted keywords and sentiment words, a keyword list of scenic spots is generated.

4. The method for semantic analysis of cultural and tourism content according to claim 1, characterized in that: Based on the semantic template under the scenic spot classification, the audio features, video features and text features are weightedly fused through the attention mechanism to generate multimodal semantic features including: Calculation of similarity between features: Calculate the similarity S between text and audio ta Calculate the similarity S between text and video tv : Where d represents the vector dimension, F t,i represents the i-th dimension text vector, F a,i represents the i-th dimension speech vector, F v,i represents the i-th dimension video vector; Dynamically assign weights to audio and video features: Among them, α ta represents the text speech weight, α tv Represents the text video weight; The text features, weighted audio features and weighted video features are fused to generate multimodal semantic features: Weighted Audio Features: F a '=a ta ·F a Weighted video features: F v '=a tv ·F v Among them, F a ' represents the weighted audio feature, F v ' represents weighted video features; The text features, weighted audio features and weighted video features are fused to generate multimodal semantic features F m : F m =W t ·F t +W a ·F a′ +W v ·F v′ Where W t , W a , W v is the weight matrix to be trained.

5. The method for semantic analysis of cultural and tourism content according to claim 1, characterized in that: The sentiment classification and topic extraction of the multimodal semantic features include: Multimodal semantic features are used as the input of the sentiment classification model, and a trained sentiment classification model is used to classify the multimodal features; the model judges the overall emotional tendency of tourists based on the emotional signals in the semantic features; and the tourists' perceptions are divided into different thematic categories based on the semantic templates and multimodal features.

6. A cultural and tourism content semantic analysis system, characterized in that: The system includes the following modules: A collection module, used for collecting text data and audio and video content related to a plurality of scenic spots, and classifying the text data and audio and video content according to a pre-built scenic spot classification library; A preprocessing module is used to preprocess the text data under each scenic spot classification, wherein the preprocessing includes denoising, word segmentation and semantic feature extraction to generate semantic templates and keywords of the scenic spot; The extraction module is used to extract audio stream and video frame data from the audio and video content, transcribe the audio stream into text using speech recognition technology, and extract features of the audio and video content using a deep learning model to obtain audio features and video features respectively; A fusion module, for weightedly fusing the audio features, video features and text features through an attention mechanism based on the semantic templates under the scenic spot classification to generate multimodal semantic features; The analysis module is used to perform sentiment classification and topic extraction on the multimodal semantic features, and output semantic tags and analysis results related to the scenic spots.

7. The cultural and tourism content semantic analysis system according to claim 6, characterized in that: The classifying the text data and the audio and video content according to the pre-built scenic spot classification library includes: The preprocessed text data is matched and classified according to the geographical location, keywords and context information in the scenic spot classification library; The audio content is classified by using the result of audio transcription and the keyword matching rules of the scenic spot classification library; Extract video key frames, match them with the identifying image features in the scenic spot classification library through the image recognition model, identify the specific scenic spots in the video shooting scene, and classify them according to the scenic spots.

8. The cultural and tourism content semantic analysis system according to claim 6, characterized in that: The semantic template and keywords for generating the scenic spot include: Based on the results of word segmentation and semantic feature extraction, a deep learning model is used to generate a semantic template of the scenic spot, which represents the context and logical relationship of the semantics related to the scenic spot. Based on the extracted keywords and sentiment words, a keyword list of scenic spots is generated.

9. The cultural and tourism content semantic analysis system according to claim 6, characterized in that: Based on the semantic template under the scenic spot classification, the audio features, video features and text features are weightedly fused through the attention mechanism to generate multimodal semantic features including: Calculation of similarity between features: Calculate the similarity S between text and audio ta Calculate the similarity S between text and video tv : Where d represents the vector dimension, F t,i represents the i-th dimension text vector, F a,i represents the i-th dimension speech vector, F v,i represents the i-th dimension video vector; Dynamically assign weights to audio and video features: Among them, α ta represents the text speech weight, α tv Represents the text video weight; The text features, weighted audio features and weighted video features are fused to generate multimodal semantic features: Weighted Audio Features: F a '=a ta ·F a Weighted video features: F v '=a tv ·F v Among them, F a ' represents the weighted audio feature, F v ' represents weighted video features; The text features, weighted audio features and weighted video features are fused to generate multimodal semantic features F m : F m =W t ·F t +W a ·F a′ +W v ·F v′ Where W t , W a , W v is the weight matrix to be trained.

10. The cultural and tourism content semantic analysis system according to claim 6, characterized in that: The sentiment classification and topic extraction of the multimodal semantic features include: Multimodal semantic features are used as the input of the sentiment classification model, and a trained sentiment classification model is used to classify the multimodal features; the model judges the overall emotional tendency of tourists based on the emotional signals in the semantic features; and the tourists' perceptions are divided into different thematic categories based on the semantic templates and multimodal features.

Images