Cross-modal travel knowledge graph construction system based on large language model

Abstract

The invention discloses a cross-modal travel knowledge graph construction system based on a large language model. The cross-modal travel knowledge graph construction system comprises a data source module, a data access module, a change identification module, an incremental updating module, a verification feedback module, a core storage and application module and a technical base module. According to the method, optimization is carried out around dynamic updating accuracy, real-time efficiency and scene adaptability, and four dimensions of hidden conflict resolution, real-time processing efficiency, small and medium main body access and multi-modal deep fusion are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph construction and processing, in particular to a cross-modal cultural and tourism knowledge graph construction system based on a large language model. BACKGROUND

[0002] The cultural and tourism industry is a comprehensive industry based on cultural resources, integrating tourism and cultural elements to provide cultural experiences and tourism services. Its core is to promote tourism product upgrading through cultural creativity, forming a cultural + tourism industry closed loop, which covers historical sites, folk experience, non-heritage research, etc. It creates differentiated experiences through cultural symbols and uses digital technology to enhance the flow attraction capacity of scenic spots and develop immersive experience projects to promote the transformation of smart tourism.

[0003] In particular, in 2025, the state explicitly listed the cultural and tourism industry as a pillar industry, promoting industrial upgrading through the creation of demonstration parks and the activation of non-heritage, and smart tourism and immersive experiences have become the focus of transformation.

[0004] Therefore, as a dynamic and information-rich field, the construction of the cultural and tourism industry's knowledge graph is crucial for integrating and utilizing relevant information.

[0005] Chinese invention patent CN120542539A "Cross-modal cultural and tourism knowledge graph construction system based on large language model" proposes the following technical solution: including dynamic real-time acquisition and processing module, intelligent knowledge verification and fusion module, blockchain trusted record and multi-source verification module, adaptive update strategy formulation module, and strong learning optimization module. The technical solution uses multi-source acquisition and cloud computing to reduce the average knowledge update time from 10 days to 2 days, solves the timeliness problem, combines crowdsourcing and professional review and model algorithms to improve review accuracy, ensures the completeness and accuracy of the graph, uses blockchain technology to enhance the credibility and authority of the knowledge graph, establishes a model based on multi-dimensional data and designs a strategy to meet different user needs, introduces reinforcement learning algorithms and a scoring mechanism to increase satisfaction from 60% to 80%, improve practicality, and better serve users.

[0006] Although the above technical solution has certain advantages, it still has the following problems: It does not address the missing implicit association problem, such as not adjusting the recommended route after updating the scenic spot; it relies too much on cloud computing for high-concurrency processing and does not specify real-time delays, with an average knowledge update time of 2 days; it does not mention a small and medium-sized subject access solution, relying only on official API or third-party interface integration, which is difficult for small and medium-sized scenic spots to adapt.

[0007] Therefore, a cross-modal cultural and tourism knowledge graph construction system based on a large language model is proposed. SUMMARY

[0008] The present application aims to at least solve one of the above technical problems to some extent, and improves the four dimensions of implicit conflict resolution, real-time processing efficiency, small and medium-sized subject access, and multi-modal deep fusion.

[0009] To achieve the above purpose, the first aspect of the present application provides a cross-modal tourism knowledge graph construction system based on a large language model, which comprises a data source module, a data access module, a change identification module, an incremental update module, a verification feedback module, a core storage and application module, and a technical base module. The data source module is configured to provide multi-type original tourism data. The data access module comprises a multi-source data access adapter and a data preprocessing unit, wherein the multi-source data access adapter uses Flink stream processing to adapt to multi-source interfaces to realize low-latency access, and the data preprocessing unit converts heterogeneous data into text, visual vector, timestamp, and filters irrelevant content. The change identification module comprises a cross-modal change extraction unit and a change priority determination unit, wherein the cross-modal change extraction unit improves the credibility of the change candidate set through a tourism fine-tuning LLM, a visual model, and a Cross-Attention module, and the change priority determination unit uses three-dimensional scoring to divide the changes into high, medium, and low priority and determine the processing time limit. The incremental update module comprises an incremental update execution unit and a double conflict detection unit, wherein the incremental update execution unit performs corresponding operations according to node addition, relationship adjustment, and attribute deletion, and the double conflict detection unit detects explicit / implicit conflicts through a rule base and a causal reasoning LLM to generate conflict-free update content. The verification feedback module comprises a multi-dimensional automatic verification unit and a manual review and feedback closed loop unit, wherein the multi-dimensional automatic verification unit uses a tourism fine-tuning CLIP and RDFox for automatic verification, and the manual review and feedback closed loop unit manually confirms the high-priority update and receives user feedback to return and trigger the update again. The core storage and application module comprises a tourism knowledge graph based on Neo4j and a graph application end, the Neo4j-based tourism knowledge graph supports incremental writing and version backtracking, and the graph application end provides tourism recommendation, route planning, and a knowledge correction entry. The technology foundation module includes a real-time computing support unit, a large language model support unit, a vision technology support unit, and a graph technology support unit. The real-time computing support unit provides Flink stream processing capabilities, the large language model support unit provides the Qwen-1.8B / Llama3-70B model, the vision technology support unit provides MobileNetV3 / ViT vision processing capabilities, and the graph technology support unit provides Neo4j / RDFox graph tools.

[0010] In addition, the cross-modal cultural tourism knowledge graph construction system based on a large language model proposed above according to the present invention may also have the following additional technical features: As a further description of the above technical solution: The data preprocessing unit further includes a format normalization subunit and a noise filtering subunit; The format standardization subunit uses ViT to extract frames from images / videos and convert them into 512-dimensional visual vectors, and to split text into a structure of object ID, change type, effective time, and scope of influence. The noise filtering subunit filters unchanging text using the Cultural Tourism Fine-tuning DistilBERT model and removes duplicate images using the DHash image hashing algorithm.

[0011] As a further description of the above technical solution: The cross-modal change extraction unit includes a text change extraction subunit, a visual change recognition subunit, and a cross-modal fusion subunit; The text change extraction subunit uses the Qwen-1.8B model and change unit to extract the Prompt output structured change unit; The visual change recognition subunit uses MobileNetV to classify image / video frames and outputs feature labels and 0-1 confidence scores. The cross-modal fusion subunit uses the Cross-Attention model to calculate similarity and retains candidate sets with a similarity of ≥0.8.

[0012] As a further description of the above technical solution: The three-dimensional scoring rules of the change priority determination unit include: Impact Scope Dimension (4 points): Assigned based on the scope of cultural and tourism objects covered by the change, with global coverage scoring 4 points, local area coverage scoring 2 points, and single-point facility coverage scoring 1 point; Timeliness dimension (3 points): Assigned according to the change effective period: 3 points for immediate effect, 2 points for effect within 7 days, and 1 point for effect after 7 days; User relevance dimension (3 points): Assigned based on the degree of impact of changes on user decisions, with core services scoring 3 points, auxiliary services scoring 2 points, and information supplementation scoring 1 point; Priority is determined based on the total score: 8-10 points are designated as high priority with a processing time of ≤10 minutes; 5-7 points are designated as medium priority with a processing time of ≤1 hour; and 1-4 points are designated as low priority with a processing time of ≤4 hours.

[0013] As a further description of the above technical solution: The incremental update execution unit adds node configuration change time and source, automatically associates with parent class and initializes surrounding elements, adjusts relationships to effective time and strength, retains historical versions and marks expiration time, deletes attributes and marks expiration status and time, and records the trigger data source number.

[0014] As a further description of the above technical solution: The dual conflict detection unit further includes an explicit conflict detection subunit and a implicit conflict detection subunit; The explicit conflict detection subunit is based on conflict trigger verification in the cultural and tourism rule base; The implicit conflict detection subunit adopts the Llama3-70B model, inputs updated information and existing associations, outputs association suggestions and impact prompts, and makes adjustments.

[0015] As a further description of the above technical solution: The verification feedback module adopts the CLIP model. When the similarity is <0.6, manual verification is triggered. Knowledge logic verification uses the RDFox tool to check ontology constraints. If there is a violation, the process is paused. User feedback is submitted through the knowledge error correction entry. After standardization, the feedback is sent back to trigger the update again.

[0016] As a further description of the above technical solution: The real-time computing support unit achieves an end-to-end latency of less than 10 seconds for data access and flow through the Flink stream processing framework, ensuring real-time processing of high-priority changes; The large language model support unit uses the Qwen-1.8B model for text change extraction, the Llama3-70B model for causal inference, and the DistilBERT model for noise filtering. All models are fine-tuned using corpora from the cultural and tourism field. The visual technology support unit uses the MobileNetV3 model for visual change recognition, the ViT model for visual feature extraction, and the CLIP model for cross-modal consistency verification. The model training data includes images specific to cultural and tourism scenarios. The Neo4j graph database of the graph technology support unit adopts an incremental write mode to avoid the performance loss caused by full updates, and uses the RDFox tool for graph ontology constraint verification to ensure the logical consistency of the graph.

[0017] The beneficial effects of this invention are: The cross-modal cultural tourism knowledge graph construction system based on a large language model according to the present invention optimizes dynamic update accuracy, real-time efficiency, and scenario adaptability, and improves four dimensions: implicit conflict resolution, real-time processing efficiency, access for small and medium-sized entities, and deep multimodal fusion. Explicit conflicts are resolved using a rules engine, such as when the same attraction opens and closes at the same time. Implicit conflicts are analyzed using a cultural tourism fine-tuning LLM, such as when a new glass walkway is not associated with surrounding parking lots / entrances, and the association is automatically added. The Flink stream processing framework is used to implement multi-source data with an end-to-end latency of <10 seconds. It is combined with Neo4j incremental write mode and updates are divided into three priorities: high, medium and low, according to the scope of impact, timeliness and user relevance. For small and medium-sized scenic spots that lack API development capabilities, a lightweight data submission tool is provided for mini-programs / web pages, which supports manual uploading of change information and the system automatically completes format standardization and preprocessing. The entire model was fine-tuned using a cultural and tourism corpus, and the Cross-Attention module was used to achieve dimensional alignment between abstract semantics and concrete visuals.

[0018] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a schematic diagram illustrating the principle of the present invention; Figure 2 This is a schematic diagram of the system processing flow of the present invention; Figure 3 This is a schematic diagram of the text and visual data conversion process of the present invention. Detailed Implementation

[0020] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0021] The following description, in conjunction with the accompanying drawings, illustrates an embodiment of the present invention: a cross-modal cultural tourism knowledge graph construction system based on a large language model.

[0022] like Figures 1-3 As shown in the figure, a cross-modal cultural tourism knowledge graph construction system based on a large language model according to an embodiment of the present invention includes a data source module, a data access module, a change identification module, an incremental update module, a verification feedback module, a core storage and application module, and a technical foundation module. The modules form a closed-loop update system through data flow and technical support. The data source module is used to provide various types of raw cultural and tourism data, including official notices and real-time visitor flow data, as well as short videos, guide texts, and real-shot images from UGC platforms, and ticket status from third-party OTA platforms and transportation connection change data from navigation platforms. To better process the data, the data preprocessing unit includes a format normalization subunit and a noise filtering subunit: The format standardization subunit uses the ViT visual feature extraction model to extract frames from real-shot images and short videos and transform them into 512-dimensional visual feature vectors. At the same time, it decomposes text data according to the fields of object ID, change type, effective time and scope of impact to form structured text fragments. The noise filtering subunit adopts the cultural tourism fine-tuned DistilBERT model to filter non-changing information such as user emotional descriptions and irrelevant chatter in the text. It uses the DHash image hashing algorithm to calculate the image hash value and delete duplicate images to reduce redundant computation.

[0023] The data access module includes a multi-source data access adapter and a data preprocessing unit. The multi-source data access adapter uses the Flink stream processing framework to achieve low-latency access to multi-source data and is compatible with official APIs, UGC distributed crawlers and third-party platform interfaces. The data preprocessing unit converts the heterogeneous data into a standardized format of text fragments + visual feature vectors + timestamps, while filtering out irrelevant text and duplicate images. The change identification module includes a cross-modal change extraction unit and a change priority determination unit. The cross-modal change extraction unit extracts text structured change units through the cultural tourism fine-tuning LLM, identifies visual change features by combining a lightweight visual classification model, calculates association similarity using the Cross-Attention module, and filters a set of high-credibility change candidates. The change priority determination unit scores change tasks based on three dimensions: impact scope, timeliness, and user relevance, and divides change tasks into high, medium, and low priorities and sets corresponding processing timelines. To further differentiate them, the cross-modal change extraction unit includes a text change extraction subunit, a visual change recognition subunit, and a cross-modal fusion subunit; The text change extraction subunit adopts the cultural tourism fine-tuning Qwen-1.8B model. The change unit extracts the Prompt-guided model output, which contains the object ID, change type, effective time, and scope of impact. The visual change recognition subunit adopts the cultural tourism fine-tuned MobileNetV3 model to classify images and videos by extracting frames, and outputs change feature labels such as fences / new signs / park closure notices and feature confidence scores in the 0-1 range; The cross-modal fusion sub-unit uses the Cross-Attention module to calculate the association similarity between the text change unit and the visual feature label, and retains only the change information with a similarity ≥ 0.8 to form a high-confidence change candidate set; To better assess the three-dimensional scores, the three-dimensional scoring rules for the change priority determination unit are as follows: Impact Scope Dimension (4 points): Values ​​are assigned based on the scope of the cultural and tourism objects covered by the change; global coverage scores 4 points, local area coverage scores 2 points, and single-point facility coverage scores 1 point. Timeliness Dimension (3 points): Values ​​are assigned based on the change's effective period; immediate effectiveness scores 3 points, effectiveness within 7 days scores 2 points, and effectiveness after 7 days scores 1 point. User Relevance Dimension (3 points): Values ​​are assigned based on the degree of impact of the change on user decisions; core services, such as scenic area opening and closing, score 3 points; auxiliary services, such as facility adjustments, score 2 points; information supplementation, such as landscape introduction updates, score 1 point. Priority is determined based on the total score: 8-10 points are designated as high priority, with a processing time ≤ 10 minutes; 5-7 points are designated as medium priority, with a processing time ≤ 1 hour; 1-4 points are designated as low priority, with a processing time ≤ 4 hours.

[0024] The incremental update module includes an incremental update execution unit and a dual conflict detection unit. The incremental update execution unit performs the following operations for three types of changes: adding nodes and associating with parent classes, modifying relationship attributes and retaining historical versions, and marking attributes as invalid and tracing data sources, respectively. The dual conflict detection unit detects explicit conflicts through the cultural and tourism rule base and combines causal reasoning LLM analysis to identify the lack of implicit associations and indirect effects, thereby generating conflict-free update content. To perform the three types of changes, the specific operational logic of the incremental update execution unit includes: node addition operation, which configures the change time and basic attributes of the data source when adding a node, automatically associates the parent node based on the predefined cultural tourism classification system, and initializes the associated placeholders of surrounding nodes; relationship adjustment operation, which locates the existing relationship between the target nodes, modifies the effective time and association strength attributes of the relationship, and retains the historical relationship version and marks the expiration time, supporting backtracking query; attribute deletion operation, which does not physically delete the attribute, but only marks the expiration status and expiration time, and records the trigger data source number of the attribute deletion for easy traceability and verification. To specify the detection process, the dual conflict detection unit includes an explicit conflict detection subunit and an implicit conflict detection subunit; The explicit conflict detection subunit is based on a rule base in the cultural and tourism field, which includes source priority rules (where official data > UGC data > third-party data), time consistency rules (the same attribute cannot have two effective times at the same time), and attribute legality rules (opening time must conform to the HH:MM-HH:MM format). When a conflict is detected, a secondary verification of the data source is automatically triggered. The latent conflict detection subunit adopts the cultural tourism fine-tuning Llama 3-70B model. It takes updated node information and existing graph relationships as input and outputs suggestions for potential missing relationships. For example, if a new facility is added, the entrance / parking lot relationship needs to be added, and the indirect impact prompts are provided. If a scenic spot is closed, the recommended route needs to be adjusted. The system automatically supplements or adjusts the graph relationships based on the suggestions.

[0025] The verification and feedback module includes a multi-dimensional automatic verification unit and a manual review and feedback closed-loop unit. The multi-dimensional automatic verification unit uses the cultural tourism fine-tuning CLIP model to verify the consistency between text and visual information and uses the RDFox tool to verify the graph ontology constraints. The manual review and feedback closed-loop unit manually confirms high-priority updates and receives user error correction feedback from the graph application terminal, which is then sent back to the change recognition layer to trigger the update again. The specific implementation of the verification feedback includes: cross-modal consistency verification, which uses the cultural tourism fine-tuning CLIP model (using the ViT-B / 32 architecture) to calculate the similarity between the updated text information and the corresponding visual feature vector. When the similarity is <0.6, it is marked as needing review and manual verification is triggered; knowledge logic verification, which uses the RDFox graph consistency verification tool to check whether the updated graph conforms to the ontology constraints of the cultural tourism field, including that scenic spot nodes must be associated with the opening time attribute and facility nodes must belong to a certain scenic spot node. If violations are found, the update is suspended and the error type is indicated; user feedback closed loop, where the knowledge error correction entry on the graph application side supports users to submit error types (attribute errors / relationship missing) + supporting data (text / image). The system converts the feedback information into standardized data and sends it back to the change recognition layer to re-trigger the update process.

[0026] The core storage and application modules include a cultural tourism knowledge graph and a graph application client based on Neo4j. The cultural tourism knowledge graph supports incremental writing and historical version backtracking, while the graph application client provides cultural tourism recommendation and route planning functions, and sets up a knowledge correction entry point.

[0027] The technology foundation module includes a real-time computing support unit, a large language model support unit, a vision technology support unit, and a graph technology support unit, which provide each layer with Flink stream processing capabilities, Qwen-1.8B / Llama 3-70B cultural tourism fine-tuning model, MobileNetV3 / ViT vision processing capabilities, and Neo4j / RDFox graph tool support. In the aforementioned units, the real-time computing support unit uses the Flink stream processing framework to achieve an end-to-end latency of less than 10 seconds for data access and flow, ensuring real-time processing of high-priority changes; the large language model support unit uses the Qwen-1.8B model for text change extraction, the Llama3-70B model for causal inference, and the DistilBERT model for noise filtering. All models use corpora from the cultural and tourism field, such as scenic spot notices, travel guides, and intangible cultural heritage introductions, and then perform fine-tuning; the visual technology support unit uses the MobileNetV3 model for visual change recognition, the ViT model for visual feature extraction, and the CLIP model for cross-modal consistency verification. The model training data includes images specific to cultural and tourism scenarios, such as ancient buildings, intangible cultural heritage techniques, and scenic spot facilities; the graph technology support unit uses the Neo4j graph database in an incremental write mode to avoid performance loss caused by full updates, and uses the RDFox tool for graph ontology constraint verification to ensure the logical consistency of the graph. Example 2

[0028] The following is an implementation through a specific scenario: (Specific implementation is carried out by personnel in this technical field) Hardware support is provided by deploying a system environment, using a distributed server cluster, including 3 data access servers, 4 model inference servers, and 2 graph storage servers, to ensure cluster bandwidth ≥10Gbps, thereby ensuring real-time flow of multi-source data. In order to cooperate with the stream processing Flink 1.17, the system used is CentOS 7.9, the graph database is Neo4j 5.12, the graph verification tool is RDFox 6.0, and all large language models and visual models are trained and deployed based on PyTorch 2.0 framework.

[0029] Implementation scenario: First, taking the popular Grand Canyon Glass Skywalk as an example, we first collect various types of cultural and tourism data through the data source module: We can access opening notices and real-time visitor flow data through official channels (API), including 12 fields such as opening hours, flow restriction rules, and safety notices. The real-time visitor flow data is set as a JSON stream, ensuring a sampling frequency of one record every 5 minutes, for a total of 24 records (this needs to be collected 1 hour before the update); We can also collect UGC data by using a Scrapy-Redis distributed crawler to legally crawl UGC content on popular software (Douyin / Xiaohongshu, etc.) with the keyword #glassskywalk, to obtain the required short videos (100-300, MP4 format), real-shot photos (800, JPG format), and 1200 paragraphs of travel guide text (plain text, average length 120 fields); We can also connect with third-party data, such as Ctrip OTA interface and Gaode Map API, to obtain ticket reservation data (500-600 records, including reservations and verifications), parking lot data (10-20 records), and shuttle bus timetable data (3-5 records).

[0030] Secondly, the data access module performs standardized processing and low-latency streaming, using the Flink 1.17 stream processing framework. Three parallel access tasks are configured to correspond to three types of data sources: For official channel data, a long polling + incremental fetch strategy is used, with an interface call frequency of once every 5 seconds, ensuring end-to-end data latency ≤ 5 seconds; for UGC crawler tasks, a distributed node system with 8 nodes is used, with a single node crawling rate of 10 frames per second, and video frame extraction using FFmpeg (extracting 1 keyframe per second), ensuring end-to-end latency ≤ 8 seconds; for third-party interface tasks, a subscription-push model is used, with push notifications triggered when data is updated, ensuring end-to-end latency ≤ 6 seconds. This enables a peak throughput of 120 messages per second, while meeting the requirement of a real-time computing support unit end-to-end latency of less than 10 seconds.

[0031] This paper performs a unified transformation of text and visual data. The standardization method used is to extract the official PDF text using PyPDF2. Then, through regular expressions and the Cultural Tourism Micro DistilBERT model, all text data is decomposed into structured fragments (e.g., object ID-change type-effective time-scope of impact) Figure 3 (as shown) The visual standardization uses the ViT-B / 32 visual model, which extracts frames from images and videos and transforms them into 512-dimensional visual feature vectors, which satisfy the normalization formula: Vvis =[ v 1, v 2,..., v 512], of which vi =∑ j =1512 xj2 xi ( xi (for the original output values ​​of the model), ensure vi ∈[0,1].

[0032] Noise filtering was performed using a fine-tuned DistilBERT model (trained with 80,000 text clips from the 2022-2025 Grand Canyon Glass Walkway travel guide). A binary classification task (effectively changed text / noisy text) was used to filter out noise such as user emotional descriptions and irrelevant chatter, achieving a filtering rate of 35% and a noise recognition accuracy of 98.3%. Image noise filtering is performed by using the DHash algorithm to calculate the hash value of a 16×16 grayscale image. Images with a distance ≤3 are considered duplicates. 260 duplicate / similar images are filtered out, and 740 valid images are retained. The filtering time is ≤2 seconds.

[0033] Cross-modal change extraction was performed, with text change extraction using the Qwen-1.8B model fine-tuned for cultural tourism (fine-tuning data: 5000 text entries of new / changed scenic facilities, 10 training rounds, learning rate 2e-5). The Prompt project guided the model to output structured change units, resulting in a text change extraction accuracy of 92% (900 out of 975 valid text entries were consistent with the official notification), and a single text processing time of ≤0.3 seconds. Visual change recognition uses a fine-tuned MobileNetV3 model for cultural tourism (training data: 2000 images of scenic facilities, including 10 categories of labels such as boardwalks, viewing platforms, and parking lots, with an accuracy of 95.6%) to classify visual data and output feature labels for glass boardwalks (such as transparent glass material / cliff-attached structure) and confidence scores (average confidence score of 0.88). The processing time for a single image is ≤0.1 seconds. The cross-modal attention module is used to calculate the correlation similarity between text semantic vectors and visual feature vectors, employing the cosine similarity formula: Sim =∥ Vtext ∥⋅∥ Vvis ∥ Vtext ⋅ Vvis in Vtext This is the 768-dimensional text semantic vector output by the Qwen-1.8B model. Vvis For the 512-dimensional visual feature vector output by the ViT model, a similarity threshold of ≥0.8 was set, and 890 high-confidence change candidate sets were selected, with a candidate set accuracy of over 90%.

[0034] Determine the change priority, construct a priority scoring model, and assign weights as follows: W 1 = 0.4 (range of influence)W 2 = 0.3 (timeliness) W 3 = 0.3 (User Relevance), the scoring formula is: S = W 1⋅ A + W 2⋅ B + W 3⋅ C Where A, B, and C are the original scores (1-4 points) for the three dimensions, and S is the weighted total score. The scoring process for the newly added scene of the glass walkway in this embodiment is as follows: Scope of impact (A): Local area coverage, 3 points → weighted score 1.2 points; Timeliness (B): Effective immediately, 3 points → weighted score 0.9 points; User Relevance (C): Core Service (New Facility), 3 points → Weighted Score 0.9 points; Total Score S =3.0 (corresponding to the original total score of 9 points), judged as high priority, with a processing time requirement of ≤10 minutes.

[0035] Incremental updates are executed by calling Neo4j's Cypher statement `(n:facility{object ID:"Grand Canyon Glass Walkway", change time:"2024-06-01 07:50:00", data source:"official API", opening hours:"08:00-17:00", visitor limit:"2000 people / day"})CREATE` to create a node. Using a predefined cultural tourism classification system (scenic area → region → facility three-level classification), the node is automatically associated with the parent node "West Sea Grand Canyon". The statement `MATCH(a:region{name:"Grand Canyon"}), (b:facility{object ID:"Grand Canyon Glass Walkway"})CREATE (a)-[r:contains{association strength:0.95}]->(b)` is executed. Then, placeholders for surrounding nodes are initialized, reserving relation fields such as entrance, parking lot, and emergency exit. Simultaneously, based on the scenic area's geographic information data, a new node is added: Glass Walkway - 5-minute walk - Grand Canyon Entrance. The glass walkway - 15-minute walk - parking lot relationship has an initial association strength of 0.8 (to be corrected based on user feedback). This step takes 7 minutes and 20 seconds from candidate set input to update completion, which meets the high priority timeliness requirements.

[0036] A dual detection mechanism of rule engine + LLM causal reasoning is adopted for explicit conflict detection: the source priority rule is official data > UGC data > third-party data; the time consistency rule is that the same attribute cannot have two non-expired effective times at the same time; the attribute legality rule requires the open time format to match a regular expression. The rule engine performs the verification through Python's Pyrules library. No explicit conflicts were detected in this embodiment. To detect implicit conflicts, a fine-tuned Llama 3-70B model for cultural tourism was used (fine-tuning data: 2000 cultural tourism knowledge association cases, 5 training rounds). The input consisted of updated node information and existing graph associations. The model output inference results: 1. Missing association between the glass walkway and the accessible passage; 2. The emergency exit attribute is unclear and needs to be supplemented. The system automatically performed association supplementation, adding: Glass walkway - accessible passage: None (labeled confidence level 0.6), and number of glass walkway - emergency exits: 2 (labeled confidence level 0.8), to avoid implicit logical conflicts.

[0037] Multi-dimensional automatic verification was performed. Cross-modal consistency verification adopted the cultural tourism fine-tuning CLIP model (ViT-B / 32 architecture, fine-tuning data: 50,000 pairs of cultural tourism text-image pairs). The similarity between the updated text and visual data was calculated (the formula is the same as Sim in Section 3.1). In this embodiment, the average similarity was 0.85, which is ≥ 0.6 threshold, and the verification was passed. For knowledge logic verification, the ontology file (OWL format) of the cultural tourism field was loaded using the RDFox 6.0 tool to verify the graph constraints: facility-type nodes must be associated with object ID, belonging region, and opening time attributes; the subject of the containment relationship must be a region class, and the object must be a facility class. After verification, this embodiment has no constraint violations and the verification is passed.

[0038] A closed-loop manual review and feedback process is implemented. For manual review, high-priority updates are pushed to three scenic area knowledge reviewers via WebSocket. The review interface displays the changes, data source credentials, and relationships. Reviewers complete the review by agreeing, rejecting, or making notes. In this embodiment, the three people agree and confirm within 2 minutes, and the update officially takes effect. By utilizing user feedback to create a closed loop, a knowledge correction entry point was set up in the scenic area's APP (map application terminal), allowing users to submit error types and supporting data. In this example, two valid feedbacks were received: the feedback content stated that the glass walkway was actually open until 17:30, and the supporting data was a photo taken at 17:20 (with a time watermark); the feedback content stated that it takes 20 minutes to walk from the parking lot to the walkway, and the supporting data was a video of the walking time. In response to the above feedback, the system converts the feedback data into a standardized format and sends it back to the change identification module, triggering the update process again: the opening hours are corrected to 08:00-17:30, the walking time is 20 minutes, and the entire error correction process takes 45 minutes, meeting the response requirement of ≤1 hour.

[0039] In summary, the cross-modal cultural tourism knowledge graph construction system based on a large language model according to embodiments of the present invention uses cultural tourism corpora from boardwalk scenic areas and similar scenic areas for fine-tuning of all core models, thereby improving the model's recognition accuracy for cultural tourism-specific entities such as glass boardwalks by 26.4%. It employs an implicit conflict reasoning mechanism, leveraging the causal reasoning capabilities of Llama 3-70B to automatically supplement implicit associations such as emergency exits and barrier-free access, solving the knowledge fragmentation problem of traditional solutions. The lightweight access tool supports manual uploading of changed data by small and medium-sized scenic areas, eliminating the need for technical integration and improving access efficiency by 3 times.

[0040] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0041] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0042] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A cross-modal cultural tourism knowledge graph construction system based on a large language model, comprising a data source module, a data access module, a change identification module, an incremental update module, a verification feedback module, a core storage and application module, and a technical foundation module, characterized in that: The data source module is used to provide various types of raw cultural and tourism data. The data access module includes a multi-source data access adapter and a data preprocessing unit. The multi-source data access adapter uses Flink stream processing to adapt to multi-source interfaces to achieve low-latency access. The data preprocessing unit converts heterogeneous data into text, visual vectors, and timestamps and filters out irrelevant content. The change identification module includes a cross-modal change extraction unit and a change priority determination unit. The cross-modal change extraction unit improves the credibility of the change candidate set by using cultural tourism fine-tuning LLM, visual model, and Cross-Attention module. The change priority determination unit uses a three-dimensional scoring system to classify changes into high, medium, and low priorities and determines the processing time. The incremental update module includes an incremental update execution unit and a dual conflict detection unit. The incremental update execution unit performs corresponding operations according to node addition, relationship adjustment, and attribute deletion. The dual conflict detection unit detects explicit / implicit conflicts through a rule base and causal inference LLM to generate conflict-free update content. The verification feedback module includes a multi-dimensional automatic verification unit and a manual review and feedback closed-loop unit. The multi-dimensional automatic verification unit uses the cultural tourism fine-tuning CLIP and RDFox for automatic verification. The manual review and feedback closed-loop unit manually confirms high-priority updates and receives user error correction feedback to trigger updates again. The core storage and application module includes a cultural tourism knowledge graph based on Neo4j and a graph application client. The Neo4j cultural tourism knowledge graph supports incremental writing and version rollback. The graph application client provides cultural tourism recommendations, route planning, and sets up knowledge correction entry points. The technology foundation module includes a real-time computing support unit, a large language model support unit, a vision technology support unit, and a graph technology support unit. The real-time computing support unit provides Flink stream processing capabilities, the large language model support unit provides the Qwen-1.8B / Llama3-70B model, the vision technology support unit provides MobileNetV3 / ViT vision processing capabilities, and the graph technology support unit provides Neo4j / RDFox graph tools.

2. The cross-modal cultural tourism knowledge graph construction system based on a large language model according to claim 1, characterized in that: The data preprocessing unit further includes a format normalization subunit and a noise filtering subunit; The format standardization subunit uses ViT to extract frames from images / videos and convert them into 512-dimensional visual vectors, and to split text into a structure of object ID, change type, effective time, and scope of influence. The noise filtering subunit filters unchanging text using the Cultural Tourism Fine-tuning DistilBERT model and removes duplicate images using the DHash image hashing algorithm.

3. The cross-modal cultural tourism knowledge graph construction system based on a large language model according to claim 1, characterized in that: The cross-modal change extraction unit includes a text change extraction subunit, a visual change recognition subunit, and a cross-modal fusion subunit; The text change extraction subunit uses the Qwen-1.8B model and change unit to extract the Prompt output structured change unit; The visual change recognition subunit uses MobileNetV to classify image / video frames and outputs feature labels and 0-1 confidence scores. The cross-modal fusion subunit uses the Cross-Attention model to calculate similarity and retains candidate sets with a similarity of ≥0.

8.

4. The cross-modal cultural tourism knowledge graph construction system based on a large language model according to claim 1, characterized in that: The three-dimensional scoring rules of the change priority determination unit include: Impact Scope Dimension (4 points): Assigned based on the scope of cultural and tourism objects covered by the change, with global coverage scoring 4 points, local area coverage scoring 2 points, and single-point facility coverage scoring 1 point; Timeliness dimension (3 points): Assigned according to the change effective period: 3 points for immediate effect, 2 points for effect within 7 days, and 1 point for effect after 7 days; User relevance dimension (3 points): Assigned according to the degree of impact of changes on user decisions, core services are worth 3 points, auxiliary services are worth 2 points, and information supplementation is worth 1 point; Priority is determined based on the total score: 8-10 points are designated as high priority with a processing time of ≤10 minutes; 5-7 points are designated as medium priority with a processing time of ≤1 hour; and 1-4 points are designated as low priority with a processing time of ≤4 hours.

5. The cross-modal cultural tourism knowledge graph construction system based on a large language model according to claim 1, characterized in that: The incremental update execution unit adds node configuration change time and source, automatically associates with parent class and initializes surrounding elements, adjusts relationships to effective time and strength, retains historical versions and marks expiration time, deletes attributes and marks expiration status and time, and records the trigger data source number.

6. The cross-modal cultural tourism knowledge graph construction system based on a large language model according to claim 1, characterized in that: The dual conflict detection unit further includes an explicit conflict detection subunit and a implicit conflict detection subunit; The explicit conflict detection subunit is based on conflict trigger verification in the cultural and tourism rule base; The implicit conflict detection subunit adopts the Llama3-70B model, inputs updated information and existing associations, outputs association suggestions and impact prompts, and makes adjustments.

7. The cross-modal cultural tourism knowledge graph construction system based on a large language model according to claim 1, characterized in that: The verification feedback module adopts the CLIP model. When the similarity is <0.6, manual verification is triggered. Knowledge logic verification uses the RDFox tool to check ontology constraints. If there is a violation, the process is paused. User feedback is submitted through the knowledge error correction entry. After standardization, the feedback is sent back to trigger the update again.

8. The cross-modal cultural tourism knowledge graph construction system based on a large language model according to claim 1, characterized in that: The real-time computing support unit achieves an end-to-end latency of less than 10 seconds for data access and flow through the Flink stream processing framework, ensuring real-time processing of high-priority changes; The large language model support unit uses the Qwen-1.8B model for text change extraction, the Llama3-70B model for causal inference, and the DistilBERT model for noise filtering. All models are fine-tuned using corpora from the cultural and tourism field. The visual technology support unit uses the MobileNetV3 model for visual change recognition, the ViT model for visual feature extraction, and the CLIP model for cross-modal consistency verification. The model training data includes images specific to cultural and tourism scenarios. The Neo4j graph database of the graph technology support unit adopts an incremental write mode to avoid the performance loss caused by full updates, and uses the RDFox tool for graph ontology constraint verification to ensure the logical consistency of the graph.

Images