Personalized scenic spot content generation method, system and equipment based on multi-modal knowledge graph, and medium

Abstract

The invention relates to the technical field of smart tourism, in particular to a personalized scenic spot content generation method, system and device based on a multi-modal knowledge graph and a medium, and the method comprises the steps: collecting multi-modal data of a scenic spot, and constructing a structured multi-modal scenic spot knowledge graph based on the multi-modal data; obtaining personalized data provided by the tourist, and constructing a personalized portrait of the tourist based on the personalized data; in response to an operation of acquiring an image in a scenic spot, identifying the scenery in the image, and extracting a corresponding initial knowledge subset from the multi-modal scenic spot knowledge graph according to an identification result; performing personalized screening and expansion on the knowledge nodes in the initial knowledge subset by calculating the matching degree between the knowledge nodes in the initial knowledge subset and the tourist interest in the personalized portrait of the tourist, and dynamically binding multi-modal resources to generate a personalized knowledge graph; and presenting the personalized knowledge graph. According to the invention, the multi-mode scenic spot knowledge experience of which the structure is matched with the interest and the knowledge background can be provided for tourists.

Description

Technical Field

[0001] This application relates to the field of smart tourism technology, specifically to a method, system, device, and medium for generating personalized scenic spot content based on multimodal knowledge graphs. Background Technology

[0002] As the digital transformation of the cultural and tourism industry continues to deepen, tourists' demand for intelligent and personalized guided tour services is growing. Utilizing artificial intelligence technology to enhance the scenic area experience, especially by integrating multimodal data to provide tourists with in-depth and interactive content services, has become an important direction for industry development.

[0003] In existing technologies, some scenic areas have already applied intelligent service solutions for attraction information. For example, some solutions construct knowledge graphs in the tourism field to extract attractions, entities, and their relationships from online text in a structured manner, providing support for information retrieval. Other solutions utilize multimodal large models to integrate data such as text, images, and voice to achieve the recognition of scenery and standardized voice or text-based explanations. In addition, there are technologies that focus on providing personalized tourism information recommendations to users by analyzing user search behavior and combining it with knowledge graphs.

[0004] However, existing intelligent service solutions for tourist attractions still have several shortcomings. First, existing solutions largely rely on user-initiated text queries or triggers, failing to build an end-to-end real-time response link from tourists taking images of scenery on-site to automatically generating structured knowledge content, thus hindering ease of use. Second, when providing information services, existing solutions typically output uniform or simple preference-based content, failing to deeply integrate the specific semantic features of currently identified scenery with tourists' multi-dimensional and dynamically updated personalized profiles for accurate content selection and adaptation. Third, while existing technologies can link multimodal data, they mostly output it as independent explanatory materials, failing to treat these text, image, and audio resources as attributes or related objects, deeply binding and structurally organizing them into an interactive knowledge graph network, thereby limiting the depth and relevance of knowledge presentation. Summary of the Invention

[0005] To address the technical problems of existing intelligent scenic spot information service solutions, such as reliance on user-initiated triggering, failure to accurately adapt content based on specific scenic object semantics and multi-dimensional user profiles, and lack of deep structural integration of multimodal resources into knowledge networks, this application provides a method, system, device, and medium for generating personalized scenic spot content based on a multimodal knowledge graph. By constructing a multimodal knowledge graph and deeply integrating dynamic user profiles, an end-to-end personalized closed loop from scenic object recognition to content generation is achieved, thereby providing tourists with a clearly structured, accurately matched multimodal scenic spot knowledge experience that is adapted to their interests and knowledge background.

[0006] Firstly, this application provides a method for generating personalized tourist attraction content based on a multimodal knowledge graph, comprising the following steps: Collect multimodal data on tourist attractions and construct a structured multimodal knowledge graph of tourist attractions based on the multimodal data; Acquire personalized data provided by tourists and build personalized tourist profiles based on the personalized data; In response to the operation of acquiring images of scenic spots, the system identifies the objects in the images and extracts the corresponding initial knowledge subset from the multimodal scenic spot knowledge graph based on the identification results. By calculating the matching degree between knowledge nodes in the initial knowledge subset and tourist interests in the personalized tourist profile, the knowledge nodes in the initial knowledge subset are individually filtered and expanded, and multimodal resources are dynamically bound to generate a personalized knowledge graph. Present personalized knowledge graphs.

[0007] It should be further noted that the multimodal data of a scenic spot includes text data, image data, voice data, 3D model data, and geospatial data related to the scenic spot as a whole and its various features.

[0008] It should be further noted that the text data includes: official introductions, historical documents, travelogues and guides, and academic materials; Image data includes: overall photos of the scenic spot, overall photos of the scenery, close-up details of the scenery, and relevant historical images; The audio data includes: professional explanation audio, dialect version narration, and folk custom audio; The 3D model data includes: structural models of ancient buildings and 3D scanned models of cultural relics. Geospatial data includes: coordinates of attractions and scenic spots, tour routes, and spatial relationships between attractions and scenic spots.

[0009] It should be further explained that the collection of multimodal data of scenic spots specifically includes: automatically collecting relevant data of scenic spots through the open API interface of the scenic spot or compliant web crawlers, and then manually supplementing and annotating the data for review and correction to obtain the multimodal data of the scenic spots.

[0010] It should be further noted that the process of constructing a structured multimodal scenic spot knowledge graph based on multimodal data includes knowledge extraction, semantic mapping, and graph construction. Knowledge extraction includes: High-dimensional feature vectors of text are obtained by extracting features from text data based on pre-trained language models. Named entity recognition technology is used to extract scenic spot entities and attributes from high-dimensional feature vectors of text. By using relation extraction technology, the relationships between scenic spot entities are extracted based on high-dimensional features; Semantic mapping includes: Semantic mapping between image data, voice data and scenic spot entities is performed using cross-modal semantic alignment technology to obtain the binding relationship between images, voice and scenic spot entities; Map construction includes: Based on the extracted scenic spot entities, scenic spot attributes, relationships, and binding relationships, a graph structure data is generated with entities as nodes and relationships as edges; The graph structure data is associated with the corresponding multimodal data and stored in RDF format to complete the construction of a structured multimodal scenic spot knowledge graph.

[0011] It should be further noted that the physical entities of a tourist attraction include buildings, cultural relics, and natural landscapes; The attributes of a tourist attraction include its age, materials, and function; The relationships between attractions include historical connections, spatial connections, and causal connections.

[0012] It should be further explained that cross-modal semantic alignment technology is used to perform semantic mapping between image data, voice data, and scenic spot entities, thereby obtaining the binding relationship between images, voice, and scenic spot entities, specifically including: Image data is input into an image encoder, speech data is input into a speech encoder, and text descriptions corresponding to scenic spot entities are input into a text encoder to obtain image feature vectors, speech feature vectors, and text feature vectors, respectively. By using a contrastive learning algorithm, image feature vectors, speech feature vectors and corresponding text feature vectors are aligned in the joint embedding space, minimizing the distance between matching sample pairs and maximizing the distance between non-matching sample pairs. The association weights between image feature vectors, speech feature vectors and text feature vectors are calculated based on the attention mechanism. Based on the association weights, image data and speech data are bound to semantically similar scenic spot entities, generating the binding relationship between images, speech and scenic spot entities.

[0013] It should be further noted that before generating the spectral structure data, a conflict resolution process is also included, specifically: Assign credibility weights to each data source; Detect whether there are conflicting values ​​for the same attribute or relation of the same entity in scenic spot attributes and semantic relations from different sources; When conflicting values ​​exist, the confidence weight difference is calculated. If the confidence weight difference is higher than the preset confidence difference threshold, the value of the data source with the highest confidence weight is adopted first. Otherwise, the conflicting values ​​are retained as multi-valued attributes or multi-valued relationships.

[0014] It should be further noted that the allocation of credibility weights is based on a base weight set according to the data source type, and is calculated in conjunction with additional weight adjustment rules, wherein: The base weight of the official data from the scenic area is 0.9; The base weight of academic literature is 0.7, and the additional weight increases by 0.1 when the number of citations is ≥50, and the additional weight increases by 0.05 when the author is a leading expert in the field, with the final weight capped at 0.8; The base weight of a legitimate travelogue platform is 0.5, and the weight increases by 0.05 when the travelogue has ≥100,000 views, and by 0.03 when it contains a professional certification mark, with a final weight cap of 0.6. The base weight for user-generated content (UGC) from ordinary users is 0.2.

[0015] It should be further noted that the multimodal scenic spot knowledge graph supports incremental updates; Incremental updates include: In response to updates to the scenic area's multimodal data, the knowledge extraction and semantic mapping process is automatically triggered to obtain newly added or changed knowledge, including scenic spot entities, scenic spot attributes, and related or binding relationships. The newly added or changed knowledge is integrated with the existing content in the multimodal scenic spot knowledge graph and conflict resolution is performed to obtain incremental data; Incremental data is incrementally synchronized to the stored multimodal scenic spot knowledge graph, thereby enabling incremental updates of the multimodal scenic spot knowledge graph.

[0016] It should be further noted that the personalized data provided by tourists includes: Basic information and interest tags provided by tourists; Data on tourists' browsing behavior when using designated mobile applications, collected with the tourists' consent.

[0017] It should be further noted that basic information includes age, occupation, and purpose of travel; Interest tags include at least one of the following: history and culture, architectural aesthetics, natural science, folk customs, photography, parent-child interaction, food exploration, and leisure and entertainment. Visitor behavior data includes photo recordings, map browsing trajectory, dwell time, click preferences, and questions asked.

[0018] It should be further explained that building personalized tourist profiles based on personalized data specifically includes: Based on the collected basic information, interest tags and tour behavior data, a multi-dimensional tourist profile model is generated, which includes basic attribute dimensions, interest preference dimensions, knowledge background dimensions and tour progress dimensions, as a personalized tourist profile. Among them, the basic attribute dimension contains basic information provided to tourists; The content of the interest preference dimension is an interest weight vector generated based on the interest tags and travel purposes selected by tourists. The knowledge background dimension is determined by analyzing the proportion of professional documents in the visitors' historical browsing content, which establishes the knowledge background level. The tour progress dimension provides real-time tour progress status, generated and updated in real time based on GPS positioning data and tour route planning.

[0019] It should be further explained that the interest weight vector is generated through the following steps: Based on the interest tags actively selected by tourists, an initial interest weight vector is generated. The initial interest weight vector takes the optional interest tags as dimensions, and the weight value corresponding to each optional interest tag is taken as the value of the corresponding dimension. Among them, the dimensions corresponding to the interest tags selected by tourists are assigned high weight values, while the dimensions corresponding to the interest tags not selected by tourists are assigned low weight values. Based on the tourists' travel purposes, the weight values ​​of the corresponding dimensions in the baseline interest weight vector are adjusted, and then normalized to obtain the interest preference vector.

[0020] It should be further noted that the dimension of the interest weight vector is fixed at 8 dimensions, corresponding to history and culture, architectural aesthetics, natural science, folk customs, photography, parent-child interaction, food exploration, and leisure and entertainment. When generating the initial weights, the initial weights of the interest tags selected by tourists are set to 0.8, and the initial weights of the unselected tags are set to 0.2, and then normalization is performed. The weighting is adjusted according to the purpose of the trip: when the purpose of the trip is study tour, the weight of the history and culture, architectural aesthetics, and natural science popularization dimensions is increased by 0.1 each, while the weight of the leisure and entertainment dimension is decreased by 0.1; when the purpose of the trip is photography, the weight of the photography check-in dimension is increased by 0.2, and the weight of other dimensions is reduced proportionally.

[0021] It should be further noted that the interest preference vector supports dynamic updates, which are achieved through the following steps: Collect real-time interactive behavior data of tourists to obtain a set of behavior records; Based on the correspondence between behavior type and interest tag, the quantitative impact value of the current behavior on the current interest preference vector is calculated from the behavior record set; Using the time decay coefficient, the current interest preference vector and the quantified influence value are fused and calculated through the weight update formula to obtain the updated interest weight vector; The updated interest weight vector is normalized, and the weight range of a single dimension is constrained to generate an updated interest preference vector.

[0022] It should be further explained that the weight update formula is as follows:

[0023] Indicates the first Each optional interest tag corresponds to a dimension in The weight of each moment; Indicates the first Each optional interest tag corresponds to a dimension in The weight of each moment; This represents the preset time decay coefficient; Indicates in Time and The first time between moments The intensity coefficient of interactive behavior. =1; Indicates the first Class interaction behavior and the first The matching degree of each optional interest tag is a preset value.

[0024] It should be further explained that, according to Time and Duration settings between moments: corresponding to 1 day =0.9, corresponding to days 1-7 =0.7, corresponding to 7-30 days =0.5, corresponding to 30 days or more =0.3: Interactive behaviors include: clicking a node, with a corresponding intensity coefficient of 0.3; The intensity coefficient for staying and browsing is 0.15; Extended nodes, corresponding to a strength coefficient of 0.25; Voice questioning corresponds to an intensity coefficient of 0.4; The shared content has an intensity coefficient of 0.2. The weight constraint for a single dimension is that the minimum value is not less than 0.05 and the maximum value is not more than 0.6.

[0025] It should be further explained that in the process of building personalized tourist profiles and processing interactive behavior data, federated learning technology is used for privacy protection. Privacy protection includes: de-identifying the user's original data locally on the mobile device, adding differential privacy noise to the gradients generated by local model training and then homomorphically encrypting them, and then uploading the encrypted gradients to the cloud for aggregation. The anonymization process includes: retaining only the surname for tourists, replacing the middle 4 digits of mobile phone numbers with "****", and retaining only the first 6 and last 4 digits of ID card numbers; reducing the accuracy of GPS coordinates in behavioral data to the 100-meter level, and adding ±3% random noise to dwell time and click frequency. Before uploading the gradient, Gaussian noise conforming to differential privacy is added with a noise intensity of ε=1.0 and δ=1e-5, and then encrypted using the Paillier homomorphic encryption algorithm.

[0026] It should be further explained that the operation of acquiring images of scenic spots specifically refers to taking single or continuous photos of the scenery in the scenic area using the mobile device's shooting function.

[0027] It should be further noted that before recognizing objects in the image, an image preprocessing step is included, specifically: The captured images are subjected to illumination correction, noise removal, and image scaling and normalization to obtain preprocessed images; The sharpness of the pre-processed image is calculated. If the sharpness is lower than the preset sharpness threshold, a shooting failure is indicated.

[0028] It should be further explained that recognizing objects in an image specifically includes: The image is input into a lightweight scene recognition model for inference to obtain scene recognition results, which include scene name, scene type, recognition confidence, and the location information of the scene in the image.

[0029] It should be further noted that the lightweight scene recognition model is an optimized version of the YOLOv8 model, with optimizations including: Channel pruning was performed on the C2f module in the backbone network of the YOLOv8 model, pruning low-contribution residual channels by a pruning ratio of 40%. The convolution kernels of the spatial pyramid pooling module are pruned, removing redundant 1x1 convolution kernels by a pruning ratio of 20%. Low-weight convolutional kernels in the detection head branch are pruned. Convolutional kernels with absolute weight values ​​below 1e-4 are pruned at a rate of 30%. The pruned model was quantized using INT8, reducing the model parameter precision from FP32 to INT8. INT8 quantization employs a symmetric quantization scheme with a quantization range of [-127, 127]. 1000 images from the scenic area vertical dataset are used as the calibration set, and accuracy loss is fine-tuned through knowledge distillation. The quantized model was fine-tuned using a scenic area vertical dataset to ensure that the final model size was no more than 50MB, the inference time on mobile devices was no more than 1 second, and the recognition accuracy was no less than 93%.

[0030] It should be further noted that, based on the recognition results, a corresponding initial knowledge subset is extracted from the multimodal scenic spot knowledge graph, specifically including: Based on the identified scenic spot names, query the multimodal scenic spot knowledge graph to obtain the corresponding scenic spot entities as core nodes; Centered on the core node, extract all associated nodes and relationships within its one-hop range to form an initial knowledge subset.

[0031] It should be further explained that the knowledge nodes are personalized by calculating the matching degree between the knowledge nodes in the initial knowledge subset and the tourist interests in the personalized tourist profile. This personalized filtering of knowledge nodes specifically includes: The matching degree is calculated by matching the interest preference vector in the tourist's personalized profile with the type vector of the knowledge node in the initial knowledge subset, and the matching score of each knowledge node is obtained. Filter out knowledge nodes with matching scores no lower than a preset matching threshold, and sort them from highest to lowest matching score to obtain the filtered set of knowledge nodes.

[0032] It should be further noted that the matching degree is calculated using the cosine similarity method, and the calculation formula is as follows:

[0033] in, A vector representing tourists' interest preferences; This vector represents the type of a knowledge node. The vector takes a value of 1 in the corresponding interest dimension based on the type of the knowledge node, and a value of 0 in the other dimensions. Representing vectors and Cosine similarity; This represents the preset correlation strength coefficient. =0.2; This indicates the strength of the association between a knowledge node and the currently captured scene; the core node's... =1.0, the first-level associated node =0.8, second-level associated nodes =0.5.

[0034] It should be further explained that the selected set of knowledge nodes undergoes knowledge expansion, specifically including: The graph neural network algorithm is used to mine the implicit relationships between nodes in the filtered set of knowledge nodes; Based on the strength of the association, expand the associated knowledge nodes with an association strength not lower than a preset association threshold and add them to the knowledge node set to form an expanded knowledge node set.

[0035] It should be further explained that dynamic binding of multimodal resources specifically includes: The tourist type is determined based on the travel purpose and knowledge background level in the tourist's personalized profile. Then, based on the tourist type, the appropriate multimodal resource version is dynamically selected for each knowledge node in the expanded knowledge node set.

[0036] It should be further noted that the types of tourists include: study tour participants, leisure travelers, photographers, families, and those seeking in-depth cultural experiences.

[0037] It should be further explained that generating personalized knowledge graphs specifically includes: Using the core node corresponding to the currently captured scene as the center, the expanded set of knowledge nodes is used as the associated nodes to construct a graph structure; Bind each node in the graph to its corresponding dynamically selected multimodal resource; Generate personalized knowledge graph data that includes graph structure information and bound multimodal resource information.

[0038] It should be further noted that at least one of the following presentation methods is used to present the personalized knowledge graph: Visualization: The personalized knowledge graph is visualized using a force-directed layout algorithm. Entities are represented by nodes and relationships by edges. Entity types are distinguished by node color and relationship strength is represented by edge thickness. Node scaling, translation, expansion and collapse operations are supported. Voice presentation: In response to a user clicking on a node in the knowledge graph, the voice explanation resources bound to that node are triggered and played, and the explanation style is adapted based on the visitor's personalized profile; Augmented reality presentation: Real-time landscape is captured through mobile camera, and node information in personalized knowledge graph is spatially overlaid with corresponding physical objects, presented in the form of floating labels or virtual models; Interactive 3D Model Presentation: Provides 3D models of key scenes, supporting user-interactive operations such as rotation, scaling, and sectioning.

[0039] It should be further explained that the tour guide style is adapted based on the tourist's personalized profile, specifically including: When tourists' purpose of travel is study or their knowledge background level is high, a professional and rigorous audio guide is appropriate. When tourists' travel purpose is leisure or their knowledge background level is average, a lively and easy-to-understand audio guide is appropriate.

[0040] It should be further explained that the spatial overlay in augmented reality specifically includes: By combining GPS and inertial measurement unit (IMU) sensor data from mobile devices with visual SLAM technology, the attitude of the mobile phone camera can be tracked and located in real time. By precisely aligning the virtual information of knowledge graph nodes with the actual coordinates of physical objects, a stable overlay of virtual information onto the real scene can be achieved.

[0041] It should be further noted that the personalized knowledge graph presentation supports offline mode, including: When the network is interrupted, it automatically switches to offline mode and presents the generated personalized knowledge graph based on the graph data and multimodal resources cached locally on the mobile device.

[0042] It should be further noted that while presenting personalized knowledge graphs, user interaction behavior data is also collected. Interactive behavior data includes: node click records, node dwell time, graph browsing path, voice question content, AR interactive operations, and content sharing and saving operations.

[0043] It should be further explained that the collected interaction behavior data is used to update the personalized profile of tourists. The update includes feeding the interaction behavior data back to the tourist's interest preference vector through an incremental learning algorithm, and dynamically adjusting its interest weights.

[0044] Secondly, this application provides a personalized tourist attraction content generation system based on a multimodal knowledge graph, used to implement the aforementioned personalized tourist attraction content generation method, including: A multimodal data acquisition module is used to collect multimodal data of scenic spots; The scenic spot knowledge graph construction module is used to build a structured multimodal scenic spot knowledge graph based on multimodal data; The personalized data acquisition module is used to acquire personalized data provided by tourists. The personalized profile building module is used to build personalized profiles of tourists based on personalized data. The scene recognition module is used to identify the scenes in the images in response to the operation of acquiring images of the scenic spot; The initial knowledge subset extraction module is used to extract the corresponding initial knowledge subset from the multimodal scenic spot knowledge graph based on the image recognition results; The personalized knowledge graph generation module is used to personalize and expand the knowledge nodes in the initial knowledge subset by calculating the matching degree between the knowledge nodes in the initial knowledge subset and the tourist interests in the personalized tourist profile, and dynamically bind multimodal resources to generate a personalized knowledge graph. The presentation module is used to present personalized knowledge graphs.

[0045] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described personalized scenic spot content generation method.

[0046] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described personalized scenic spot content generation method.

[0047] As can be seen from the above technical solutions, this application has the following advantages: 1. This application constructs a structured multimodal scenic spot knowledge graph by collecting multimodal data of scenic spots, and performs personalized filtering, expansion and binding of the initial knowledge subset extracted after identifying scenic spots based on the personalized profile of tourists, and finally generates and presents a personalized knowledge graph. This solves the problem of existing technologies relying on users to actively query and lacking an end-to-end automatic response link from visual interaction to knowledge generation. It realizes the convenient experience of "generating knowledge by shooting", which enables tourists to obtain a personalized knowledge network that is deeply adapted to their interests, has a clear structure and rich content through simple shooting operations, which significantly improves the knowledge, fun and personalized experience of visiting.

[0048] 2. This application constructs a structured knowledge graph containing multimodal data such as text, images, voice, and 3D models, and dynamically binds and adapts multimodal resources when generating personalized knowledge graphs, so that knowledge can be presented in a richer and more user-preferred form, thus meeting the diverse perception and learning needs of tourists.

[0049] 3. This application identifies scenery in response to tourists' shooting operations and extracts an initial knowledge subset from a multimodal scenic spot knowledge graph accordingly, achieving a seamless experience of "shooting is associating knowledge", which greatly reduces the operational threshold and cognitive burden for tourists to obtain in-depth information.

[0050] 4. This application solves the problems of existing solutions that output uniform or simple content and fail to deeply integrate the specific semantic features of the currently identified scene with the multi-dimensional personalized profile of tourists to achieve accurate content adaptation by calculating the matching degree between knowledge nodes in the initial knowledge subset based on the personalized profile of tourists, and performing personalized filtering and expansion of knowledge nodes. This achieves refined and dynamic customization of knowledge presentation and ensures that the generated content is highly relevant to the tourists' real-time interests and the tour scene.

[0051] 5. This application generates and presents a personalized knowledge graph containing graph structure information and bound multimodal resource information by dynamically binding personalized filtering and expanded knowledge nodes with resources such as text, images, and voice. This solves the problem of existing technologies that output multimodal resources as independent materials and fail to deeply bind and structurally organize them into an interactive knowledge network. This allows knowledge to be presented in a three-dimensional and explorable form in the form of an interconnected network, significantly enhancing the depth and interactivity of knowledge services. Attached Figure Description

[0052] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of a method for generating personalized scenic spot content based on a multimodal knowledge graph in one embodiment of this application.

[0054] Figure 2 This is a schematic block diagram of a personalized scenic spot content generation system based on multimodal knowledge graph in one embodiment of this application.

[0055] Figure 3 This is a schematic diagram of the hardware structure of an electronic device in one embodiment of this application. Detailed Implementation

[0056] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] The personalized scenic spot content generation method of this application will be described in detail below. Specific details such as particular system structures and technologies are presented for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details.

[0058] In the personalized attraction content generation method involved in this application, the term "comprising" indicates the presence of the described feature, whole, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0059] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0060] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0061] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0062] The personalized attraction content generation method provided in this application embodiment is executed by a computer device, and correspondingly, the personalized attraction content generation system based on multimodal knowledge graph runs in the computer device.

[0063] Figure 1 This is a flowchart illustrating a method for generating personalized tourist attraction content based on a multimodal knowledge graph, according to an embodiment of this application. Figure 1 The implementing entity can be a personalized attraction content generation system. Depending on different needs, the order of the steps in this flowchart can be changed, and some can be omitted.

[0064] like Figure 1 As shown, the personalized tourist attraction content generation method based on multimodal knowledge graph includes: Step S1: Collect multimodal data of scenic spots and construct a structured multimodal scenic spot knowledge graph based on the multimodal data.

[0065] In some specific embodiments, the multimodal data of a scenic spot includes text data, image data, voice data, 3D model data, and geospatial data related to the scenic spot as a whole and its various features.

[0066] In some specific embodiments, the text data includes: official introductions, historical documents, travelogues and guides, and academic materials; Image data includes: overall photos of the scenic spot, overall photos of the scenery, close-up details of the scenery, and relevant historical images; The audio data includes: professional explanation audio, dialect version narration, and folk custom audio; The 3D model data includes: structural models of ancient buildings and 3D scanned models of cultural relics. Geospatial data includes: coordinates of attractions and scenic spots, tour routes, and spatial relationships between attractions and scenic spots.

[0067] In some specific embodiments, collecting multimodal data of scenic spots includes: automatically collecting relevant data of scenic spots through the scenic spot's open API interface or compliant web crawler, and then manually supplementing and annotating the data for review and correction to obtain multimodal data of scenic spots.

[0068] In some specific embodiments, the construction of a structured multimodal scenic spot knowledge graph based on multimodal data includes knowledge extraction, semantic mapping, and graph construction processes in sequence. Knowledge extraction includes: High-dimensional feature vectors of text are obtained by extracting features from text data based on pre-trained language models. Named entity recognition technology is used to extract scenic spot entities and attributes from high-dimensional feature vectors of text. By using relation extraction technology, the relationships between scenic spot entities are extracted based on high-dimensional features; Semantic mapping includes: Semantic mapping between image data, voice data and scenic spot entities is performed using cross-modal semantic alignment technology to obtain the binding relationship between images, voice and scenic spot entities; Map construction includes: Based on the extracted scenic spot entities, scenic spot attributes, relationships, and binding relationships, a graph structure data is generated with entities as nodes and relationships as edges; The graph structure data is associated with the corresponding multimodal data and stored in RDF format to complete the construction of a structured multimodal scenic spot knowledge graph.

[0069] In some specific embodiments, the entities of a scenic spot include buildings, cultural relics, and natural landscapes; The attributes of a tourist attraction include its age, materials, and function; The relationships between attractions include historical connections, spatial connections, and causal connections.

[0070] In some specific embodiments, cross-modal semantic alignment technology is used to perform semantic mapping between image data, voice data, and scenic spot entities to obtain the binding relationship between images, voice, and scenic spot entities, specifically including: Image data is input into an image encoder, speech data is input into a speech encoder, and text descriptions corresponding to scenic spot entities are input into a text encoder to obtain image feature vectors, speech feature vectors, and text feature vectors, respectively. By using a contrastive learning algorithm, image feature vectors, speech feature vectors and corresponding text feature vectors are aligned in the joint embedding space, minimizing the distance between matching sample pairs and maximizing the distance between non-matching sample pairs. The association weights between image feature vectors, speech feature vectors and text feature vectors are calculated based on the attention mechanism. Based on the association weights, image data and speech data are bound to semantically similar scenic spot entities, generating the binding relationship between images, speech and scenic spot entities.

[0071] In some specific embodiments, a conflict resolution process is included before generating the map structure data, specifically including: Assign credibility weights to each data source; Detect whether there are conflicting values ​​for the same attribute or relation of the same entity in scenic spot attributes and semantic relations from different sources; When conflicting values ​​exist, the confidence weight difference is calculated. If the confidence weight difference is higher than the preset confidence difference threshold, the value of the data source with the highest confidence weight is adopted first. Otherwise, the conflicting values ​​are retained as multi-valued attributes or multi-valued relationships.

[0072] In some specific embodiments, the assignment of credibility weights is based on a base weight set according to the data source type, and calculated in conjunction with additional weight adjustment rules, wherein: The base weight of the official data from the scenic area is 0.9; The base weight of academic literature is 0.7, and the additional weight increases by 0.1 when the number of citations is ≥50, and the additional weight increases by 0.05 when the author is a leading expert in the field, with the final weight capped at 0.8; The base weight of a legitimate travelogue platform is 0.5, and the weight increases by 0.05 when the travelogue has ≥100,000 views, and by 0.03 when it contains a professional certification mark, with a final weight cap of 0.6. The base weight for user-generated content (UGC) from ordinary users is 0.2.

[0073] In one specific embodiment, the credibility weight allocation rule table for credibility weight allocation is shown in Table 1.

[0074] Table 1 Credibility Weight Allocation Table

[0075] In some specific embodiments, the multimodal scenic spot knowledge graph supports incremental updates; Incremental updates include: In response to updates to the scenic area's multimodal data, the knowledge extraction and semantic mapping process is automatically triggered to obtain newly added or changed knowledge, including scenic spot entities, scenic spot attributes, and related or binding relationships. The newly added or changed knowledge is integrated with the existing content in the multimodal scenic spot knowledge graph and conflict resolution is performed to obtain incremental data; Incremental data is incrementally synchronized to the stored multimodal scenic spot knowledge graph, thereby enabling incremental updates of the multimodal scenic spot knowledge graph.

[0076] Step S2: Obtain personalized data provided by tourists and construct personalized tourist profiles based on the personalized data.

[0077] In some specific embodiments, the personalized data provided by tourists includes: Basic information and interest tags provided by tourists; Data on tourists' browsing behavior when using designated mobile applications, collected with the tourists' consent.

[0078] In some specific embodiments, the basic information includes age, occupation, and purpose of travel; Interest tags include at least one of the following: history and culture, architectural aesthetics, natural science, folk customs, photography, parent-child interaction, food exploration, and leisure and entertainment. Visitor behavior data includes photo recordings, map browsing trajectory, dwell time, click preferences, and questions asked.

[0079] In some specific embodiments, a personalized tourist profile is constructed based on personalized data, specifically including: Based on the collected basic information, interest tags and tour behavior data, a multi-dimensional tourist profile model is generated, which includes basic attribute dimensions, interest preference dimensions, knowledge background dimensions and tour progress dimensions, as a personalized tourist profile. Among them, the basic attribute dimension contains basic information provided to tourists; The content of the interest preference dimension is an interest weight vector generated based on the interest tags and travel purposes selected by tourists. The knowledge background dimension is determined by analyzing the proportion of professional documents in the visitors' historical browsing content, which establishes the knowledge background level. The tour progress dimension provides real-time tour progress status, generated and updated in real time based on GPS positioning data and tour route planning.

[0080] In some specific embodiments, the interest weight vector is generated through the following steps: Based on the interest tags actively selected by tourists, an initial interest weight vector is generated. The initial interest weight vector takes the optional interest tags as dimensions, and the weight value corresponding to each optional interest tag is taken as the value of the corresponding dimension. Among them, the dimensions corresponding to the interest tags selected by tourists are assigned high weight values, while the dimensions corresponding to the interest tags not selected by tourists are assigned low weight values. Based on the tourists' travel purposes, the weight values ​​of the corresponding dimensions in the baseline interest weight vector are adjusted, and then normalized to obtain the interest preference vector.

[0081] In some specific embodiments, the dimension of the interest weight vector is fixed at 8 dimensions, corresponding to history and culture, architectural aesthetics, natural science, folk customs, photography, parent-child interaction, food exploration, and leisure and entertainment. When generating the initial weights, the initial weights of the interest tags selected by tourists are set to 0.8, and the initial weights of the unselected tags are set to 0.2, and then normalization is performed. The weighting is adjusted according to the purpose of the trip: when the purpose of the trip is study tour, the weight of the history and culture, architectural aesthetics, and natural science popularization dimensions is increased by 0.1 each, while the weight of the leisure and entertainment dimension is decreased by 0.1; when the purpose of the trip is photography, the weight of the photography check-in dimension is increased by 0.2, and the weight of other dimensions is reduced proportionally.

[0082] In some specific embodiments, the interest preference vector supports dynamic updates, which are achieved through the following steps: Collect real-time interactive behavior data of tourists to obtain a set of behavior records; Based on the correspondence between behavior type and interest tag, the quantitative impact value of the current behavior on the current interest preference vector is calculated from the behavior record set; Using the time decay coefficient, the current interest preference vector and the quantified influence value are fused and calculated through the weight update formula to obtain the updated interest weight vector; The updated interest weight vector is normalized, and the weight range of a single dimension is constrained to generate an updated interest preference vector.

[0083] In some specific embodiments, the weight update formula is:

[0084] Indicates the first Each optional interest tag corresponds to a dimension in The weight of each moment; Indicates the first Each optional interest tag corresponds to a dimension in The weight of each moment; This represents the preset time decay coefficient; Indicates in Time and The first time between moments The intensity coefficient of interactive behavior. =1; Indicates the first Class interaction behavior and the first The matching degree of each optional interest tag is a preset value.

[0085] In some specific embodiments, according to Time and Duration settings between moments: corresponding to 1 day =0.9, corresponding to days 1-7 =0.7, corresponding to 7-30 days =0.5, corresponding to 30 days or more =0.3: Interactive behaviors include: clicking a node, with a corresponding intensity coefficient of 0.3; The intensity coefficient for staying and browsing is 0.15; Extended nodes, corresponding to a strength coefficient of 0.25; Voice questioning corresponds to an intensity coefficient of 0.4; The shared content has an intensity coefficient of 0.2. The weight constraint for a single dimension is that the minimum value is not less than 0.05 and the maximum value is not more than 0.6.

[0086] In one specific embodiment, the weight update rule table for weight updating is shown in Table 2.

[0087]

[0088] In some specific embodiments, in the process of constructing personalized tourist profiles and processing interactive behavior data, federated learning technology is used for privacy protection. Privacy protection includes: desensitizing the user's original data locally on the mobile device, adding differential privacy noise to the gradients generated by local model training and then homomorphically encrypting them, and then uploading the encrypted gradients to the cloud for aggregation. The anonymization process includes: retaining only the surname for tourists, replacing the middle 4 digits of mobile phone numbers with "****", and retaining only the first 6 and last 4 digits of ID card numbers; reducing the accuracy of GPS coordinates in behavioral data to the 100-meter level, and adding ±3% random noise to dwell time and click frequency. Before uploading the gradient, Gaussian noise conforming to differential privacy is added with a noise intensity of ε=1.0 and δ=1e-5, and then encrypted using the Paillier homomorphic encryption algorithm.

[0089] Step S3: In response to the operation of acquiring images of scenic spots, identify the scenery in the images, and extract the corresponding initial knowledge subset from the multimodal scenic spot knowledge graph based on the identification results.

[0090] In some specific embodiments, the operation of acquiring images of scenic spots is specifically to take single or continuous photos of the scenic spots using the mobile device's shooting function.

[0091] In some specific embodiments, an image preprocessing step is included before identifying objects in the image, specifically including: The captured images are subjected to illumination correction, noise removal, and image scaling and normalization to obtain preprocessed images; The sharpness of the pre-processed image is calculated. If the sharpness is lower than the preset sharpness threshold, a shooting failure is indicated.

[0092] In some specific embodiments, identifying objects in an image specifically includes: The image is input into a lightweight scene recognition model for inference to obtain scene recognition results, which include scene name, scene type, recognition confidence, and the location information of the scene in the image.

[0093] In some specific embodiments, the lightweight scene recognition model is a model obtained by optimizing the YOLOv8 model, and the optimization includes: Channel pruning was performed on the C2f module in the backbone network of the YOLOv8 model, pruning low-contribution residual channels by a pruning ratio of 40%. The convolution kernels of the spatial pyramid pooling module are pruned, removing redundant 1x1 convolution kernels by a pruning ratio of 20%. Low-weight convolutional kernels in the detection head branch are pruned. Convolutional kernels with absolute weight values ​​below 1e-4 are pruned at a rate of 30%. The pruned model was quantized using INT8, reducing the model parameter precision from FP32 to INT8. INT8 quantization employs a symmetric quantization scheme with a quantization range of [-127, 127]. 1000 images from the scenic area vertical dataset are used as the calibration set, and accuracy loss is fine-tuned through knowledge distillation. The quantized model was fine-tuned using a scenic area vertical dataset to ensure that the final model size was no more than 50MB, the inference time on mobile devices was no more than 1 second, and the recognition accuracy was no less than 93%.

[0094] In one specific embodiment, the specific parameter table for INT8 symmetric quantization is shown in Table 3.

[0095] Table 3 Specific Parameters for INT8 Symmetric Quantization

[0096] In some specific embodiments, an initial knowledge subset is extracted from the multimodal scenic spot knowledge graph based on the recognition results, specifically including: Based on the identified scenic spot names, query the multimodal scenic spot knowledge graph to obtain the corresponding scenic spot entities as core nodes; Centered on the core node, extract all associated nodes and relationships within its one-hop range to form an initial knowledge subset.

[0097] Step S4: By calculating the matching degree between the knowledge nodes in the initial knowledge subset and the tourist interests in the tourist's personalized profile, the knowledge nodes in the initial knowledge subset are personalized and expanded, and multimodal resources are dynamically bound to generate a personalized knowledge graph.

[0098] In some specific embodiments, knowledge nodes are personalized by calculating the matching degree between knowledge nodes in the initial knowledge subset and tourist interests in the personalized tourist profile. This includes: The matching degree is calculated by matching the interest preference vector in the tourist's personalized profile with the type vector of the knowledge node in the initial knowledge subset, and the matching score of each knowledge node is obtained. Filter out knowledge nodes with matching scores no lower than a preset matching threshold, and sort them from highest to lowest matching score to obtain the filtered set of knowledge nodes.

[0099] In some specific embodiments, the matching degree is calculated using the cosine similarity method, and the calculation formula is as follows:

[0100] in, A vector representing tourists' interest preferences; This vector represents the type of a knowledge node. The vector takes a value of 1 in the corresponding interest dimension based on the type of the knowledge node, and a value of 0 in the other dimensions. Representing vectors and Cosine similarity; This represents the preset correlation strength coefficient. =0.2; This indicates the strength of the association between a knowledge node and the currently captured scene; the core node's... =1.0, the first-level associated node =0.8, second-level associated nodes =0.5.

[0101] In some specific embodiments, the filtered set of knowledge nodes is expanded, specifically including: The graph neural network algorithm is used to mine the implicit relationships between nodes in the filtered set of knowledge nodes; Based on the strength of the association, expand the associated knowledge nodes with an association strength not lower than a preset association threshold and add them to the knowledge node set to form an expanded knowledge node set.

[0102] In some specific embodiments, dynamic binding of multimodal resources specifically includes: The tourist type is determined based on the travel purpose and knowledge background level in the tourist's personalized profile. Then, based on the tourist type, the appropriate multimodal resource version is dynamically selected for each knowledge node in the expanded knowledge node set.

[0103] In some specific embodiments, tourist types include: study tour users, leisure users, photography users, family users, and in-depth cultural users.

[0104] In one specific embodiment, the tourist type determination rule table based on the travel purpose and knowledge background level in the tourist's personalized profile is shown in Table 4.

[0105] Table 4. Rules for Determining Tourist Types

[0106] In one specific embodiment, the rule for dynamically selecting the appropriate multimodal resource version based on each knowledge node in the expanded set of knowledge nodes for visitor type includes: For study tour users, priority is given to binding professional version resources, including: academic explanations (with reference annotations), professional explanations (at a moderate pace, including professional terminology), 3D models + scanned copies of historical artifacts + high-definition detailed images; For casual users, priority will be given to binding popular resources, including: popular explanations (≤300 words), vivid and popular explanations (including interesting stories), and high-definition real-life pictures + short videos; For photography users, priority is given to binding photography-adapted resources, including: shooting guides (best angles, lighting suggestions), shooting technique breakdowns, shooting location marking maps + lighting effect comparison maps + AR shooting templates; For families with children, priority will be given to integrating parent-child interactive resources, including: nursery rhyme-style explanations (approximately 200 words, including metaphors and simplified versions), cartoon-style explanations, and 3D models + interactive Q&A animations. For users with a deep understanding of culture, priority is given to binding the professional version with extended resources, including: professional explanations + related cultural extension content, professional explanations + dialect versions available, 3D models + historical scene restoration + related scenic spot navigation.

[0107] In some specific embodiments, the rules also include: core resources (text + voice) must be bound, and multimedia resources are bound according to the priority of tourist type (e.g., study tour users are given priority for 3D models, and photography users are given priority for shooting location maps). Users can manually switch resource versions (e.g., casual users can choose to view the professional version of the text), and the switching record is synchronized to the visitor profile, affecting the subsequent resource recommendation weight.

[0108] In some specific embodiments, generating personalized knowledge graphs specifically includes: Using the core node corresponding to the currently captured scene as the center, the expanded set of knowledge nodes is used as the associated nodes to construct a graph structure; Bind each node in the graph to its corresponding dynamically selected multimodal resource; Generate personalized knowledge graph data that includes graph structure information and bound multimodal resource information.

[0109] Step S5: Present the personalized knowledge graph.

[0110] In some specific embodiments, personalized knowledge graphs are presented using at least one of the following presentation methods: Visualization: The personalized knowledge graph is visualized using a force-directed layout algorithm. Entities are represented by nodes and relationships by edges. Entity types are distinguished by node color and relationship strength is represented by edge thickness. Node scaling, translation, expansion and collapse operations are supported. Voice presentation: In response to a user clicking on a node in the knowledge graph, the voice explanation resources bound to that node are triggered and played, and the explanation style is adapted based on the visitor's personalized profile; Augmented reality presentation: Real-time landscape is captured through mobile camera, and node information in personalized knowledge graph is spatially overlaid with corresponding physical objects, presented in the form of floating labels or virtual models; Interactive 3D Model Presentation: Provides 3D models of key scenes, supporting user-interactive operations such as rotation, scaling, and sectioning.

[0111] In some specific embodiments, the explanation style is adapted based on the tourist's personalized profile, specifically including: When tourists' purpose of travel is study or their knowledge background level is high, a professional and rigorous audio guide is appropriate. When tourists' travel purpose is leisure or their knowledge background level is average, a lively and easy-to-understand audio guide is appropriate.

[0112] In some specific embodiments, spatial overlay in augmented reality presentation includes: By combining GPS and inertial measurement unit (IMU) sensor data from mobile devices with visual SLAM technology, the attitude of the mobile phone camera can be tracked and located in real time. By precisely aligning the virtual information of knowledge graph nodes with the actual coordinates of physical objects, a stable overlay of virtual information onto the real scene can be achieved.

[0113] In some specific embodiments, the presentation of personalized knowledge graphs supports offline mode, including: When the network is interrupted, it automatically switches to offline mode and presents the generated personalized knowledge graph based on the graph data and multimodal resources cached locally on the mobile device.

[0114] In some specific embodiments, user interaction behavior data is also collected while presenting personalized knowledge graphs; Interactive behavior data includes: node click records, node dwell time, graph browsing path, voice question content, AR interactive operations, and content sharing and saving operations.

[0115] In some specific embodiments, the collected interaction behavior data is used to update the personalized profile of tourists. The update includes: feeding the interaction behavior data back to the tourist's interest preference vector through an incremental learning algorithm, and dynamically adjusting its interest weights.

[0116] The following are embodiments of a personalized tourist attraction content generation system based on a multimodal knowledge graph provided in this application. This personalized tourist attraction content generation system based on a multimodal knowledge graph belongs to the same inventive concept as the personalized tourist attraction content generation methods in the above embodiments. For details not described in detail in the embodiments of the personalized tourist attraction content generation system, please refer to the embodiments of the personalized tourist attraction content generation methods based on a multimodal knowledge graph described above.

[0117] like Figure 2 As shown, the personalized tourist attraction content generation system based on multimodal knowledge graph includes: A multimodal data acquisition module is used to collect multimodal data of scenic spots; The scenic spot knowledge graph construction module is used to build a structured multimodal scenic spot knowledge graph based on multimodal data; The personalized data acquisition module is used to acquire personalized data provided by tourists. The personalized profile building module is used to build personalized profiles of tourists based on personalized data. The scene recognition module is used to identify the scenes in the images in response to the operation of acquiring images of the scenic spot; The initial knowledge subset extraction module is used to extract the corresponding initial knowledge subset from the multimodal scenic spot knowledge graph based on the image recognition results; The personalized knowledge graph generation module is used to personalize and expand the knowledge nodes in the initial knowledge subset by calculating the matching degree between the knowledge nodes in the initial knowledge subset and the tourist interests in the personalized tourist profile, and dynamically bind multimodal resources to generate a personalized knowledge graph. The presentation module is used to present personalized knowledge graphs.

[0118] The personalized tourist attraction content generation system in this embodiment is used to implement a method for generating personalized tourist attraction content based on a multimodal knowledge graph.

[0119] This application also provides an electronic device for implementing the various embodiments of this application. Figure 3 To illustrate the hardware structure of an electronic device according to various embodiments of this application, as shown in the following diagram... Figure 3 As shown, the electronic device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor.

[0120] Those skilled in the art will understand that the electronic device structure involved in the embodiments of this application does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0121] In embodiments of this application, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0122] In this application embodiment, the processor can be implemented using at least one of an Application-Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a processor, a controller, a microcontroller, a microprocessor, or an electronic unit designed to perform the functions described herein. In some cases, such implementations can be implemented within a controller. For software implementations, implementations such as processes or functions can be implemented with separate software modules that allow the performance of at least one function or operation. The software code can be implemented by a software application (or program) written in any suitable programming language, and the software code can be stored in memory and executed by the controller.

[0123] In addition, the electronic device includes some functional modules not shown, which will not be described in detail here.

[0124] Those skilled in the art will understand that the various aspects of the electronic device provided in this application can be implemented as a system, method, or program product. Therefore, the various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0125] This application also provides a storage medium storing a program product capable of implementing a method for generating personalized tourist attraction content based on a multimodal knowledge graph. In some possible implementations, various aspects of this application can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this application.

[0126] The storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example,, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating personalized scenic spot content based on multimodal knowledge graphs, characterized in that, include: Collect multimodal data on tourist attractions and construct a structured multimodal knowledge graph of tourist attractions based on the multimodal data; Acquire personalized data provided by tourists and build personalized tourist profiles based on the personalized data; In response to the operation of acquiring images of scenic spots, the system identifies the objects in the images and extracts the corresponding initial knowledge subset from the multimodal scenic spot knowledge graph based on the identification results. By calculating the matching degree between knowledge nodes in the initial knowledge subset and tourist interests in the personalized tourist profile, the knowledge nodes in the initial knowledge subset are individually filtered and expanded, and multimodal resources are dynamically bound to generate a personalized knowledge graph. Present personalized knowledge graphs.

2. The personalized scenic spot content generation method as described in claim 1, characterized in that, Multimodal data of scenic spots include text data, image data, voice data, 3D model data, and geospatial data related to the overall scenic spot and its various features.

3. The personalized scenic spot content generation method as described in claim 2, characterized in that, The process of constructing a structured multimodal scenic spot knowledge graph based on multimodal data includes knowledge extraction, semantic mapping, and graph construction. Knowledge extraction includes: High-dimensional feature vectors of text are obtained by extracting features from text data based on pre-trained language models. Named entity recognition technology is used to extract scenic spot entities and attributes from high-dimensional feature vectors of text. By using relation extraction technology, the relationships between scenic spot entities are extracted based on high-dimensional features; Semantic mapping includes: Semantic mapping between image data, voice data and scenic spot entities is performed using cross-modal semantic alignment technology to obtain the binding relationship between images, voice and scenic spot entities; Map construction includes: Based on the extracted scenic spot entities, scenic spot attributes, relationships, and binding relationships, a graph structure data is generated with entities as nodes and relationships as edges; The graph structure data is associated with the corresponding multimodal data and stored in RDF format to complete the construction of a structured multimodal scenic spot knowledge graph.

4. The personalized scenic spot content generation method as described in claim 1, characterized in that, Personalized data provided by tourists includes: Basic information and interest tags provided by tourists; Data on tourists' browsing behavior when using designated mobile applications, collected with the tourists' consent.

5. The personalized scenic spot content generation method as described in claim 4, characterized in that, Personalized tourist profiles are built based on personalized data, specifically including: Based on the collected basic information, interest tags and tour behavior data, a multi-dimensional tourist profile model is generated, which includes basic attribute dimensions, interest preference dimensions, knowledge background dimensions and tour progress dimensions, as a personalized tourist profile. Among them, the basic attribute dimension contains basic information provided to tourists; The content of the interest preference dimension is an interest weight vector generated based on the interest tags and travel purposes selected by tourists. The knowledge background dimension is determined by analyzing the proportion of professional documents in the visitors' historical browsing content, which establishes the knowledge background level. The tour progress dimension provides real-time tour progress status, generated and updated in real time based on GPS positioning data and tour route planning.

6. The personalized scenic spot content generation method as described in claim 5, characterized in that, By calculating the matching degree between knowledge nodes in the initial knowledge subset and tourist interests in the personalized tourist profile, knowledge nodes are individually filtered, specifically including: The matching degree is calculated by matching the interest preference vector in the tourist's personalized profile with the type vector of the knowledge node in the initial knowledge subset, and the matching score of each knowledge node is obtained. Filter out knowledge nodes with matching scores no lower than a preset matching threshold, and sort them from highest to lowest matching score to obtain the filtered set of knowledge nodes.

7. The personalized scenic spot content generation method as described in claim 1, characterized in that, Dynamic binding of multimodal resources includes: The tourist type is determined based on the travel purpose and knowledge background level in the tourist's personalized profile. Then, based on the tourist type, the appropriate multimodal resource version is dynamically selected for each knowledge node in the expanded knowledge node set.

8. A personalized scenic spot content generation system based on multimodal knowledge graph, characterized in that, To implement the personalized attraction content generation method as described in any one of claims 1-7, the method includes: A multimodal data acquisition module is used to collect multimodal data of scenic spots; The scenic spot knowledge graph construction module is used to build a structured multimodal scenic spot knowledge graph based on multimodal data; The personalized data acquisition module is used to acquire personalized data provided by tourists. The personalized profile building module is used to build personalized profiles of tourists based on personalized data. The scene recognition module is used to identify the scenes in the images in response to the operation of acquiring images of the scenic spot; The initial knowledge subset extraction module is used to extract the corresponding initial knowledge subset from the multimodal scenic spot knowledge graph based on the image recognition results; The personalized knowledge graph generation module is used to personalize and expand the knowledge nodes in the initial knowledge subset by calculating the matching degree between the knowledge nodes in the initial knowledge subset and the tourist interests in the personalized tourist profile, and dynamically bind multimodal resources to generate a personalized knowledge graph. The presentation module is used to present personalized knowledge graphs.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes a computer program, it implements the steps of the personalized attraction content generation method as described in any one of claims 1-7.

10. A storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the steps of the personalized attraction content generation method as described in any one of claims 1-7.

Images