A panoramic interactive content automatic generation method and system of a fusion multi-modal large model

Abstract

The application discloses a kind of fusion multi-modal big model panoramic interactive content automatic generation method and system.Method includes: receiving unstructured multi-modal demand, reconstructing to generate initial panoramic interactive language script linked list;For current scene, utilize panoramic feature vector calculation panoramic chart spherical equidistant slice attention weighting algorithm, through cross-modal matching degree retrieval, control flow conversion reuse existing material or trigger diffusion model adaptive generation visual continuous brand-new panoramic chart;According to industry theme probability mask matrix decision hot spot list, call big model synthesis multi-modal hot spot media;In spherical coordinate system, predefine coordinate is configured to hot spot, and spiral line step optimization collision detection is carried out to remaining hot spot using spherical great circle distance constraint, complete unobstructed space layout;Finally, standard panoramic interactive engineering package is packaged and exported, and artificial collaborative editing is supported.The application significantly improves the automation level and cross-scene visual coherence of panoramic interactive content full-link generation.

Description

Technical Field

[0001] This invention belongs to the field of panoramic interactive content generation technology, specifically relating to a method and system for automatically generating panoramic interactive content by integrating multimodal large models. This invention can automatically complete the entire process of panoramic interactive content generation based on user needs, from "scene planning—panoramic generation—hotspot layout—interactive arrangement—content output—collaborative editing," and is applicable to application scenarios such as cultural tourism guidance, interactive teaching, industrial simulation training, and virtual exhibition halls based on panoramic technology. Background Technology

[0002] Panoramic interactive content typically consists of multiple 360° panoramic scenes and interactive hotspots embedded within them. Users can freely roam between different scenes and interact with the hotspot content using a mouse, touch device, or VR terminal.

[0003] Currently, the production of panoramic interactive content mainly relies on manual operation of panoramic editors. This requires manual creation of panoramas, arrangement of scenes, preparation of hotspot materials, and layout of hotspots, which has problems such as high learning threshold, time-consuming and labor-intensive content editing, and low degree of automation. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for automatically generating panoramic interactive content by integrating multimodal large models, the core implementation of which is as follows.

[0005] Step S1: User Input and Semantic Parsing Receive unstructured multimodal requirement information input by the user, the requirement information including natural language prompts, documents or images; call a multimodal large model to perform topic recognition on the requirement information and output topic categories; The multimodal large model is invoked to perform semantic decomposition and structural reconstruction of the required information, generating panoramic initial data information containing scene sequence number, scene name, scene description, and scene prompt words, and constructing a scene temporal linked list; a structured panoramic interactive language script is defined, and the generated panoramic initial data information is saved.

[0006] Step S2: Panoramic Scene Retrieval and Generation A pre-constructed panoramic material vector database is constructed, which contains panoramic description information and corresponding panoramic feature vectors; For the current scene to be generated, cross-modal retrieval and matching are performed by combining the scene description in the initial panoramic data information with the panoramic material description and feature vector in the panoramic material vector database, and the comprehensive matching score is calculated. The system iterates through the panoramic materials in the panoramic material library and filters for the material with the highest overall matching degree. When the highest overall matching degree is greater than or equal to a preset hard threshold, the panoramic resource path corresponding to the matching material is directly reused and written into the corresponding field in the panoramic interactive language script. When the highest overall matching degree is less than the preset hard threshold, the system proceeds to the large model AI panoramic image generation, which calls the adaptive guided diffusion model based on reference image condition control to generate the corresponding panoramic image and writes the storage path of the generated panoramic image into the corresponding field in the panoramic interactive language script.

[0007] Step S3: Automatic planning of trending content The system sets up a hotspot probability mask matrix under different industry themes. Based on the theme category output in step S1, the system performs probability filtering on the preset hotspot type set according to the weight value corresponding to each hotspot type, automatically filtering out the hotspot types with low probability and retaining the hotspot types with high weight values, thereby finally determining the list of interactive hotspots to be generated for the current scenario. Then, the multimodal large model and corresponding media generation algorithm are invoked to automatically generate multimodal hotspot material content corresponding to each hotspot type in the interactive hotspot list; and the storage path of the generated structured data and multimedia files are associated with the hotspot material content and written into the corresponding hotspot information list field in the panoramic interactive language script.

[0008] Step S4: Hotspot Space Layout Planning Based on the current panoramic image, establish a three-dimensional spatial coordinate system, and build a hotspot spatial layout model within the panoramic scene under the three-dimensional spatial coordinate system; set predefined spatial coordinates for different types of hotspots; For the remaining hotspots without predefined spatial coordinates, an iterative collision detection algorithm based on distance constraints is used to calculate their final spatial coordinates in order to avoid spatial occlusion collisions between hotspots. All hotspot coordinate data are then stored in the corresponding fields in the panoramic interactive language script.

[0009] Step S5: Automatic generation of panoramic interactive project The panoramic images, trending media resources, and structured configuration files generated in the above process are packaged together to form a panoramic interactive project package. The extracted final data information is then restructured and stored in the project's structured file.

[0010] Step S6: Collaborative Editing The automatically generated panoramic interactive project package is pushed to the visual panoramic editor; the collaborative editing mode is entered in response to the user's adjustment needs; the secondary editing and adjustment instructions input by the user through interactive operations in the visual panoramic editor are received to correct the automatically generated content. The secondary editing and adjustment instructions include adding or deleting scenes, adjusting the scene order, modifying hotspot information, and adjusting the spatial coordinate position of hotspots.

[0011] The panoramic interactive content automatic generation method based on this invention can fully integrate multimodal large model technology to automate the panoramic scene, hotspot materials, hotspot layout, and the entire project engineering of panoramic interactive content, reducing the complexity of manual operations and thus improving content production efficiency and reducing production costs. Simultaneously, this invention also utilizes a cross-modal retrieval and matching algorithm to allow users to fully and rationally select and use their existing panoramic scene materials, facilitating users to continuously accumulate materials within their industry, improving content production efficiency and industry-specific relevance, while also enhancing cross-scene visual coherence. Attached Figure Description

[0012] The invention will now be further described with reference to the accompanying drawings.

[0013] Figure 1 This is the overall flowchart of the method for automatically generating panoramic interactive content.

[0014] Figure 2 This is a schematic diagram of panoramic feature vector generation.

[0015] Figure 3 It is a cross-modal retrieval and matching calculation process for the panoramic material library.

[0016] Figure 4 This is a schematic diagram of the layout plan for hotspot spaces. Detailed Implementation

[0017] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0018] like Figure 1 The diagram shown is the overall flowchart of the system designed in this invention, which includes the following steps:

[0019] Step S1: Input Module: User Input and Semantic Parsing The system receives unstructured multimodal request information from users (including natural language prompts, documents, images, etc.), and performs deep understanding through a multimodal model, including:

[0020] S101: Content Theme Recognition Call the multimodal large model to identify the topic to which the content belongs and output the topic category C_theme (such as C1-culture and tourism, C2-education, C3-industrial training, C4-virtual exhibition hall, etc.).

[0021] S102: Reconstructing User Needs The multimodal large model is invoked to decompose and reconstruct the input content, and combined with the theme category C_theme, the structured panoramic initial data information is output, including scene number SceneID, scene name SceneName, scene description Description, and scene prompt word Prompt. This data constructs a scene temporal linked list. The system defines a structured panoramic interactive language script called SceneNode to save the generated initial panoramic data information: SceneNode(i)={SceneID(i), SceneName(i), Description(i), Prompt(i),HotspotArrayList(i), SceneURL(i)} in, SceneID(i) is the scene number; SceneName(i) is the scene name; Description(i) is the scene description; Prompt(i) is a prompt word generated by optimizing the panoramic image generated from the large model. Its typical structure is as follows: [Theme C_theme] + [Detailed Description] + [Style Parameters] + 360-degree equirectangularpanorama, VR seamless sphere, HDR; HotspotArrayList(i) is a list of hotspot information; SceneURL(i) is the path to the panoramic resource.

[0022] Step S2: Panoramic Scene Retrieval and Generation

[0023] S201: Panoramic Material Library Search and Matching To avoid the illusions caused by raw images from general large models and to save computing power costs, the system introduces a panorama-specific cross-modal retrieval and matching algorithm.

[0024] 1. Panoramic material database construction Panoramic image data is pre-acquired, and a panoramic image vector database (SceneDB) is established. Each panoramic image data item includes a panoramic ID, a panoramic file path (SceneDB_URL), a panoramic description (SceneDB_Dec), and a panoramic feature vector (SceneDB_VM). That is, SceneDB = {SceneDB_ID,SceneDB_URL,SceneDB_Dec,SceneDB_VM} in: SceneDB_ID is the identifier for panoramic footage; SceneDB_URL is the path to the panoramic footage; SceneDB_Dec is a description of the panoramic footage; SceneDB_VM is the panoramic feature vector; Among them, such as Figure 2 As shown, the specific calculation method for the panoramic feature vector (SceneDB_VM) is as follows: a) Using the spherical isometric slicing algorithm, the panoramic image is spherically reprojected into six perspective viewports: front, back, left, right, top, and bottom. The corresponding visual features are extracted by a cross-modal visual encoder (preferably CLIP visual encoder): F1, F2, F3, F4, F5, and F6. b) The system sets attention weights A1, A2, A3, A4, A5, and A6 based on the visual importance of each viewport within the user's default viewing path, where: Typically, the top viewport F5 and bottom viewport F6 in panoramic view have fewer core visual elements, so the system sets them as low-weighted viewports and the others as high-weighted viewports. c) The formula for calculating the panoramic feature vector generated by fusion is:

[0025] 2. Panoramic material retrieval During system operation, suitable materials are first retrieved from the panoramic material library. The matching degree Sim is calculated by combining the scene's description(i) in SceneNode(i) with the panoramic images [SceneDB_Dec, SceneDB_VM] in the SceneDB library. Figure 3 As shown, the matching degree Sim is calculated as follows: a) Text semantic mapping V_node_text: Using a text encoder, Description(i) is transformed into a text feature vector V_node_text; similarly, the scene description (SceneDB_Dec) of the kth material in the material library is transformed into a text feature vector V_text(k); b) Cross-modal visual mapping V_node_vis: Using a cross-modal text encoder (preferably a CLIP text encoder) that shares a joint semantic space with the cross-modal visual encoder described above, Description(i) is transformed into a cross-modal visual control vector V_node_vis; c) The system calculates the similarity between the current requirement and the k-th material in the material library from two dimensions: "textual semantic similarity" and "cross-modal visual similarity". Sim_text, a text semantic similarity function, calculates the degree of fit between the current scene description vector and the scene description vector in the material library using the cosine similarity formula. Sim_visual cross-modal visual similarity: Calculates the fit between the cross-modal visual mapping vector describing the current scene and the panoramic feature vector of the k-th element in the panoramic material library using the cosine similarity formula. The system uses a linear weighted fusion algorithm to calculate the final comprehensive matching score Sim between the current scene and the material: Sim = w_t × Sim_text + w_v × Sim_visual Wherein, w_t and w_v are system-preset static weight coefficients, satisfying w_t + w_v = 1. In this embodiment, w_t = 0.5 and w_v = 0.5 are preferably set, that is, balancing the accuracy of text meaning with the consistency of the visual style of the panoramic image.

[0026] The system calculates the overall matching degree between the scene to be generated and all panoramic materials in the material library, and selects the material with the highest overall matching degree score and its maximum value (Sim_max). The system sets the following material matching conditions: Sim_max ≥ Th (the hard threshold Th is set to 0.8 by default) In other words, after determining the material that meets the hit criteria, the system writes the "SceneDB_URL" of the material into the SceneURL(i) field of SceneNode(i) and skips the subsequent generation of the S202 large model AI panoramic image.

[0027] If Sim_max < Th, it is determined that there are no materials in the material library that meet the conditions, and then proceed to step S202.

[0028] S202: Large Model AI Panoramic Image Generation When triggering AI-generated images, in order to ensure visual continuity between temporal scenes, the system does not use pure text-generated images, but instead adopts an adaptive guided diffusion model based on reference image condition control.

[0029] When generating the panoramic image of the current scene SceneNode(i), the system automatically extracts the panoramic image of the previously confirmed scene SceneNode(i-1) as a structural and color reference image. The image features of SceneNode(i-1) are used as a condition vector and injected into the latent space of the diffusion generation model together with the current scene's Prompt(i). By controlling the redrawing intensity parameter (Denoising Strength between [0.4, 0.6]), the two scenes are made to maintain a strong correlation in overall tone, ambient lighting, and geometric structure, thereby eliminating visual abrupt changes in traditional generation as much as possible.

[0030] Specifically: when i=1, that is, when generating the first panoramic image, the system directly calls the multimodal large model based on Prompt(1) to generate the image.

[0031] After the panoramic image is generated in step S202, the system saves the newly generated panoramic image to the asset folder and writes the storage path of the image into the SceneURL(i) field of the corresponding SceneNode(i).

[0032] Step S3: Automatic planning of trending content The system automatically plans interactive hotspots based on scene content.

[0033] S301 Hotspot Type Decision Based on the industry theme code C_theme output in step S101, the system sets up a hotspot probability mask matrix M_attr for different industry themes, M_attr = {m1,m2,...mn}, where each mi represents the generation weight of the corresponding hotspot type, and n is the total number of preset interactive hotspot types.

[0034] During the hotspot type decision-making process, the system performs probability screening on the preset hotspot type set according to the weight value corresponding to each hotspot type, automatically filtering out hotspot types with low probability and retaining hotspot types with a weight value greater than 0.5, thereby finally determining the list of interactive hotspots to be generated for the current scenario.

[0035] for example: Under the teaching theme, the focus is on selecting topics such as "Q&A hot topics" and "digital human video hot topics"; Under the theme of museums, the focus is on selecting "multilingual audio and video highlights of cultural relics"; In industrial training, the focus is on selecting "textual hot topics, Q&A hot topics, and video hot topics"; Under the theme of the virtual exhibition hall, the focus is on selecting "image hotspots" and "digital human video hotspots".

[0036] S302 Hot Topic Content Generation Once the hot topic type is determined, the system calls the multimodal large model to generate the corresponding hot topic content: Q&A Hot Topics: The large model automatically generates structured QA pairs (including correct answers and explanations) that conform to the teaching syllabus based on the scenario description; Voice / Digital Human Narration Highlights: After the large model generates the narration text, it automatically calls the speech algorithm to synthesize the audio stream. Simultaneously, the audio stream is synchronously input into the waveform attention-based digital human-driven model (Audio-to-Video) to synthesize a digital human narration video file with accurate lip-sync and facial micro-expressions, and an alpha transparency channel in real time; Image Hotspots: Based on the scene content, the large model extracts the content description of image hotspots and submits them to the multimodal large model for image generation; Video Highlights: Based on the scene content, the large model extracts video script descriptions and submits multimodal large models to generate videos.

[0037] The system associates the generated structured QA data and the storage paths of multimedia files (audio, images, and videos) as hotspot content and writes them into the HotspotArrayList(i) field of the corresponding SceneNode(i).

[0038] Step S4: Hotspot Space Layout Planning The system establishes a hotspot optimization model in a spherical coordinate system. In the spherical space of the panoramic scene, the hotspot coordinates are: Pi(Hi,Vi), where: Hi is the yaw angle [-180°, 180°]; Vi is the pitch angle [-90°, 90°].

[0039] like Figure 4 As shown, in order to ensure that the icons corresponding to the hotspots are reasonably distributed in the spherical space of the panoramic scene without obstruction and with reasonable interaction, the system adopts the following method for layout to determine the hotspot distribution coordinates in each scene.

[0040] 1. Scene transition hotspots: These are typically used to allow for sequential transitions between multiple panoramic scenes according to a timeline, enabling scene navigation and switching. The coordinates of these hotspots are preferably set near (0°, -30°), i.e., close to the ground in the panoramic image, to align with user experience. 2. Q&A Hotspots: Preferably, the coordinates are set around (-90°, 0°), so that users only answer questions when they rotate clockwise from the main viewpoint (0°, 0°) to the last area, ensuring that all knowledge elements in the scene have been visited and experienced by the user; 3. Digital Human Hotspot: Preferably set on the right side of the main viewpoint near the ground, i.e., around (20°, -15°), so that when the user enters the scene, they can have a visual effect of being greeted by a real person from the side. 4. For other hotspots in a panoramic scene, the system uses an iterative collision detection algorithm based on spherical distance constraints to calculate their coordinates: Set the initial coordinates of the remaining hotspots to the user's default visual front area (0°, 0°), and calculate the great circle distance d between them and the fixed hotspots (such as scene transition, Q&A, and digital human hotspots). Given two hotspots: Pi(Hi,Vi) and Pj(Hj,Vj), calculate the spherical angular distance d between the two hotspots: To avoid hotspot occlusion and collision, a minimum angular distance threshold θ is set between hotspots: θ = 15° If d ≥ θ, then retain the initial coordinates; If d < θ (an obstruction collision occurs), the system performs a spiral step search traversal on the sphere with a preset yaw angle step size ∆H (e.g., 5°) and pitch angle step size ∆V (e.g., 5°) until a set of blank coordinate regions satisfying d(Pi, Pj) ≥ θ is found, and this region is established as the final coordinate of the hot spot. After all hotspot coordinates are determined, the hotspot coordinate data is stored in the HotspotArrayList(i) field of SceneNode(i).

[0041] Step S5: The panoramic interactive project is automatically generated.

[0042] The system will package the panoramic content data generated in the above process into a unified panoramic interactive project package: 1. Place static resources (scenes, images, audio, video, etc.) into a pre-defined standardized engineering asset folder, and update the HotspotArrayList and SceneURL data in SceneNode; 2. Extract the final data information such as SceneID, SceneName, HotspotArrayList, and SceneURL from SceneNode, restructure it, and store it in the project's structured file.

[0043] Step S6: Collaborative Editing The system will push the generated panoramic interactive project to the visual panoramic editor.

[0044] If users need to adjust the content, they can edit it manually in the visual panorama editor. They can perform operations such as adding or deleting scenes, adjusting the scene order, modifying hotspot information, and adjusting the spatial coordinates of hotspots through intuitive drag-and-drop and mouse clicks.

[0045] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A method for automatically generating panoramic interactive content by integrating multimodal large models, characterized in that, Includes the following steps: Step S1: User Input and Semantic Parsing Receive unstructured multimodal requirement information input by the user, the requirement information including natural language prompts, documents or images; call a multimodal large model to perform topic recognition on the requirement information and output topic categories; The multimodal large model is invoked to perform semantic decomposition and structural reconstruction of the requirement information, generating panoramic initial data information containing scene sequence number, scene name, scene description, and scene prompt words, and constructing a scene temporal linked list; Define a structured panoramic interactive language script to save the generated initial panoramic data information; Step S2: Panoramic Scene Retrieval and Generation A pre-constructed panoramic material vector database is constructed, which contains panoramic description information and corresponding panoramic feature vectors; For the current scene to be generated, cross-modal retrieval and matching are performed between the scene description in the initial panoramic data and the description and feature vector of the panoramic material in the panoramic material vector database, and the comprehensive matching score is calculated; the panoramic materials in the panoramic material library are traversed and the maximum comprehensive matching score is obtained. When the maximum overall matching degree is greater than or equal to the preset hard threshold, the panoramic resource path corresponding to the matching material is directly reused and written into the corresponding field in the panoramic interactive language script. When the maximum comprehensive matching degree is less than the preset hard threshold, the process flows to the generation of large model AI panoramic image, calls the adaptive guided diffusion model based on reference image condition control to generate the corresponding panoramic image, and writes the storage path of the generated panoramic image into the corresponding field in the panoramic interactive language script. Step S3: Automatic planning of trending content The system sets up hotspot probability mask matrices under different industry themes. Based on the theme categories output in step S1, the system performs probability filtering on the preset hotspot type set according to the weight values ​​corresponding to each hotspot type, automatically filtering out hotspot types with low probability and retaining hotspot types with high weight values, thereby finally determining the interactive hotspot list to be generated for the current scenario; then, it calls the multimodal large model and the corresponding media generation algorithm to automatically generate multimodal hotspot material content corresponding to each hotspot type in the interactive hotspot list; The generated structured data and the storage path of the multimedia files are then associated with the hotspot material content and written into the corresponding hotspot information list field in the panoramic interactive language script; Step S4: Hotspot Space Layout Planning Based on the current panoramic image, a spherical coordinate system is established, and a hotspot spatial layout model within the panoramic scene is built under the spherical coordinate system. Predefined spatial coordinates are set for different types of hotspots. For other hotspots without predefined spatial coordinates, an iterative collision detection algorithm based on distance constraints is used to calculate their final spatial coordinates to avoid spatial occlusion collisions between hotspots. All hotspot coordinate data are stored in the corresponding fields in the panoramic interactive language script. Step S5: Automatic generation of panoramic interactive project The panoramic images, hot media resources, and structured configuration files generated in the above process are packaged together to generate a panoramic interactive project package. The extracted final data information is then restructured and stored in the project's structured file. Step S6: Collaborative Editing The automatically generated panoramic interactive project package is pushed to the visual panoramic editor; the collaborative editing mode is entered in response to the user's adjustment needs; the secondary editing and adjustment instructions input by the user through interactive operations in the visual panoramic editor are received to correct the automatically generated content. The secondary editing and adjustment instructions include adding or deleting scenes, adjusting the scene order, modifying hotspot information, and adjusting the spatial coordinate position of hotspots.

2. The method for automatically generating panoramic interactive content by fusing multimodal large models according to claim 1, characterized in that: The data structure of the panoramic interactive language script SceneNode(i) is defined as follows: SceneNode(i)={SceneID(i),SceneName(i),Description(i),Prompt(i),HotspotArrayList(i),SceneURL(i)} Where: SceneID(i) is the scene number; SceneName(i) is the scene name; Description(i) is the scene description; Prompt(i) is the prompt word generated by optimizing the panoramic image generated by the multimodal large model; HotspotArrayList(i) is the hotspot information list; SceneURL(i) is the panoramic resource path; The content structure of the prompt word Prompt(i) is: [Theme C_theme] + [Detailed description] + [Style parameters] + 360-degree equirectangular panorama, VR seamless sphere, HDR.

3. The method for automatically generating panoramic interactive content by fusing multimodal large models according to claim 1, characterized in that: The structure of the panoramic material vector database SceneDB is defined as follows: SceneDB={SceneDB_ID,SceneDB_URL,SceneDB_Dec,SceneDB_VM} Where: SceneDB_ID is the panoramic material identifier; SceneDB_URL is the panoramic material path; SceneDB_Dec is the panoramic material description; and SceneDB_VM is the panoramic feature vector.

4. The method for automatically generating panoramic interactive content by fusing multimodal large models according to claim 3, characterized in that: The specific calculation method for the panoramic feature vector SceneDB_VM includes: a) Using the spherical isometric slicing algorithm, the panoramic image is reprojected onto a spherical perspective to generate six perspective viewports: front, back, left, right, top, and bottom. b) Use a cross-modal visual encoder to extract the visual features corresponding to the six perspective viewports: F1, F2, F3, F4, F5, and F6 respectively; c) Based on the visual importance of each viewport in the default viewing path, set corresponding attention weights: A1, A2, A3, A4, A5, A6, while satisfying the following constraints: , d) A panoramic feature vector is generated using a visual attention-weighted fusion method. The calculation formula is as follows: , Preferably, the cross-modal visual encoder is a CLIP visual encoder.

5. The method for automatically generating panoramic interactive content by fusing multimodal large models according to claim 4, characterized in that, The cross-modal retrieval matching method for calculating the comprehensive matching score Sim in step S2 specifically includes: Text semantic mapping: The current scene description Description(i) is transformed into a text feature vector V_node_text using a text encoder; at the same time, the scene description SceneDB_Dec of the kth material in the material library is transformed into a text feature vector V_text(k); Cross-modal visual mapping: Using a cross-modal text encoder that shares a joint semantic space with the cross-modal visual encoder, Description(i) is transformed into a cross-modal visual control vector V_node_vis; Dual-path similarity calculation: The cosine similarity formula is used to calculate the text semantic similarity Sim_text and the cross-modal visual similarity Sim_visual, respectively. The calculation formulas are as follows: , , Overall Match Score Fusion: The final overall match score Sim is calculated using a linear weighted fusion algorithm. Sim = w_t×Sim_text + w_v×Sim_visual Where w_t and w_v are the system-preset static weight coefficients, satisfying the constraint w_t + w_v = 1; Preferably, the cross-modal text encoder is a CLIP text encoder, and the weight coefficients w_t = 0.5 and w_v = 0.5 are set.

6. The method for automatically generating panoramic interactive content by fusing multimodal large models according to claim 1, characterized in that: The control mechanism for generating the large-scale AI panoramic image in step S2 specifically includes: When scene number i>1, the system automatically extracts the panoramic image of the previously confirmed scene SceneNode(i−1) as a structure and color reference image; the image features of the reference image are used as a conditional control vector and injected into the latent space of the diffusion generation model together with the prompt word Prompt(i) of the current scene, so as to maintain the visual continuity between the two scenes by controlling the redraw intensity parameter. When scene number i=1, the first panoramic image is generated by directly calling the multimodal large model cold start based on the prompt word Prompt(1).

7. The method for automatically generating panoramic interactive content by fusing multimodal large models according to claim 1, characterized in that: In step S3, the industry hotspot probability mask matrix M_attr is defined as follows: M_attr = {m1,m2,...mn}, where each mi represents the generation weight of the corresponding hotspot type, and n is the total number of preset interactive hotspot types. The system automatically retains hotspot types with a weight value greater than 0.5 to determine the final list of interactive hotspots; The method for generating trending content includes: Targeting common Q&A topics: A multimodal large model is invoked to automatically generate structured QA pairs based on scenario descriptions; For the hot topic of voice or digital human narration: call a multimodal large model to generate narration text, synthesize audio stream through speech algorithm and synchronously input the audio stream into the digital human driving model based on waveform attention, and synthesize digital human narration video file with transparent channel in real time; For trending images or videos: Extract content descriptions or script descriptions and submit them to a multimodal large model for automatic generation of corresponding images or videos.

8. The method for automatically generating panoramic interactive content by fusing multimodal large models according to claim 1, characterized in that: The three-dimensional spatial coordinate system in step S4 is a spherical coordinate system. The spatial coordinates of the hotspot in the spherical space of the scene are represented as Pi(Hi,Vi), where: Hi is the yaw angle; Vi is the pitch angle; the hotspot spatial layout method includes the following steps: For hot topics related to scene transitions, Q&A, and digital humans, corresponding predefined spatial feature coordinates are directly configured on the sphere. For the remaining hotspots, the initial coordinates are set to the area in front of the user's default visual field, and the spherical angular distance d between them and the hotspots with fixed coordinates is calculated using the spherical distance formula: , The minimum angular distance threshold for anti-occlusion is set to θ. If the calculated spherical angular distance satisfies d < θ, then an occlusion collision is determined. The system performs a spiral step optimization traversal on the sphere with a preset yaw angle step size ΔH and pitch angle step size ΔV until a set of blank coordinate regions satisfying d(Pi, Pj) ≥ θ is found, which is taken as the final spatial coordinates of the hot spot. Preferably, the minimum angular distance threshold θ = 15°, the yaw angle step size ΔH = 5°, and the pitch angle step size ΔV = 5°.

9. A panoramic interactive content automatic generation system integrating multimodal large models, characterized in that, include: Input module: Used to receive unstructured multimodal demand information, call the multimodal large model to complete content theme recognition and user demand structure reconstruction, build scene time-series linked list and output panoramic interactive language script; Panoramic Scene Retrieval and Generation Module: Used to build a panoramic material vector database and perform cross-modal retrieval matching when generating the current scene. Based on the calculated maximum comprehensive matching score, it controls the reuse of existing materials or triggers an adaptive guided diffusion model based on reference map conditions to automatically generate panoramic images. Automatic Hotspot Planning Module: This module is used to filter and generate an interactive hotspot list based on the hotspot probability mask matrix corresponding to the industry theme code, and call multimodal large model and multimedia generation algorithm to synthesize hotspot media files online in real time. Hotspot spatial layout module: used to establish layout optimization models for different types of hotspots in three-dimensional space. For hotspots without predefined coordinates, an iterative collision detection algorithm based on spherical distance constraints is used to complete the unobstructed coordinate optimization layout and optimization traversal. Automatic Project Packaging Module: This module packages panoramic image project resources into a unified package, restructures and extracts them, and stores them in the project's structured project file, outputting a standardized panoramic interactive project package.

Images