A virtual-real fusion digital experience hall construction method

Abstract

This invention discloses a method for constructing a virtual-real integrated digital experience hall, relating to the field of data processing technology. The method includes: constructing a network layer to generate an exhibit resource pool by collecting open-source exhibits without boundaries through an open internet window; constructing a data layer to establish exhibit big data by structuring the exhibit resource pool; constructing a visual layer to transform two-dimensional images into virtual 3D dynamic display content; constructing a scene layer to generate immersive experience content by loading virtual scene environment information; and constructing a configuration layer to generate a virtual-real integrated, borderless digital experience hall by dynamically classifying and grouping the immersive experience content. This method solves the technical problems of limited resource sources, monotonous display formats, lack of immersion and interactivity, and difficulty in dynamic management and borderless access in existing technologies, achieving the technical effects of unlimited exhibit resources, intelligent data processing, immersive display formats, and personalized experience scenarios.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method for constructing a digital experience hall that integrates virtual and real elements. Background Technology

[0002] Currently, various digital experience halls, particularly those focused on technology, follow the structure of physical exhibitions. Limited by factors such as physical space, exhibit protection, cost, and accessibility, they lack a digital experience architecture that showcases the advantages of virtual experiences. This makes it difficult to meet the growing demand for personalized, interactive, and borderless experiences, resulting in limited experiential effects. Firstly, the number of exhibits is limited; digital exhibits follow the same collection model as physical exhibits, limiting the scope of exhibit submissions and invitations. Secondly, there is a lack of scenarios; digital exhibits follow the simple object display model of physical exhibits. Creating real-world scenarios is costly and can only be statically loaded, while physical exhibits generally do not load scenarios, making it difficult to stimulate creativity. Thirdly, exhibition booths are fixed; digital experience halls follow the fixed booth model of traditional science and technology museums, placing digital exhibits like physical exhibits in their booths, which cannot be moved or reused in other exhibition areas, lacking flexibility. Fourthly, exhibition categories are singular and rigid; digital experience halls follow the single-category physical exhibition model of traditional science and technology museums, such as chronological classification of industrial technology exhibits, which cannot adapt to the needs of different visitor groups. Creating new configurations would require disrupting and overturning the existing exhibition layout and rebuilding the exhibition project, resulting in high costs.

[0003] At present, the relevant technologies suffer from technical problems such as limited resource sources, limited display formats, lack of immersive and interactive experience, and difficulty in dynamic management and unrestricted access. Summary of the Invention

[0004] This application provides a method for constructing a digital experience hall that integrates virtual and real elements, which solves the technical problems of limited resource sources, single display forms, lack of immersion and interactivity, difficulty in dynamic management and boundless access in the existing technology, and achieves the technical effects of unlimited exhibit resources, intelligent data processing, immersive display forms and personalized experience scenarios.

[0005] This application provides a method for constructing a virtual-real integrated digital experience hall. The method includes: constructing a network layer, which generates an exhibit resource pool by collecting open-source exhibits through an open window on the Internet; constructing a data layer, which establishes exhibit big data by structuring the exhibit resource pool; constructing a visual layer, which converts two-dimensional images into virtual 3D dynamic display content according to the exhibit big data; constructing a scene layer, which generates immersive experience content by loading virtual scene environment information onto the virtual 3D dynamic display content; and constructing a configuration layer, which generates a virtual-real integrated borderless digital experience hall by dynamically classifying and grouping the immersive experience content.

[0006] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further performs the following processing: 1) Configuring a distributed web crawler cluster through an open internet window to automatically crawl first open-source digital exhibit data; 2) Constructing a crowdsourcing submission portal to review actively submitted data packets, and using approved submission data packets as second open-source digital exhibit data; 3) Collaboratively aggregating the first and second open-source digital exhibit data through federated learning to construct a global recommendation model; 4) Performing data mining and filtering through the global recommendation model to construct exhibit metadata, and uploading the exhibit metadata to a central server for data normalization processing to generate the exhibit resource pool.

[0007] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further performs the following processes: initializing a federated learning system, configuring multiple data provider nodes, including a first data provider node and a second data provider node, wherein the first data provider node stores the first open-source digital exhibit data, and the second data provider node stores the second open-source digital exhibit data; distributing initialization training parameters to the first data provider node and the second data provider node for feature analysis to obtain multiple feature vectors; performing backpropagation according to the multiple feature vectors, recording model training update values, encrypting the model training update values ​​to obtain encrypted data packets; using a secure aggregation algorithm to perform aggregation analysis on the encrypted data packets, decrypting the aggregation results, and constructing the global recommendation model.

[0008] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further performs the following processes: cleaning multi-source heterogeneous data from the exhibit resource pool to generate standardized exhibit core metadata; automatically extracting entities based on the standardized exhibit core metadata to obtain multiple entity data; performing relationship mining based on the multiple entity data to determine data relationships and semantically associating the multiple entity data to construct an exhibit knowledge graph; and associating and mapping the graph nodes of the exhibit knowledge graph with the standardized exhibit core metadata to construct the exhibit big data.

[0009] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further performs the following processing: traversing the exhibit knowledge graph to extract the multiple entity data, the multiple entity data including two-dimensional image resource data; reconstructing the two-dimensional image resource data in 3D according to the data relationship to obtain the 3D geometric structure parameters of the exhibits; enhancing the details of the 3D geometric structure parameters of the exhibits to construct a 3D rendering task; deploying edge computing nodes, the edge computing nodes being located between the data layer and the visual layer; and assigning the 3D rendering task to the edge computing nodes for rendering to construct virtual 3D dynamic display content.

[0010] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further performs the following processing: sending a rendering request to the edge computing node; upon receiving the rendering request, the edge computing node monitors its load parameters in real time; automatically loading the 3D rendering task; when any load parameter within the edge computing node exceeds a preset load threshold, splitting the 3D rendering task, arranging multiple 3D rendering subtasks in ascending order according to their rendering impact, and migrating the highest-ranking 3D rendering subtasks to nearby idle edge nodes; retrieving user viewpoint information, dynamically adjusting the rendering precision of the remaining 3D rendering subtasks according to the user viewpoint information to obtain a real-time rendered image from the user viewpoint; and transmitting the real-time rendered image from the user viewpoint back to the user terminal to construct the virtual 3D dynamic display content.

[0011] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further performs the following processes: generating AI scenes based on the virtual 3D dynamic display content to construct background scene animations; parsing the background scene animations using natural language to generate scene description text for environmental analysis, extracting key environmental elements to construct virtual scene environment information; tracking users in real time using computer vision to obtain multimodal user data, performing interaction analysis based on the multimodal user data to obtain a user behavior time-series dataset; performing user preference analysis based on the user behavior time-series dataset to construct a user interest profile; using the user interest profile as an index to retrieve the exhibit knowledge graph and generate exhibit configuration recommendation information; dynamically adjusting the virtual scene environment information according to the user behavior time-series dataset and the exhibit configuration recommendation information to obtain target virtual scene environment information; and fusing and rendering the virtual 3D dynamic display content with the target virtual scene environment information to generate the immersive experience content.

[0012] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further performs the following processing: mapping and analyzing the user behavior time-series dataset with the virtual scene environment information to construct behavior-scene mapping rules; matching and analyzing the exhibit configuration recommendation information with the virtual scene environment information to construct recommendation-scene adaptation rules; performing multi-objective optimization on the user behavior time-series dataset and the exhibit configuration recommendation information according to the behavior-scene mapping rules and the recommendation-scene adaptation rules to generate multi-dimensional scene parameter adjustment instructions; and performing conflict optimization on the virtual scene environment information through the multi-dimensional scene parameter adjustment instructions to obtain the target virtual scene environment information.

[0013] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further includes the following processes: dynamically classifying the immersive experience content into multiple dimensions to determine multi-configuration data; introducing an emotion computing module to capture users in real time and obtain multiple emotion indicators; continuously analyzing and comparing users based on the multiple emotion indicators to obtain continuous emotional state data of users; performing exhibit interaction analysis based on the continuous emotional state data of users combined with the user behavior time-series dataset to determine exhibit recommendation strategies; executing the exhibit recommendation strategies according to the multi-configuration data and performing experience analysis to generate experience quality parameters for testing and evaluation; defining multiple exhibition configuration modes based on the experience evaluation results to construct the virtual-real integrated borderless digital experience hall.

[0014] In a possible implementation, the method for constructing a virtual-real integrated digital experience hall further performs the following processing: performing feature analysis based on the user's continuous emotional state data to construct an emotional state temporal feature vector; performing feature analysis based on the user's behavioral temporal dataset to construct a behavioral temporal feature vector; employing an attention mechanism to assign weights to the emotional state temporal feature vector and the behavioral temporal feature vector, determining multiple feature weight coefficients; fusing and analyzing the user's continuous emotional state data and the user's behavioral temporal dataset according to the multiple feature weight coefficients to construct a joint representation vector; performing preference identification analysis on the user based on the joint representation vector to determine the user's interest preference vector and calculate the exhibit content matching degree; performing emotional need analysis on the user based on the joint representation vector to determine the user's emotional need vector and calculate the exhibit emotional suitability degree; balancing and optimizing the exhibit content matching degree and exhibit emotional suitability degree to construct a recommendation sequence for recommendation feedback and generate a recommendation score; and dynamically adjusting the recommendation sequence according to the recommendation score to construct the exhibit recommendation strategy.

[0015] This application proposes a method for constructing a virtual-real integrated digital experience hall. The method comprises: a network layer, which uses the internet to collect open-source exhibits without boundaries to generate an exhibit resource pool; a data layer, which structures the exhibit resource pool to create exhibit big data; a visual layer, which transforms 2D images into virtual 3D dynamic display content; a scene layer, which loads virtual scene environment information to generate immersive experience content; and a configuration layer, which dynamically classifies and groups the immersive experience content to generate a virtual-real integrated, borderless digital experience hall. This method solves the technical problems of limited resource sources, monotonous display formats, lack of immersion and interactivity, and difficulty in dynamic management and borderless access in existing technologies. It achieves the technical effects of unlimited exhibit resources, intelligent data processing, immersive display formats, and personalized experience scenarios. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. Flowcharts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 This is a flowchart illustrating a method for constructing a digital experience center that integrates virtual and real elements, as provided in an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating the process of establishing exhibit big data in a method for constructing a virtual-real integrated digital experience hall, as provided in an embodiment of this application. Detailed Implementation

[0019] The above description is merely an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below.

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description of this application will be provided in conjunction with the accompanying drawings. The described embodiments should not be considered as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the following description, references to "some embodiments" describe a subset of all possible embodiments. However, it is understood that "some embodiments" can be the same or different subsets of all possible embodiments and can be combined with each other without conflict. The terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, product, or server that includes a series of steps is not necessarily limited to those steps explicitly listed, but may include other steps not explicitly listed or inherent to such processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only.

[0022] This application provides a method for constructing a digital experience center that integrates virtual and real elements, such as... Figure 1 As shown, the method includes: Step S100: Construct a network layer, which is obtained by collecting open-source exhibits through the Internet Open Window to generate an exhibit resource pool.

[0023] Preferably, a network layer responsible for data collection and aggregation is constructed in the digital experience hall. Open-source exhibits are collected without boundaries through the Internet open window to generate an exhibit resource pool. Here, the Internet open window refers to standardized data access interfaces and collection tools that are open to external data sources, such as API interfaces, web crawlers / spiders, etc. Digital resources whose copyright status allows free access, use and modification are collected as target data, which may include works in the public domain, open digital collections of museums / institutions and user-generated content. Finally, a massive collection of digital exhibit data is obtained to generate an exhibit resource pool.

[0024] Furthermore, step S100 also includes step S110, configuring a distributed web crawler cluster through an internet open window to automatically crawl the first open-source digital exhibit data; step S120, constructing a crowdsourcing submission portal to review the actively submitted submission data packets, and using the approved submission data packets as the second open-source digital exhibit data; step S130, collaboratively aggregating the first open-source digital exhibit data and the second open-source digital exhibit data through federated learning to construct a global solicitation model; step S140, performing data mining and filtering through the global solicitation model to construct exhibit metadata, uploading the exhibit metadata to the central server for data normalization processing, and generating the exhibit resource pool.

[0025] Preferably, a distributed web crawler cluster is configured through an internet open window to automatically traverse and crawl open-source digital content such as exhibit information, images, and videos from the internet according to preset rules such as website list, crawl depth, and file type, to determine the first open-source digital exhibit data; an API interface is built as a crowdsourcing submission entry point to review the submission data packages of digital exhibit files actively submitted by users, including reviewing file type, virus scanning, content quality, and copyright compliance, and then using the submitted data packages that pass the review as the second open-source digital exhibit data.

[0026] Preferably, federated learning is used to collaboratively aggregate the first and second open-source digital exhibit data. Specifically, the first and second open-source digital exhibit data may be stored in different locations or managed by different entities. Federated learning does not require centralizing the data to a central server. Instead, the machine learning model parameters are sent to the local nodes storing the first and second open-source digital exhibit data respectively. The local nodes use their own data to calculate model updates, and then send the encrypted updates to the central server for aggregation, generating a global recommendation model that can identify the features of all data sources for data filtering and recommendation.

[0027] Preferably, a global recommendation model is used to perform data mining and screening on the first and second open-source digital exhibit data. This can identify high-quality, highly relevant, and valuable exhibits and automatically filter out low-quality, irrelevant, or duplicate content. Then, key descriptive information is extracted from the selected high-quality exhibits to construct exhibit metadata. The exhibit metadata is then uploaded to the central server for data normalization processing, which converts exhibit metadata extracted from different sources into a standardized format. Finally, a high-quality, standardized, easily searchable, and manageable structured data set is obtained, generating an exhibit resource pool.

[0028] Furthermore, step S130 also includes step S131, initializing the federated learning system and configuring multiple data provider nodes, including a first data provider node and a second data provider node, wherein the first data provider node stores the first open-source digital exhibit data and the second data provider node stores the second open-source digital exhibit data; step S132, distributing initialization training parameters to the first data provider node and the second data provider node for feature analysis to obtain multiple feature vectors; step S133, performing backpropagation according to the multiple feature vectors, recording the model training update values, encrypting the model training update values ​​to obtain encrypted data packets; step S134, using a secure aggregation algorithm to perform aggregation analysis on the encrypted data packets, decrypting the aggregation results, and constructing the global recommendation model.

[0029] Preferably, the federated learning system is initialized and configured with multiple data provider nodes. The data provider nodes are the participating parties in the collaboration. Each node stores its own private dataset locally, including a first data provider node and a second data provider node. The first data provider node stores the first open-source digital exhibit data, and the second data provider node stores the second open-source digital exhibit data. The central server generates an initial model based on a neural network and sends the initial training parameters, such as the weights and parameters of the initial model, to the first and second data provider nodes respectively. Each node uses its own private data to perform feature analysis on the initial training parameters locally, including forward propagation and loss calculation, thereby obtaining multiple feature vectors, i.e., how to adjust the initial training parameters to better fit its local data.

[0030] Preferably, backpropagation is performed using multiple feature vectors to calculate and determine the update values ​​required for initializing the training parameters at each layer, specifying the magnitude and direction of the model parameter adjustments. Each node records the calculated gradient update values ​​and encrypts them using homomorphic encryption or differential privacy to form encrypted data packets. These encrypted data packets are then transmitted to a central server, where a secure aggregation algorithm is used to perform aggregation analysis. This involves calculating a weighted average while the data is encrypted to obtain the aggregation result, which is then decrypted to obtain the aggregated update values. The central server uses these aggregated update values ​​to update the initial model, and this process is repeated multiple times to determine a high-quality global recommendation model.

[0031] Step S200: Construct a data layer. The data layer establishes exhibit big data by performing structured processing on the exhibit resource pool.

[0032] Furthermore, such as Figure 2 As shown, step S200 further includes step S210, cleaning multi-source heterogeneous data in the exhibit resource pool to generate standardized exhibit core metadata; step S220, automatically extracting entities based on the standardized exhibit core metadata to obtain multiple entity data; step S230, performing relationship mining based on the multiple entity data to determine data relationships and semantically associating the multiple entity data to construct an exhibit knowledge graph; and step S240, associating and mapping the graph nodes of the exhibit knowledge graph with the standardized exhibit core metadata to construct the exhibit big data.

[0033] Preferably, a data layer is constructed to perform structured processing on the exhibit resource pool and generate exhibit big data through knowledge graph association. Specifically, the exhibit resource pool is cleaned from multiple heterogeneous sources, including deduplication, error correction, filling missing values, and format standardization, to generate standardized exhibit core metadata, where each row represents an exhibit and each column represents a core attribute. Then, entities are automatically extracted based on the standardized exhibit core metadata, that is, named entities are identified and extracted from the standardized exhibit core metadata using natural language processing, and classified into multiple entity data.

[0034] Preferably, a rule engine or relation extraction model is used to mine relationships among multiple entity data, determine data relationships, and semantically associate multiple entity data. That is, data relationships are stored in the form of subject-verb-object triples to generate an interconnected semantic network, thereby obtaining an exhibit knowledge graph, where nodes are entity data and edges are data relationships. Finally, the graph nodes of the exhibit knowledge graph are associated and mapped with standardized exhibit core metadata. That is, for each entity node of the exhibit knowledge graph, a pointer is established to point to the detailed data of the standardized exhibit core metadata, obtaining an exhibit data fusion body. Finally, exhibit big data, including attribute data and relation data, is determined, which can be directly used for advanced intelligent analysis.

[0035] Step S300: Construct a visual layer, which is obtained by converting two-dimensional images into virtual 3D dynamic display content according to the exhibit big data.

[0036] Step S300 further includes step S310, traversing the exhibit knowledge graph to extract the multiple entity data, the multiple entity data including two-dimensional image resource data; step S320, reconstructing the two-dimensional image resource data in 3D according to the data relationship to obtain the exhibit's 3D geometric structure parameters; step S330, enhancing the details of the exhibit's 3D geometric structure parameters to construct a 3D rendering task; step S340, deploying edge computing nodes, the edge computing nodes being located between the data layer and the visual layer; step S350, assigning the 3D rendering task to the edge computing nodes for rendering to construct virtual 3D dynamic display content.

[0037] Preferably, a key transformation layer is constructed to convert two-dimensional images into 3D dynamic display content that can be displayed in virtual space and dynamically interact with users using exhibit big data. As a visual layer, specifically, the exhibit knowledge graph is traversed according to the semantic guidance of the knowledge graph to identify and extract multiple entity data containing two-dimensional image resource data, and all image resources are intelligently aggregated for a single exhibit using the correlation of the exhibit knowledge graph. Then, the two-dimensional image resource data is reconstructed in 3D according to the data relationship, including using multi-view stereo vision analysis to analyze the parallax between multiple images of entities in the exhibit knowledge graph from different angles, calculating and determining the depth information of each pixel in the image, and then outputting a three-dimensional mesh containing parameters such as vertex coordinates, normal vectors, and patch indices, thereby obtaining the 3D geometric structure parameters of the exhibit and accurately describing the three-dimensional shape and contour of the exhibit.

[0038] Preferably, the 3D geometric parameters of the exhibits are enhanced with texture mapping. This involves precisely wrapping the original image as a texture map around the 3D geometric mesh and assigning it realistic colors and patterns. Then, corresponding material properties are set based on metadata to ensure the correct visual effect under lighting. Finally, the geometric mesh, texture map, material properties, and related data are packaged into a complete rendering task description package. Edge computing nodes, equipped with GPUs and other graphics rendering engines, are deployed between the data layer and the visual layer. These nodes serve as an intermediate computing layer for graphics rendering calculations, solving the problems of high latency in cloud rendering and insufficient computing power on user terminal devices, thereby providing users with a real-time and smooth interactive experience. 3D rendering tasks are distributed to edge computing nodes for rendering. The graphics rendering engines of the edge computing nodes receive tasks and perform real-time rendering, calculating each frame and outputting an interactive, dynamic graphics stream as virtual 3D dynamic display content. Each user interaction triggers a real-time re-rendering, generating new images and streaming them to the user's terminal device over the network.

[0039] Furthermore, step S350 also includes step S351, sending a rendering request to the edge computing node; when the edge computing node receives the rendering request, it monitors the load parameters of the edge computing node in real time; step S352, automatically loading the 3D rendering task; when any load parameter in the edge computing node exceeds a preset load threshold, the 3D rendering task is split, and multiple 3D rendering subtasks are arranged in ascending order according to their rendering impact, and the 3D rendering subtasks with higher priority are migrated to nearby idle edge nodes; step S353, retrieving user viewpoint information, and dynamically adjusting the rendering accuracy of the remaining 3D rendering subtasks according to the user viewpoint information to obtain a real-time rendered image from the user viewpoint; step S354, transmitting the real-time rendered image from the user viewpoint back to the user terminal to construct the virtual 3D dynamic display content.

[0040] Preferably, the central scheduler or user terminal sends a rendering request to the edge computing node, requesting the rendering of a certain 3D model. After receiving the rendering request, the edge computing node monitors the load parameters of the edge computing node in real time, including GPU utilization, CPU utilization, memory usage, and network bandwidth. It automatically loads the 3D rendering task and starts rendering. If any load parameter in the edge computing node is higher than the preset load threshold, it means that the node is about to be overloaded and cannot guarantee the rendering frame rate. Then, the 3D rendering task is divided according to the scene area, and the complete rendering task is broken down into multiple smaller, independently processable 3D rendering subtasks. The contribution or computational complexity of each 3D rendering subtask to the final image quality is evaluated and sorted from low to high. That is, the 3D rendering subtasks with the lowest impact are placed at the top. The 3D rendering subtasks at the top of the list with the least impact on the current visual experience are migrated to other nearby idle edge nodes with lower loads for execution via the network.

[0041] Preferably, the user's viewpoint information is retrieved through the terminal device's sensors, specifically the central area of ​​the image currently being viewed by the user. The remaining 3D rendering subtasks are then dynamically adjusted in terms of rendering precision based on this viewpoint information. Specifically, the remaining 3D rendering subtasks processed at this node are rendered at full resolution and high precision, while the rendering precision is significantly reduced for the edges of the user's field of view, such as by lowering resolution and reducing lighting effects. Finally, the high-precision rendered central area and the low-precision rendered edge areas are combined to create the final real-time rendered image from the user's viewpoint. This significantly reduces the GPU's computational load without the user perceiving any loss in image quality, thereby significantly improving the rendering frame rate and ensuring real-time performance. Finally, the real-time rendered image from the user's viewpoint is encoded with low latency via a high-speed network and streamed back to the user's terminal device, such as a computer, mobile phone, or VR headset, to obtain the final virtual 3D dynamic display content.

[0042] Step S400: Construct a scene layer, which is obtained by loading virtual scene environment information onto the virtual 3D dynamic display content to generate immersive experience content.

[0043] Step S400 further includes step S410, generating an AI scene based on the virtual 3D dynamic display content to construct a background scene animation; step S420, performing natural language parsing on the background scene animation to generate scene description text for environmental analysis, extracting key environmental elements to construct virtual scene environment information; step S430, tracking users in real time using computer vision to obtain multimodal user data, performing interaction analysis based on the multimodal user data to obtain a user behavior time-series dataset; step S440, performing user preference analysis based on the user behavior time-series dataset to construct a user interest profile; step S450, using the user interest profile as an index to retrieve the exhibit knowledge graph and generate exhibit configuration recommendation information; step S460, dynamically adjusting the virtual scene environment information according to the user behavior time-series dataset and the exhibit configuration recommendation information to obtain target virtual scene environment information; and step S470, fusing and rendering the virtual 3D dynamic display content with the target virtual scene environment information to generate the immersive experience content.

[0044] Preferably, a scene layer is constructed to load virtual scene environment information and interaction logic onto the virtual 3D dynamic display content, which is used to generate immersive experience content. Specifically, the AI ​​model is driven by the virtual 3D dynamic display content to generate AI scene, i.e., to create background scene animation; then, video description generation or image recognition is used to perform natural language parsing on the background scene animation to generate scene description text; then, natural language processing is used to perform environmental analysis on the scene description text to extract key environmental elements and formally construct virtual scene environment information, which may include spatial environment, atmosphere information, narrative information, and interaction rules.

[0045] Preferably, user data is obtained through real-time tracking using computer vision devices such as cameras and VR headset sensors. This data includes the user's gaze focus, body position, gestures, and movement path, generating multimodal user data. This multimodal user data is then subjected to interactive analysis. For example, it can identify when a user is gazing at exhibit A, when a user lingers in front of exhibit B for 10 seconds, or when a user makes a zoom-in gesture. This data is then arranged chronologically to form a user behavior time-series dataset, revealing the continuous interaction process between the user and the virtual world. Next, user preference analysis is performed based on the user behavior time-series dataset to infer stable user interest patterns, which are then quantified into labels or vectors to construct a user interest profile. Finally, the user interest profile is used as a search query condition for semantic retrieval in the exhibit knowledge graph. The search results are then transformed into exhibit configuration recommendation information to determine the content to be displayed to the user, thereby improving the user experience.

[0046] Preferably, the virtual scene environment information is dynamically adjusted based on the user behavior time-series dataset and exhibit configuration recommendation information. This involves dynamically optimizing the current scene information by integrating real-time user behavior and system recommendations, thereby generating multiple adjustment commands to further optimize the virtual scene environment information. Examples include changing lighting focus, loading new exhibit models, and triggering relevant explanatory text, resulting in the optimized target virtual scene environment information. Finally, the virtual 3D dynamic display content is fused and rendered with the target virtual scene environment information. This involves combining the exhibit models included in the virtual 3D dynamic display content with the lighting, background, and sound elements included in the target virtual scene environment information, calculating the relationships between light and shadow, materials, sound, and space, and rendering immersive experience content—a real-time responsive audiovisual output stream tailored to the user.

[0047] Furthermore, step S460 also includes step S461, mapping and analyzing the user behavior time-series dataset with the virtual scene environment information to construct behavior-scene mapping rules; step S462, matching and analyzing the exhibit configuration recommendation information with the virtual scene environment information to construct recommendation-scene adaptation rules; step S463, performing multi-objective optimization on the user behavior time-series dataset and the exhibit configuration recommendation information according to the behavior-scene mapping rules and the recommendation-scene adaptation rules to generate multi-dimensional scene parameter adjustment instructions; step S464, performing conflict optimization on the virtual scene environment information through the multi-dimensional scene parameter adjustment instructions to obtain the target virtual scene environment information.

[0048] Preferably, the user behavior time-series dataset is mapped and analyzed with virtual scene environment information to determine the correspondence between user behavior and scene environment adjustment information, and then behavior-scene mapping rules are constructed. These rules may include the correspondence between user gaze focus and scene lighting focus, with longer gaze time corresponding to higher lighting intensity in the area; matching rules between user movement speed and scene change rate, with faster movement speed leading to faster scene switching; and association rules between user interaction frequency and scene detail level, with frequently interacting areas automatically improving rendering accuracy.

[0049] Preferably, the exhibit configuration recommendation information is matched and analyzed with the virtual scene environment information. That is, the compatibility between the recommended exhibit attributes and the current virtual scene environment information is analyzed, and then recommendation-scene adaptation rules are constructed to present the recommended content appropriately. This may include automatically matching the corresponding historical background scene according to the era characteristics of the recommended exhibits; adjusting the color tone and decoration style of the scene based on the cultural attributes of the recommended exhibits; and optimizing the lighting reflection parameters and material performance effects of the scene according to the material characteristics of the recommended exhibits.

[0050] Preferably, a user's real-time behavior may trigger multiple rules simultaneously, potentially conflicting with rules triggered by the recommendation system. Therefore, based on particle swarm optimization (PSO) and following behavior-scene mapping rules and recommendation-scene adaptation rules, multi-objective optimization is performed on the user behavior time-series dataset and exhibit configuration recommendation information. All rules are considered as objectives that must be satisfied simultaneously, and the optimal solution that satisfies as many rules as possible is calculated and generated as a multi-dimensional scene parameter adjustment instruction. Finally, conflict optimization is performed, that is, the virtual scene environment information is updated and corrected through the multi-dimensional scene parameter adjustment instructions to obtain the target virtual scene environment information, incorporating the best response to user behavior and system recommendations, thereby creating a truly coherent, comfortable, and personalized immersive experience.

[0051] Step S500: Construct a configuration layer, which generates a boundless digital experience hall that blends the virtual and real worlds by dynamically classifying and grouping the immersive experience content.

[0052] Step S500 further includes step S510, dynamically classifying the immersive experience content in multiple dimensions to determine multi-configuration data; step S520, introducing an emotion computing module to capture users in real time and obtain multiple emotion indicators; step S530, continuously analyzing and comparing users based on the multiple emotion indicators to obtain continuous emotional state data of users; step S540, performing exhibit interaction analysis based on the continuous emotional state data of users combined with the user behavior time series dataset to determine exhibit recommendation strategies; step S550, executing the exhibit recommendation strategy according to the multi-configuration data and performing experience analysis, generating experience quality parameters for testing and evaluation, defining multiple exhibition configuration modes based on the experience evaluation results, and constructing the virtual-real integrated borderless digital experience hall.

[0053] Preferably, a configuration layer is constructed to dynamically classify and group immersive experience content, generating a vast, organic, adaptive, and unbounded ultimate digital experience hall. Specifically, the immersive experience content is dynamically classified in multiple dimensions, i.e., real-time analysis of the immersive experience content automatically divides it into different categories, such as by content dimension or by experience dimension, thereby determining multi-configuration data, including multiple possible exhibition configuration schemes. Then, an emotion computing module is introduced to capture users in real time, i.e., through facial expression recognition, voice emotion analysis, physiological signal monitoring, interactive behavior analysis, etc., to capture users' emotional indicators in real time, such as excitement, confusion, and boredom. Then, users are continuously analyzed and compared based on multiple emotion indicators, such as plotting multiple emotion indicators as curves changing over time, and obtaining continuous emotional state data of users by analyzing the curves, thereby judging the user's overall experience trajectory.

[0054] Preferably, interactive analysis of exhibits is conducted by combining continuous user emotional state data with time-series datasets of user behavior. For example, the analysis found that users' excitement and pleasure levels significantly increased whenever interactive puzzle content appeared. Based on the results of the interactive analysis, a more advanced recommendation strategy was developed as an exhibit recommendation strategy, achieving a leap from content recommendation to experience mode recommendation. Next, an exhibit recommendation strategy is executed based on multi-configuration data, and experience analysis is conducted. Specifically, based on multi-configuration data, several new exhibition routes are actually combined according to the new exhibit recommendation strategy and launched to users for experience analysis. Then, comprehensive data of the new exhibition routes is collected to generate quantitative experience quality parameters, such as average dwell time, average score of user sentiment curve, tour completion rate, and sharing rate. The experience quality parameters are then tested and evaluated, that is, the experience quality parameters of different configuration routes are compared and their advantages and disadvantages are evaluated to obtain experience evaluation results. Finally, based on the experience evaluation results, various exhibition configuration modes are defined, such as efficient guided tour mode, deep immersion mode, and social interaction mode. Ultimately, a seamless digital experience hall integrating virtual and real elements is constructed, which can provide different high-quality experiences for different users at different times, thereby realizing unlimited exhibit resources, intelligent data processing, immersive display forms, and personalized experience scenarios.

[0055] Furthermore, step S540 also includes step S541, performing feature analysis based on the user's continuous emotional state data to construct an emotional state temporal feature vector; step S542, performing feature analysis based on the user's behavior temporal dataset to construct a behavior temporal feature vector; step S543, using an attention mechanism to assign weights to the emotional state temporal feature vector and the behavior temporal feature vector, determining multiple feature weight coefficients; and step S544, fusing and analyzing the user's continuous emotional state data and the user's behavior temporal dataset according to the multiple feature weight coefficients to construct... Step S545: Construct a joint representation vector; Step S546: Analyze user preferences based on the joint representation vector to determine user interest preference vectors and calculate exhibit content matching degree; Step S547: Analyze user emotional needs based on the joint representation vector to determine user emotional need vectors and calculate exhibit emotional suitability degree; Step S548: Balance and optimize the exhibit content matching degree and exhibit emotional suitability degree, construct a recommendation sequence for recommendation feedback, and generate a recommendation score; Step S549: Dynamically adjust the recommendation sequence according to the recommendation score to construct the exhibit recommendation strategy.

[0056] Preferably, time series analysis is used to perform feature analysis on continuous user emotional state data, constructing emotional state time series feature vectors from fluctuating emotional data over a period of time, numerically describing the user's emotional change patterns; similarly, time series feature analysis is performed on the user behavior time series dataset, transforming it into a fixed-length behavior time series feature vector, numerically describing the user's behavior patterns; an attention mechanism is used to analyze and assign weights to the emotional state time series feature vectors and behavior time series feature vectors, that is, to analyze whether the user's emotional signal or behavioral signal is more important at the current moment, and then output multiple feature weight coefficients, representing the proportion of different parts of the emotional and behavioral vectors; then, based on the multiple feature weight coefficients, the continuous user emotional state data and the user behavior time series dataset are fused and analyzed, that is, the emotional state time series feature vectors and behavior time series feature vectors are weighted and fused to generate a joint representation vector, representing the deeply integrated user state.

[0057] Preferably, based on a neural network, user preference identification and analysis are performed using joint representation vectors to predict users' long-term interests and preferences, which are then represented as user interest preference vectors. The cosine similarity between this vector and the content feature vector of each exhibit in the knowledge graph is then calculated to determine the exhibit content matching degree, representing the degree to which the exhibit matches the user's interest topics. Similarly, based on the joint representation vectors, emotional needs analysis is performed on users, analyzing their current instantaneous emotional needs and quantifying the user's emotional need vector. The similarity between this vector and the emotional experience feature vector that each exhibit can bring is also calculated to obtain the exhibit emotional fit degree, representing the degree to which the exhibit matches the user's current emotional needs. Next, a balance optimization is performed, i.e., a weighted average of the exhibit content matching degree and the exhibit emotional fit degree is applied to calculate and generate recommendation scores for candidate exhibits. All candidate exhibits are then scored and ranked to obtain a recommendation sequence. Finally, based on reinforcement learning, the recommendation sequence is dynamically adjusted in reverse according to the recommendation score. For example, if the recommended content receives positive user interaction, it means that the strategy of balancing content matching and emotional fit in this recommendation is successful. In turn, feedback signals are obtained and the recommendation sequence is dynamically adjusted. Through self-iteration, a dynamic exhibit recommendation strategy is generated, which greatly improves immersion and satisfaction.

[0058] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A method for constructing a digital experience hall that integrates virtual and real elements, characterized in that, The method includes: A network layer is constructed, which generates an exhibit resource pool by unboundedly soliciting open-source exhibits through the Internet's open window. A data layer is constructed, which establishes big data on exhibits by structuring the exhibit resource pool; A visual layer is constructed, which is obtained by converting two-dimensional images into virtual 3D dynamic display content according to the big data of the exhibits; A scene layer is constructed, which generates immersive experience content by loading virtual scene environment information onto the virtual 3D dynamic display content; A configuration layer is constructed, which dynamically classifies and groups the immersive experience content to generate a boundless digital experience hall that blends the virtual and real worlds.

2. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 1, characterized in that, The process by which the network layer generates an exhibit resource pool by unboundedly soliciting open-source exhibits through the Internet open window includes the following methods: Configure a distributed web crawler cluster through the Internet open window to automatically crawl the data of the first open source digital exhibits; A crowdsourcing submission portal is established to review the actively submitted data packages, and the approved data packages are used as the second source of digital exhibit data. The first open-source digital exhibit data and the second open-source digital exhibit data are collaboratively aggregated through federated learning to construct a global recommendation model; Data mining and filtering are performed using the global recommendation model to construct exhibit metadata. The exhibit metadata is then uploaded to the central server for data normalization processing to generate the exhibit resource pool.

3. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 2, characterized in that, The first open-source digital exhibit data and the second open-source digital exhibit data are collaboratively aggregated through federated learning to construct a global recommendation model. The method includes: Initialize the federated learning system and configure multiple data provider nodes, including a first data provider node and a second data provider node, wherein the first data provider node stores the first open-source digital exhibit data and the second data provider node stores the second open-source digital exhibit data; Initialization training parameters are distributed to the first data provider node and the second data provider node for feature analysis to obtain multiple feature vectors; Backpropagation is performed according to the multiple feature vectors to record the model training update values. The model training update values ​​are then encrypted to obtain an encrypted data packet. The encrypted data packets are aggregated and analyzed using a secure aggregation algorithm. The aggregation results are then decrypted to construct the global recommendation model.

4. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 1, characterized in that, The data layer establishes the process of acquiring big data about the exhibits by structuring the exhibit resource pool. The method includes: The exhibit resource pool is subjected to multi-source heterogeneous data cleaning to generate standardized core metadata of exhibits; Entities are automatically extracted based on the core metadata of the standardized exhibits to obtain multiple entity data. Based on the multiple entity data, relationship mining is performed to determine the data relationships and semantic associations are made with the multiple entity data to construct an exhibit knowledge graph. The exhibit big data is constructed by associating and mapping the graph nodes of the exhibit knowledge graph with the standardized exhibit core metadata.

5. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 4, characterized in that, The visual layer is obtained by converting two-dimensional images into virtual 3D dynamic display content according to the exhibit big data. The method includes: The knowledge graph of the exhibits is traversed to extract the multiple entity data, which includes two-dimensional image resource data. The two-dimensional image resource data is reconstructed into 3D according to the data relationship to obtain the 3D geometric structure parameters of the exhibits. The 3D geometric parameters of the exhibits are enhanced in detail to construct a 3D rendering task. Deploy edge computing nodes, which are located between the data layer and the visual layer; The 3D rendering task is assigned to the edge computing node for rendering to construct virtual 3D dynamic display content.

6. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 5, characterized in that, The method of assigning the 3D rendering task to the edge computing node for rendering and constructing virtual 3D dynamic display content includes: A rendering request is sent to the edge computing node. When the edge computing node receives the rendering request, its load parameters are monitored in real time. The 3D rendering task is automatically loaded. When any load parameter in the edge computing node is higher than the preset load threshold, the 3D rendering task is split, and multiple 3D rendering subtasks are arranged in ascending order according to their rendering impact. The 3D rendering subtasks with the highest order are migrated to nearby idle edge nodes. Retrieve user viewpoint information and dynamically adjust the rendering precision of the remaining 3D rendering subtasks according to the user viewpoint information to obtain a real-time rendered image from the user viewpoint. The user's viewpoint is rendered in real time and transmitted back to the user's terminal to construct the virtual 3D dynamic display content.

7. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 5, characterized in that, The process by which the scene layer generates immersive experience content by loading virtual scene environment information onto the virtual 3D dynamic display content includes: AI scene generation is performed based on the virtual 3D dynamic display content to construct background scene animation; Natural language parsing is performed on the background scene animation to generate scene description text for environmental analysis, and key environmental elements are extracted to construct virtual scene environment information; Users are tracked in real time using computer vision to obtain multimodal user data. Interaction analysis is then performed based on the multimodal user data to obtain a time-series dataset of user behavior. Based on the aforementioned user behavior time-series dataset, user preference analysis is performed to construct user interest profiles; The user interest profile is used as an index to retrieve the exhibit knowledge graph and generate exhibit configuration recommendation information. The virtual scene environment information is dynamically adjusted based on the user behavior time series dataset and the exhibit configuration recommendation information to obtain the target virtual scene environment information; The virtual 3D dynamic display content is fused and rendered with the target virtual scene environment information to generate the immersive experience content.

8. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 7, characterized in that, The method involves dynamically adjusting the virtual scene environment information based on the user behavior time-series dataset and the exhibit configuration recommendation information to obtain the target virtual scene environment information. The user behavior time series dataset is mapped and analyzed with the virtual scene environment information to construct behavior-scene mapping rules; The exhibit configuration recommendation information is matched and analyzed with the virtual scene environment information to construct recommendation-scene adaptation rules; The user behavior time series dataset and the exhibit configuration recommendation information are optimized in a multi-objective manner according to the behavior-scene mapping rules and the recommendation-scene adaptation rules to generate multi-dimensional scene parameter adjustment instructions. The virtual scene environment information is optimized by adjusting the multi-dimensional scene parameters to obtain the target virtual scene environment information.

9. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 7, characterized in that, The configuration layer generates a seamless digital experience hall that blends the virtual and real worlds by dynamically classifying and grouping the immersive experience content. The method includes: The immersive experience content is dynamically classified in multiple dimensions to determine multi-configuration data; An emotion computing module is introduced to capture user emotions in real time and obtain multiple emotion indicators; The user's continuous emotional state data is obtained by continuously analyzing and comparing the multiple emotion indicators. Based on the user's continuous emotional state data and the user's time-series behavior dataset, perform exhibit interaction analysis to determine exhibit recommendation strategies; The exhibit recommendation strategy is executed according to the multi-configuration data, and experience analysis is performed to generate experience quality parameters for testing and evaluation. Based on the experience evaluation results, multiple exhibition configuration modes are defined to construct the virtual-real integrated borderless digital experience hall.

10. The method for constructing a digital experience hall that integrates virtual and real elements as described in claim 9, characterized in that, Based on the user's continuous emotional state data and the user's time-series behavior dataset, exhibit interaction analysis is performed to determine exhibit recommendation strategies. The methods include: Based on the user's continuous emotional state data, feature analysis is performed to construct a temporal feature vector of emotional state. Based on the aforementioned user behavior time-series dataset, feature analysis is performed to construct a behavior time-series feature vector. An attention mechanism is used to assign weights to the temporal feature vector of the emotional state and the temporal feature vector of the behavior, and multiple feature weight coefficients are determined. The user's continuous emotional state data and the user's time-series behavior dataset are fused and analyzed according to the multiple feature weight coefficients to construct a joint representation vector; Based on the joint representation vector, user preference identification and analysis are performed to determine the user interest preference vector and calculate the content matching degree of the exhibits. Based on the joint representation vector, the user's emotional needs are analyzed, and the emotional fit of the exhibits is calculated by determining the user's emotional needs vector. The matching degree of the exhibit content and the emotional fit of the exhibit are balanced and optimized, a recommendation sequence is constructed for recommendation feedback, and a recommendation score is generated. The recommended sequence is dynamically adjusted in reverse according to the recommended score to construct the exhibit recommendation strategy.

Images