A Metaverse Application Construction Method

Abstract

The present invention discloses a method for constructing a metaverse application, including: uploading digital content and optimizing it; creating a new XR application and entering the XR application editor; creating a new subscene in the newly created XR application by inputting the subscene name and baseboard data; completing the layout of the scene digital content through the XR application editor; completing the arrangement of subscene interaction instructions through the XR application editor; drawing the program interface through the interface editor and adding interaction instructions to the interface elements; setting the basic information required for XR application construction and generating a data file of the XR application construction model; generating an XR application QR code based on the application ID, XR application type, target operating platform type and the data file of the XR application construction model; building a runtime engine and pre-installing it in a terminal device; scanning the XR application QR code through the runtime engine of the terminal device to start and use the XR application. The present invention can reduce the cost and cycle of constructing a metaverse application.

Description

Technical Field

[0001] This invention relates to the field of metaverse technology, and in particular to a method for constructing metaverse applications. Background Technology

[0002] Metaverse applications are developed using Extended Reality (XR) technology. They create and present digital content that blends with the real world, providing users with an immersive and interactive experience. Generally, building a metaverse application involves knowledge and skills from multiple fields, such as art design, 3D modeling, animation design, interaction design, and programming. Therefore, collaboration among professionals from various disciplines and fields is essential to completing a metaverse application. Furthermore, traditional metaverse application development requires a complete software development process, including requirements gathering, product design, software development, functional testing, and deployment—a lengthy and labor-intensive process. This illustrates the numerous challenges involved in launching a metaverse application, including high technical difficulty, challenges in building a talent pool, long development cycles, and high costs. Therefore, there is an urgent need to design a methodology for building metaverse applications. Summary of the Invention

[0003] In view of this, the purpose of this invention is to propose a method for constructing metaverse applications, reduce the cost and cycle of constructing metaverse applications, enrich the metaverse application ecosystem, and bring people a more diverse and exciting digital life experience.

[0004] To achieve the above-mentioned technical objectives, the technical solution adopted by this invention is as follows:

[0005] This invention provides a method for constructing metaverse applications, comprising the following steps:

[0006] Step 1: XR application creators upload different types of digital content to the XR application building platform. The digital content optimizer optimizes the different types of digital content and stores it in the digital content repository.

[0007] Step 2: Create a new XR application on the XR application building platform. After selecting the preset XR application type, enter the application name and launch cover information, and enter the XR application editor.

[0008] Step 3: Create a new sub-scene in the newly created XR application by entering the sub-scene name and base plate data;

[0009] Step 4: Complete the scene digital content layout using the XR application editor;

[0010] Step 5: Arrange the sub-scene interaction instructions using the XR application editor. In the 3D view or resource management tree, select the object that needs to be interacted with and add the interaction instructions in sequence. Repeat steps 3 to 5 as needed to complete the scene layout of the entire XR application.

[0011] Step 6: Use the interface editor to draw the program interface and add interactive commands to the interface elements;

[0012] Step 7: Set the basic information required for building the XR application, generate the data file of the XR application building model, and store it in the application building model repository;

[0013] Step 8: Generate an XR application QR code based on the application ID, XR application type, target operating platform type, and XR application building model data file;

[0014] Step 9: Build the runtime engine and pre-install it on the terminal device;

[0015] Step 10: Scan the XR application QR code with the terminal device's operating engine to launch and use the XR application.

[0016] Furthermore, in step 1, a digital content optimizer optimizes different types of digital content, including images, videos, audio, and 3D models, specifically including:

[0017] Step 11: For images, restrict the upload to only images of the specified format, extract image thumbnails using Thumbnailator, compress the images, and limit the image resolution to within the specified size.

[0018] Step 12: For videos, restrict uploads to only videos of a specified format. Extract the first frame image using ffmpeg and compress the image using thumbnailator to generate a video thumbnail.

[0019] Step 13: For audio, restrict the upload to only audio files of the specified format, and use the system's built-in default image as the audio image;

[0020] Step 14: For 3D models, restrict the upload to only 3D models in the specified format, and convert all uploaded 3D models to GLTF format.

[0021] Furthermore, after step 14, the method further includes: optimizing the 3D model for different operating platforms, with the specific steps as follows:

[0022] Step 141: For the browser runtime environment, compress the mesh surfaces of static 3D models using the Draco algorithm, compress the mesh surfaces and animations of animated 3D models using the Meshopt algorithm, and convert the textures to ktx2 format;

[0023] Step 142: For WeChat Mini Programs, compress the mesh surfaces of the 3D model using the Draco algorithm and convert the textures to WebP format;

[0024] Step 143: For terminal and glasses applications, compress the mesh surface of the 3D model using the Draco algorithm and convert the texture to ktx2 format;

[0025] Step 144: When the platform encounters problems loading the compressed 3D model, switch to loading the original GLTF format 3D model.

[0026] Furthermore, step 3 specifically includes:

[0027] Step 31: Create a sub-scene in the newly created XR application;

[0028] Step 32: Enter the name of the newly created sub-scene;

[0029] Step 33: Input the base plate data in the newly created sub-scene. The type of the base plate data is determined according to the XR application type. Each sub-scene contains one set of base plate data, which is loaded by the XR application editor.

[0030] The XR application types include: image AR, planar AR, face AR, human body AR, surround view AR, object AR, visual positioning AR, VR roaming, and VR model viewing;

[0031] If the selected XR application type is image AR, then the base plate data is a recognition image;

[0032] If the selected XR application type is planar AR, then the base plate data is a virtual plane;

[0033] If the selected XR application type is face AR, then the base data is a standard white film of a human head;

[0034] If the selected XR application type is human AR, then the substrate data is a standard human white film;

[0035] If the selected XR application type is surround view AR, then the base plate data is a skybox model and metadata information, which includes: skybox zoom size and height;

[0036] If the selected XR application type is physical AR, then the base plate data is object feature point cloud data, which is obtained by three-dimensional reconstruction of multiple consecutive photos of the object.

[0037] If the selected XR application type is visual positioning AR, then the base plate data consists of a spatial base plate white film, feature point cloud data, panoramic viewpoints, and metadata information; wherein, the metadata information includes: city coordinate offset, relative coordinate offset value, center point latitude and longitude, and global rotation amount;

[0038] If the selected XR application type is VR roaming, the base plate data consists of a detailed city model and metadata information, wherein the metadata information includes: city coordinate offset, relative coordinate offset value and global rotation amount;

[0039] If the selected XR application type is VR model viewing, then the base plate data is an empty object;

[0040] Step 34: Place the base plate data at the origin of the world coordinate system in the 3D view;

[0041] Step 35: Set the scaling factor of the base plate data to achieve the ratio of the base plate space scale to the running engine space scale as preset.

[0042] Furthermore, step 4 specifically includes:

[0043] Step 41: Using the categorization and search functions provided by the XR application editor, retrieve the required digital content from the digital content warehouse and add it to the 3D view;

[0044] Step 42: Using the base plate data as a reference, determine the relative placement, relative scaling, relative rotation angle, and model animation playback status of the digital content.

[0045] Furthermore, step 5 involves arranging sub-scene interaction instructions using an XR application editor, specifically including:

[0046] Step 51: Select a number and, in the interaction design panel, select an event based on the event type;

[0047] Step 52: After selecting to trigger the event, determine the action instructions to be executed;

[0048] Step 53: Determine whether the action command requires selecting an object to be operated on; if it does not require selecting an object to be operated on, enter the corresponding operation information and complete the interaction creation; if it requires selecting an object to be operated on, select the operable object in the current scene through the drop-down box, enter the corresponding operation information and complete the interaction creation.

[0049] Furthermore, step 7 specifically includes:

[0050] Step 71: Generate the basic information required for the XR application building model based on the application ID, application name, XR application type, and launch cover information, and store it under the application information node of the XR application building model;

[0051] Step 72: Create a scene storage object and store the sub-scene name, sub-scene cover information, and order information in the scene object;

[0052] Step 73: Traverse the data content under the environment directory node in the scene structure tree, including: various lights, background textures and HDR information, and store them under the environment node of the scene object;

[0053] Step 74: Store the base plate data address and metadata information contained in the scene under the base plate node of the scene object;

[0054] Step 75: Recursively traverse the digital content directory in the scene structure tree, store the directory as a directory object, and store the name, address, pose, parameter settings, animation and interaction instruction information of the digital content as digital content objects, and store them under the digital content node of the scene object according to the tree structure.

[0055] Step 76: Repeat steps 72 to 75 until an array containing all sub-scene storage objects of the XR application is obtained, and store the array under the scene content node of the XR application building model;

[0056] Step 77: Convert all the data used to build the model in the XR application into a JSON format string using a JSON serialization algorithm;

[0057] Step 78: Use a combination of multiple encryption algorithms to perform symmetric encryption on the string, and write the encrypted data into a file as a binary stream, which will serve as the data file for building the XR application model;

[0058] Step 79: Store the data file of the XR application building model in the application building model repository.

[0059] Furthermore, step 8 specifically includes:

[0060] Step 81: Obtain the download address of the data file in the Internet environment based on the storage location of the XR application building model data file in the application building model repository;

[0061] Step 82: Determine the scheme value and application page launch path based on the target operating platform type and XR application type;

[0062] Step 83: Generate a record in the database version record table. The record information includes: version ID, application ID, data file download address, and application page launch path; the version ID, data file download address, and application page launch path are in one-to-one correspondence.

[0063] Step 84: Generate the application launch link based on the scheme value, application page launch path, version ID, and application ID, according to the URL Scheme protocol format;

[0064] Step 85: Generate an XR application QR code based on the application's wake-up link, according to the QR code encoding method.

[0065] Furthermore, step 9, which involves building the runtime engine, specifically includes:

[0066] Step 91: Build the runtime engine for native applications. Develop application templates for all XR application types using Unity and AR engine technologies, and package and output installation packages for different systems as the runtime engine for native applications.

[0067] Step 92: Build the runtime engine for the glasses application. Based on the application template, integrate the glasses SDK and package and output the glasses version installation package as the runtime engine for the glasses application.

[0068] Step 93: Build the runtime engine of the WeChat Mini Program application. Develop application templates for all XR application types using Visionkit and Three.js technologies, package and publish the WeChat Mini Program application as its runtime engine.

[0069] Step 94: Build the runtime engine for H5 page applications. Develop application templates for all XR application types using WebXR API and Three.js technology to serve as the runtime engine for H5 page applications.

[0070] Furthermore, step 10 specifically includes:

[0071] Step 101: If the terminal device uses a native application or a glasses application, the application template page of the pre-installed runtime engine will be invoked through the application invocation link contained in the XR application QR code.

[0072] Step 102: If the terminal device uses an H5 page application, the browser will be launched and the application template page of the pre-installed runtime engine will be entered through the application launch link contained in the XR application QR code.

[0073] Step 103: If the terminal device uses a WeChat mini-program application, the WeChat mini-program will be launched through the application launch link contained in the XR application QR code and the application template page of the pre-installed runtime engine will be entered.

[0074] Step 104: Parse the parameters carried by the application invocation link to obtain the application ID and version ID. Then, query the server based on the application ID to obtain the application name, launch cover information, and XR application type of the XR application. Query the server based on the version ID to obtain the data file download address.

[0075] Step 105: Based on the obtained application name and launch cover information, render the UI interface to present the launch loading interface of the XR application;

[0076] Step 106: Download the data file for building the XR application model from the data file download address, decrypt it, and read its contents;

[0077] Step 107: Based on the target operating platform type, XR application type, scene and environment data, initialize the XR engine and configure the XR engine data. The browser environment is WebXR API, the WeChat mini program environment is VisionKit, the Android native application and glasses application environment is AR Core, the iOS environment is AR Kit, and the HarmonyOS system environment is AR Engine.

[0078] Step 108: Parse the data under the scene content nodes in the XR application construction model. Traverse the scene list and parse the environment, base, and digital content node data under the scene content nodes in sequence. Set the 3D engine HDR, background texture, and lighting through the environment node data; set the XR engine and configure the corresponding data and parameters through the base node data; download digital content from the server through the digital content node data, instantiate the digital content object by the 3D engine, and set the pose and two-sided rendering attributes. If it is a 3D model, the animation objects contained in the model need to be instantiated; attach events to the digital content object according to the interaction command configuration.

[0079] Step 109: Analyze the program interface node data in the XR application construction model, draw the interface through the 3D engine, and attach events;

[0080] Step 1010: The XR application is initialized. Enter the XR application and use it.

[0081] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0082] 1. This invention provides a method for building metaverse applications, enabling zero-code, configurable construction of metaverse XR applications. By designing a standard XR application building model to store scene information, interaction information, and environmental information of the metaverse application, a pre-installed runtime engine parses the XR application building model to generate a metaverse application adapted to the terminal, achieving one-time creation and multi-terminal deployment. This significantly improves production efficiency, shortens the production cycle, and allows enterprises to obtain benefits from large-scale applications.

[0083] 2. The XR application editor included in the metaverse application building system provided by this invention allows users to quickly get started without having to spend time and effort installing cumbersome game engines, development tools, and other software, even for non-professionals, greatly reducing the barrier to entry for metaverse application construction. It also provides capabilities such as moving, rotating, scaling, quickly aligning to a plane, interactive arrangement, and animation control of 3D models, enabling efficient construction of the 3D virtual scenes required for metaverse application building.

[0084] 3. The 3D model optimization scheme provided by this invention can significantly reduce the size of 3D models without compromising display quality, and improve the transmission efficiency of 3D model data in an internet network environment. Furthermore, when displaying and rendering 3D models on the client side, the memory usage of the optimized model will be significantly reduced, allowing more users to smoothly use the Metaverse application. Attached Figure Description

[0085] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0086] Figure 1 This is an execution flowchart of a metaverse application construction method provided in an embodiment of the present invention.

[0087] Figure 2 This is a schematic diagram illustrating the principle of a metaverse application construction system provided in an embodiment of the present invention.

[0088] Figure 3 This is a schematic diagram of the interactive instruction design in a metaverse application construction method provided by an embodiment of the present invention.

[0089] Figure 4 This is a schematic diagram of the application construction model design in a metaverse application construction method provided by an embodiment of the present invention. Detailed Implementation

[0090] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0091] Please see Figures 1-4 The present invention provides a method for constructing a metaverse application, comprising the following steps:

[0092] Step 1: XR application creators upload different types of digital content to the XR application building platform. The digital content optimizer optimizes the different types of digital content and stores it in the digital content repository; providing digital content as materials for easy use when creating XR applications later.

[0093] In this embodiment, step 1 involves a digital content optimizer optimizing different types of digital content, including images, videos, audio, and 3D models, specifically including:

[0094] Step 11: For images, restrict the upload to only images in the specified format (e.g., jpg, png). Use Thumbnailator (a Google open-source, excellent third-party Java library for image processing that can compress images with very little code) to extract image thumbnails and compress the images, limiting the image resolution to the specified size (e.g., 2048*2048).

[0095] Step 12: For videos, restrict uploads to only those in a specified format (e.g., mp4). Extract the first frame image using ffmpeg (FFmpeg is an open-source computer program that can record, convert, and stream digital audio and video). Compress the image using thumbnailator to generate a video thumbnail.

[0096] Step 13: For audio, restrict the upload to only audio files in a specified format (e.g., mp3 format), and use the system's built-in default image as the audio image;

[0097] Step 14: For 3D models, restrict uploads to only those in specified formats (e.g., FBX, OBJ). Use fbx2gltf (a powerful command-line tool for converting FBX to GLTF format) or obj2gltf (an open-source tool for converting OBJ to GLTF format) to convert all uploaded FBX or OBJ 3D models to GLTF format. Optimizing different types of digital content using specific methods improves optimization efficiency.

[0098] In this embodiment, after step 14, the method further includes: optimizing the 3D model for different operating platforms, with the specific steps as follows:

[0099] Step 141: For the browser runtime environment, compress the mesh surfaces of static 3D models using the Draco algorithm (Draco is a library for compressing and decompressing 3D geometric meshes and point clouds), compress the mesh surfaces and animations of animated 3D models using the Meshopt (mesh optimization toolkit) algorithm, and convert the textures to ktx2 format;

[0100] Step 142: For WeChat Mini Programs, compress the mesh surfaces of the 3D model using the Draco algorithm and convert the textures to WebP format;

[0101] Step 143: For terminal and glasses applications, compress the mesh surface of the 3D model using the Draco algorithm and convert the texture to ktx2 format;

[0102] Step 144: When the platform encounters problems loading the compressed 3D model, switch to loading the original GLTF format 3D model.

[0103] Optimizing 3D models using different methods for different operating platforms can accelerate the loading of 3D models on clients via the internet. This 3D model optimization scheme can significantly reduce the size of 3D models without compromising display quality, and improve the transmission efficiency of 3D model data over the internet. Furthermore, when displaying and rendering 3D models on the client side, the optimized model will significantly reduce memory usage, allowing more users to smoothly use the Metaverse application.

[0104] Step 2: Create a new XR application on the XR application building platform. After selecting a preset XR application type, enter the application name and launch cover information, and enter the XR application editor. The preset XR application types include: Image AR, 2D AR, Face AR, Human Body AR, Surround View AR, Object AR, Visual Positioning AR, VR Roaming, and VR Model Viewing. Complete the creation of the XR application based on the XR application type, application name, and launch cover information for convenient subsequent use.

[0105] Step 3: Create a new sub-scene in the newly created XR application by entering the sub-scene name and base plate data;

[0106] In this embodiment, step 3 specifically includes:

[0107] Step 31: Create a sub-scene in the newly created XR application; at least one sub-scene can be created in an XR application, which can contain corresponding data;

[0108] Step 32: Enter the name of the sub-scene in the newly created sub-scene; each sub-scene has a unique name to distinguish different sub-scenes;

[0109] Step 33: Input the base plate data in the newly created sub-scene. The type of the base plate data is determined according to the XR application type. Each sub-scene contains one set of base plate data, which is loaded by the XR application editor.

[0110] The XR application types include: image AR, planar AR, face AR, human body AR, surround view AR, object AR, visual positioning AR, VR roaming, and VR model viewing;

[0111] If the selected XR application type is image AR, then the base plate data is a recognition image;

[0112] If the selected XR application type is planar AR, then the base plate data is a virtual plane;

[0113] If the selected XR application type is face AR, then the base data is a standard white film of a human head;

[0114] If the selected XR application type is human AR, then the substrate data is a standard human white film;

[0115] If the selected XR application type is surround view AR, then the base plate data is a skybox model and metadata information, which includes: skybox zoom size and height;

[0116] If the selected XR application type is physical AR, then the base plate data is object feature point cloud data, which is obtained by three-dimensional reconstruction of multiple consecutive photos of the object.

[0117] If the selected XR application type is visual positioning AR, then the base plate data consists of a spatial base plate white film, feature point cloud data, panoramic viewpoints, and metadata information; wherein, the metadata information includes: city coordinate offset, relative coordinate offset value, center point latitude and longitude, and global rotation amount;

[0118] If the selected XR application type is VR roaming, the base plate data consists of a detailed city model and metadata information, wherein the metadata information includes: city coordinate offset, relative coordinate offset value and global rotation amount;

[0119] If the selected XR application type is VR model viewing, then the base plate data is an empty object;

[0120] The type of base plate data is determined based on the type of XR application, thus achieving consistency between the XR application type and the base plate data type, which facilitates subsequent operations.

[0121] Step 34: Place the base plate data at the origin (0,0) of the 3D view world coordinate system;

[0122] Step 35: Set the scaling factor of the base plate data to achieve a preset ratio between the base plate space scale and the running engine space scale; for example, set the scaling factor of the base plate data to 1 so as to achieve a 1:1 ratio between the base plate space scale and the running engine space scale.

[0123] Step 4: Complete the scene digital content layout using the XR application editor;

[0124] In this embodiment, step 4 specifically includes:

[0125] Step 41: Using the classification and search functions provided by the XR application editor, obtain the required digital content (such as 3D models, images, videos, etc.) from the digital content warehouse and add it to the 3D view;

[0126] Step 42: Using the base plate data as a reference, determine the relative placement, relative scaling, relative rotation angle, and model animation playback status of the digital content.

[0127] The XR application editor has the following functions:

[0128] S1, Scene Management: Allows users to view a list of sub-scenes, create and delete sub-scenes, switch the currently active sub-scene, and edit the sub-scene name and image.

[0129] S2, Baseboard Repository: Based on the newly created XR application type, the baseboard repository recommends baseboard data of the corresponding type.

[0130] S3, Material Library: This is a digital content repository that categorizes materials into 3D models, images, videos, audio, and special effects. Each category is further divided into public materials and personal materials. Users can add the materials they need through the upload function.

[0131] S4, Convenient Toolbar: Enables visual movement, rotation, and scaling of digital content; has functions for cloning object instances, showing and hiding objects, and deleting objects; and has alignment tools to quickly align digital content to the base plane.

[0132] S5, Scene Structure Tree Panel: Displays the entire scene data as a resource tree. Default divisions include three directories: Environment, Baseboard, and Digital Content. The Environment directory contains the scene's camera, ambient light, and directional light by default; the Baseboard directory stores the current scene's baseboard data; and the Digital Content directory stores user-added digital content such as images, videos, and 3D models, allowing users to show, hide, delete, and replace digital content. A new directory creation function is provided, allowing users to move and categorize digital content into selected directories.

[0133] S6, Object Properties Panel: Allows you to view and modify the property information of digital content, including: layout, visibility, two-sided rendering, view frustum clipping, rendering order, animation control, etc.

[0134] S7, Global Toolbar: Features undo, redo, save, and preview functions. The preview function can simulate the application's operation within the system in a VR format and generate a QR code for experiencing the XR application. Scanning the code allows users to experience the application.

[0135] The XR application editor allows users to quickly get started without the hassle of installing cumbersome game engines and development tools, even for non-professionals, significantly lowering the barrier to entry for building metaverse applications. It also provides capabilities such as moving, rotating, scaling, quickly aligning 3D models to a plane, interactive arrangement, and animation control, enabling efficient construction of the 3D virtual scenes needed for metaverse application development.

[0136] Step 5: Arrange the sub-scene interaction instructions using the XR application editor; in the 3D view or resource management tree, select the objects that need to be interacted with, and add interaction instructions in sequence to make the final XR application content interactive. Repeat steps 3 to 5 as needed to complete the scene layout of the entire XR application;

[0137] In this embodiment, step 5 involves arranging sub-scene interaction instructions using an XR application editor, specifically including:

[0138] Step 51: Select a number and, in the interaction design panel, select an event based on the event type;

[0139] Step 52: After selecting to trigger the event, determine the action instructions to be executed;

[0140] Step 53: Determine whether the action command requires selecting an object to be operated on; if it does not require selecting an object to be operated on, enter the corresponding operation information and complete the interaction creation; if it requires selecting an object to be operated on, select the operable object in the current scene through the drop-down box, enter the corresponding operation information and complete the interaction creation.

[0141] The interactive instructions are designed as follows:

[0142] 1. A complete interaction definition is as follows:

[0143] M 交互 =E 事件 +A 动作 +O 操作对象 +I 操作信息

[0144] 2. The preset events include two types:

[0145] ① Object events: Click (on_click), Double click (on_double_click), Model animation ends (on_model_animations_end), Media playback ends (on_media_play_end);

[0146] ② Global events: when the page loads (on_load), after a certain period of time (on_time_out), and when data loading is complete (on_model_loaded);

[0147] 3. Preset action commands include: open link (open_url), play multimedia (play_media), stop playing multimedia (stop_play_media), play model animation (play_model_animations), stop playing model animation (stop_model_animations), show object (show_object), hide object (hide_object), message notification (show_msg), move object (move_object), rotate object (rotate_object), scale object (scale_object), and launch mini-program (call_wx_app);

[0148] 4. Preset object filters, i.e., operation object selectors, including: none, animated models, all objects, all videos, all models, all audio, and all images;

[0149] 5. Interaction commands are isolated according to scene. The execution rule for each sub-scene interaction command is to prioritize the execution of interactions with global events, followed by interactions with object events. For similar interactions, they are executed in the order they were added.

[0150] Step 6: Using the interface editor, draw the program interface, store the interface elements contained in the program interface in the program interface node of the XR application building model, and add interactive instructions to the interface elements.

[0151] Step 7: Set the basic information required for building XR applications, generate the data file of the XR application building model, and store it in the application building model repository. By designing a standard XR application building model, scene information, interaction information, and environmental information of the metaverse application are stored. Then, the pre-installed runtime engine is responsible for parsing the XR application building model and generating metaverse applications adapted to the terminals, enabling creation on one platform and publishing on multiple platforms. This greatly improves production efficiency, shortens the production cycle, and allows enterprises to obtain benefits from large-scale applications.

[0152] In this embodiment, step 7 specifically includes:

[0153] Step 71: Generate the basic information required for the XR application building model based on the application ID, application name, XR application type, and launch cover information, and store it under the application information node of the XR application building model;

[0154] Step 72: Create a scene storage object and store the sub-scene name, sub-scene cover information, and order information in the scene object;

[0155] Step 73: Traverse the data content under the environment directory node in the scene structure tree, including: various lights, background textures and HDR information, and store them under the environment node of the scene object;

[0156] Step 74: Store the base plate data address and metadata information contained in the scene under the base plate node of the scene object;

[0157] Step 75: Recursively traverse the digital content directory in the scene structure tree, store the directory as a directory object, and store the name, address, pose, parameter settings, animation and interaction instruction information of the digital content as digital content objects, and store them under the digital content node of the scene object according to the tree structure.

[0158] Step 76: Repeat steps 72 to 75 until an array containing all sub-scene storage objects of the XR application is obtained, and store the array under the scene content node of the XR application building model;

[0159] Step 77: Convert all the data used to build the model in the XR application into a JSON format string using a JSON serialization algorithm;

[0160] Step 78: Use a combination of multiple encryption algorithms to perform symmetric encryption on the string, and write the encrypted data into a file as a binary stream, which will serve as the data file for building the XR application model;

[0161] Step 79: Store the data file of the XR application building model in the application building model repository.

[0162] Step 8: Generate an XR application QR code based on the application ID, XR application type, target operating platform type, and XR application building model data file;

[0163] In this embodiment, step 8 specifically includes:

[0164] Step 81: Obtain the download address of the data file in the Internet environment based on the storage location of the XR application building model data file in the application building model repository;

[0165] Step 82: Determine the scheme value and application page invocation path based on the target operating platform type and XR application type. For example, the scheme for the WeChat mini program environment is "weixin", the scheme for the web environment is "https", and the scheme for Android, iOS, HarmonyOS and glasses native application environments is "XREngine" (the identifier of the running engine application).

[0166] Step 83: Generate a record in the database version record table. The record information includes: version ID, application ID, data file download address, and application page launch path; the version ID, data file download address, and application page launch path are in one-to-one correspondence.

[0167] Step 84: Generate an application launch link based on the scheme value, application page launch path, version ID, and application ID, using the URL Scheme protocol format. A URL Scheme is a mechanism for launching applications on mobile devices via links or buttons. It is a standardized protocol that can be used to transfer data and commands between applications.

[0168] For example: URL scheme=${scheme}: / / ${path} / ? appid=${appid}&versionid=${versionid}

[0169] Among them, URL scheme This indicates the application launch link, scheme indicates the application identifier of the current environment's running engine, path indicates the application page launch path, appid indicates the application ID, and versionid indicates the version ID;

[0170] Step 85: Generate an XR application QR code based on the application's wake-up link, according to the QR code encoding method.

[0171] Step 9: Build the runtime engine and pre-install it on the terminal device;

[0172] In this embodiment, step 9, building the runtime engine, specifically includes:

[0173] Step 91: Build the runtime engine for native applications. Develop application templates for all XR application types using Unity and AR engine technologies (AR Core, AR Kit, AREngine), and package them into installation packages for different systems (Android, iOS, and HarmonyOS) as the runtime engine for native applications.

[0174] Step 92: Build the runtime engine for the glasses application. Based on the application template, integrate the glasses SDK and package and output the glasses version installation package as the runtime engine for the glasses application.

[0175] Step 93: Build the runtime engine of the WeChat Mini Program application. Develop application templates for all XR application types using Visionkit and Three.js technologies, package and publish the WeChat Mini Program application as its runtime engine.

[0176] Step 94: Build the runtime engine for H5 page applications. Develop application templates for all XR application types using WebXR API and Three.js technology to serve as the runtime engine for H5 page applications.

[0177] Step 10: Scan the XR application QR code with the terminal device's operating engine to launch and use the XR application.

[0178] In this embodiment, step 10 specifically includes:

[0179] Step 101: If the terminal device uses a native application or a glasses application, the application template page of the pre-installed runtime engine will be invoked through the application invocation link contained in the XR application QR code.

[0180] Step 102: If the terminal device uses an H5 page application, the browser will be launched and the application template page of the pre-installed runtime engine will be entered through the application launch link contained in the XR application QR code.

[0181] Step 103: If the terminal device uses a WeChat mini-program application, the WeChat mini-program will be launched through the application launch link contained in the XR application QR code and the application template page of the pre-installed runtime engine will be entered.

[0182] Step 104: Parse the parameters carried by the application invocation link to obtain the application ID and version ID. Then, query the server based on the application ID to obtain the application name, launch cover information, and XR application type of the XR application. Query the server based on the version ID to obtain the data file download address.

[0183] Step 105: Based on the obtained application name and launch cover information, render the UI interface to present the launch loading interface of the XR application;

[0184] Step 106: Download the data file for building the XR application model from the data file download address, decrypt it, and read its contents;

[0185] Step 107: Based on the target operating platform type, XR application type, scene and environment data, initialize the XR engine and configure the XR engine data. The browser environment is WebXR API, the WeChat mini program environment is VisionKit, the Android native application and glasses application environment is AR Core, the iOS environment is AR Kit, and the HarmonyOS system environment is AR Engine.

[0186] Step 108: Parse the data under the scene content nodes in the XR application construction model. Traverse the scene list and parse the environment, base, and digital content node data under the scene content nodes in sequence. Set the 3D engine HDR, background texture, and lighting through the environment node data; set the XR engine and configure the corresponding data and parameters through the base node data; download digital content from the server through the digital content node data, instantiate the digital content object by the 3D engine, and set the pose and two-sided rendering attributes. If it is a 3D model, the animation objects contained in the model need to be instantiated; attach events to the digital content object according to the interaction command configuration.

[0187] Step 109: Analyze the program interface node data in the XR application construction model, draw the interface through the 3D engine, and attach events;

[0188] Step 1010: The XR application is initialized. Enter the XR application and use it.

[0189] The above descriptions are only some embodiments of the present invention and do not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for constructing metaverse applications, characterized in that, Includes the following steps: Step 1: XR application creators upload different types of digital content to the XR application building platform. The digital content optimizer optimizes the different types of digital content and stores it in the digital content repository. Step 2: Create a new XR application on the XR application building platform. After selecting the preset XR application type, enter the application name and launch cover information, and enter the XR application editor. Step 3: Create a new sub-scene in the newly created XR application by entering the sub-scene name and base plate data; Step 4: Complete the scene digital content layout using the XR application editor; Step 5: Arrange the sub-scene interaction instructions using the XR application editor. In the 3D view or resource management tree, select the object that needs to be interacted with and add the interaction instructions in sequence. Repeat steps 3 to 5 as needed to complete the scene layout of the entire XR application. Step 6: Use the interface editor to draw the program interface and add interactive commands to the interface elements; Step 7: Set the basic information required for building the XR application, generate the data file of the XR application building model, and store it in the application building model repository; Step 8: Generate an XR application QR code based on the application ID, XR application type, target operating platform type, and XR application building model data file; Step 9: Build the runtime engine and pre-install it on the terminal device; Step 10: Scan the XR application QR code with the terminal device's operating engine to launch and use the XR application.

2. The method for constructing a metaverse application as described in claim 1, characterized in that, In step 1, the digital content optimizer optimizes different types of digital content, including images, videos, audio, and 3D models, specifically including: Step 11: For images, restrict the upload to only images of the specified format, extract image thumbnails using Thumbnailator, compress the images, and limit the image resolution to within the specified size. Step 12: For videos, restrict uploads to only videos of a specified format. Extract the first frame image using ffmpeg and compress the image using thumbnailator to generate a video thumbnail. Step 13: For audio, restrict the upload to only audio files of the specified format, and use the system's built-in default image as the audio image; Step 14: For 3D models, restrict the upload to only 3D models in the specified format, and convert all uploaded 3D models to GLTF format.

3. The method for constructing a metaverse application as described in claim 2, characterized in that, Following step 14, the process further includes optimizing the 3D model for different operating platforms, with the specific steps as follows: Step 141: For the browser runtime environment, compress the mesh surfaces of static 3D models using the Draco algorithm, compress the mesh surfaces and animations of animated 3D models using the Meshopt algorithm, and convert the textures to ktx2 format; Step 142: For WeChat Mini Programs, compress the mesh surfaces of the 3D model using the Draco algorithm and convert the textures to WebP format; Step 143: For terminal and glasses applications, compress the mesh surface of the 3D model using the Draco algorithm and convert the texture to ktx2 format; Step 144: When the platform encounters problems loading the compressed 3D model, switch to loading the original GLTF format 3D model.

4. The method for constructing a metaverse application as described in claim 1, characterized in that, Step 3 specifically includes: Step 31: Create a sub-scene in the newly created XR application; Step 32: Enter the name of the newly created sub-scene; Step 33: Input the base plate data in the newly created sub-scene. The type of the base plate data is determined according to the XR application type. Each sub-scene contains one set of base plate data, which is loaded by the XR application editor. The XR application types include: image AR, planar AR, face AR, human body AR, surround view AR, object AR, visual positioning AR, VR roaming, and VR model viewing; If the selected XR application type is image AR, then the base plate data is a recognition image; If the selected XR application type is planar AR, then the base plate data is a virtual plane; If the selected XR application type is face AR, then the base data is a standard white film of a human head; If the selected XR application type is human AR, then the substrate data is a standard human white film; If the selected XR application type is surround view AR, then the base plate data is a skybox model and metadata information, which includes: skybox zoom size and height; If the selected XR application type is physical AR, then the base plate data is object feature point cloud data, which is obtained by three-dimensional reconstruction of multiple consecutive photos of the object. If the selected XR application type is visual positioning AR, then the base plate data consists of a spatial base plate white film, feature point cloud data, panoramic viewpoints, and metadata information; wherein, the metadata information includes: city coordinate offset, relative coordinate offset value, center point latitude and longitude, and global rotation amount; If the selected XR application type is VR roaming, the base plate data consists of a detailed city model and metadata information, wherein the metadata information includes: city coordinate offset, relative coordinate offset value and global rotation amount; If the selected XR application type is VR model viewing, then the base plate data is an empty object; Step 34: Place the base plate data at the origin of the world coordinate system in the 3D view; Step 35: Set the scaling factor of the base plate data to achieve the ratio of the base plate space scale to the running engine space scale as preset.

5. The method for constructing a metaverse application as described in claim 1, characterized in that, Step 4 specifically includes: Step 41: Using the categorization and search functions provided by the XR application editor, retrieve the required digital content from the digital content warehouse and add it to the 3D view; Step 42: Using the base plate data as a reference, determine the relative placement, relative scaling, relative rotation angle, and model animation playback status of the digital content.

6. The method for constructing a metaverse application as described in claim 1, characterized in that, Step 5 involves arranging sub-scene interaction instructions using an XR application editor, specifically including: Step 51: Select a number and, in the interaction design panel, select an event based on the event type; Step 52: After selecting to trigger the event, determine the action instructions to be executed; Step 53: Determine whether the action command requires selecting an object to be operated on; if it does not require selecting an object to be operated on, enter the corresponding operation information and complete the interaction creation; if it requires selecting an object to be operated on, select the operable object in the current scene through the drop-down box, enter the corresponding operation information and complete the interaction creation.

7. The method for constructing a metaverse application as described in claim 1, characterized in that, Step 7 specifically includes: Step 71: Generate the basic information required for the XR application building model based on the application ID, application name, XR application type, and launch cover information, and store it under the application information node of the XR application building model; Step 72: Create a scene storage object and store the sub-scene name, sub-scene cover information, and order information in the scene object; Step 73: Traverse the data content under the environment directory node in the scene structure tree, including: various lights, background textures and HDR information, and store them under the environment node of the scene object; Step 74: Store the base plate data address and metadata information contained in the scene under the base plate node of the scene object; Step 75: Recursively traverse the digital content directory in the scene structure tree, store the directory as a directory object, and store the name, address, pose, parameter settings, animation and interaction instruction information of the digital content as digital content objects, and store them under the digital content node of the scene object according to the tree structure. Step 76: Repeat steps 72 to 75 until an array containing all sub-scene storage objects of the XR application is obtained, and store the array under the scene content node of the XR application building model; Step 77: Convert all the data used to build the model in the XR application into a JSON format string using a JSON serialization algorithm; Step 78: Use a combination of multiple encryption algorithms to perform symmetric encryption on the string, and write the encrypted data into a file as a binary stream, which will serve as the data file for building the XR application model; Step 79: Store the data file of the XR application building model in the application building model repository.

8. The method for constructing a metaverse application as described in claim 7, characterized in that, Step 8 specifically includes: Step 81: Obtain the download address of the data file in the Internet environment based on the storage location of the XR application building model data file in the application building model repository; Step 82: Determine the scheme value and application page launch path based on the target operating platform type and XR application type; Step 83: Generate a record in the database version record table. The record information includes: version ID, application ID, data file download address, and application page launch path; the version ID, data file download address, and application page launch path are in one-to-one correspondence. Step 84: Generate the application launch link based on the scheme value, application page launch path, version ID, and application ID, according to the URL Scheme protocol format; Step 85: Generate an XR application QR code based on the application's wake-up link, according to the QR code encoding method.

9. The method for constructing a metaverse application as described in claim 7, characterized in that, Step 9, which involves building the runtime engine, specifically includes: Step 91: Build the runtime engine for native applications. Develop application templates for all XR application types using Unity and AR engine technologies, and package and output installation packages for different systems as the runtime engine for native applications. Step 92: Build the runtime engine for the glasses application. Based on the application template, integrate the glasses SDK and package and output the glasses version installation package as the runtime engine for the glasses application. Step 93: Build the runtime engine of the WeChat Mini Program application. Develop application templates for all XR application types using Visionkit and Three.js technologies, package and publish the WeChat Mini Program application as its runtime engine. Step 94: Build the runtime engine for H5 page applications. Develop application templates for all XR application types using WebXR API and Three.js technology to serve as the runtime engine for H5 page applications.

10. A method for constructing a metaverse application as described in claim 7, characterized in that, Step 10 specifically includes: Step 101: If the terminal device uses a native application or a glasses application, the application template page of the pre-installed runtime engine will be invoked through the application invocation link contained in the XR application QR code. Step 102: If the terminal device uses an H5 page application, the browser will be launched and the application template page of the pre-installed runtime engine will be entered through the application launch link contained in the XR application QR code. Step 103: If the terminal device uses a WeChat mini-program application, the WeChat mini-program will be launched through the application launch link contained in the XR application QR code and the application template page of the pre-installed runtime engine will be entered. Step 104: Parse the parameters carried by the application invocation link to obtain the application ID and version ID. Then, query the server based on the application ID to obtain the application name, launch cover information, and XR application type of the XR application. Query the server based on the version ID to obtain the data file download address. Step 105: Based on the obtained application name and launch cover information, render the UI interface to present the launch loading interface of the XR application; Step 106: Download the data file for building the XR application model from the data file download address, decrypt it, and read its contents; Step 107: Based on the target operating platform type, XR application type, scene and environment data, initialize the XR engine and configure the XR engine data. The browser environment is WebXR API, the WeChat mini program environment is VisionKit, the Android native application and glasses application environment is AR Core, the iOS environment is AR Kit, and the HarmonyOS system environment is AR Engine. Step 108: Parse the data under the scene content nodes in the XR application construction model. Traverse the scene list and parse the environment, base, and digital content node data under the scene content nodes in sequence. Set the 3D engine HDR, background texture, and lighting through the environment node data; set the XR engine and configure the corresponding data and parameters through the base node data; download digital content from the server through the digital content node data, instantiate the digital content object by the 3D engine, and set the pose and two-sided rendering attributes. If it is a 3D model, the animation objects contained in the model need to be instantiated; attach events to the digital content object according to the interaction command configuration. Step 109: Analyze the program interface node data in the XR application construction model, draw the interface through the 3D engine, and attach events; Step 1010: The XR application is initialized. Enter the XR application and use it.

Images