Metacosm application construction method

Abstract

The invention discloses a meta universe application construction method. The method comprises the following steps: uploading and optimizing digital content; newly establishing an XR application, and entering an XR application editor; newly establishing a sub-scene in the newly established XR application by inputting the name of the sub-scene and the base plate data; scene digital content arrangement is completed through an XR application editor; sub-scene interaction instruction arrangement is completed through an XR application editor; drawing a program interface through an interface editor, and adding an interaction instruction for interface elements; basic information required by XR application construction is set, and a data file of an XR application construction model is generated; according to the application ID, the XR application type, the target operation platform type and the data file of the XR application construction model, generating an XR application two-dimensional code; constructing a running engine, and pre-installing the running engine in the terminal equipment; and scanning the two-dimensional code of the XR application through an operation engine of the terminal equipment, and starting and using the XR application. According to the method, the cost and the period of element universe application construction can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of metaverse technology, and in particular to a metaverse application construction method. Background Art

[0002] Metaverse applications are applications developed using Extended Reality (XR) technology. They create and present digital content that is integrated with the real world to provide users with an immersive interactive experience. Generally speaking, the construction of Metaverse applications involves knowledge and skills in multiple fields, such as art design, 3D modeling, animation design, interactive design, program development, etc. Therefore, it requires the collaboration of talents from multiple professions and fields to complete the construction of Metaverse applications. On the other hand, traditional Metaverse application development requires a complete software development process, including demand research, product design, software development, functional testing, and deployment. The process is long and requires a large amount of manpower. This also shows that the advent of a Metaverse application faces many challenges, which are manifested in high technical difficulty, difficulty in building a talent team, long creation cycle, and high cost. Therefore, it is urgent to design a method for building Metaverse applications. Summary of the invention

[0003] In view of this, the purpose of the present invention is to propose a metaverse application construction method to reduce the cost and cycle of metaverse application construction, enrich the metaverse application ecology, and bring people a more diverse and exciting digital life experience.

[0004] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0005] The present invention provides a method for constructing a metaverse application, comprising the following steps:

[0006] Step 1: XR application creators upload different types of digital content to the XR application construction platform, and the digital content optimizer optimizes different types of digital content and stores them in the digital content warehouse;

[0007] Step 2: Create a new XR application on the XR application building platform. After selecting the preset XR application type, enter the application name and startup cover information, and enter the XR application editor.

[0008] Step 3: Create a new sub-scene in the newly created XR application by entering the sub-scene name and base plate data;

[0009] Step 4: Complete the layout of the scene digital content through the XR application editor;

[0010] Step 5. Use the XR application editor to complete the arrangement of sub-scene interaction instructions. In the 3D view or resource management tree, select the objects that need to be interacted with and add interaction instructions in sequence. Repeat steps 3 to 5 according to the number of sub-scenes required to complete the scene layout of the entire XR application.

[0011] Step 6: Use the interface editor to draw the program interface and add interactive instructions for the interface elements;

[0012] Step 7: Set the basic information required for XR application construction, generate the data file of the XR application construction model, and store it in the application construction model warehouse;

[0013] Step 8: Generate an XR application QR code based on the application ID, XR application type, target operating platform type, and data file of the XR application construction model;

[0014] Step 9: Build the runtime engine and pre-install it in the terminal device;

[0015] Step 10: Scan the XR application QR code through the runtime engine of the terminal device to start and use the XR application.

[0016] Furthermore, in step 1, the digital content optimizer optimizes different types of digital content, wherein the different types of digital content include pictures, videos, audios and three-dimensional models, specifically including:

[0017] Step 11: For pictures, limit the upload to only pictures of a set format, extract picture thumbnails through thumbnailator, compress the pictures, and limit the picture resolution to a set size;

[0018] Step 12: For videos, only videos of a set format are allowed to be uploaded. The first frame is extracted through ffmpeg, and the image is compressed through thumbnailator to generate a video thumbnail.

[0019] Step 13: For audio, only audio files in a set format are allowed to be uploaded, and the system's built-in default image is used as the audio image;

[0020] Step 14: For 3D models, only 3D models in a set format are allowed to be uploaded, and the uploaded 3D models are uniformly converted into 3D models in GLTF format.

[0021] Furthermore, after step 14, the method further includes: optimizing the three-dimensional model for different operating platforms, and the specific steps are as follows:

[0022] Step 141: for the browser running environment, compress the mesh surface of the static 3D model by using the Draco algorithm, compress the mesh surface and animation of the 3D model with animation by using the Meshopt algorithm, and convert the texture into the ktx2 format;

[0023] Step 142: For the WeChat applet, compress the mesh surface of the three-dimensional model using the Draco algorithm, and convert the texture into a webp format;

[0024] Step 143: For terminal and glasses applications, compress the mesh surface of the 3D model using the Draco algorithm, and convert the texture into a ktx2 format;

[0025] Step 144: When a problem occurs when the running platform loads the compressed 3D model, switch to loading the original 3D model in GLTF format.

[0026] Furthermore, the step 3 specifically includes:

[0027] Step 31: Create a new sub-scene in the newly created XR application;

[0028] Step 32: Enter the sub-scene name in the newly created sub-scene;

[0029] Step 33: Input the baseplate data in the newly created sub-scene. The type of the baseplate data is determined according to the XR application type. Each sub-scene contains a copy of the baseplate data, and the baseplate data is loaded by the XR application editor.

[0030] Among them, the XR application types include: picture AR, plane AR, face AR, body AR, surround AR, physical AR, visual positioning AR, VR roaming and VR model viewing;

[0031] If the selected XR application type is image AR, the base plate data is an identification image;

[0032] If the selected XR application type is planar AR, the base plate data is a virtual plane;

[0033] If the selected XR application type is face AR, the base plate data is a standard human head portrait white film;

[0034] If the selected XR application type is human AR, the base plate data is a standard human white film;

[0035] If the selected XR application type is surround AR, the base plate data is a skybox model and metadata information, and the metadata information includes: skybox scaling size and height;

[0036] If the selected XR application type is physical AR, the base plate data is feature point cloud data of the object, which is obtained by three-dimensional reconstruction of multiple consecutive photos of the object;

[0037] If the selected XR application type is visual positioning AR, the base plate data consists of a space base plate white film, feature point cloud data, panoramic points and metadata information; wherein the metadata information includes: city coordinate offset, relative coordinate offset value, center point longitude and latitude and global rotation;

[0038] If the selected XR application type is VR roaming, the base plate data is composed of a detailed city model and metadata information, wherein the metadata information includes: city coordinate offset, relative coordinate offset value and global rotation;

[0039] If the selected XR application type is VR model viewing, the base plate data is an empty object;

[0040] Step 34, placing the base plate data at the coordinate origin position of the three-dimensional view world coordinate system;

[0041] Step 35: Set the scaling factor of the base plate data to achieve a ratio between the base plate space scale and the running engine space scale as a preset value.

[0042] Furthermore, the step 4 specifically includes:

[0043] Step 41: Obtain required digital content from the digital content warehouse through the classification and search functions provided by the XR application editor, and add it to the three-dimensional view;

[0044] Step 42: Using the base plate data as a reference, determine the relative placement position, relative scaling size, relative rotation angle and model animation playback status of the digital content.

[0045] Furthermore, in step 5, the sub-scene interaction instruction arrangement is completed through the XR application editor, which specifically includes:

[0046] Step 51, select a digital content, and select an event according to the event type in the interaction design panel;

[0047] Step 52: After selecting to trigger the event, determine the action instruction to be executed;

[0048] Step 53, determine whether the action instruction requires the selection of an operation object; if the operation object does not need to be selected, enter the corresponding operation information to complete the interactive creation; if the operation object needs to be selected, select the operable object in the current scene through the drop-down box, enter the corresponding operation information to complete the interactive creation.

[0049] Furthermore, the step 7 specifically includes:

[0050] Step 71: Generate basic information required for XR application construction model according to application ID, application name, XR application type and startup cover information, and store it under the application information node of XR application construction model;

[0051] Step 72: Create a scene storage object, and store the sub-scene name, sub-scene cover information and sequence information in the scene object;

[0052] Step 73, traverse the data content under the environment directory node in the scene structure tree, including: various types of lights, background maps and HDR information, and store them under the environment node of the scene object;

[0053] Step 74: store the baseboard data address and metadata information contained in the scene under the baseboard node of the scene object;

[0054] Step 75, recursively traverse the digital content directory in the scene structure tree, store the directory as a directory object, store the name, address, position, parameter setting, animation and interactive instruction information of the digital content as a digital content object, and store it under the digital content node of the scene object according to the tree structure;

[0055] Step 76, looping step 72 to step 75 until an array containing all sub-scene storage objects of the XR application is obtained, and the array is stored under the scene content node of the XR application construction model;

[0056] Step 77: Convert all data of the XR application model into a string in JSON format through the JSON serialization algorithm.

[0057] Step 78: Use a combination of multiple encryption algorithms to symmetrically encrypt the string, and write the encrypted data into a file in the form of a binary stream as a data file for building the XR application model;

[0058] Step 79: Store the data file of the XR application construction model in the application construction model warehouse.

[0059] Furthermore, the step 8 specifically includes:

[0060] Step 81: Obtain a data file download address in an Internet environment according to the storage location of the data file of the XR application construction model in the application construction model warehouse;

[0061] Step 82: Determine the scheme value and the application page calling path according to the target operating platform type and the XR application type;

[0062] Step 83: Generate a record in the database version record table, the record information includes: version ID, application ID, data file download address and application page call path; the version ID, data file download address and application page call path are one-to-one corresponding;

[0063] Step 84: Generate an application invocation link based on the URL Scheme protocol format according to the scheme value, the application page invocation path, the version ID, and the application ID;

[0064] Step 85: Generate an XR application QR code according to the application call link in accordance with the QR code encoding method.

[0065] Furthermore, the construction of the running engine in step 9 specifically includes:

[0066] Step 91: Build a runtime engine for native applications, develop application templates for all XR application types through Unity and AR engine technology, and package and output installation packages for different systems as the runtime engine for native applications;

[0067] Step 92: constructing a running engine for the glasses application. Based on the application template, integrating the glasses SDK, and packaging and outputting a glasses version installation package as the running engine for the glasses application.

[0068] Step 93: Build a running engine for the WeChat applet application, develop application templates for all XR application types through Visionkit and Three.js technologies, package and publish the WeChat applet application as the running engine for the WeChat applet application;

[0069] Step 94: Build a runtime engine for the H5 page application, and develop application templates for all XR application types through the WebXR API and Three.js technology as the runtime engine for the H5 page application.

[0070] Furthermore, the step 10 specifically includes:

[0071] Step 101: If the terminal device uses a native application or a glasses application, the application template page of the pre-installed runtime engine of the local device is invoked through the application invocation link contained in the XR application QR code;

[0072] Step 102: If the terminal device uses an H5 page application, the application invocation link included in the XR application QR code is used to invoke the browser and enter the application template page of the pre-installed runtime engine;

[0073] Step 103: If the terminal device uses a WeChat applet application, the WeChat applet is invoked through the application invocation link contained in the XR application QR code and the application template page of the pre-installed runtime engine is entered;

[0074] Step 104: parse the parameters carried by the application call link to obtain the application ID and version ID, and then query the server according to the application ID to obtain the application name, startup cover information and XR application type of the XR application; query the server according to the version ID to obtain the data file download address;

[0075] Step 105: Render a UI interface according to the obtained application name and startup cover information to present a startup loading interface of the XR application;

[0076] Step 106: Download the data file of the XR application construction model according to the data file download address, decrypt and read the content;

[0077] Step 107: Initialize the XR engine and configure the XR engine data according to the target operating platform type, XR application type, scene and environment data. The browser environment is WebXR API, the WeChat applet environment is VisionKit, the Android native application and glasses application environment is AR Core, the iOS environment is AR Kit, and the Hongmeng system environment is AR Engine.

[0078] Step 108: parse the data under the scene content node in the XR application construction model, traverse the scene list, parse the environment, base plate, and digital content node data under the scene content node in turn, set the 3D engine HDR, background map, and lighting through the environment node data; set the data and parameters corresponding to the XR engine and configuration engine through the base plate node data; download digital content from the server through the digital content node data, instantiate the digital content object by the 3D engine, and set the posture and double-sided rendering attributes. If it is a 3D model, the animation object contained in the model needs to be instantiated; according to the interactive instruction configuration, the digital content object is hooked up with the event;

[0079] Step 109: Parse the program interface node data in the XR application construction model, draw the interface through the 3D engine, and hook up events;

[0080] Step 1010: After the XR application is initialized, enter the XR application and use it.

[0081] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0082] 1. The present invention provides a method for building a metaverse application, which can realize a zero-code, configurable way to build a metaverse XR application. By designing a set of standard XR application construction models, the scene information, interaction information, environmental information, etc. of the metaverse application are stored. Then, the pre-installed runtime engine is responsible for parsing the XR application construction model, generating a metaverse application adapted to the terminal, and realizing one-end creation and multi-end publishing. It greatly improves production efficiency, shortens production cycles, and allows enterprises to obtain large-scale application benefits.

[0083] 2. The XR application editor included in the Metaverse application construction system provided by the present invention can save users from the time and effort of installing bulky game engines, development tools and other software. Even non-professionals can quickly get started, greatly reducing the threshold for Metaverse application construction. At the same time, it also provides the ability to move, rotate, scale, quickly align to a plane, interactively arrange, and control animations of 3D models, so as to achieve efficient construction of the 3D virtual scene required for Metaverse application construction.

[0084] 3. The 3D model optimization solution provided by the present invention can significantly reduce the volume of the 3D model without reducing the display effect, and improve the transmission efficiency of the 3D model data in the Internet network environment. On the other hand, when the client displays the rendered 3D model, the memory occupied by the model optimized by this solution will also be significantly reduced, so that more users can use the Metaverse application smoothly. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0086] Figure 1 It is an execution flow chart of a metaverse application construction method provided by an embodiment of the present invention.

[0087] Figure 2 It is a schematic diagram of the principle of a metaverse application construction system provided by an embodiment of the present invention.

[0088] Figure 3 It is a schematic diagram of interactive instruction design in a metaverse application construction method provided in an embodiment of the present invention.

[0089] Figure 4 It is a schematic diagram of the design of an application construction model in a metaverse application construction method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0090] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0091] See also Figure 1-Figure 4 A method for constructing a metaverse application of the present invention comprises the following steps:

[0092] Step 1. XR application creators upload different types of digital content to the XR application construction platform. The digital content optimizer optimizes different types of digital content and stores them in the digital content warehouse. Digital content is provided as material to facilitate subsequent calls when creating XR applications.

[0093] In this embodiment, in step 1, the digital content optimizer optimizes different types of digital content, where the different types of digital content include pictures, videos, audios, and three-dimensional models, specifically including:

[0094] Step 11. For images, limit the upload to only images in a set format (such as jpg, png), extract image thumbnails through thumbnailator (thumbnailator is a third-party Java library for excellent image processing open sourced by Google, which can achieve image compression with very little code), and compress the images, limiting the image resolution to a set size (such as 2048*2048);

[0095] Step 12: For videos, limit the upload to only videos in a set format (e.g., mp4 format), extract the first frame through ffmpeg (FFmpeg is an open source computer program that can be used to record, convert digital audio and video, and convert them into streams), and compress the image through thumbnailator to generate a video thumbnail;

[0096] Step 13: For audio, only audio files in a set format (such as mp3 format) are allowed to be uploaded, and the system's built-in default picture is used as the audio picture;

[0097] Step 14: For 3D models, only 3D models in set formats (such as FBX and OBJ formats) are allowed to be uploaded. The uploaded FBX or OBJ format 3D models are uniformly converted to GLTF format 3D models through fbx2gltf (fbx2gltf is a powerful command line tool for converting FBX file formats of 3D model assets to GLTF file formats) or obj2gltf (obj2gltf is an open source tool for converting OBJ format of 3D models to GLTF format). Optimizing different types of digital content in specific ways can improve optimization efficiency.

[0098] In this embodiment, after step 14, the method further includes: optimizing the three-dimensional model for different operating platforms, and the specific steps are as follows:

[0099] Step 141: for the browser running environment, compress the mesh surface of the static 3D model by using the Draco (Draco is a library for compressing and decompressing 3D geometric meshes and point clouds) algorithm, and compress the mesh surface and animation of the 3D model with animation by using the Meshopt (mesh optimization toolkit) algorithm, and convert the texture into ktx2 format;

[0100] Step 142: For the WeChat applet, compress the mesh surface of the three-dimensional model using the Draco algorithm, and convert the texture into a webp format;

[0101] Step 143: For terminal and glasses applications, compress the mesh surface of the 3D model using the Draco algorithm, and convert the texture into a ktx2 format;

[0102] Step 144: When a problem occurs when the running platform loads the compressed 3D model, switch to loading the original 3D model in GLTF format.

[0103] Optimizing 3D models in different ways for different operating platforms can speed up the efficiency of clients loading 3D models through the Internet. The 3D model optimization solution can significantly reduce the size of 3D models without reducing the display effect, and improve the transmission efficiency of 3D model data in the Internet network environment. On the other hand, when the client displays and renders the 3D model, the model optimized by this solution will significantly reduce the memory usage, allowing more users to use the Metaverse application smoothly.

[0104] Step 2: Create a new XR application on the XR application building platform. After selecting the preset XR application type, enter the application name and startup cover information, and enter the XR application editor; the preset XR application types include: image AR, plane AR, face AR, body AR, surround AR, physical AR, visual positioning AR, VR roaming, and VR model viewing. Complete the creation of the XR application based on the XR application type, application name, and startup cover information for easy subsequent use.

[0105] Step 3: Create a new sub-scene in the newly created XR application by entering the sub-scene name and base plate data;

[0106] In this embodiment, step 3 specifically includes:

[0107] Step 31: Create a new sub-scene in the newly created XR application. At least one sub-scene can be created in an XR application, which can contain corresponding data.

[0108] Step 32: Enter a sub-scene name in the newly created sub-scene; each sub-scene has a unique sub-scene name, which is used to distinguish different sub-scenes;

[0109] Step 33: Input the baseplate data in the newly created sub-scene. The type of the baseplate data is determined according to the XR application type. Each sub-scene contains a copy of the baseplate data, and the baseplate data is loaded by the XR application editor.

[0110] Among them, the XR application types include: picture AR, plane AR, face AR, body AR, surround AR, physical AR, visual positioning AR, VR roaming and VR model viewing;

[0111] If the selected XR application type is image AR, the base plate data is an identification image;

[0112] If the selected XR application type is planar AR, the base plate data is a virtual plane;

[0113] If the selected XR application type is face AR, the base plate data is a standard human head portrait white film;

[0114] If the selected XR application type is human AR, the base plate data is a standard human white film;

[0115] If the selected XR application type is surround AR, the base plate data is a skybox model and metadata information, and the metadata information includes: skybox scaling size and height;

[0116] If the selected XR application type is physical AR, the base plate data is feature point cloud data of the object, which is obtained by three-dimensional reconstruction of multiple consecutive photos of the object;

[0117] If the selected XR application type is visual positioning AR, the base plate data consists of a space base plate white film, feature point cloud data, panoramic points and metadata information; wherein the metadata information includes: city coordinate offset, relative coordinate offset value, center point longitude and latitude and global rotation;

[0118] If the selected XR application type is VR roaming, the base plate data is composed of a detailed city model and metadata information, wherein the metadata information includes: city coordinate offset, relative coordinate offset value and global rotation;

[0119] If the selected XR application type is VR model viewing, the base plate data is an empty object;

[0120] The type of the corresponding baseboard data is determined according to the XR application type, so as to unify the types of the XR application type and the baseboard data, and facilitate subsequent operations.

[0121] Step 34: placing the base plate data at the coordinate origin position (0,0) of the three-dimensional view world coordinate system;

[0122] Step 35, setting the scaling factor of the base plate data to achieve a preset ratio between the base plate space scale and the running engine space scale; for example, setting the scaling factor of the base plate data to 1 to achieve a 1:1 ratio between the base plate space scale and the running engine space scale.

[0123] Step 4: Complete the layout of the scene digital content through the XR application editor;

[0124] In this embodiment, step 4 specifically includes:

[0125] Step 41: Obtain required digital content (such as 3D models, pictures, videos, etc.) from the digital content warehouse through the classification and search functions provided by the XR application editor, and add it to the 3D view;

[0126] Step 42: Using the base plate data as a reference, determine the relative placement position, relative scaling size, relative rotation angle and model animation playback status of the digital content.

[0127] The XR application editor has the following functions:

[0128] S1, Scene Management: Ability to view sub-scene list, create and delete sub-scenes, switch currently activated sub-scene, edit sub-scene name and picture.

[0129] S2, baseboard warehouse: Based on the type of newly created XR application, the baseboard warehouse recommends the corresponding type of baseboard data.

[0130] S3, material library: that is, a digital content warehouse, which divides materials into 3D models, pictures, videos, audios, and special effects. Each category is further divided into public materials and my materials. Users can add the required materials through the upload function.

[0131] S4, convenient toolbar: It can visually move, rotate, and scale digital content. It has the functions of cloning object instances, showing and hiding objects, and deleting objects. It has an alignment tool that can quickly align digital content to the base plane.

[0132] S5, scene structure tree panel: can display the entire scene data content in the form of a resource tree. Default division: environment, baseplate, digital content three directories. Among them, the environment directory has the camera, ambient light, and parallel light of the scene by default; the baseplate directory stores the baseplate data of the current scene; the digital content directory stores the digital content such as pictures, videos, and 3D models added by the user, and can show, hide, delete, and replace digital content. Provide a new directory function, which can move and classify digital content to the selected directory.

[0133] S6, Object Properties Panel: can view and modify the property information of digital content, including: layout, visibility, double-sided rendering, frustum clipping, rendering order, animation control, etc.

[0134] S7, global toolbar: with undo, redo, save, and preview functions. The preview function can simulate the application running effect in the system in the form of VR, and generate an experience QR code, which can be scanned to experience the XR application.

[0135] The XR application editor allows users to quickly get started without having to spend time and effort installing bulky game engines, development tools and other software, even non-professionals can do so, greatly reducing the threshold for building Metaverse applications. At the same time, it also provides the ability to move, rotate, scale, quickly align to a plane, interactively arrange, and control animations for 3D models, enabling efficient construction of 3D virtual scenes required for Metaverse application construction.

[0136] Step 5. Use the XR application editor to complete the arrangement of sub-scene interaction instructions; in the 3D view or resource management tree, select the objects that need to be interacted with, and add interaction instructions in sequence to make the final XR application content interactive. Repeat steps 3 to 5 according to the number of sub-scenes required to complete the scene layout of the entire XR application;

[0137] In this embodiment, the sub-scene interaction instruction arrangement is completed by the XR application editor in step 5, which specifically includes:

[0138] Step 51, select a digital content, and select an event according to the event type in the interaction design panel;

[0139] Step 52: After selecting to trigger the event, determine the action instruction to be executed;

[0140] Step 53, determine whether the action instruction requires the selection of an operation object; if the operation object does not need to be selected, enter the corresponding operation information to complete the interactive creation; if the operation object needs to be selected, select the operable object in the current scene through the drop-down box, enter the corresponding operation information to complete the interactive creation.

[0141] The interactive instructions are designed as follows:

[0142] 1. A complete interaction definition is as follows:

[0143] M 交互 =E 事件 +A 动作 +O 操作对象 +I 操作信息

[0144] 2. The preset events include two categories:

[0145] ① Object events: single click (on_click), double click (on_double_click), when the model animation ends (on_model_animations_end), when the media ends (on_media_play_end);

[0146] ② Global events: when the page is loaded (on_load), after a period of time (on_time_out), when the data is loaded (on_model_loaded);

[0147] 3. The preset action commands include: open link (open_url), play multimedia (play_media), stop playing multimedia (stop_play_media), play model animation (play_model_animations), stop playing model animation (stop_model_animations), show object (show_object), hide object (hide_object), message prompt (show_msg), move object (move_object), rotate object (rotate_object), scale object (scale_object), call applet (call_wx_app);

[0148] 4. Preset object filters, i.e., operation object selectors, including: none, animated models, all objects, all videos, all models, all audios, all pictures;

[0149] 5. Interaction instructions are isolated according to scenes. The execution rule of interaction instructions in each sub-scene is to prioritize interactions with global events, followed by interactions with object events. Interactions of the same type are executed in the order they are added.

[0150] Step 6: Draw the program interface through the interface editor, store the interface elements contained in the program interface under the program interface node of the XR application construction model, and add interactive instructions for the interface elements;

[0151] Step 7: Set the basic information required for XR application construction, generate the data file of the XR application construction model, and store it in the application construction model warehouse; design a set of standard XR application construction models to store the scene information, interaction information, environment information, etc. of the Metaverse application, and then use the pre-installed runtime engine to parse the XR application construction model and generate a Metaverse application that is compatible with the terminal, so as to achieve one-end creation and multi-end release. This greatly improves production efficiency, shortens production cycles, and allows enterprises to obtain large-scale application benefits.

[0152] In this embodiment, step 7 specifically includes:

[0153] Step 71: Generate basic information required for XR application construction model according to application ID, application name, XR application type and startup cover information, and store it under the application information node of XR application construction model;

[0154] Step 72: Create a scene storage object, and store the sub-scene name, sub-scene cover information and sequence information in the scene object;

[0155] Step 73, traverse the data content under the environment directory node in the scene structure tree, including: various types of lights, background maps and HDR information, and store them under the environment node of the scene object;

[0156] Step 74: store the baseboard data address and metadata information contained in the scene under the baseboard node of the scene object;

[0157] Step 75, recursively traverse the digital content directory in the scene structure tree, store the directory as a directory object, store the name, address, position, parameter setting, animation and interaction instruction information of the digital content as a digital content object, and store it under the digital content node of the scene object according to the tree structure;

[0158] Step 76, looping step 72 to step 75 until an array containing all sub-scene storage objects of the XR application is obtained, and the array is stored under the scene content node of the XR application construction model;

[0159] Step 77: Convert all data of the XR application model into a string in JSON format through the JSON serialization algorithm.

[0160] Step 78: Use a combination of multiple encryption algorithms to symmetrically encrypt the string, and write the encrypted data into a file in the form of a binary stream as a data file for building the XR application model;

[0161] Step 79: Store the data file of the XR application construction model in the application construction model warehouse.

[0162] Step 8: Generate an XR application QR code based on the application ID, XR application type, target operating platform type, and data file of the XR application construction model;

[0163] In this embodiment, step 8 specifically includes:

[0164] Step 81: Obtain a data file download address in an Internet environment according to the storage location of the data file of the XR application construction model in the application construction model warehouse;

[0165] Step 82: Determine the scheme value and the application page invocation path according to the target operating platform type and the XR application type. For example, the scheme of the WeChat applet environment is "weixin", the web environment is "https", and the Android, iOS, Hongmeng and glasses native application environment is "XREngine" (the identifier of the operating engine application).

[0166] Step 83: Generate a record in the database version record table, the record information includes: version ID, application ID, data file download address and application page call path; the version ID, data file download address and application page call path are one-to-one corresponding;

[0167] Step 84: Generate an application invocation link based on the URL Scheme protocol format according to the scheme value, the application page invocation path, the version ID, and the application ID; URL Scheme is a mechanism for calling an application through a link or button on a mobile device, and is a standardized protocol that can be used to transfer data and commands between applications.

[0168] Such as: URL scheme=${scheme}: / / ${path} / ? appid=${appid}&versionid=${versionid}

[0169] Among them, URL scheme Indicates the application call link, scheme indicates the current environment running engine application ID, path indicates the application page call path, appid indicates the application ID, and versionid indicates the version ID;

[0170] Step 85: Generate an XR application QR code according to the application call link in accordance with the QR code encoding method.

[0171] Step 9: Build the runtime engine and pre-install it in the terminal device;

[0172] In this embodiment, building the execution engine in step 9 specifically includes:

[0173] Step 91: Build a runtime engine for native applications, develop application templates for all XR application types through Unity and AR engine technologies (AR Core, AR Kit, AREngine), and package and output installation packages for different systems (Android, iOS, and Hongmeng system platforms) as the runtime engine for native applications;

[0174] Step 92: constructing a running engine for the glasses application. Based on the application template, integrating the glasses SDK, and packaging and outputting a glasses version installation package as the running engine for the glasses application.

[0175] Step 93: Build a running engine for the WeChat applet application, develop application templates for all XR application types through Visionkit and Three.js technologies, package and publish the WeChat applet application as the running engine for the WeChat applet application;

[0176] Step 94: Build a runtime engine for the H5 page application, and develop application templates for all XR application types through the WebXR API and Three.js technology as the runtime engine for the H5 page application.

[0177] Step 10: Scan the XR application QR code through the runtime engine of the terminal device to start and use the XR application.

[0178] In this embodiment, the step 10 specifically includes:

[0179] Step 101: If the terminal device uses a native application or a glasses application, the application template page of the pre-installed runtime engine of the local device is invoked through the application invocation link contained in the XR application QR code;

[0180] Step 102: If the terminal device uses an H5 page application, the application invocation link included in the XR application QR code is used to invoke the browser and enter the application template page of the pre-installed runtime engine;

[0181] Step 103: If the terminal device uses a WeChat applet application, the WeChat applet is invoked through the application invocation link contained in the XR application QR code and the application template page of the pre-installed runtime engine is entered;

[0182] Step 104: parse the parameters carried by the application call link to obtain the application ID and version ID, and then query the server according to the application ID to obtain the application name, startup cover information and XR application type of the XR application; query the server according to the version ID to obtain the data file download address;

[0183] Step 105: Render a UI interface according to the obtained application name and startup cover information to present a startup loading interface of the XR application;

[0184] Step 106: Download the data file of the XR application construction model according to the data file download address, decrypt and read the content;

[0185] Step 107: Initialize the XR engine and configure the XR engine data according to the target operating platform type, XR application type, scene and environment data. The browser environment is WebXR API, the WeChat applet environment is VisionKit, the Android native application and glasses application environment is AR Core, the iOS environment is AR Kit, and the Hongmeng system environment is AR Engine.

[0186] Step 108: parse the data under the scene content node in the XR application construction model, traverse the scene list, parse the environment, base plate, and digital content node data under the scene content node in turn, set the 3D engine HDR, background map, and lighting through the environment node data; set the data and parameters corresponding to the XR engine and configuration engine through the base plate node data; download digital content from the server through the digital content node data, instantiate the digital content object by the 3D engine, and set the posture and double-sided rendering attributes. If it is a 3D model, the animation object contained in the model needs to be instantiated; according to the interactive instruction configuration, the digital content object is hooked up with the event;

[0187] Step 109: Parse the program interface node data in the XR application construction model, draw the interface through the 3D engine, and hook up events;

[0188] Step 1010: After the XR application is initialized, enter the XR application and use it.

[0189] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for constructing a metaverse application, characterized in that: The steps include: Step 1: XR application creators upload different types of digital content to the XR application construction platform, and the digital content optimizer optimizes different types of digital content and stores them in the digital content warehouse; Step 2: Create a new XR application on the XR application building platform. After selecting the preset XR application type, enter the application name and startup cover information, and enter the XR application editor. Step 3: Create a new sub-scene in the newly created XR application by entering the sub-scene name and base plate data; Step 4: Complete the layout of the scene digital content through the XR application editor; Step 5. Use the XR application editor to complete the arrangement of sub-scene interaction instructions. In the 3D view or resource management tree, select the objects that need to be interacted with and add interaction instructions in sequence. Repeat steps 3 to 5 according to the number of sub-scenes required to complete the scene layout of the entire XR application. Step 6: Use the interface editor to draw the program interface and add interactive instructions for the interface elements; Step 7: Set the basic information required for XR application construction, generate the data file of the XR application construction model, and store it in the application construction model warehouse; Step 8: Generate an XR application QR code based on the application ID, XR application type, target operating platform type, and data file of the XR application construction model; Step 9: Build the runtime engine and pre-install it in the terminal device; Step 10: Scan the XR application QR code through the runtime engine of the terminal device to start and use the XR application.

2. A method for constructing a metaverse application according to claim 1, characterized in that: In step 1, the digital content optimizer optimizes different types of digital content, wherein the different types of digital content include pictures, videos, audios and three-dimensional models, specifically including: Step 11: For pictures, limit the upload to only pictures of a set format, extract picture thumbnails through thumbnailator, compress the pictures, and limit the picture resolution to a set size; Step 12: For videos, only videos of a set format are allowed to be uploaded. The first frame is extracted through ffmpeg, and the image is compressed through thumbnailator to generate a video thumbnail. Step 13: For audio, only audio files in a set format are allowed to be uploaded, and the system's built-in default image is used as the audio image; Step 14: For 3D models, only 3D models in a set format are allowed to be uploaded, and the uploaded 3D models are uniformly converted into 3D models in GLTF format.

3. A method for constructing a metaverse application as claimed in claim 2, characterized in that: After step 14, the following further includes: optimizing the three-dimensional model for different operating platforms, the specific steps are as follows: Step 141: for the browser running environment, compress the mesh surface of the static 3D model by using the Draco algorithm, compress the mesh surface and animation of the 3D model with animation by using the Meshopt algorithm, and convert the texture into the ktx2 format; Step 142: For the WeChat applet, compress the mesh surface of the three-dimensional model using the Draco algorithm, and convert the texture into a webp format; Step 143: For terminal and glasses applications, compress the mesh surface of the 3D model using the Draco algorithm, and convert the texture into a ktx2 format; Step 144: When a problem occurs when the running platform loads the compressed 3D model, switch to loading the original 3D model in GLTF format.

4. A method for constructing a metaverse application according to claim 1, characterized in that: The step 3 specifically includes: Step 31: Create a new sub-scene in the newly created XR application; Step 32: Enter the sub-scene name in the newly created sub-scene; Step 33: Input the baseplate data in the newly created sub-scene. The type of the baseplate data is determined according to the XR application type. Each sub-scene contains a copy of the baseplate data, and the baseplate data is loaded by the XR application editor. Among them, the XR application types include: picture AR, plane AR, face AR, body AR, surround AR, physical AR, visual positioning AR, VR roaming and VR model viewing; If the selected XR application type is image AR, the base plate data is an identification image; If the selected XR application type is planar AR, the base plate data is a virtual plane; If the selected XR application type is face AR, the base plate data is a standard human head portrait white film; If the selected XR application type is human AR, the base plate data is a standard human white film; If the selected XR application type is surround AR, the base plate data is a skybox model and metadata information, and the metadata information includes: skybox scaling size and height; If the selected XR application type is physical AR, the base plate data is feature point cloud data of the object, which is obtained by three-dimensional reconstruction of multiple consecutive photos of the object; If the selected XR application type is visual positioning AR, the base plate data consists of a space base plate white film, feature point cloud data, panoramic points and metadata information; wherein the metadata information includes: city coordinate offset, relative coordinate offset value, center point longitude and latitude and global rotation; If the selected XR application type is VR roaming, the base plate data is composed of a detailed city model and metadata information, wherein the metadata information includes: city coordinate offset, relative coordinate offset value and global rotation; If the selected XR application type is VR model viewing, the base plate data is an empty object; Step 34, placing the base plate data at the coordinate origin position of the three-dimensional view world coordinate system; Step 35: Set the scaling factor of the base plate data to achieve a ratio between the base plate space scale and the running engine space scale as a preset value.

5. A method for constructing a metaverse application according to claim 1, characterized in that: The step 4 specifically includes: Step 41: Obtain required digital content from the digital content warehouse through the classification and search functions provided by the XR application editor, and add it to the three-dimensional view; Step 42: Using the base plate data as a reference, determine the relative placement position, relative scaling size, relative rotation angle and model animation playback status of the digital content.

6. A method for constructing a metaverse application according to claim 1, characterized in that: In step 5, the sub-scene interaction instruction arrangement is completed through the XR application editor, specifically including: Step 51, select a digital content, and select an event according to the event type in the interaction design panel; Step 52: After selecting to trigger the event, determine the action instruction to be executed; Step 53, determine whether the action instruction requires the selection of an operation object; if the operation object does not need to be selected, enter the corresponding operation information to complete the interactive creation; if the operation object needs to be selected, select the operable object in the current scene through the drop-down box, enter the corresponding operation information to complete the interactive creation.

7. A method for constructing a metaverse application according to claim 1, characterized in that: The step 7 specifically includes: Step 71: Generate basic information required for XR application construction model according to application ID, application name, XR application type and startup cover information, and store it under the application information node of XR application construction model; Step 72: Create a scene storage object, and store the sub-scene name, sub-scene cover information and sequence information in the scene object; Step 73, traverse the data content under the environment directory node in the scene structure tree, including: various types of lights, background maps and HDR information, and store them under the environment node of the scene object; Step 74: store the baseboard data address and metadata information contained in the scene under the baseboard node of the scene object; Step 75, recursively traverse the digital content directory in the scene structure tree, store the directory as a directory object, store the name, address, position, parameter setting, animation and interaction instruction information of the digital content as a digital content object, and store it under the digital content node of the scene object according to the tree structure; Step 76, looping step 72 to step 75 until an array containing all sub-scene storage objects of the XR application is obtained, and the array is stored under the scene content node of the XR application construction model; Step 77: Convert all data of the XR application model into a string in JSON format through the JSON serialization algorithm. Step 78: Use a combination of multiple encryption algorithms to symmetrically encrypt the string, and write the encrypted data into a file in the form of a binary stream as a data file for building the XR application model; Step 79: Store the data file of the XR application construction model in the application construction model warehouse.

8. A method for constructing a metaverse application as claimed in claim 7, characterized in that: The step 8 specifically includes: Step 81: Obtain a data file download address in an Internet environment according to the storage location of the data file of the XR application construction model in the application construction model warehouse; Step 82: Determine the scheme value and the application page calling path according to the target operating platform type and the XR application type; Step 83: Generate a record in the database version record table, the record information includes: version ID, application ID, data file download address and application page call path; the version ID, data file download address and application page call path are one-to-one corresponding; Step 84: Generate an application invocation link based on the URL Scheme protocol format according to the scheme value, the application page invocation path, the version ID, and the application ID; Step 85: Generate an XR application QR code according to the application call link in accordance with the QR code encoding method.

9. A method for constructing a metaverse application as claimed in claim 7, characterized in that: The step 9 constructs the running engine, specifically including: Step 91: Build a runtime engine for native applications, develop application templates for all XR application types through Unity and AR engine technology, and package and output installation packages for different systems as the runtime engine for native applications; Step 92: constructing a running engine for the glasses application. Based on the application template, integrating the glasses SDK, and packaging and outputting a glasses version installation package as the running engine for the glasses application. Step 93: Build a running engine for the WeChat applet application, develop application templates for all XR application types through Visionkit and Three.js technologies, package and publish the WeChat applet application as the running engine for the WeChat applet application; Step 94: Build a runtime engine for the H5 page application, and develop application templates for all XR application types through the WebXR API and Three.js technology as the runtime engine for the H5 page application.

10. A method for constructing a metaverse application according to claim 7, characterized in that: The step 10 specifically includes: Step 101: If the terminal device uses a native application or a glasses application, the application template page of the running engine pre-installed on the local device is invoked through the application invocation link contained in the XR application QR code; Step 102: If the terminal device uses an H5 page application, the application invocation link included in the XR application QR code is used to invoke the browser and enter the application template page of the pre-installed runtime engine; Step 103: If the terminal device uses a WeChat applet application, the WeChat applet is invoked through the application invocation link contained in the XR application QR code and the application template page of the pre-installed runtime engine is entered; Step 104: parse the parameters carried by the application call link to obtain the application ID and version ID, and then query the server according to the application ID to obtain the application name, startup cover information and XR application type of the XR application; query the server according to the version ID to obtain the data file download address; Step 105: Render a UI interface according to the obtained application name and startup cover information to present a startup loading interface of the XR application; Step 106: Download the data file of the XR application construction model according to the data file download address, decrypt and read the content; Step 107: Initialize the XR engine and configure the XR engine data according to the target operating platform type, XR application type, scene and environment data. The browser environment is WebXR API, the WeChat applet environment is VisionKit, the Android native application and glasses application environment is AR Core, the iOS environment is AR Kit, and the Hongmeng system environment is AR Engine. Step 108: parse the data under the scene content node in the XR application construction model, traverse the scene list, parse the environment, base plate, and digital content node data under the scene content node in turn, set the 3D engine HDR, background map, and lighting through the environment node data; set the data and parameters corresponding to the XR engine and configuration engine through the base plate node data; download digital content from the server through the digital content node data, instantiate the digital content object by the 3D engine, and set the posture and double-sided rendering attributes. If it is a 3D model, the animation object contained in the model needs to be instantiated; according to the interactive instruction configuration, the digital content object is hooked up with the event; Step 109: Parse the program interface node data in the XR application construction model, draw the interface through the 3D engine, and hook up events; Step 1010: After the XR application is initialized, enter the XR application and use it.

Images