Child application processing method and apparatus, computer device, and storage medium

By analyzing and converting the predefined tags in the page structure file to extend the real-life interaction framework elements and component information, an extended real-life interaction page is generated, which solves the problem of high threshold for the development of sub-applications in the existing technology, and realizes the interaction between real and virtual scenes. Functions reduce development complexity and cost.

WO2024230324A1PCT designated stage expired Publication Date: 2024-11-14TENCENT TECHNOLOGY (SHENZHEN) CO LTD

Patent Information

Application Number
PCT/CN2024/082397
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-06
Filing Date
2024-03-19
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

In the prior art, the development threshold for extended real-life interactive sub-applications is relatively high, and it is necessary to have relevant knowledge of a third-party rendering engine, and users need to have more pre-emptive knowledge for custom development.

Method used

By obtaining the page structure file of the sub-app, parsing and converting predefined tags into elements and component information in the extended reality interaction framework, generating an extended reality interaction page, realizing the interaction between real scenes and virtual scenes, lowering the development threshold and improving flexibility.

Benefits of technology

The development of extended real-life interactive sub-applications can be realized without introducing third-party tools, which reduces the threshold for developers to extend real-life interactive sub-applications, provides the functions of interacting with real scenes and virtual scenes, and improves development efficiency and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024082397_14112024_PF_FP_ABST
    Figure CN2024082397_14112024_PF_FP_ABST
Patent Text Reader

Abstract

A child application processing method, comprising: obtaining a page structure file corresponding to a child application, the page structure file being used for describing the structure of an extended reality interaction page to be generated of the child application (S202); parsing the page structure file to obtain predefined tags in the page structure file, and respectively converting the predefined tags into elements in an extended reality interaction framework (S204); obtaining attribute information corresponding to the predefined tags from the page structure file (S206); for each predefined tag, converting the attribute information corresponding to the predefined tag into component information bound to the element into which the predefined tag is converted (S208); and generating an extended reality interaction page of the child application according to the elements in the extended reality interaction framework and the component information bound to the elements, the extended reality interaction page being used for displaying the interaction between a real scene and a virtual scene (S210).
Need to check novelty before this filing date? Find Prior Art

Description

Sub-application processing method, device, computer equipment and storage medium

[0001] Related applications

[0002] This application claims priority to Chinese patent application number 2023105046867, filed on May 6, 2023, entitled “Sub-application processing method, apparatus, computer device and storage medium,” the entire text of which is hereby incorporated by reference. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a sub-application processing method, apparatus, computer equipment, storage medium, and computer program product. Background Art

[0004] With the development of computer technology, more and more applications are providing convenience for users' work and life. Applications can also be divided into parent applications and child applications. Parent applications are applications that can run independently, while child applications are applications that can be used without downloading and installing, but they need to run on the parent application.

[0005] Currently, there is a sub-application development solution open to users, allowing them to develop customized sub-applications. For example, online merchants in the Internet of Things can develop exclusive sub-applications to provide services to consumers.

[0006] However, custom development of mini-programs currently requires a significant amount of prior knowledge. For example, developing sub-apps for extended reality interactions requires integrating third-party rendering engines with the sub-app's visual positioning capabilities. This requires developers to have relevant knowledge of third-party rendering engines, making sub-app development a relatively high barrier to entry.

[0007] Summary of the Invention

[0008] According to various embodiments provided in the present application, a sub-application processing method, apparatus, computer device, computer-readable storage medium, and computer program product are provided.

[0009] In one aspect, the present application provides a sub-application processing method, which is executed by a computer device, and the method includes:

[0010] Obtaining a page structure file corresponding to the sub-application, wherein the page structure file is used to describe the structure of the to-be-generated extended reality interaction page of the sub-application;

[0011] Parsing the page structure file to obtain predefined tags in the page structure file, and converting each predefined tag into an element in the extended reality interaction framework;

[0012] Acquire attribute information corresponding to each of the predefined tags from the page structure file;

[0013] For each of the predefined tags, converting the attribute information corresponding to the predefined tag into component information bound to the converted element of the predefined tag; and

[0014] An extended reality interaction page of the sub-application is generated according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements, and the extended reality interaction page is used to display the interaction between the real scene and the virtual scene.

[0015] On the other hand, the present application also provides a sub-application processing device. The device includes:

[0016] An acquisition module, configured to acquire a page structure file corresponding to a sub-application, wherein the page structure file is used to describe the structure of an extended reality interaction page to be generated for the sub-application;

[0017] An element generation module, configured to parse the page structure file, obtain predefined tags in the page structure file, and convert the predefined tags into elements in the extended reality interaction framework;

[0018] A component information generation module is configured to obtain attribute information corresponding to the predefined tags from the page structure file; for each predefined tag, convert the attribute information corresponding to the predefined tag into component information bound to the converted element of the predefined tag; and

[0019] A page generation module is used to generate an extended reality interaction page of the sub-application based on the elements in the extended reality interaction framework and the component information bound to the elements, wherein the extended reality interaction page is used to display the interaction between the real scene and the virtual scene.

[0020] On the other hand, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the method embodiments of the present application when executing the computer program.

[0021] In another aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of each method embodiment of the present application.

[0022] On the other hand, the present application further provides a computer program product, which includes a computer program that, when executed by a processor, performs the steps of each method embodiment of the present application.

[0023] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the disclosed drawings without any creative work.

[0025] FIG1 is an application environment diagram of a sub-application processing method according to an embodiment;

[0026] FIG2 is a schematic diagram of a flow chart of a method for processing a sub-application in one embodiment;

[0027] FIG3 is a schematic diagram of code for a custom component according to an embodiment;

[0028] FIG4 is a schematic diagram of a code for calling a customized component in another embodiment;

[0029] FIG5 is a schematic diagram of a process for converting attribute information corresponding to each predefined tag into component information bound to a corresponding element in one embodiment;

[0030] FIG6 is a schematic diagram of sending component configuration data to a system corresponding to a component in one embodiment;

[0031] FIG7 is a schematic diagram of a process for driving each system to process received component configuration data, forming corresponding page content, and obtaining an extended reality interaction page for a sub-application in one embodiment;

[0032] FIG8 is a schematic diagram of converting predefined tags in a page structure file into elements in an extended reality interaction framework in one embodiment;

[0033] FIG9 is an extended reality interaction page of a sub-application in one embodiment;

[0034] FIG10 is a schematic diagram of the architecture of a method for processing a sub-application in one embodiment;

[0035] FIG11 is a schematic diagram of writing a wxml file in one embodiment;

[0036] FIG12 is a schematic diagram of writing a js file in one embodiment;

[0037] FIG13 is a schematic diagram of an extended reality interaction effect achieved by combining a wxml file and a js file with an extended reality interaction framework in one embodiment;

[0038] FIG14 is a structural block diagram of a sub-application processing device according to an embodiment;

[0039] FIG15 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0041] The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. For example, it can be applied to the field of artificial intelligence (AI) technology, wherein artificial intelligence is the theory, method, technology and application system of using digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. The solution provided in the embodiments of the present application relates to a sub-application processing method of artificial intelligence, which is specifically described through the following embodiments.

[0042] The sub-application processing method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. In which, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other servers. The terminal 102 obtains the page structure file corresponding to the sub-application, and the page structure file is used to describe the structure of the extended reality interaction page to be generated for the sub-application. The terminal 102 sends the page structure file to the server 104. The server 104 parses the page structure file, obtains each predefined tag in the page structure file, and converts each predefined tag into an element in the extended reality interaction framework. The server 104 obtains the attribute information corresponding to each predefined tag from the page structure file, and for each predefined tag, converts the attribute information corresponding to each predefined tag into component information bound to the element converted by the predefined tag. The server 104 generates the extended reality interaction page of the sub-application based on each element in the extended reality interaction framework and the component information bound to each element. The sub-application runs on the parent application, which runs on terminal 102. When the sub-application runs, the extended reality interaction page is used to display the interaction between the real scene and the virtual scene. Terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices can include smart watches, smart bracelets, head-mounted devices, etc. Server 104 can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0043] In one embodiment, as shown in FIG2 , a sub-application processing method is provided. The method is described by taking the method as applied to FIG1 (the computer device may be the terminal or server in FIG1 ) as an example, and includes the following steps:

[0044] Step S202: Obtain a page structure file corresponding to the sub-application. The page structure file is used to describe the structure of the to-be-generated extended reality interaction page of the sub-application.

[0045] A sub-application is an application that cannot run independently and requires the help of other applications. A parent application is an application that can run independently and provide an operating environment for the sub-application.

[0046] The page structure file describes the structure of the sub-application's to-be-generated extended reality interaction page. This refers to the page that needs to be generated. This page enables interaction between real and virtual scenes. The page structure file can be a sub-application's wxml file. wxml (WeiXin Markup Language) files are a markup language designed by the sub-application framework. Combined with the sub-application's basic components and event system, they construct the sub-application's page structure.

[0047] In this embodiment, a user can log in to a development application running on a terminal using a development application account, or log in to a web version of the development application using a development application account. The terminal then displays the development application's development project management page. When a user triggers a development project creation operation, the terminal responds by creating a sub-application development project. The user then writes page structure files in the development environment corresponding to the sub-application development project. This user is referred to as a developer.

[0048] Step S204: parsing the page structure file to obtain predefined tags in the page structure file, and converting the predefined tags into elements in the extended reality interaction framework.

[0049] A predefined tag is a predefined tag, and each predefined tag may have the same label. The label of a predefined tag can be used to distinguish it from other tags. For example, a predefined tag is a tag marked with "XR-".

[0050] In this embodiment, the predefined label may be a user-defined label.

[0051] The extended reality interaction framework is the XR (Extended Reality, XR) interaction framework, which is used to build sub-applications of the extended reality interaction class.

[0052] XR (Extended Reality) is a brand-new set of technological concepts. The "X" stands for both expansion (Xtended) and the unknown variable (X), while the "R" stands for reality. It collectively refers to the three technologies of AR (augmented reality), VR (virtual reality), and MR (mixed reality). These technologies are applied together in different scenarios or industries, blending the virtual and real worlds in various combinations to create more creative possibilities. For example, in the video field, AR and MR technologies are used simultaneously to detect camera movement and lens angles, and then partially fill in the real environment with virtual images, thereby supplementing and expanding the scene. In other words, virtual space is superimposed on the real space, allowing limited space to be infinitely extended and achieving a perfect combination of virtual and real images. XR extended reality technology combines the real and virtual spaces.

[0053] Virtual Reality (VR) refers to a completely virtual personal experience.

[0054] Augmented Reality (AR) refers to the overlay of virtual images on top of reality. AR technology calculates the position and angle of camera images in real time and adds corresponding images, videos, and 3D (Three Dimensions) models. This technology embeds the virtual world on the screen into the real world and enables user interaction.

[0055] Mixed Reality (MR) refers to virtual images that can actively interact based on reality, which can enhance the realism of the user experience.

[0056] The Extended Reality Interaction Framework is an entity-component-system framework, comprising entities, components, and systems. These entities, components, and systems enable the implementation of Extended Reality Interaction sub-applications. These sub-applications can include, but are not limited to, virtual reality, augmented reality, and mixed reality sub-applications.

[0057] The extended reality interaction framework may be a framework including an ECS (Entity-Component-System) architecture, where an entity is an element.

[0058] An entity represents a basic unit in the XR Interaction Framework. An entity can be identified by the XR Interaction Framework and can also have several components mounted on it.

[0059] A component is a component mounted on an entity, which carries the properties of a certain part of the entity. This property can be a pure data structure without functions.

[0060] A system is a whole formed by the interconnected and interactive nature of its parts, each of which performs a specific function. It focuses on entities with specific attributes, specifically entities with specific components. It processes data related to these attributes.

[0061] Different systems have different functions. For example, the rendering system has the function of page rendering, and the animation system has the function of generating animation content.

[0062] Each basic unit in an extended reality interaction scene is an entity, and each entity is composed of one or more components. Each component only contains data representing its characteristics, that is, there are no methods in the component. For example: the component MoveComponent related to the movement function contains properties such as speed, position, and orientation. An entity that has a MoveComponent component means that the entity has the ability to move. The system is a tool used to process a collection of entities with one or more identical components. The system only has behavior, that is, there is no data in the system. In this example, the system that handles movement only focuses on entities with the ability to move. It will traverse all entities with MoveComponent components and update the entity's position based on the speed, position, orientation and other related component configuration data. Extended reality interaction scenes are such as game scenes.

[0063] There is a one-to-many relationship between entities and components. The capabilities of an entity depend on the components it has. By dynamically adding or removing components, the behavior of the entity can be changed while the extended reality interaction scene is running.

[0064] In the ECS architecture, entities act as the "carrier" of components, representing a collection of components. Entities do not contain data or business logic. To easily distinguish between entities, they are typically represented in code using data structures, such as an ID. All components that make up the same entity are associated with that entity's ID, making it clear which components belong to the entity. This allows access to associated components and their bound configuration data through the entity.

[0065] Because entities are collections of components, you can dynamically add or remove components from them at runtime. For example, if a player in a game is an entity and loses the ability to move due to something like falling unconscious, simply removing the movement component from the entity will render it immobile.

[0066] Components are bound to component configuration data, but the components themselves cannot change this bound configuration data; this must be done through the system. The bound configuration data describes a specific characteristic of an entity; for the component to function, it must be loaded onto the entity. The system truly contains the business logic. The object of interest to the system is a collection of one or more components, and the system uses this collection to capture the entity that contains all components in the collection. The system can also operate on these entities, for example, deleting the entity, adding or removing components from the entity, and changing the bound configuration data.

[0067] In this embodiment, the computer device parses the page structure file to obtain the tags in the page structure file. The tags may include predefined tags and tags other than the predefined tags. The predefined tags can be converted into the extended reality interaction framework, while the remaining tags cannot be converted into the extended reality interaction framework.

[0068] The computer device classifies the tags to select predefined tags. The computer device can obtain a preset extended reality interaction framework and convert each predefined tag into an element in the extended reality interaction framework. A predefined tag can be converted into an element in the extended reality interaction framework.

[0069] In this embodiment, the extended reality interaction framework includes a preconfigured element library containing multiple predefined tags and elements matching each predefined tag. After obtaining each predefined tag from the page structure file, the computer device searches the element library for each predefined tag in the page structure file, and then searches the element library for an element matching the predefined tag. The found element is converted into an element in the extended reality interaction framework as the element corresponding to the predefined tag. The found element is then added to the extended reality interaction framework.

[0070] Step S206: Acquire attribute information corresponding to each predefined tag from the page structure file.

[0071] Step S208 : for each predefined tag, convert the attribute information corresponding to the predefined tag into component information bound to the converted element of the predefined tag.

[0072] The attribute information corresponding to the predefined tag represents the attributes of the predefined tag in the extended reality interaction page. A predefined tag may correspond to one or more attribute information.

[0073] Component information represents the properties of an element in the XR interaction framework. An element can correspond to one or more components.

[0074] In this embodiment, for each predefined tag, the computer device obtains attribute information corresponding to the predefined tag from the page structure file. The computer device determines the element formed by the conversion of the predefined tag in the extended reality interaction framework, and converts the attribute information corresponding to the predefined tag into component information bound to the element formed by the conversion of the predefined tag.

[0075] In this embodiment, the attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag and the attribute data corresponding to the tag attribute. The component information bound to the element includes the component associated with the element and the component configuration data bound to the component.

[0076] Step S208: Generate an extended reality interaction page of the sub-application based on each element in the extended reality interaction framework and the component information bound to each element. The extended reality interaction page is used to display the interaction between the real scene and the virtual scene.

[0077] A virtual scene is a scene in which virtual objects move or perform interactive actions when a virtual interactive application is running. This virtual scene can be a simulation of the real world, a virtual environment that is partially simulated and partially fictional, or a purely fictional virtual environment. For example, a virtual scene can be any of two-dimensional, two-and-a-half-dimensional, and three-dimensional. Specifically, a virtual scene can be, but is not limited to, a mobile game scene, a PC game scene, an augmented reality scene, a virtual reality scene, or an interactive scene with virtual objects in mixed reality. Users can control the movement of virtual objects or perform interactive actions within this virtual scene.

[0078] A virtual object is an object that can be presented in a virtual scene and can be an object that moves. These virtual objects are not limited to virtual characters, virtual animals, virtual plants, or virtual environments. For example, a virtual object can be a person, animal, or plant displayed in a virtual scene. A virtual scene can include multiple virtual objects, each of which has its own shape and volume within the virtual scene and can occupy a portion of the space within the virtual scene.

[0079] In this implementation, a computer device generates an extended reality interaction page based on the elements in the extended reality interaction framework and the component information bound to each element, thereby obtaining a sub-application. This sub-application can then display the page content of the extended reality interaction page and is a sub-application of the extended reality interaction type. Users can run this sub-application within the parent application and interact between real and virtual scenes on the sub-application's extended reality interaction page.

[0080] The above-mentioned sub-application processing method, apparatus, computer device, computer-readable storage medium and computer program product obtain a page structure file corresponding to the sub-application. The obtained page structure file is used to describe the structure of the extended reality interaction page to be generated of the sub-application, so that the page layout of the extended reality interaction page to be generated can be known through the page structure file. The page structure file is parsed to obtain each predefined tag in the page structure file. The obtained predefined tags are predefined by the user and can be converted to specific tags used in the extended reality interaction framework, so that each predefined tag can be converted into an element in the extended reality interaction framework. The attribute information corresponding to each predefined tag is obtained from the page structure file. For each predefined tag, the attribute information corresponding to the predefined tag is converted into component information bound to the element converted by the predefined tag, so that the predefined tags and corresponding attribute information in the page structure file can be converted into elements and component information in the extended reality interaction framework, and the correspondence between each element and each component information in the extended reality interaction framework follows the correspondence between the predefined tags and attribute information in the page structure file. Based on the elements of the XR Interaction Framework and the component information bound to each element, an XR interaction page for the sub-application is generated. This allows the creation of XR interaction sub-applications within the XR Interaction Framework itself, eliminating the need for third-party tools. This simplifies the development of these sub-applications and effectively lowers the barrier to entry for developers. Furthermore, this XR interaction page provides interaction between real and virtual scenes, creating a human-computer interaction environment that combines real and virtual scenes for users through the sub-application.

[0081] In one embodiment, a user can create custom elements and components within the extended reality interaction framework and generate predefined tags that match each element. When creating a sub-application, a page structure file is generated using the predefined tags that match the custom elements, so that the computer device can automatically convert the predefined tags in the page structure file into matching elements within the extended reality interaction framework.

[0082] In this embodiment, users can create custom elements and components within the extended reality interaction framework, and generate predefined tags that match each element and tag attributes that match each component. When creating a sub-application, the predefined tags that match the custom elements and tag attributes that match the components are used to generate a page structure file, allowing the computer device to automatically convert the predefined tags in the page structure file into matching elements within the extended reality interaction framework and the tag attributes into matching components within the extended reality interaction framework.

[0083] In this embodiment, the extended reality interaction framework provides users with the function of generating elements and components, allowing users to customize the generation of required elements and components, thereby improving the flexibility and scalability of the extended reality interaction framework.

[0084] In one embodiment, the sub-application processing method not only lowers the entry threshold for sub-application development, but also improves the flexibility and scalability of sub-applications. Developers only need to use interfaces such as "registerComponent" to define components and elements by themselves, which is equivalent to customizing the tags and attributes in "wxml". As shown in Figure 3, it is the code for customizing components. Through the registration mechanism of XR-FRAMR, developers register the "auto-rotate" component, and use this component as an attribute in "wxml" to allow the 3D node corresponding to the mounted element to automatically rotate every frame. "auto-rotate-touchable-gltf" shows how to pre-combine the functions of multiple components into an element. This element integrates automatic rotation, interactivity and GlTF rendering capabilities for easy subsequent use. As shown in Figure 4, this is the code for calling customized components in the process of generating sub-applications.

[0085] In one embodiment, the attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag and the attribute data corresponding to the tag attribute; the component information bound to each element includes the component associated with each element and the component configuration data bound to the component.

[0086] For each predefined tag, the attribute information corresponding to the predefined tag is converted into component information bound to the element converted by the predefined tag, including:

[0087] For each predefined tag, a component associated with the element converted by the predefined tag is generated according to the tag attribute corresponding to the predefined tag; for each tag attribute, based on the attribute data corresponding to the tag attribute, component configuration data bound to the component matching the tag attribute is generated in the extended reality interaction framework.

[0088] A component is associated with an element, indicating that the component belongs to the element. For example, component a associated with element A means that component a belongs to element A. A component can belong to at least one element.

[0089] In this embodiment, the computer device converts each predefined tag into an element in the extended reality interaction framework. A predefined tag may correspond to one or more tag attributes in the page structure file.

[0090] The computer device can obtain each tag attribute corresponding to the predefined tag from the page structure file, and based on each tag attribute, generate each component associated with the element converted by the predefined tag in the extended reality interaction framework. Each tag attribute can generate one component.

[0091] For the attribute data corresponding to each tag attribute, the computer device generates component configuration data bound to the component matching the targeted tag attribute in the extended reality interaction framework based on the attribute data corresponding to the targeted tag attribute.

[0092] In this embodiment, each predefined tag in the page structure file is converted into an element in the extended reality interaction framework, the tag attributes corresponding to the predefined tags are converted into components associated with the elements, and the attribute data corresponding to the tag attributes are converted into component configuration data bound to the components, so that the various parts describing the structure of the extended reality interaction page can be converted into the extended reality interaction framework. Subsequently, the extended reality interaction page can be directly generated by using the elements, components and component configuration data through the extended reality interaction framework, so that the development of extended reality interactive sub-applications can be realized without the need for additional processing by third-party tools, thereby lowering the development threshold of extended reality interactive sub-applications.

[0093] In one embodiment, as shown in FIG5 , for each of the predefined tags, the attribute information corresponding to the predefined tag is converted into component information bound to the element converted by the predefined tag, including:

[0094] Step S502: Generate a property adding instruction for a component in the extended reality interaction framework based on the tag attributes corresponding to each predefined tag.

[0095] The attribute addition instruction is used to configure attributes for components in the extended reality interaction framework. Attributes in the extended reality interaction framework are represented by components, and the attributes of components in the extended reality interaction framework are represented by component configuration data.

[0096] In this embodiment, the computer device can generate an attribute adding instruction according to each predefined tag and the tag attribute corresponding to the predefined tag, and the attribute adding instruction instructs adding component configuration data for the component in the extended reality interaction framework.

[0097] In this embodiment, the computer device sends the attribute addition instruction to the template engine. The attribute addition instruction is used to instruct the template engine to convert each tag attribute into a component in the extended reality interaction framework. The template engine is an engine tool that combines the page structure and the data to be displayed to generate the extended reality interaction page. The template engine can run on the server or the client, and can be directly parsed into markup language on the server and then passed to the client after completion.

[0098] Step S504 : Based on the attribute adding instruction, components matching each tag attribute are obtained from the component library of the extended reality interaction framework.

[0099] In this embodiment, the component library of the extended reality interaction framework stores multiple components. The components in the component library can be user-defined components or common components in the extended reality interaction framework.

[0100] The computer device adds instructions based on the attribute, and matches each tag attribute with each component in the component library of the extended reality interaction framework to obtain a component that matches each tag attribute.

[0101] Step S506 : For each tag attribute, an association is established between the component that matches the tag attribute and the element converted from the predefined tag corresponding to the tag attribute.

[0102] In this embodiment, for each tag attribute, the computer device determines the component that matches the tag attribute, and determines the element converted by the predefined tag corresponding to the tag attribute, and associates the determined component and element to obtain component information bound to the element converted by each predefined tag.

[0103] Step S508 : Based on the attribute data corresponding to the targeted tag attribute, component configuration data bound to the component matching the targeted tag attribute is generated.

[0104] For each tag attribute, the computer device determines the attribute data corresponding to the tag attribute and determines the component that matches the tag attribute. The computer device uses the determined attribute data as the component configuration data of the determined component and binds the determined component and the component configuration data of the determined component.

[0105] In this embodiment, the template engine obtains the page structure file corresponding to the sub-application and parses the page structure file to obtain the predefined tags and the tag attributes corresponding to each predefined tag. The template engine generates an attribute addition instruction based on the tag attributes corresponding to each predefined tag. The attribute addition instruction is sent to the backend of the extended reality interaction framework, instructing the backend of the extended reality interaction framework to search for components that match each tag attribute and associate the found components with the corresponding elements.

[0106] In this embodiment, based on the tag attributes corresponding to each predefined tag, attribute addition instructions are generated for components in the extended reality interaction framework. Based on the attribute addition instructions, components matching each tag attribute are obtained from the component library of the extended reality interaction framework, thereby converting the tag attributes in the page structure file into corresponding components in the extended reality interaction framework. For each tag attribute, an association is established between the component matching the tag attribute and the element converted by the predefined tag corresponding to the tag attribute, so that the correspondence between the component and the element is consistent with the correspondence between the predefined tag and the tag attribute, thereby ensuring the accuracy of the converted data. In addition, based on the attribute data corresponding to the tag attribute, component configuration data bound to the component matching the tag attribute is generated. The attribute data corresponding to the tag attribute can be migrated to the component in the extended reality interaction framework, so that the binding relationship between the component and the component configuration data is consistent with the correspondence between the tag attribute and the attribute data, thereby ensuring the accuracy of the converted data.

[0107] In one embodiment, an extended reality interaction framework includes a plurality of systems, each system corresponding to at least one component, and each system is configured to process component configuration data of a corresponding component;

[0108] Generate the sub-application's extended reality interaction page based on the elements in the extended reality interaction framework and the component information bound to each element, including:

[0109] Determine the system corresponding to each component in the extended reality interaction framework; for each component, send the component configuration data bound to the targeted component to the system corresponding to the targeted component; drive each system to process the received component configuration data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0110] In this implementation, the extended reality interaction framework includes multiple systems, each of which is used to process component configuration data for a corresponding component. For example, the rendering system is used to process component configuration data related to rendering, the animation system is used to process component configuration data related to animation content, and the AR system is used to process component configuration data related to AR.

[0111] After the elements, tag attributes, and attribute information in the page structure file are converted into corresponding elements, components, and component configuration data, the corresponding system of each component is determined. A system can correspond to one or more components.

[0112] For each component, the computer device sends the component configuration data bound to the component to the corresponding system, driving each system to process the received component configuration data and generate the corresponding page content. Each page content forms the extended reality interaction page of the sub-application, thereby obtaining an extended reality interaction sub-application.

[0113] For example, as shown in Figure 6, the extended reality interaction framework includes systems A, B, C, and D. After converting the predefined tags, tag attributes corresponding to the predefined tags, and attribute data corresponding to the tag attributes in the page structure file into the elements, element-associated components, and component configuration data bound to the components in the extended reality interaction framework, the component a corresponding to system A is determined, and the component configuration data of component a is sent to system A. The component b corresponding to system B is determined, and the component configuration data of component b is sent to system B. The component c corresponding to system C is determined, and the component configuration data of component c is sent to system C. The component d corresponding to system D is determined, and the component configuration data of component b is sent to system d.

[0114] In this embodiment, the system focuses on the corresponding component configuration data, rather than the component configuration data bound to all components corresponding to an element. For example, in a game sub-application, there are multiple objects, each with corresponding skills, and each skill is configured with corresponding skill data. The extended reality interaction framework includes a skill system and a movement system, but the skill system only focuses on and processes the skill data of each object, and the movement system only focuses on and processes the displacement data of each object.

[0115] In this embodiment, the extended reality interaction framework integrates multiple systems, each corresponding to at least one component, enabling each system to process the component configuration data of the corresponding component. After converting each predefined tag, the tag attributes corresponding to the predefined tags, and the attribute data corresponding to the tag attributes in the page structure file into the configuration data of each element, the component associated with the element, and the component binding to the component in the extended reality interaction framework, the system corresponding to each component in the extended reality interaction framework is determined to accurately determine which component data needs to be sent to which system for processing. Thus, for each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component. Each system is driven to execute the processing of the received component configuration data, so that the corresponding page content is formed through the processing of the corresponding component configuration data by each system. This allows the construction of subroutines of the extended reality interaction class to be directly implemented in the extended reality interaction framework without the need for external tools, reducing the development cost of the subroutines of the extended reality interaction class and improving the efficiency of the subroutines of the extended reality interaction class.

[0116] In one embodiment, as shown in FIG7 , driving each system to process the received component configuration data and form corresponding page content to obtain an extended reality interaction page of the sub-application includes:

[0117] Step S702: Obtain the sub-reference code of each system in the extended reality interaction framework.

[0118] Step S704 : For each component, fill the component configuration data bound to the component into the sub-reference code of the system corresponding to the component to obtain the component code.

[0119] Among them, the sub-reference code is the code used to drive the system operation to implement the corresponding functions in the sub-application.

[0120] In this embodiment, the computer obtains the sub-reference code of each system in the extended reality interaction framework. For each component, the sub-reference code of the system corresponding to the component is adjusted according to the component configuration data bound to the component to obtain the component code.

[0121] In this embodiment, for each component, the computer device fills the component configuration data bound to the component into the sub-reference code of the system corresponding to the component to obtain the component code corresponding to the component.

[0122] Step S706: Drive each system to run its own component code respectively to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0123] Specifically, the computer device drives each system to run its own component code to form corresponding page content, and each page content forms an extended reality interaction page, thereby obtaining an extended reality interaction type sub-application.

[0124] In this embodiment, the extended reality interaction framework has a corresponding reference framework code. The extended reality interaction framework includes multiple systems, and the reference framework code includes a sub-reference code corresponding to each system.

[0125] In this embodiment, the sub-reference code for each system in the extended reality interaction framework is obtained, and each component configuration data is populated into the sub-reference code of the corresponding system, so that the component code can be automatically generated without the user having to write code. Each system is driven to run its own component code separately, forming the corresponding page content to obtain the extended reality interaction page of the sub-application, thereby building a user-defined extended reality interaction type sub-application, which can shorten the development cycle and improve development efficiency. Moreover, by adding elements and components and binding component configuration data, users with less knowledge of extended reality interaction type sub-application development can also develop various parts and functions of the sub-application, making the development of extended reality interaction type sub-applications simpler.

[0126] In one embodiment, the plurality of systems include at least a rendering system, and each component includes at least a component corresponding to the rendering system;

[0127] For each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component, including: obtaining page rendering data from the component configuration data bound to the component of the corresponding rendering system; and sending the page rendering data to the rendering system;

[0128] Driving each system to execute the received component configuration data to form corresponding page content to obtain the extended reality interaction page of the sub-application, including: controlling the rendering system to render the page according to the page rendering data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0129] Page rendering is the process of completing the layout and drawing of the requested page resources based on certain rules. Page resources include, but are not limited to, text, images, animations, videos, and audio.

[0130] The multiple systems in the extended reality interaction framework include a rendering system, and each component includes at least a component corresponding to the rendering system.

[0131] In this implementation, the computer device determines the system corresponding to each component in the extended reality interaction framework. If multiple systems include a rendering system, and the components of the extended reality interaction framework include a component corresponding to the rendering system, the computer device obtains page rendering data from the component configuration data bound to the component. The computer device then sends the page rendering data to the rendering system.

[0132] The computer device controls the rendering system to perform page rendering according to the page rendering data, forming corresponding page content to obtain the extended reality interaction page of the sub-application.

[0133] In this embodiment, each component includes at least a component corresponding to the rendering system, and the component configuration data bound to the component corresponding to the rendering system includes page rendering data. The computer device sends the page rendering data bound to the component corresponding to the rendering system to the rendering system.

[0134] In this embodiment, page rendering can be real-time page rendering. Real-time page rendering refers to rendering three-dimensional data into two-dimensional bitmaps based on graphics algorithms and displaying these bitmaps in real time. Its essence is the real-time calculation and output of image data, requiring the rendering and display of a picture in a short period of time, while also rendering and displaying the next picture.

[0135] In this embodiment, the rendering system is integrated into the extended reality interaction framework to obtain page rendering data from the component configuration data bound to the component corresponding to the rendering system in the extended reality interaction framework, and the page rendering data is sent to the rendering system in the extended reality interaction framework, so that the rendering system uses the page rendering data to perform page rendering and form corresponding page content. When developing sub-applications of the extended reality interaction class, there is no need to rely on a third-party rendering engine. Instead, page rendering can be quickly implemented directly through the rendering system that comes with the extended reality interaction framework. This makes it easy for developers who do not understand the relevant knowledge of third-party rendering engines to develop sub-applications of the extended reality interaction class, thereby lowering the threshold for developing sub-applications of the extended reality interaction class.

[0136] In one embodiment, the plurality of systems further include an animation system, and each component further includes a component corresponding to the animation system; the method further includes:

[0137] Obtain animation data from the component configuration data bound to the component of the corresponding animation system, and send the animation data to the animation system;

[0138] Control the rendering system to render the page according to the page rendering data, form corresponding page content, and obtain the extended reality interaction page of the sub-application, including:

[0139] The rendering system is controlled to perform page rendering according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data, forming page content including the animation content, so as to obtain an extended reality interaction page of the sub-application.

[0140] The animation data is data used to generate animation content in the extended reality interactive page, such as dynamic people, dynamic objects, etc.

[0141] Multiple systems in the extended reality interaction framework also include an animation system, and each component also includes a component corresponding to the animation system.

[0142] In this implementation, the computer device determines the system corresponding to each component in the extended reality interaction framework. When multiple systems include an animation system, and each component includes a component corresponding to the animation system, the computer device obtains animation data from the component configuration data bound to the component corresponding to the animation system and sends the animation data to the animation system.

[0143] The computer device controls the rendering system to render the page based on the page rendering data, and controls the animation system to generate animation content based on the animation data, thereby obtaining static page content and dynamic page content. The dynamic page content is the animation content. The static page content and the dynamic page content form the extended reality interaction page of the sub-application, thereby obtaining an extended reality interaction sub-application.

[0144] In this embodiment, each component includes at least a component corresponding to the animation system, and the component configuration data bound to the component corresponding to the animation system includes animation data. The computer device sends the animation data bound to the component corresponding to the animation system to the animation system.

[0145] In this embodiment, an animation system is also integrated into the extended reality interaction framework, and each component also includes a component corresponding to the animation system. Animation data is obtained from the component configuration data bound to the component corresponding to the animation system, and the animation data is sent to the animation system. The rendering system is controlled to render the page according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data to generate dynamic page content and static page content. Therefore, when developing sub-applications of the extended reality interaction class, there is no need to rely on third-party related engines. Page rendering and animation generation can be quickly achieved directly through the rendering system and animation system provided in the extended reality interaction framework. This allows developers to develop sub-applications of the extended reality interaction class without having relevant knowledge of third-party related engines, thereby lowering the threshold for developing sub-applications of the extended reality interaction class and making the development of sub-applications of the extended reality interaction class simpler.

[0146] In one embodiment, converting each predefined tag into an element in an extended reality interaction framework includes:

[0147] According to each predefined tag, an element creation instruction for the extended reality interaction framework is generated; based on the element creation instruction, elements matching each predefined tag are obtained from the element library of the extended reality interaction framework, and the elements matching each predefined tag are added to the extended reality interaction framework.

[0148] The element creation instruction converts the predefined tags in the page structure file into elements in the extended reality interaction framework. The element library stores multiple elements, which can be user-defined elements or common elements in the extended reality interaction framework.

[0149] In this embodiment, the computer device generates an element creation instruction for the extended reality interaction framework based on each predefined tag. The computer device determines the element library preset by the extended reality interaction framework and, based on the element creation instruction, matches each predefined tag with each element in the element library, obtains an element that matches each predefined tag, and then adds the obtained element to the extended reality interaction framework. As shown in Figure 8, the predefined tags in the page structure file are XR-Tag A, XR-Tag B, XR-Tag C, and XR-Tag D, which are converted into elements A, B, C, and D in the extended reality interaction framework, respectively.

[0150] In this embodiment, according to each predefined tag, an element creation instruction for the extended reality interaction framework is generated, and based on the element creation instruction, elements matching each predefined tag are respectively obtained from the element library of the extended reality interaction framework, and the obtained elements are added to the extended reality interaction framework. This can achieve accurate conversion between the predefined tags in the page structure file and the elements in the extended reality interaction framework, thereby generating an extended reality interaction page based on the elements in the extended reality interaction framework.

[0151] In one embodiment, generating an extended reality interaction page of a sub-application based on elements in an extended reality interaction framework and component information bound to the elements includes:

[0152] Obtain the reference frame code corresponding to the extended reality interaction framework; adjust the reference frame code based on the elements in the extended reality interaction framework and the component information bound to the elements to obtain the page frame code of the sub-application. The sub-application page frame code is used to generate the extended reality interaction page of the sub-application.

[0153] In this implementation, the extended reality interaction framework is preconfigured with reference framework code. This reference framework code is used to create sub-applications based on the extended reality interaction framework's structure. Based on the elements in the extended reality interaction framework and the component information bound to those elements, the computer device adjusts the corresponding reference elements and component information in the reference framework code to obtain the sub-application's page framework code. The computer device executes this page framework code to generate the extended reality interaction page, thereby obtaining the sub-application.

[0154] In this embodiment, the computer device fills the corresponding positions in the reference frame code according to the elements in the extended reality interaction framework and the component information bound to the elements to obtain the page frame code of the sub-application.

[0155] In this embodiment, the computer device may obtain a page style file that describes at least one of a local style or a global style for the sub-application's extended reality interaction page. The computer device may generate the sub-application's extended reality interaction page based on the page style file and the sub-application's page framework code.

[0156] In this embodiment, the extended reality interaction framework has a corresponding reference framework code. The extended reality interaction framework includes multiple systems, and the reference framework code includes a sub-reference code corresponding to each system. According to the elements in the extended reality interaction framework and the component information bound to the elements, the reference framework code is adjusted to obtain the page framework code of the sub-application, including:

[0157] For each component, the component configuration data bound to the component is filled into the sub-reference code of the system corresponding to the component to obtain the component code; each system is driven to run its own component code respectively to form the corresponding page content to obtain the extended reality interaction page of the sub-application.

[0158] In this embodiment, the reference framework code corresponding to the extended reality interaction framework is obtained. Based on the elements in the extended reality interaction framework and the component information bound to the elements, the reference framework code is adjusted to obtain the page framework code for the sub-application. This allows the automatic generation of user-defined sub-applications of extended reality interaction types without the need for user code, shortening the development cycle and improving development efficiency. Furthermore, by adding elements and components and binding component configuration data, users without code skills can also develop their own sub-applications, making sub-application development easier.

[0159] In one embodiment, the method further comprises:

[0160] Call the camera in the extended reality interaction page to capture the real scene through the camera; obtain the pre-stored 3D model material, and generate a 3D virtual object based on the 3D model material; synthesize the real scene and the 3D virtual object, and display the enhanced scene picture obtained by the synthesis on the extended reality interaction page.

[0161] In this implementation, after a sub-application of the extended reality interaction class is generated, the sub-application is run in the parent application.

[0162] The parent application provides an operating environment for the sub-application. After the sub-application is generated, the user can trigger the parent application identifier. In response to the trigger operation of the parent application, the terminal jumps to the parent application and runs the sub-application in the operating environment provided by the parent application.

[0163] The extended reality interaction page is displayed in the sub-application, which provides an interface for calling the terminal's camera. Users can use this interface to call the camera in the extended reality interaction page and use the camera to capture the real scene.

[0164] The computer device pre-stores 3D model material, which is used to generate a corresponding 3D virtual object. The 3D virtual object can be a person or object in a 3D space. The computer device generates the corresponding 3D virtual object using the 3D model material.

[0165] The computer device synthesizes the real scene and the three-dimensional virtual object to obtain an augmented scene image. The computer device outputs the augmented scene image to the extended reality interaction page of the sub-application to display the augmented scene image on the extended reality interaction page.

[0166] In this embodiment, a camera call function is provided within the extended reality interaction page, allowing users to capture real scenes through the camera. Pre-stored 3D model materials are retrieved, and after generating 3D virtual objects based on the 3D model materials, the real scene and the 3D virtual objects are synthesized to obtain an augmented scene image. This augmented scene image can be displayed within a sub-application, allowing the sub-application to present a scene image that combines the real and virtual. This provides users with a function that combines the real and virtual through the extended reality interaction sub-application, expanding the sub-application's interactive capabilities.

[0167] In one embodiment, a real scene and a virtual object are synthesized, and an enhanced scene obtained by the synthesis is displayed on an extended reality interaction page, including:

[0168] The camera's posture information is obtained, and based on the camera's posture information, a simulated scene that simulates the real scene is generated; the three-dimensional virtual object is integrated into the simulated scene to obtain an enhanced scene image displayed on the extended reality interaction page.

[0169] In this embodiment, the computer device obtains camera pose information, which represents the camera's three-dimensional spatial data. Based on the camera pose information, the computer device generates a virtual scene that simulates the real scene. This virtual scene is a simulation of the real scene. The computer device integrates the generated three-dimensional virtual object into the simulated scene, resulting in an augmented scene image that includes the simulated scene and the three-dimensional virtual scene. This augmented scene image is displayed on the sub-application's extended reality interaction page.

[0170] In this embodiment, the posture information of the camera is obtained, and a virtual scene matching the real scene is generated based on the posture information of the camera to integrate the three-dimensional virtual scene into the virtual scene, thereby obtaining an enhanced scene picture displayed on the extended reality interaction page, so that virtual content can be superimposed in real time on the real scene captured by the camera, seamlessly combining the virtual content with the real scene, and enhancing the visual effect of virtual reality interaction.

[0171] In this embodiment, a sub-application processing method is provided, which is applied to a computer device and includes:

[0172] Get the page structure file corresponding to the sub-application. The page structure file is used to describe the structure of the extended reality interaction page to be generated for the sub-application.

[0173] The page structure file is parsed to obtain each predefined tag in the page structure file; and an element creation instruction for the extended reality interaction framework is generated according to each predefined tag.

[0174] Based on the element creation instruction, elements matching each predefined tag are obtained from the element library of the extended reality interaction framework.

[0175] From the page structure file, obtain the tag attributes corresponding to each predefined tag and the attribute data corresponding to each tag attribute.

[0176] Based on the tag attributes corresponding to each predefined tag, attribute addition instructions are generated for components in the extended reality interaction framework. Based on the attribute addition instructions, components matching each tag attribute are retrieved from the extended reality interaction framework's component library. For each tag attribute, an association is established between the component matching the targeted tag attribute and the element converted from the predefined tag corresponding to the targeted tag attribute. Based on the attribute data corresponding to the targeted tag attribute, the extended reality interaction framework generates component configuration data bound to the component matching the targeted tag attribute.

[0177] In the extended reality interaction framework, identify the system that corresponds to each component.

[0178] For each component, the component configuration data bound to the component is sent to the system corresponding to the component.

[0179] Obtain the sub-reference code for each system in the extended reality interaction framework. For each component, populate the component configuration data bound to the target component into the sub-reference code of the system corresponding to the target component to obtain the component code. Drive each system to run its own component code, generate the corresponding page content, and obtain the extended reality interaction page of the sub-application; the sub-application runs on the parent application.

[0180] Furthermore, when multiple systems include a rendering system and an animation system, and each component includes a component corresponding to the rendering system and a component corresponding to the animation system, page rendering data is obtained from the component configuration data bound to the component, and the page rendering data is sent to the rendering system; animation data is obtained from the component configuration data bound to the component corresponding to the animation system, and the animation data is sent to the animation system; the rendering system is controlled to render the page according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data, forming an extended reality interaction page of the sub-application, and obtaining a sub-application of the extended reality interaction class.

[0181] After generating the sub-application, call the camera in the extended reality interaction page to capture the real scene through the camera; obtain the pre-stored three-dimensional model material, and generate a three-dimensional virtual object based on the three-dimensional model material; synthesize the real scene and the three-dimensional virtual object, and display the enhanced scene picture obtained by the synthesis on the extended reality interaction page.

[0182] In this embodiment, by obtaining the page structure file corresponding to the sub-application, and the page structure file is used to describe the structure of the extended reality interaction page to be generated, the page layout of the sub-application page to be generated can be known through the page structure file. The page structure file is parsed to obtain each predefined tag in the page structure file. The obtained predefined tags are predefined tags that can be converted into elements in the extended reality interaction framework. Each predefined tag is converted into an element in the extended reality interaction framework, the tag attributes corresponding to the predefined tags are converted into components associated with the elements, and the attribute data corresponding to the tag attributes are converted into component configuration data bound to the components, so that the various parts describing the extended reality interaction page structure can be converted into the extended reality interaction framework, so that the extended reality interaction page can be directly generated using the elements, components and component configuration data through the extended reality interaction framework.

[0183] The extended reality interaction framework includes multiple systems, each corresponding to at least one component, so that each system can process the component configuration data of the corresponding component. After converting each predefined tag in the page structure file, the tag attributes corresponding to the predefined tags, and the attribute data corresponding to the tag attributes into the configuration data of each element, the component associated with the element, and the component binding to the component in the extended reality interaction framework, the system corresponding to each component in the extended reality interaction framework is determined to accurately determine which components need to be sent to which system for processing. Thus, for each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component. Each system is driven to execute the processing of the received component configuration data, so that the corresponding page content is formed through the processing of the corresponding component configuration data by each system. This allows the construction of subroutines of the extended reality interaction class to be directly implemented in the extended reality interaction framework without the need for external tools, thereby reducing the development threshold and development cost of subroutines of the extended reality interaction class and improving the efficiency of subroutines of the extended reality interaction class.

[0184] It's understood that this sub-application generation method is applicable to generating any extended reality interactive sub-application, such as big data sub-applications, artificial intelligence sub-applications, and IoT sub-applications. This sub-application generation method is applicable to both mobile and web applications, allowing users to develop sub-applications using the mobile or web development application. Figure 9 shows an extended reality interactive page generated by a user using this sub-application generation method. This extended reality interactive page can be used for face recognition.

[0185] In one embodiment, an application scenario of a sub-application processing method is provided, which is specifically applied to the development scenario of sub-applications of the extended reality interaction class. The sub-application of the extended reality interaction class is the XR applet, and the extended reality interaction framework includes the ECS framework. As shown in Figure 10, in this embodiment, the development of the XR applet is divided into three parts. The first part is to parse the page structure file of the applet into structured data and instructions. The page structure file is wxml. The second part is to convert the structured data into elements and components of the extended reality interaction framework. The ECS framework is the ECS framework, and the elements are entities in the ECS framework. The third part is the logic driven by each system and finally rendered. The specific processing process is as follows:

[0186] The first part is the top portion of the architecture diagram. The Extended Reality Interaction Framework pre-defines a special class of native Mini Program tags, known as predefined tags, that begin with "xr-." During parsing, the template engine first retrieves the code snippet from the page structure file (wxml). It then analyzes and categorizes the tags within the wxml file to separate tags beginning with "xr-" from other common UI tags.

[0187] Generate a call instruction similar to "createElement" according to each predefined tag, that is, an element creation instruction. Determine the tag attribute corresponding to each predefined tag in the wxml file, and generate a call instruction similar to "addAttribute" according to the tag attribute corresponding to each predefined tag, that is, an attribute addition instruction. The template engine forwards the element creation instruction and attribute addition instruction to the "xr-frame" backend of the ECS framework. This extended reality interaction framework is specially implemented for the template engine. After receiving these instructions, the "xr-frame" backend will automatically create elements corresponding to the predefined tags in the ECS framework, as well as components that match the tag attributes. Among them, the mini-program tags correspond to the elements in ECS, and the mini-program attributes correspond to the components in ECS.

[0188] The second section is the middle of the architecture diagram. In the "xr-frame" backend, developers can register elements and components using methods such as "registerElement" and "registerComponent" to customize components and elements within the ECS framework. When the "xr-frame" backend receives instructions for creating elements and adding attributes, it uses a table lookup to locate the elements and components indicated by the template engine instructions. For example, the predefined tag "xr-camera" corresponds to the "Camera" element. After locating these elements and components, the ECS framework further parses the attribute data passed by the template engine, which can be of string type. The data parser converts these attribute data into corresponding ECS ​​data types, such as "number" and "array." This generates the corresponding component configuration data for each component, triggering the component lifecycle methods "onAdd" and "onUpdate" to update the component configuration data and bind it to the component. Once the component is ready, it is processed by the following systems.

[0189] The third part is the lower half of the architecture diagram. Each system in the ECS framework receives the component configuration data generated in the second step to drive the entire logic loop. The ECS framework mainly uses a callback for each frame to drive all systems mounted on the scene. Among them, the rendering system includes a programmable rendering pipeline, and the developer decides how to cull the scene, assemble the rendering queue, and render. The resource system is responsible for managing the loading of all resources, the animation system drives all frame animations and model animations in the scene, and the AR system drives the camera image rendering of the visual positioning tool (VisionKit) and automatic matching of identification points. It can also include logic systems, physical systems, particle systems, etc., and can also include other systems.

[0190] The rendering system includes a graphics backend (Web Graphics Library, WebGL), which is a 3D graphics protocol.

[0191] The asset system includes custom loaders, allowing developers to customize the loading of Graphics Language Transmission Format (GLTF), environment data (EnvData), and various textures. The asset system also includes a command-line tool, xr-frame-cli, which integrates environment data generation and glTF optimization.

[0192] In addition, each system has its own code. After each system receives the component configuration data of the corresponding component, it fills the component configuration data of the corresponding component into the code and runs it. Then, after calculation processing, the calculation results are updated to the component, so that the entity has specific functions and can perform specific operations.

[0193] Through the sub-application processing method of this embodiment, the cost for developers to develop 3D applications and XR applications in mini-programs is greatly reduced, while also providing developers with a better experience.

[0194] In this embodiment, the main steps for developers to develop 3D mini-programs and XR mini-programs using the sub-application processing method are as follows:

[0195] Create a small program component and set the third-party renderer "renderer" to "xr-frame" in the "json" configuration.

[0196] As shown in Figure 11, open the page structure file wxml and write code using markup language like a normal applet component; you can also write some javascript code as needed, as shown in Figure 12.

[0197] Run the debugger on a real device.

[0198] For developers, the entire development process is almost the same as the process of developing ordinary mini-programs. However, since the tags written are predefined xr tags, through the wxml file shown in Figure 11 and the js file shown in Figure 12, a three-dimensional effect that integrates three-dimensional virtual objects and real scenes can be presented as shown in Figure 13.

[0199] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0200] Based on the same inventive concept, the embodiments of the present application also provide a sub-application processing device for implementing the sub-application processing method involved above. The implementation solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations in the one or more sub-application processing device embodiments provided below can be found in the above-mentioned limitations on the sub-application processing method, and will not be repeated here.

[0201] In one embodiment, as shown in FIG14 , a sub-application processing device is provided, including: an acquisition module 1402 , an element generation module 1404 , a component information generation module 1406 , and a page generation module 1408 , wherein:

[0202] The acquisition module 1402 is used to acquire a page structure file corresponding to the sub-application, where the page structure file is used to describe the structure of the to-be-generated extended reality interaction page of the sub-application.

[0203] The element generation module 1404 is used to parse the page structure file, obtain predefined tags in the page structure file, and convert the predefined tags into elements in the extended reality interaction framework.

[0204] The component information generation module 1406 is used to obtain attribute information corresponding to predefined tags from the page structure file; for each predefined tag, convert the attribute information corresponding to the predefined tag into component information bound to the converted element of the predefined tag.

[0205] The page generation module 1408 is used to generate an extended reality interaction page for the sub-application based on the elements in the extended reality interaction framework and the component information bound to the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0206] In this embodiment, by obtaining the page structure file corresponding to the sub-application, the obtained page structure file is used to describe the structure of the extended reality interaction page to be generated of the sub-application, so that the page layout of the sub-application to be generated can be known through the page structure file. The page structure file is parsed to obtain each predefined tag in the page structure file. The obtained predefined tags are predefined by the user and can be converted into tags used in the extended reality interaction framework, so that each predefined tag can be converted into an element in the extended reality interaction framework. The attribute information corresponding to each predefined tag is obtained from the page structure file. For each predefined tag, the attribute information corresponding to the predefined tag is converted into component information bound to the element converted by the predefined tag, so that the predefined tags and corresponding attribute information in the page structure file can be converted into elements and component information in the extended reality interaction framework, and the correspondence between each element and each component information in the extended reality interaction framework follows the direct correspondence between the predefined tags and attribute information in the page structure file. Based on the elements of the XR Interaction Framework and the component information bound to each element, an XR interaction page for the sub-application is generated. This allows the creation of XR interactive sub-applications within the XR Interaction Framework itself, without the need for third-party tools. This simplifies the development of XR interactive sub-applications and effectively lowers the barrier to entry for developers. Furthermore, this XR interaction page provides interaction between real and virtual scenes, allowing users to experience a human-computer interaction environment that combines real and virtual scenes through the sub-application.

[0207] In one embodiment, the attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag and the attribute data corresponding to the tag attribute; the component information bound to each element includes the component associated with each element and the component configuration data bound to the component.

[0208] In this embodiment, each predefined tag in the page structure file is converted into an element in the extended reality interaction framework, the tag attributes corresponding to the predefined tags are converted into components associated with the elements, and the attribute data corresponding to the tag attributes are converted into component configuration data bound to the components, so that the various parts describing the structure of the extended reality interaction page can be converted into the extended reality interaction framework, so that the extended reality interaction page can be directly generated using the elements, components and component configuration data through the extended reality interaction framework, so that the development of extended reality interactive sub-applications can be realized without the need for additional third-party processing, thereby lowering the development threshold of extended reality interactive sub-applications.

[0209] In one embodiment, the component information generation module 1406 is further used to generate attribute addition instructions for components in the extended reality interaction framework based on the tag attributes corresponding to each predefined tag; based on the attribute addition instructions, obtain components that match each tag attribute from the component library of the extended reality interaction framework; for each tag attribute, establish an association between the component that matches the targeted tag attribute and the element converted by the predefined tag corresponding to the targeted tag attribute; based on the attribute data corresponding to the targeted tag attribute, generate component configuration data bound to the component that matches the targeted tag attribute.

[0210] In this embodiment, based on the tag attributes corresponding to each predefined tag, attribute adding instructions are generated for components in the extended reality interaction framework, so that based on the attribute adding instructions, components matching each tag attribute are respectively obtained from the component library of the extended reality interaction framework, so that the tag attributes in the page structure file can be converted into corresponding components in the extended reality interaction framework. For each tag attribute, an association is established between the component matching the tag attribute and the element converted from the corresponding predefined tag, so that the correspondence between the component and the element is consistent with the correspondence between the predefined tag and the tag attribute, thereby ensuring the accuracy of the converted data.

[0211] In one embodiment, an extended reality interaction framework includes a plurality of systems, each system corresponding to at least one component, and each system is configured to process component configuration data of a corresponding component;

[0212] The page generation module 1408 is also used to determine the system corresponding to each component in the extended reality interaction framework; for each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component; and each system is driven to process the received component configuration data to form the corresponding page content to obtain the extended reality interaction page of the sub-application.

[0213] In this embodiment, the extended reality interaction framework includes multiple systems, each corresponding to at least one component, and each system is capable of processing component configuration data for the corresponding component. After converting each predefined tag, the tag attributes corresponding to the predefined tags, and the attribute data corresponding to the tag attributes in the page structure file into each element, the component associated with the element, and the component configuration data bound to the component in the extended reality interaction framework, the system corresponding to each component in the extended reality interaction framework is determined to accurately determine which components need to be sent to which system for processing. Thus, for each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component. Each system is driven to execute the processing of the received component configuration data, so that the corresponding page content is formed through the processing of the corresponding component configuration data by each system. This allows the construction of subroutines of the extended reality interaction class to be directly implemented in the extended reality interaction framework without the need for external tools, reducing the development cost of the subroutines of the extended reality interaction class and improving the efficiency of the subroutines of the extended reality interaction class.

[0214] In one embodiment, the page generation module 1408 is also used to obtain the sub-reference code of each system in the extended reality interaction framework; for each component, the component configuration data bound to the component is filled into the sub-reference code of the system corresponding to the component to obtain the component code; and each system is driven to run its own component code respectively to form the corresponding page content to obtain the extended reality interaction page of the sub-application.

[0215] In this embodiment, the sub-reference code of each system in the extended reality interaction framework is obtained. For each component, the component configuration data bound to the target component is filled into the sub-reference code of the system corresponding to the target component to obtain the component code, so that the corresponding component code can be automatically generated without the user having to write code. Each system is driven to run its own component code separately to form the corresponding page content to obtain the extended reality interaction page of the sub-application, thereby building a user-defined extended reality interaction type sub-application, which can shorten the development cycle and improve development efficiency. Moreover, by adding elements and components and binding component configuration data, users who do not have a deep understanding of the development of extended reality interaction type sub-applications can also complete the development of various parts and functions of the sub-application, making the development of extended reality interaction type sub-applications simpler.

[0216] In one embodiment, the plurality of systems include a rendering system, and each component includes at least a component corresponding to the rendering system;

[0217] The page generation module 1408 is also used to obtain page rendering data from the component configuration data bound to the components of the corresponding rendering system; send the page rendering data to the rendering system; control the rendering system to perform page rendering according to the page rendering data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0218] In this embodiment, page rendering data is obtained from the component configuration data bound to the component of the corresponding rendering system, and the page rendering data is sent to the rendering system. The rendering system is controlled to render the page according to the page rendering data to form corresponding page content. When developing sub-applications of the extended reality interaction class, there is no need to rely on a third-party rendering engine. The page rendering can be performed directly through the rendering system provided in the extended reality interaction framework. This makes it easy for developers who do not have relevant knowledge of third-party rendering engines to develop sub-applications of the extended reality interaction class, thereby lowering the threshold for developing sub-applications of the extended reality interaction class.

[0219] In one embodiment, multiple systems also include an animation system, and each component also includes a component of the corresponding animation system; the page generation module 1408 is also used to obtain animation data from the component configuration data bound to the component of the corresponding animation system, and send the animation data to the animation system; control the rendering system to render the page according to the page rendering data, and control the animation system to generate animation content according to the animation data, forming page content including animation content, so as to obtain the extended reality interaction page of the sub-application.

[0220] In this embodiment, multiple systems also include an animation system, and each component also includes a component of the corresponding animation system. Animation data is obtained from the component configuration data bound to the component of the corresponding animation system, and the animation data is sent to the animation system. The rendering system is controlled to render the page according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data to generate dynamic page content and static page content. Therefore, when developing sub-applications of the extended reality interaction type, there is no need to rely on third-party related engines. Page rendering and animation generation can be performed directly through the rendering system and animation system provided in the extended reality interaction framework. Therefore, developers do not need to have relevant knowledge of third-party related engines to develop sub-applications of the extended reality interaction type, thereby lowering the threshold for developing sub-applications of the extended reality interaction type.

[0221] In one embodiment, the element generation module 1404 is further configured to generate element creation instructions for the extended reality interaction framework based on each predefined tag; and based on the element creation instructions, obtain elements matching each predefined tag from the element library of the extended reality interaction framework.

[0222] In this embodiment, element creation instructions for the extended reality interaction framework are generated according to each predefined tag, so that elements matching each predefined tag are obtained from the element library of the extended reality interaction framework based on the element creation instructions, so that each predefined tag in the page structure file can be accurately converted into an element in the extended reality interaction framework.

[0223] In one embodiment, the page generation module 1408 is also used to obtain the reference frame code corresponding to the extended reality interaction framework; adjust the reference frame code according to the elements in the extended reality interaction framework and the component information bound to the elements to obtain the page frame code of the sub-application; the page frame code of the sub-application is used to generate the extended reality interaction page of the sub-application.

[0224] In this embodiment, the reference framework code corresponding to the extended reality interaction framework is obtained. Based on the elements in the extended reality interaction framework and the component information bound to the elements, the reference framework code is adjusted to obtain the page framework code for the sub-application. This allows the automatic generation of user-defined sub-applications of extended reality interaction types without the need for user code, shortening the development cycle and improving development efficiency. Furthermore, by adding elements and components and binding component configuration data, users without code skills can also develop their own sub-applications, making sub-application development easier.

[0225] In one embodiment, the device also includes an interaction module; the interaction module is used to call a camera in the extended reality interaction page to capture the real scene through the camera; obtain pre-stored three-dimensional model materials, generate a three-dimensional virtual object based on the three-dimensional model materials; synthesize the real scene with the three-dimensional virtual object, and display the synthesized enhanced scene picture on the extended reality interaction page.

[0226] In this embodiment, a camera is called in the extended reality interaction page, and the real scene is captured by the camera to obtain pre-stored three-dimensional model materials. A three-dimensional virtual object is generated based on the three-dimensional model materials, and the real scene and the three-dimensional virtual object are synthesized so that the synthesized enhanced scene picture can be displayed in the sub-application. The sub-application can present a scene picture that combines the real and virtual, thereby providing users with a function of combining the real environment and the virtual environment through the extended reality interaction sub-application, thereby expanding the interactive capabilities of the sub-application.

[0227] In one embodiment, the interaction module is used to obtain the posture information of the camera, generate a virtual scene that matches the real scene based on the posture information of the camera, and integrate the three-dimensional virtual scene into the virtual scene to obtain an enhanced scene picture displayed on the extended reality interaction page.

[0228] In this embodiment, the posture information of the camera is obtained, and a virtual scene matching the real scene is generated based on the posture information of the camera, so as to integrate the three-dimensional virtual scene into the virtual scene, and obtain an enhanced scene picture displayed on the extended reality interaction page, so that virtual content can be superimposed on the real scene of the camera in real time, and the virtual content can be seamlessly combined with the real environment to form a visual effect that is indistinguishable from the real thing.

[0229] Each module in the above-mentioned sub-application processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0230] In one embodiment, a computer device is provided, which may be a terminal or a server. Taking a terminal as an example, its internal structure diagram may be shown in Figure 15 . The computer device includes a processor, memory, an input interface, an output interface, a communication interface, a display unit, and an input device. The processor, memory, input interface, and output interface are connected via a system bus, while the communication interface, display unit, and input device are connected to the system bus via the input interface and output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input interface and output interface of the computer device are used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal via wired or wireless communication. The wireless communication may be implemented via Wi-Fi, a mobile cellular network, NFC (near field communication), or other technologies. When executed by the processor, the computer program implements a sub-application processing method. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0231] Those skilled in the art will understand that the structure shown in FIG15 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0232] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0233] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0234] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0235] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0236] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0237] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0238] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A sub-application processing method, executed by a computer device, the method comprising: Obtaining a page structure file corresponding to the sub-application, wherein the page structure file is used to describe the structure of the to-be-generated extended reality interaction page of the sub-application; Parsing the page structure file to obtain each predefined tag in the page structure file, and converting each predefined tag into an element in the extended reality interaction framework; Acquire attribute information corresponding to each of the predefined tags from the page structure file; For each of the predefined tags, converting the attribute information corresponding to the predefined tag into component information bound to the element converted by the predefined tag; and According to each of the elements in the extended reality interaction framework and the component information bound to each of the elements, an extended reality interaction page of the sub-application is generated, and the extended reality interaction page is used to display the interaction between the real scene and the virtual scene.

2. According to the method described in claim 1, the attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag, and the attribute data corresponding to the tag attribute; the component information bound to each of the elements includes the components associated with each of the elements, and the component configuration data bound to the components.

3. The method according to claim 1 or 2, wherein for each of the predefined tags, converting the attribute information corresponding to the predefined tag into component information bound to the element converted by the predefined tag comprises: Based on the tag attributes corresponding to each of the predefined tags, generating a property adding instruction for a component in the extended reality interaction framework; Based on the attribute adding instruction, respectively obtaining components matching each of the tag attributes from the component library of the extended reality interaction framework; For each of the tag attributes, an association is established between a component matched with the tag attribute and an element converted by a predefined tag corresponding to the tag attribute; and Based on the attribute data corresponding to the targeted tag attribute, component configuration data bound to the component matching the targeted tag attribute is generated.

4. The method according to any one of claims 1 to 3, wherein the extended reality interaction framework comprises a plurality of systems, each of the systems corresponds to at least one component, and each of the systems is used to process component configuration data of the corresponding component; The generating of the extended reality interaction page of the sub-application according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements includes: Determine a system corresponding to each of the components in the extended reality interaction framework; For each of the components, the component configuration data bound to the targeted component is sent to a system corresponding to the targeted component; and The systems are driven to process the received component configuration data to form corresponding page contents, so as to obtain an extended reality interaction page of the sub-application.

5. The method according to any one of claims 1 to 4, wherein the driving each of the systems to process the received component configuration data and generate corresponding page content to obtain the extended reality interaction page of the sub-application comprises: Obtaining respective sub-reference codes of the systems in the extended reality interaction framework; For each of the components, fill the component configuration data bound to the component into the sub-reference code of the system corresponding to the component to obtain the component code; and The systems are driven to respectively run their respective component codes to form corresponding page contents, so as to obtain the extended reality interaction page of the sub-application.

6. The method according to any one of claims 1 to 5, wherein the plurality of systems at least include a rendering system, and each of the components at least includes a component corresponding to the rendering system; For each of the components, sending the component configuration data bound to the component to the system corresponding to the component includes: Obtaining page rendering data from component configuration data bound to a component corresponding to the rendering system; Sending the page rendering data to the rendering system; and The driving each of the systems to execute the received component configuration data to form corresponding page content to obtain the extended reality interaction page of the sub-application includes: The rendering system is controlled to perform page rendering according to the page rendering data to form corresponding page content to obtain an extended reality interaction page of the sub-application.

7. The method according to any one of claims 1 to 6, wherein the plurality of systems further include an animation system, and each of the components further includes a component corresponding to the animation system; the method further includes: Acquire animation data from component configuration data bound to a component corresponding to the animation system, and send the animation data to the animation system; and The controlling the rendering system to perform page rendering according to the page rendering data to form corresponding page content to obtain an extended reality interaction page of the sub-application includes: The rendering system is controlled to perform page rendering according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data to form page content including the animation content, so as to obtain an extended reality interaction page of the sub-application.

8. The method according to any one of claims 1 to 7, wherein converting each of the predefined tags into an element in an extended reality interaction framework comprises: Generating, according to each of the predefined tags, an element creation instruction for the extended reality interaction framework; and Based on the element creation instruction, elements matching each of the predefined tags are respectively obtained from the element library of the extended reality interaction framework, and the elements matching each of the predefined tags are added to the extended reality interaction framework.

9. The method according to any one of claims 1 to 8, wherein generating an extended reality interaction page of a sub-application according to the element in the extended reality interaction framework and the component information bound to the element comprises: Obtaining a reference frame code corresponding to the extended reality interaction frame; and According to the elements in the extended reality interaction framework and the component information bound to the elements, the reference framework code is adjusted to obtain the page framework code of the sub-application, and the page framework code of the sub-application is used to generate the extended reality interaction page of the sub-application.

10. The method according to any one of claims 1 to 9, further comprising: Invoke a camera in the extended reality interaction page to capture a real scene through the camera; Acquire pre-stored three-dimensional model materials, and generate a three-dimensional virtual object based on the three-dimensional model materials; and The real scene and the three-dimensional virtual object are synthesized, and an enhanced scene picture obtained by the synthesis is displayed on the extended reality interaction page.

11. According to the method of any one of claims 1 to 10, synthesizing the real scene with the virtual object and displaying the synthesized enhanced scene picture on the extended reality interaction page comprises: Acquiring the posture information of the camera, and generating a simulated scene simulating the real scene based on the posture information of the camera; and The three-dimensional virtual object is integrated into the simulated scene to obtain an enhanced scene image displayed on the extended reality interaction page.

12. A sub-application processing device, the device comprising: An acquisition module, used to acquire a page structure file corresponding to a sub-application, wherein the page structure file is used to describe the structure of an extended reality interaction page to be generated for the sub-application; An element generation module, used to parse the page structure file, obtain predefined tags in the page structure file, and convert the predefined tags into elements in the extended reality interaction framework; A component information generation module is used to obtain the attribute information corresponding to the predefined tag from the page structure file; for each predefined tag, convert the attribute information corresponding to the predefined tag into component information bound to the element converted by the predefined tag; and A page generation module is used to generate an extended reality interaction page of a sub-application according to the elements in the extended reality interaction framework and the component information bound to the elements, wherein the extended reality interaction page is used to display the interaction between the real scene and the virtual scene.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Citation Information

Patent Citations

  • Sub-application page processing method and device and computer equipment

    CN113312046A

  • Page generation method and device

    CN113535176A

  • Public account number-based interaction method and apparatus, and computer device

    CN114095582A

  • Page mutual jump closing method and system, electronic equipment and storage medium

    CN116049598A

  • Application system for generating 3D applications

    US20180349108A1

Cited By

  • Browser webpage spatialization method and device

    CN119337014A