Sub-application processing method and device, computer equipment and storage medium

By analyzing and converting the sub-application page structure file, the extended reality interaction page is generated, which solves the problem of high development threshold in the existing technology, and realizes the development of extended reality interaction sub-applications without third-party tools, providing an environment for real and virtual interaction.

CN118918228APending Publication Date: 2024-11-08TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310504686.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-06
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The development threshold for existing extended reality interactive sub-applications is high, and it requires knowledge related to third-party rendering engines, making it difficult for users to customize development.

Method used

By obtaining the page structure file of the sub-application, parsing and converting predefined tags into elements and component information in the extended reality interaction framework, generating an extended reality interaction page. Without the need to introduce third-party tools, the sub-applications are directly implemented in the extended reality interaction framework. create.

Benefits of technology

It lowers the threshold for the development of sub-applications for extended reality interactions, simplifies the development process, realizes the interaction between real scenes and virtual scenes, and provides an environment that combines real and virtual.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918228A_ABST
    Figure CN118918228A_ABST
Patent Text Reader

Abstract

The invention relates to a sub-application processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps of obtaining a page structure file corresponding to a sub-application, wherein the page structure file is used for describing the structure of an augmented reality interaction page of the sub-application; analyzing the page structure file to obtain each predefined tag in the page structure file, and converting each predefined tag into an element in the augmented reality interaction framework; obtaining attribute information corresponding to each predefined tag from the page structure file, and respectively converting the attribute information corresponding to each predefined tag into component information bound with the corresponding element; generating an augmented reality interaction page of the sub-application according to each element in the augmented reality interaction framework and the component information bound with each element; the augmented reality interaction page is used for displaying interaction between a real scene and a virtual scene, and the child application runs on the mother application. By adopting the method, the creation of the sub-application is simpler.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a sub-application processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of computer technology, more and more applications are providing convenience for users' work and life. Applications can also be divided into parent applications and child applications. Parent applications refer to applications that can be run independently, and child applications refer to applications that can be used without downloading and installing, but the child applications need to run on the parent application.

[0003] Currently, there is a sub-application development solution open to users, which allows users to develop customized sub-applications. For example, online merchants in the Internet of Things can develop exclusive sub-applications to provide services to consumers.

[0004] However, the current custom development of mini-programs often requires a lot of prior knowledge. In the development process of sub-applications for extended reality interaction, it is necessary to introduce a third-party rendering engine, and then combine the visual positioning capabilities of the sub-application for algorithm docking, etc. This requires developers to have relevant knowledge of third-party rendering engines, and the threshold for sub-application development is relatively high. Summary of the invention

[0005] Based on this, it is necessary to provide a low-threshold extended reality interaction sub-application processing method, device, computer equipment, computer-readable storage medium and computer program product to address the above technical problems.

[0006] In a first aspect, the present application provides a sub-application processing method. The method comprises:

[0007] Obtaining a page structure file corresponding to the sub-application, wherein the page structure file is used to describe the structure of the extended reality interaction page of the sub-application;

[0008] Parsing the page structure file to obtain each predefined tag in the page structure file, and converting each predefined tag into an element in the extended reality interaction framework;

[0009] Acquire the attribute information corresponding to each of the predefined tags from the page structure file, and convert the attribute information corresponding to each of the predefined tags into component information bound to the corresponding element;

[0010] An extended reality interaction page of a sub-application is generated according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0011] In a second aspect, the present application further provides a sub-application processing device. The device comprises:

[0012] An acquisition module, used to acquire a page structure file corresponding to a sub-application, wherein the page structure file is used to describe the structure of an extended reality interaction page of the sub-application;

[0013] An element generation module, used to parse the page structure file, obtain predefined tags in the page structure file, and convert the predefined tags into elements in the extended reality interaction framework;

[0014] A component information generation module, used to obtain the attribute information corresponding to the predefined tag from the page structure file, and convert the attribute information corresponding to the predefined tag into component information bound to the element;

[0015] A page generation module is used to generate an extended reality interaction page of a sub-application based on the elements in the extended reality interaction framework and the component information bound to the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0016] In one of the embodiments, the attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag and the attribute data corresponding to the tag attribute; the component information includes the component associated with the corresponding element and the component configuration data bound to the component.

[0017] In one of the embodiments, the component information generation module is also used to generate attribute adding instructions for components in the extended reality interaction framework based on the tag attributes corresponding to each of the predefined tags; based on the attribute adding instructions, respectively obtain components matching each of the tag attributes from the component library of the extended reality interaction framework; for each of the tag attributes, establish an association between the component matching the tag attribute and the element converted from the corresponding predefined tag.

[0018] In one of the embodiments, the extended reality interaction framework includes a plurality of systems, each of the systems being used to process component configuration data of a corresponding component;

[0019] The page generation module is also used to determine the systems corresponding to each of the components in the extended reality interaction framework; for each of the components, the component configuration data bound to the component is sent to the system corresponding to the component; and each of the systems is driven to process the received component configuration data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0020] In one of the embodiments, the page generation module is also used to obtain the sub-reference code of each system in the extended reality interaction framework; fill each component configuration data into the sub-reference code of the corresponding system to obtain the component code; drive each system to run its own component code respectively to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0021] In one embodiment, the plurality of systems include a rendering system, and each of the components includes at least a component corresponding to the rendering system;

[0022] The page generation module is also used to obtain page rendering data from the component configuration data bound to the component corresponding to the rendering system; send the page rendering data to the rendering system; control the rendering system to perform page rendering according to the page rendering data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0023] In one embodiment, the multiple systems also include an animation system, and each of the components also includes a component corresponding to the animation system; the page generation module is also used to obtain animation data from component configuration data bound to the component corresponding to the animation system, and send the animation data to the animation system; control the rendering system to perform page rendering according to the page rendering data, and control the animation system to generate animation content according to the animation data, forming page content including the animation content, so as to obtain an extended reality interaction page of the sub-application.

[0024] In one of the embodiments, the element generation module is further used to generate element creation instructions for the extended reality interaction framework according to each of the predefined tags; based on the element creation instructions, obtain elements that match each of the predefined tags from the element library of the extended reality interaction framework.

[0025] In one of the embodiments, the page generation module is also used to obtain a reference frame code corresponding to the extended reality interaction framework; adjust the reference frame code according to the elements in the extended reality interaction framework and the component information bound to the elements to obtain the page frame code of the sub-application; the page frame code of the sub-application is used to generate an extended reality interaction page of the sub-application.

[0026] In one of the embodiments, the device also includes an interaction module; the interaction module is used to call a camera in the extended reality interaction page to capture a real scene through the camera; obtain pre-stored three-dimensional model materials, and generate a three-dimensional virtual object based on the three-dimensional model materials; synthesize the real scene and the three-dimensional virtual object, and display the enhanced scene picture obtained by the synthesis on the extended reality interaction page.

[0027] In one of the embodiments, the interaction module is used to obtain the posture information of the camera, generate a simulated scene that simulates the real scene based on the posture information of the camera; integrate the three-dimensional virtual object into the simulated scene, and obtain an enhanced scene picture displayed on the extended reality interaction page.

[0028] In a second aspect, the present application further provides a sub-application processing device. The device comprises:

[0029] An acquisition module, used to acquire a page structure file corresponding to a sub-application, wherein the page structure file is used to describe the structure of an extended reality interaction page of the sub-application;

[0030] An element generation module, used to parse the page structure file, obtain predefined tags in the page structure file, and convert the predefined tags into elements in the extended reality interaction framework;

[0031] A component information generation module, used to obtain the attribute information corresponding to the predefined tag from the page structure file, and convert the attribute information corresponding to the predefined tag into component information bound to the element;

[0032] A page generation module is used to generate an extended reality interaction page of a sub-application based on the elements in the extended reality interaction framework and the component information bound to the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0033] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0034] Obtaining a page structure file corresponding to the sub-application, wherein the page structure file is used to describe the structure of the extended reality interaction page of the sub-application;

[0035] Parsing the page structure file to obtain each predefined tag in the page structure file, and converting each predefined tag into an element in the extended reality interaction framework;

[0036] Acquire the attribute information corresponding to each of the predefined tags from the page structure file, and convert the attribute information corresponding to each of the predefined tags into component information bound to the corresponding element;

[0037] An extended reality interaction page of a sub-application is generated according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0039] Obtaining a page structure file corresponding to the sub-application, wherein the page structure file is used to describe the structure of the extended reality interaction page of the sub-application;

[0040] Parsing the page structure file to obtain each predefined tag in the page structure file, and converting each predefined tag into an element in the extended reality interaction framework;

[0041] Acquire the attribute information corresponding to each of the predefined tags from the page structure file, and convert the attribute information corresponding to each of the predefined tags into component information bound to the corresponding element;

[0042] An extended reality interaction page of a sub-application is generated according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0044] Obtaining a page structure file corresponding to the sub-application, wherein the page structure file is used to describe the structure of the extended reality interaction page of the sub-application;

[0045] Parsing the page structure file to obtain each predefined tag in the page structure file, and converting each predefined tag into an element in the extended reality interaction framework;

[0046] Acquire the attribute information corresponding to each of the predefined tags from the page structure file, and convert the attribute information corresponding to each of the predefined tags into component information bound to the corresponding element;

[0047] An extended reality interaction page of a sub-application is generated according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0048] The above-mentioned sub-application processing method, apparatus, computer equipment, storage medium and computer program product obtain a page structure file corresponding to the sub-application, and the obtained page structure file is used to describe the structure of the extended reality interaction page of the sub-application, so that the page layout of the sub-application to be generated can be known through the page structure file. The page structure file is parsed to obtain each predefined tag in the page structure file, and the obtained predefined tags are predefined tags that can be converted into tags used in the extended reality interaction framework, so that each predefined tag can be converted into an element in the extended reality interaction framework. The attribute information corresponding to each predefined tag is obtained from the page structure file, and the attribute information corresponding to each predefined tag is converted into component information bound to the corresponding element, so that the predefined tags and related attribute information in the page structure file can be converted into elements and component information in the extended reality interaction framework, so that the extended reality interaction page of the sub-application can be generated according to each element in the extended reality interaction framework and the component information bound to each element, so that the creation of the extended reality interactive sub-application can be realized through the extended reality interaction framework itself, without the need to introduce third-party tools for related processing, making the development of the extended reality interactive sub-application simpler, and effectively reducing the threshold for developers to get started with the development of extended reality interactive sub-applications. In addition, the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, so that the sub-application can provide users with an environment that combines the real and the virtual and the human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 An application environment diagram of a sub-application processing method in an embodiment;

[0050] Figure 2 A schematic diagram of a process flow of a sub-application processing method in an embodiment;

[0051] Figure 3 A schematic diagram of a code for a custom component according to an embodiment;

[0052] Figure 4 A schematic diagram of a code for calling a customized component in another embodiment;

[0053] Figure 5 It is a flowchart of converting the attribute information corresponding to each predefined tag into the component information bound to the corresponding element in one embodiment;

[0054] Figure 6 A schematic diagram of sending component configuration data to a system corresponding to the component in one embodiment;

[0055] Figure 7 A schematic diagram of a process of driving each system to process received component configuration data and form corresponding page content to obtain an extended reality interaction page of a sub-application in an embodiment;

[0056] Figure 8 A schematic diagram of converting each predefined tag in a page structure file into an element in an extended reality interaction framework in one embodiment;

[0057] Fig. 9 An extended reality interaction page of a sub-application in an embodiment;

[0058] Fig.10 A schematic diagram of the architecture of a method for processing a sub-application in one embodiment;

[0059] Fig.11 A schematic diagram of writing a wxml file in an embodiment;

[0060] Fig.12 A schematic diagram of writing a js file in an embodiment;

[0061] Fig.13 The extended reality interaction effect achieved by combining a wxml file and a js file with an extended reality interaction framework in one embodiment;

[0062] Fig.14 is a structural block diagram of a sub-application processing device in an embodiment;

[0063] Fig.15 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. For example, it can be applied to the field of artificial intelligence (AI) technology, where artificial intelligence is the theory, method, technology and application system of simulating, extending and expanding human intelligence using digital computers or machines controlled by digital computers, perceiving the environment, acquiring knowledge and using knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. The solution provided in the embodiments of the present application relates to a sub-application processing method of artificial intelligence, which is specifically described by the following embodiments.

[0066] The sub-application processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 obtains the page structure file corresponding to the sub-application, and the page structure file is used to describe the structure of the extended reality interaction page of the sub-application. The terminal 102 sends the page structure file to the server 104. The server 104 parses the page structure file, obtains each predefined tag in the page structure file, and converts each predefined tag into an element in the extended reality interaction framework. The server 104 obtains the attribute information corresponding to each predefined tag from the page structure file, and converts the attribute information corresponding to each predefined tag into component information bound to the corresponding element. The server 104 generates the extended reality interaction page of the sub-application according to each element in the extended reality interaction framework and the component information bound to each element. The sub-application runs on the parent application, and the parent application runs on the terminal 102. When the sub-application runs, the extended reality interaction page is used to display the interaction between the real scene and the virtual scene. The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0067] In one embodiment, Figure 2 As shown, a sub-application processing method is provided, which is applied to Figure 1 (Computer equipment can be Figure 1 The terminal or server in the example is used to illustrate, including the following steps:

[0068] Step S202: Obtain a page structure file corresponding to the sub-application, where the page structure file is used to describe the structure of the extended reality interaction page of the sub-application.

[0069] Among them, a sub-application refers to an application that cannot run independently and needs to rely on other applications to run. A parent application refers to an application that can run independently and can provide an operating environment for the sub-application to run.

[0070] The page structure file is used to describe the structure of the extended reality interaction page of the sub-application. The page structure file can be the wxml file of the sub-application. The wxml (WeiXinMarkupLanguage) file is a set of markup languages ​​designed by the sub-application framework. Combined with the basic components and event system of the sub-application, the structure of the sub-application page can be constructed.

[0071] In this embodiment, the user can log in to the development application installed on the terminal through the development application account, or the user can log in to the web version of the development application through the development application account. Then, the terminal displays the development project management page of the development application. When the user triggers the development project creation operation, the terminal responds to the development project creation operation, creates a sub-application development project, and writes a page structure file in the development environment corresponding to the sub-application development project.

[0072] Step S204: parse the page structure file, obtain each predefined tag in the page structure file, and convert each predefined tag into an element in the extended reality interaction framework.

[0073] The predefined tag is a predefined tag, and each predefined tag may have the same label. The predefined tag can be distinguished from other tags by the label. For example, the predefined tag is a tag marked with "XR-".

[0074] In this embodiment, the predefined label may be a user-defined label.

[0075] The extended reality interaction framework is the XR (Extended Reality, XR) interaction framework, which is used to build sub-applications of the extended reality interaction class.

[0076] XR extended reality is a brand-new set of technical concepts. The X stands for both extension (Xtended) and unknown variables (X), and R stands for reality. It is a general term for the three technologies of AR (augmented reality), VR (virtual reality), and MR (mixed reality), and they are applied together in different scenarios or industries, integrating the virtual world and the real world in a variety of combinations, thus realizing more possibilities for creation. For example, in the field of video, AR and MR technologies are used at the same time. By detecting the movement of the camera and the lens, the virtual environment is used to fill and cover part of the real environment, which plays the effect of supplementing and expanding the scene, that is, superimposing the virtual space in the real space, so that the limited space can be infinitely extended, and the virtual picture and the real picture can be perfectly combined. XR extended reality technology is a combination of real space and virtual space.

[0077] Virtual Reality VR refers to a completely virtual personal experience.

[0078] Augmented Reality (AR) refers to superimposing virtual images on the basis of reality. Augmented reality technology is a technology that calculates the position and angle of camera images in real time and adds corresponding images, videos, and 3D models. The goal of this technology is to put the virtual world on the real world on the screen and interact with it.

[0079] Mixed Reality (MR) refers to virtual images and personal experiences that can be actively interacted with based on reality.

[0080] The extended reality interaction framework is an "entity-component-system" framework, including entities, components, and systems. The sub-applications of the extended reality interaction class are implemented through entities, components, and systems. The sub-applications of the extended reality interaction class can be virtual reality sub-applications, augmented reality sub-applications, mixed reality sub-applications, etc.

[0081] The extended reality interaction framework may be a framework including an ECS (Entity-Component-System) architecture. An entity is an element.

[0082] An entity represents a unit in the extended reality interaction framework, can be identified by the extended reality interaction framework, and can mount several components.

[0083] Components, that is, components mounted on entities, carry the attributes of a part of the entity and are pure data structures that do not contain functions.

[0084] The system is a pure function that does not contain data. It only cares about entities with certain specific attributes (components) and processes these attribute data.

[0085] Each basic unit in the extended reality interaction scene is an entity, and each entity is composed of one or more components. Each component only contains data representing its characteristics (that is, there is no method in the component). For example, the movement-related component MoveComponent contains properties such as speed, position, and direction. Once an entity has a MoveComponent component, it can be considered that it has the ability to move. The system is a tool to process a collection of entities with one or more identical components. It only has behavior (that is, there is no data in the system). In this example, the system that handles movement only cares about entities with the ability to move. It will traverse all entities with MoveComponent components and update the entity's position based on relevant data (speed, position, direction, etc.). Extended reality interaction scenes such as game scenes.

[0086] There is a one-to-many relationship between entities and components. The capabilities of an entity depend on the components it has. By dynamically adding or removing components, the behavior of the entity can be changed when the extended reality interaction scene is running.

[0087] In the ECS architecture, entities act as the "carrier" of components. They are a collection of components and do not contain data or business logic. To distinguish different entities, they are generally represented by simple data structures at the code level, such as an ID. All components that make up the entity are associated with the ID, so that it is clear which components belong to the entity. Through the entity, you can access the components and data on the components.

[0088] Since it is a collection of components, it is possible to dynamically add a new component to an entity or remove a component from an entity at runtime. For example, if a player entity loses its ability to move for some reason (perhaps falling into a coma), simply remove the movement component from the entity to achieve the effect of being unable to move.

[0089] Components only contain data, and components themselves cannot change data. The data on the component describes a certain feature of the entity. If you want the component to work, you must load the component onto the entity. The system is what really contains the business logic. What the system cares about is a collection of one or more components. The system captures the entity with all the components in the collection through the collection. After that, you can operate these entities, delete, add or delete components on the entity, change the data on the component, etc. This is what the system does.

[0090] Specifically, the page structure file is parsed to obtain each tag in the page structure file. Each tag may include a tag that can be converted into the extended reality interaction framework and a tag that cannot be converted into the extended reality interaction framework.

[0091] The computer device classifies each tag and selects each predefined tag. The computer device can obtain an extended reality interaction framework and convert each predefined tag into a corresponding element in the extended reality interaction framework.

[0092] Step S206, acquiring the attribute information corresponding to each predefined tag from the page structure file, and converting the attribute information corresponding to each predefined tag into component information bound to the corresponding element.

[0093] The attribute information corresponding to the predefined tag represents the attribute of the predefined tag in the extended reality interaction page. A predefined tag may correspond to one or more attribute information.

[0094] Component information represents the properties of an element in the extended reality interaction framework. An element can correspond to one or more component information.

[0095] Specifically, for each predefined tag, the computer device obtains the attribute information corresponding to the predefined tag from the page structure file, determines the element formed by the predefined tag in the extended reality interaction framework, and converts the attribute information corresponding to the predefined tag into component information bound to the element.

[0096] In this embodiment, the attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag and the attribute data corresponding to the tag attribute; the component information includes the component associated with the corresponding element and the component configuration data bound to the component.

[0097] Step S208, generating an extended reality interaction page of the sub-application according to each element in the extended reality interaction framework and the component information bound to each element; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0098] Among them, the virtual scene is a scene for virtual characters to carry out activities or perform interactive actions when the virtual interactive application is running. The virtual scene can be a simulation environment of the real world, a semi-simulated and semi-fictitious virtual environment, or a purely fictitious virtual environment. For example, the virtual scene can be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, and a three-dimensional virtual scene. The virtual scene can specifically be a mobile game scene, a terminal game scene, an interactive scene of virtual characters in augmented reality, virtual reality, or mixed reality, etc., but is not limited thereto. The user can control the virtual character to move or perform interactive actions in the virtual scene.

[0099] A virtual character is an active object in a virtual scene. The active object may be a virtual object used to represent a user. The virtual object is not limited to a virtual person, a virtual animal, or a cartoon character. For example, the virtual object may be a person, an animal, or a plant displayed in a virtual scene. A virtual scene may include multiple virtual characters, each of which has its own shape and volume in the virtual scene and occupies a portion of the space in the virtual scene.

[0100] Specifically, the computer device generates an extended reality interaction page of the sub-application according to each element in the extended reality interaction framework and the component information bound to each element, thereby obtaining an extended reality interaction sub-application. The user can run the sub-application in the parent application and realize the interaction between the real scene and the virtual scene in the extended reality interaction page of the sub-application.

[0101] The above-mentioned sub-application processing method, apparatus, computer device, computer-readable storage medium and computer program product obtain a page structure file corresponding to the sub-application, and the obtained page structure file is used to describe the structure of the extended reality interaction page of the sub-application, so that the page layout of the sub-application to be generated can be known through the page structure file. The page structure file is parsed to obtain each predefined tag in the page structure file, and the obtained predefined tags are predefined tags that can be converted into tags used in the extended reality interaction framework, so that each predefined tag can be converted into an element in the extended reality interaction framework. The attribute information corresponding to each predefined tag is obtained from the page structure file, and the attribute information corresponding to each predefined tag is converted into component information bound to the corresponding element, so that the predefined tags and related attribute information in the page structure file can be converted into elements and component information in the extended reality interaction framework, so that the extended reality interaction page of the sub-application can be generated according to each element in the extended reality interaction framework and the component information bound to each element, so that the creation of the sub-application of the extended reality interaction class can be realized through the extended reality interaction framework itself, without the need to introduce third-party processing, making the development of the sub-application of the extended reality interaction class simpler, and effectively reducing the threshold for developers to get started with the development of sub-applications of the extended reality interaction class. In addition, the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, so that the sub-application can provide users with an environment that combines the real and virtual and human-computer interaction.

[0102] In one embodiment, a user can generate custom elements and components in the extended reality interaction framework, and generate predefined tags that match each element. When generating a sub-application, a page structure file is generated using predefined tags that match the custom elements, so that the computer device can automatically convert the predefined tags in the page structure file into corresponding elements in the extended reality interaction framework.

[0103] In this embodiment, the user can generate custom elements and components in the extended reality interaction framework, and generate predefined tags that match each element and attributes that match each component. When generating a sub-application, the predefined tags that match the custom elements and the attributes that match the components are used to generate a page structure file, so that the computer device can automatically convert the predefined tags in the page structure file into corresponding elements in the extended reality interaction framework and convert the attributes into corresponding components in the extended reality interaction framework.

[0104] In this embodiment, the extended reality interaction framework provides users with the function of generating elements and components, so that users can customize the generation of required elements and components, thereby improving the flexibility and extensibility of the extended reality interaction framework.

[0105] In one embodiment, the sub-application processing method has a simple entry threshold, but it also makes great efforts in flexibility and extensibility. Developers only need to use interfaces such as "registerComponent" to define components and elements by themselves, which is equivalent to customizing the tags and attributes in "wxml", such as Figure 3 As shown in the figure, it is the code for customizing components. Through the registration mechanism of XR-FRAMR, the developer registered the "auto-rotate" component, and used the component as an attribute in "wxml" to make the 3D node corresponding to the mounted element rotate automatically every frame. "auto-rotate-touchable-gltf" shows how to pre-combine the functions of multiple components into one element, which integrates automatic rotation, interactivity and GlTF rendering capabilities for subsequent use. Figure 4 The following is the code for calling the customized component during the process of generating the sub-application.

[0106] In one embodiment, the attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag and the attribute data corresponding to the tag attribute; the component information includes the component associated with the corresponding element and the component configuration data bound to the component.

[0107] The attribute information corresponding to each predefined tag is converted into component information bound to the corresponding element, including:

[0108] For each predefined tag, a component associated with the element formed by the conversion of the predefined tag is generated according to the tag attribute corresponding to the predefined tag; for each tag attribute, component configuration data bound to the corresponding component in the extended reality interaction framework is generated based on the attribute data corresponding to the tag attribute.

[0109] Specifically, the computer device converts each predefined tag into an element in the extended reality interaction framework, and a predefined tag may correspond to one or more tag attributes in the page structure file.

[0110] The computer device may obtain each tag attribute corresponding to the predefined tag from the page structure file, and based on each tag attribute, generate each component associated with the element converted by the predefined tag in the extended reality interaction framework. One tag attribute may generate one component.

[0111] For the attribute data corresponding to each tag attribute, the computer device generates component configuration data bound to the corresponding component in the extended reality interaction framework according to the data tag corresponding to the tag attribute.

[0112] In this embodiment, each predefined tag in the page structure file is converted into an element in the extended reality interaction framework, the tag attributes corresponding to the predefined tags are converted into components associated with the elements, and the attribute data corresponding to the tag attributes are converted into component configuration data bound to the components, so that the various parts of the structure describing the extended reality interaction page can be converted to the extended reality interaction framework, so that the extended reality interaction page can be directly generated by using the elements, components and component configuration data through the extended reality interaction framework, so that the development of extended reality interactive sub-applications can be realized without the need for additional processing by third-party tools, thereby lowering the development threshold of extended reality interactive sub-applications.

[0113] In one embodiment, if Figure 5 As shown, the attribute information corresponding to each predefined tag is converted into component information bound to the corresponding element, including:

[0114] Step S502: Generate a property adding instruction for a component in the extended reality interaction framework based on the tag attributes corresponding to each predefined tag.

[0115] The attribute adding instruction is an instruction for configuring attributes for components in the extended reality interaction framework. Attributes in the extended reality interaction framework are represented by components, and attribute data are represented by component configuration data.

[0116] Specifically, the computer device may generate an attribute adding instruction according to each predefined tag and the tag attributes corresponding to each predefined tag, where the attribute adding instruction indicates adding corresponding component configuration data to the component in the extended reality interaction framework.

[0117] In this embodiment, the computer device sends the attribute adding instruction to the template engine, and the attribute adding instruction is used to instruct the template engine to convert each tag attribute into each component in the extended reality interaction framework. Among them, the template engine is an engine tool that can combine the page structure and the data to be displayed to generate an extended reality interaction page. The template engine can run on the server side or on the client side, and can be directly parsed into a markup language on the server side, and then passed to the client after completion.

[0118] Step S504: based on the attribute adding instruction, components matching each tag attribute are obtained from the component library of the extended reality interaction framework.

[0119] Specifically, a plurality of components are stored in the component library of the extended reality interaction framework. The components in the component library may be custom components generated by users, or may be common components in the extended reality interaction framework.

[0120] The computer device adds instructions based on the attribute, and matches each tag attribute with each component in the component library of the extended reality interaction framework to obtain a component that matches each tag attribute.

[0121] Step S506: for each tag attribute, an association is established between the component matching the tag attribute and the element converted from the corresponding predefined tag.

[0122] Specifically, for each tag attribute, the computer device determines the component that matches the tag attribute, and associates the component that matches the tag attribute with the element converted by the corresponding predefined tag to obtain component information bound to the element converted by each predefined tag.

[0123] In this embodiment, the page structure file corresponding to the sub-application is obtained through the template engine, and the page structure file is parsed to obtain each predefined tag in the page structure file and the tag attributes corresponding to each predefined tag. The template engine generates an attribute adding instruction based on the tag attributes corresponding to each predefined tag, and sends the attribute adding instruction to the back end of the extended reality interaction framework to instruct the back end of the extended reality interaction framework to find a component matching each tag attribute and associate the found component with the corresponding element.

[0124] In this embodiment, based on the tag attributes corresponding to each predefined tag, attribute adding instructions are generated for components in the extended reality interaction framework, so that based on the attribute adding instructions, components matching each tag attribute are respectively obtained from the component library of the extended reality interaction framework, so that the tag attributes in the page structure file can be converted into corresponding components in the extended reality interaction framework, and for each tag attribute, an association is established between the component matching the tag attribute and the element converted from the corresponding predefined tag, so that the correspondence between the component and the element is consistent with the correspondence between the predefined tag and the tag attribute, thereby ensuring the accuracy of the converted data.

[0125] In one of the embodiments, an extended reality interaction framework includes a plurality of systems, each system being configured to process component configuration data of a corresponding component;

[0126] According to the elements in the extended reality interaction framework and the component information bound to each element, the extended reality interaction page of the sub-application is generated, including:

[0127] Determine the system corresponding to each component in the extended reality interaction framework; for each component, send the component configuration data bound to the targeted component to the system corresponding to the targeted component; drive each system to execute processing on the received component configuration data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0128] Specifically, the extended reality interaction framework includes multiple systems, each of which is used to process component configuration data of a corresponding component. For example, the rendering system is used to process configuration data related to rendering, the animation system is used to process configuration data related to animation content, and the AR system is used to process configuration data related to AR.

[0129] After each element, tag attribute and attribute information in the page structure file is converted into corresponding elements, components and component configuration data, the system corresponding to each component is determined. One system can correspond to multiple components.

[0130] For each component, the computer device sends the component configuration data bound to the component to the system corresponding to the component, and drives each system to process the received component configuration data to form corresponding page content. Each page content forms the extended reality interaction page of the sub-application, thereby obtaining the sub-application of the extended reality interaction class.

[0131] For example, Figure 6As shown, the extended reality interaction framework includes systems A, B, C, and D. After converting the predefined tags in the page structure file, the tag attributes corresponding to the predefined tags, and the attribute data corresponding to the tag attributes into the elements, the components associated with the elements, and the component configuration data bound to the components in the extended reality interaction framework, the component a corresponding to the system A is determined, and the component configuration data of the component a is sent to the system A; the component b corresponding to the system B is determined, and the component configuration data of the component b is sent to the system B; the component c corresponding to the system C is determined, and the component configuration data of the component c is sent to the system C; the component d corresponding to the system D is determined, and the component configuration data of the component b is sent to the system d.

[0132] In this embodiment, the system focuses on the corresponding component data instead of the component configuration data bound to all components corresponding to an element. For example, in a game sub-application, there are multiple characters, each character has corresponding skills, and each skill is configured with corresponding skill data. There are skill systems and mobile systems in the extended reality interaction framework, and the skill system only focuses on and processes the skill data of each character, and the mobile system only focuses on and processes the displacement data of each character.

[0133] In this embodiment, the extended reality interaction framework includes multiple systems, each system is used to process the component configuration data of the corresponding component. After converting each predefined tag, the tag attribute corresponding to the predefined tag, and the attribute data corresponding to the tag attribute in the page structure file into each element, the component associated with the element, and the component binding component configuration data, the system corresponding to each component in the extended reality interaction framework is determined to accurately determine which components need to be sent to which system for processing, so that for each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component. Each system is driven to execute the processing of the received component configuration data, so that the corresponding page content is formed through the processing of the corresponding component configuration data by each system, so that the construction of the subroutine of the extended reality interaction class can be directly implemented in the extended reality interaction framework without the help of external tools, which reduces the development cost of the subroutine of the extended reality interaction class and improves the efficiency of the subroutine of the extended reality interaction class.

[0134] In one embodiment, if Figure 7 As shown, each system is driven to process the received component configuration data to form corresponding page content to obtain an extended reality interaction page of the sub-application, including:

[0135] Step S702: Obtain the sub-reference code of each system in the extended reality interaction framework.

[0136] Step S704: Fill each component configuration data into the corresponding system sub-reference code to obtain the component code.

[0137] Among them, the sub-reference code is the code used to drive the system operation to realize the corresponding functions in the sub-application.

[0138] Specifically, the computer is set to obtain the sub-reference code of each system in the extended reality interaction framework. For the component configuration data of each component, according to the component configuration data, the sub-reference code of the system corresponding to the component bound to the component configuration data is adjusted to obtain the component code.

[0139] In this embodiment, for each component configuration data, the computer device fills the targeted component configuration data into the sub-reference code of the system corresponding to the component bound to the targeted component configuration data to obtain the component code corresponding to the component.

[0140] Step S706, driving each system to run its own component code respectively to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0141] Specifically, the computer device drives each system to run its own component code respectively to form corresponding page content, and each page content forms an extended reality interaction page, thereby obtaining an extended reality interaction type sub-application.

[0142] In this embodiment, the extended reality interaction framework has a corresponding reference framework code, the extended reality interaction framework includes multiple systems, and the reference framework code includes a sub-reference code corresponding to each system.

[0143] In this embodiment, the sub-reference code of each system in the extended reality interaction framework is obtained, and each component configuration data is filled into the sub-reference code of the corresponding system to obtain the component code, so that the corresponding component code can be automatically generated without the user writing code. Each system is driven to run its own component code respectively to form the corresponding page content to obtain the extended reality interaction page of the sub-application, thereby building a user-defined extended reality interaction type sub-application, which can shorten the development cycle and improve development efficiency. In addition, through the addition of elements and components, and the binding of component configuration data, users who do not have a deep understanding of the development of extended reality interaction type sub-applications can also complete the development of various parts and functions of the sub-application, making the development of extended reality interaction type sub-applications simpler.

[0144] In one embodiment, the plurality of systems include a rendering system, and each component includes at least a component corresponding to the rendering system;

[0145] For each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component, including:

[0146] Obtaining page rendering data from component configuration data bound to a component of a corresponding rendering system; and sending the page rendering data to the rendering system;

[0147] Driving each system to execute the received component configuration data to form corresponding page content to obtain the extended reality interaction page of the sub-application, including:

[0148] The rendering system is controlled to perform page rendering according to the page rendering data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0149] Among them, page rendering is the process of completing the page layout and drawing based on certain rules based on the page resources returned by the request. Page resources include, but are not limited to, text, images, animations, videos, audio, etc.

[0150] The multiple systems in the extended reality interaction framework include a rendering system, and each component includes at least a component corresponding to the rendering system.

[0151] Specifically, the computer device determines the system corresponding to each component in the extended reality interaction framework, and when multiple systems include a rendering system, and each component includes a component corresponding to the rendering system, the computer device obtains page rendering data from the component configuration data bound to the component, and sends the page rendering data to the rendering system.

[0152] The computer device controls the rendering system to perform page rendering according to the page rendering data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0153] In this embodiment, each component includes at least a component corresponding to the rendering system, and the component configuration data bound to the component corresponding to the rendering system includes page rendering data. The computer device sends the page rendering data bound to the component corresponding to the rendering system to the rendering system.

[0154] In this embodiment, page rendering can be real-time page rendering. Real-time page rendering refers to drawing three-dimensional data into two-dimensional bitmaps according to graphics algorithms, and displaying these bitmaps in real time. Its essence is the real-time calculation and output of image data, requiring the rendering and display of the picture in a short time, and rendering and displaying the next picture at the same time.

[0155] In this embodiment, page rendering data is obtained from component configuration data bound to a component of a corresponding rendering system, the page rendering data is sent to the rendering system, and the rendering system is controlled to perform page rendering according to the page rendering data to form corresponding page content. This allows developers who do not have relevant knowledge of third-party rendering engines to develop sub-applications of the extended reality interaction class without resorting to a third-party rendering engine, and can directly perform page rendering through the rendering system provided in the extended reality interaction framework. This makes it easy for developers who do not have relevant knowledge of third-party rendering engines to develop sub-applications of the extended reality interaction class, thereby lowering the threshold for developing sub-applications of the extended reality interaction class.

[0156] In one embodiment, the plurality of systems further include an animation system, and each component further includes a component corresponding to the animation system; and the method further includes:

[0157] Obtain animation data from component configuration data bound to a component of a corresponding animation system, and send the animation data to the animation system;

[0158] Control the rendering system to render the page according to the page rendering data to form corresponding page content to obtain the extended reality interaction page of the sub-application, including:

[0159] The rendering system is controlled to perform page rendering according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data to form page content including the animation content, so as to obtain an extended reality interaction page of the sub-application.

[0160] Among them, animation data is data used to generate animation content in the extended reality interaction page, such as dynamic people, dynamic objects, etc.

[0161] The multiple systems in the extended reality interaction framework also include an animation system, and each component also includes a component corresponding to the animation system.

[0162] Specifically, the computer device determines the systems corresponding to each component in the extended reality interaction framework. When multiple systems include an animation system, and each component includes a component corresponding to the animation system, the computer device obtains animation data from the component configuration data bound to the component corresponding to the animation system and sends the animation data to the animation system.

[0163] The computer device controls the rendering system to render the page according to the page rendering data, and controls the animation system to generate animation content according to the animation data, so as to obtain static page content and dynamic page content. The dynamic page content is the animation content. The static page content and the dynamic page content form the extended reality interaction page of the sub-application, thereby obtaining the sub-application of the extended reality interaction class.

[0164] In this embodiment, each component includes at least a component corresponding to the animation system, and the component configuration data bound to the component corresponding to the animation system includes animation data. The computer device sends the animation data bound to the component corresponding to the animation system to the animation system.

[0165] In this embodiment, multiple systems also include an animation system, and each component also includes a component corresponding to the animation system. Animation data is obtained from component configuration data bound to the component corresponding to the animation system, and the animation data is sent to the animation system. The rendering system is controlled to render the page according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data to generate dynamic page content and static page content. Therefore, when developing sub-applications of the extended reality interaction class, there is no need to rely on third-party related engines. Page rendering and animation generation can be performed directly through the rendering system and animation system provided in the extended reality interaction framework. Therefore, developers are not required to have relevant knowledge of third-party related engines to develop sub-applications of the extended reality interaction class, thereby lowering the threshold for developing sub-applications of the extended reality interaction class.

[0166] In one embodiment, each predefined tag is converted into an element in an extended reality interaction framework, including:

[0167] Generate element creation instructions for an extended reality interaction framework according to each predefined tag;

[0168] Based on the element creation instruction, elements matching each predefined tag are obtained from the element library of the extended reality interaction framework.

[0169] The element creation instruction is an instruction to convert the predefined tags in the page structure file into components in the extended reality interaction framework. The element library stores multiple elements, and the elements in the element library can be custom elements generated by users or common elements in the extended reality interaction framework.

[0170] Specifically, the computer device generates an element creation instruction for the extended reality interaction framework according to each predefined tag. Based on the element creation instruction, the computer device matches each predefined tag with each element in the element library of the extended reality interaction framework to obtain an element matching each predefined tag. Figure 8 As shown, the predefined tags in the page structure file are XR-tag A, XR-tag B, XR-tag C and XR-tag D, which are respectively converted into element A, element B, element C and element D in the extended reality interaction framework.

[0171] In this embodiment, element creation instructions for the extended reality interaction framework are generated according to each predefined tag, so that based on the element creation instructions, elements matching each predefined tag are obtained from the element library of the extended reality interaction framework, so that the predefined tags in the page structure file can be accurately converted into corresponding elements in the extended reality interaction framework.

[0172] In one embodiment, generating an extended reality interaction page of a sub-application according to elements in an extended reality interaction framework and component information bound to the elements includes:

[0173] Obtain the reference frame code corresponding to the extended reality interaction framework; adjust the reference frame code according to the elements in the extended reality interaction framework and the component information bound to the elements to obtain the page frame code of the sub-application; the page frame code of the sub-application is used to generate the extended reality interaction page of the sub-application.

[0174] Specifically, the extended reality interaction framework is pre-configured with a reference framework code for creating a sub-application according to the structure of the extended reality interaction framework. The computer device adjusts the corresponding reference elements and the corresponding component information in the reference framework code according to the elements in the extended reality interaction framework and the component information bound to the elements, obtains the page framework code of the sub-application, and runs the page framework code to generate an extended reality interaction page, thereby obtaining the sub-application.

[0175] In this embodiment, the computer device fills in the corresponding positions in the reference frame code according to the elements in the extended reality interaction frame and the component information bound to the elements to obtain the page frame code of the sub-application.

[0176] In this embodiment, the computer device can obtain a page style file, which is used to describe the local style or global style of the extended reality interaction page of the sub-application. The computer device can generate the extended reality interaction page of the sub-application according to the page style file and the page framework code of the sub-application.

[0177] In this embodiment, the extended reality interaction framework has a corresponding reference framework code, the extended reality interaction framework includes multiple systems, and the reference framework code includes a sub-reference code corresponding to each system; according to the elements in the extended reality interaction framework and the component information bound to the elements, the reference framework code is adjusted to obtain the page framework code of the sub-application, including:

[0178] Fill each component configuration data into the sub-reference code of the corresponding system to obtain the component code; drive each system to run its own component code respectively to form the corresponding page content to obtain the extended reality interaction page of the sub-application.

[0179] In this embodiment, the reference frame code corresponding to the extended reality interaction framework is obtained, and the reference frame code is adjusted according to the elements in the extended reality interaction framework and the component information bound to the elements to obtain the page frame code of the sub-application, so that the sub-application of the user-defined extended reality interaction type can be automatically generated without the user writing code, which can shorten the development cycle and improve development efficiency. In addition, through the addition of elements and components, and the binding of component configuration data, users who do not understand code can also develop sub-applications by themselves, making the development of sub-applications simpler.

[0180] In one embodiment, the method further comprises:

[0181] Call the camera in the extended reality interaction page to capture the real scene through the camera;

[0182] The pre-stored 3D model material is obtained, and a 3D virtual object is generated based on the 3D model material; the real scene and the 3D virtual object are synthesized, and the enhanced scene picture obtained by the synthesis is displayed on the extended reality interaction page.

[0183] Specifically, after the sub-application of the extended reality interaction class is generated, the sub-application runs in the parent application. The parent application provides an operating environment for the sub-application. After the sub-application is generated, the user can trigger the parent application identifier, and the terminal responds to the trigger operation of the parent application, jumps to the parent application, and enters the operating environment provided by the parent application to run the sub-application.

[0184] The extended reality interaction page is displayed in the sub-application, and the extended reality interaction page provides a calling interface of the terminal camera. The user can call the camera in the extended reality interaction page through the calling interface to capture the real scene through the camera.

[0185] The computer device pre-stores a 3D model material, which is used to generate a corresponding 3D virtual object. The 3D virtual object may be a virtual person or object in a 3D space. The computer device generates the corresponding 3D virtual object through the 3D model material.

[0186] The computer device synthesizes the real scene and the three-dimensional virtual object to obtain an enhanced scene picture. The computer device outputs the enhanced scene picture to the extended reality interaction page of the sub-application to display the enhanced scene picture on the extended reality interaction page.

[0187] In this embodiment, a camera is called in the extended reality interaction page, the real scene is captured by the camera, pre-stored three-dimensional model materials are obtained, three-dimensional virtual objects are generated based on the three-dimensional model materials, and the real scene and the three-dimensional virtual objects are synthesized so that the enhanced scene picture obtained by the synthesis can be displayed in the sub-application, and the scene picture that combines the real and virtual can be presented through the sub-application, thereby providing users with a real and virtual environment through the extended reality interaction sub-application.

[0188] In one embodiment, a real scene and a virtual object are synthesized, and an enhanced scene image obtained by the synthesis is displayed on an extended reality interaction page, including:

[0189] Acquire the posture information of the camera, and generate a simulated scene that simulates the real scene based on the posture information of the camera; integrate the three-dimensional virtual object into the simulated scene to obtain an enhanced scene picture displayed on the extended reality interaction page.

[0190] Specifically, the computer device obtains the posture information of the camera, and the posture information represents the three-dimensional spatial data of the camera. Based on the posture information of the camera, the computer device simulates the real scene to generate a virtual scene, and the virtual scene is a simulated scene of the real scene. The computer device integrates the generated three-dimensional virtual object into the simulated scene to obtain an enhanced scene screen including the simulated scene and the three-dimensional virtual scene. The enhanced scene screen is displayed on the extended reality interaction page of the sub-application.

[0191] In this embodiment, the posture information of the camera is obtained, and a virtual scene matching the real scene is generated based on the posture information of the camera, so as to integrate the three-dimensional virtual scene into the virtual scene, and obtain an enhanced scene picture displayed on the extended reality interaction page, so that virtual content can be superimposed on the real scene of the camera in real time, and the virtual content can be seamlessly combined with the real environment to form a visual effect that is indistinguishable from the real thing.

[0192] In this embodiment, a sub-application processing method is provided, which is applied to a computer device and includes:

[0193] Get the page structure file corresponding to the sub-application, which is used to describe the structure of the extended reality interaction page of the sub-application;

[0194] Parse the page structure file to obtain predefined tags in the page structure file; generate element creation instructions for the extended reality interaction framework according to the predefined tags;

[0195] Based on the element creation instruction, elements matching each predefined tag are obtained from the element library of the extended reality interaction framework.

[0196] Obtain the tag attributes corresponding to each predefined tag and the attribute data corresponding to the tag attributes from the page structure file;

[0197] Based on the tag attributes corresponding to each predefined tag, attribute adding instructions for components in the extended reality interaction framework are generated; based on the attribute adding instructions, components matching each tag attribute are respectively obtained from the component library of the extended reality interaction framework; for each tag attribute, an association is established between the component matching the tag attribute and the element converted from the corresponding predefined tag.

[0198] For each tag attribute, based on the attribute data corresponding to the tag attribute, component configuration data bound to a corresponding component in the extended reality interaction framework is generated;

[0199] Determine the systems that correspond to each component in the extended reality interaction framework;

[0200] For each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component;

[0201] Obtain the sub-reference code of each system in the extended reality interaction framework; fill each component configuration data into the sub-reference code of the corresponding system to obtain the component code; drive each system to run its own component code respectively to form the corresponding page content to obtain the extended reality interaction page of the sub-application; the sub-application runs on the parent application.

[0202] Furthermore, when multiple systems include a rendering system and an animation system, and each component includes a component corresponding to the rendering system and a component corresponding to the animation system, page rendering data is obtained from component configuration data bound to the component, and the page rendering data is sent to the rendering system; animation data is obtained from component configuration data bound to the component corresponding to the animation system, and the animation data is sent to the animation system; the rendering system is controlled to perform page rendering according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data, so as to form an extended reality interaction page of the sub-application and obtain a sub-application of the extended reality interaction class.

[0203] After the sub-application is generated, the camera is called in the extended reality interaction page to capture the real scene; the pre-stored 3D model material is obtained, and a 3D virtual object is generated based on the 3D model material; the real scene and the 3D virtual object are synthesized, and the enhanced scene image obtained by the synthesis is displayed on the extended reality interaction page.

[0204] In this embodiment, by obtaining the page structure file corresponding to the sub-application, the obtained page structure file is used to describe the structure of the extended reality interaction page of the sub-application, so that the page layout of the sub-application to be generated can be known through the page structure file. The page structure file is parsed to obtain each predefined tag in the page structure file. The obtained predefined tags are predefined tags that can be converted into tags used in the extended reality interaction framework, so that each predefined tag can be converted into an element in the extended reality interaction framework.

[0205] Each predefined tag in the page structure file is converted into an element in the extended reality interaction framework, the tag attributes corresponding to the predefined tags are converted into components associated with the elements, and the attribute data corresponding to the tag attributes are converted into component configuration data bound to the components, so that the various parts of the structure describing the extended reality interaction page can be converted to the extended reality interaction framework, so that the extended reality interaction page can be directly generated subsequently by using the elements, components and component configuration data through the extended reality interaction framework.

[0206] The extended reality interaction framework includes multiple systems, each of which is used to process the component configuration data of the corresponding component. After converting the predefined tags, the tag attributes corresponding to the predefined tags, and the attribute data corresponding to the tag attributes in the page structure file into the components, the components associated with the elements, and the component configuration data bound to the components in the extended reality interaction framework, the system corresponding to each component in the extended reality interaction framework is determined to accurately determine which components need to be sent to which system for processing, so that for each component, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component. Each system is driven to execute the processing of the received component configuration data, so that the corresponding page content is formed through the processing of the corresponding component configuration data by each system, so that the construction of the subroutine of the extended reality interaction class can be directly realized in the extended reality interaction framework without the help of external tools, which reduces the development threshold and development cost of the subroutine of the extended reality interaction class and improves the efficiency of the subroutine of the extended reality interaction class.

[0207] It is understandable that the sub-application generation method is applicable to the generation of any extended reality interaction sub-application, such as big data sub-application, artificial intelligence sub-application, Internet of Things sub-application, etc. The sub-application generation method is applicable to mobile terminals and also to web terminals. Users can complete the development of sub-applications in the development application of mobile terminals or web terminals. Fig. 9 As shown, it is an extended reality interaction page of a sub-application for face recognition generated by a user through a sub-application generation method.

[0208] In one embodiment, an application scenario of a sub-application processing method is provided, which is applied to the development of a sub-application of an extended reality interaction type, wherein the sub-application of the extended reality interaction type is an XR applet, and the extended reality interaction framework includes an ECS framework. Fig.10 As shown, in this embodiment, the development of the XR applet is divided into three parts. The first part is to parse the applet's page structure file wxml into structured data and instructions, the second part is to convert the structured data into entities and components of the ECS framework, and the third part is to drive the logic of each system module and finally render it. The specific processing process is as follows:

[0209] The first part is the upper part of the architecture diagram. In the extended reality interaction framework, a special type of mini-program native tags, namely predefined tags, are predefined and start with "xr-". During the parsing process, the template engine first obtains the code snippet in the page structure file wxml, analyzes the tags in the wxml file and classifies them, and separates the tags starting with "xr-" from other common UI tags.

[0210] Generate a call instruction similar to "createElement" according to each predefined tag, that is, an element creation instruction. Determine the tag attribute corresponding to each predefined tag in the wxml file, and generate a call instruction similar to "addAttribute" according to the tag attribute corresponding to each predefined tag, that is, an attribute addition instruction. The template engine forwards the element creation instruction and attribute addition instruction to the "xr-frame" backend that the ECS framework implements specifically for the template engine. After receiving these instructions, the "xr-frame" backend will create the corresponding elements and components in the ECS framework. Among them, the mini program tags correspond to the elements in ECS, and the mini program attributes correspond to the components in ECS.

[0211] The second part is the middle part of the architecture diagram. In the "xr-frame" backend, developers can use methods such as "registerElement" and "registerComponent" to register elements and components to customize the components and elements in the framework. When the "xr-frame" backend receives element creation instructions and attribute addition instructions, it will find the elements (i.e. entities) and components corresponding to the template engine instructions by looking up the table. For example, the predefined tag "xr-camera" corresponds to the "Camera" element. After finding these elements or components, the framework will further parse the attribute data sent by the template engine. The attribute data can be a string type. The types of these attribute data are converted into corresponding data through the data parser, such as "number", "array", etc., to obtain the component configuration data corresponding to each component, and then trigger the corresponding life cycle "onAdd", "onUpdate" and other component configuration data to update the component. After the component is ready, it will be handled by the following systems.

[0212] The third part is the lower half of the architecture diagram. Each system in the framework accepts the component configuration data generated in the second step to drive the entire logic loop. It mainly consists of a callback per frame to drive all systems mounted on the scene. Among them, the rendering system includes a programmable rendering pipeline, and the developer decides how to cull the scene, assemble the rendering queue, and render. The resource system is responsible for managing the loading of all resources, the animation system drives all frame animations and model animations in the scene, and the AR system drives the camera screen rendering of the visual positioning tool (VisionKit) and automatic matching of identification points. It can also include logic systems, physical systems, particle systems, and other systems.

[0213] The rendering system includes the drawing backend (Web Graphics Library, WebGL), which is a 3D drawing protocol.

[0214] The resource system includes a custom loader, and developers can customize the loading of Graphics Language Transmission Format (GLTF), environment data (EnvData), and various textures. The resource system provides a command line tool xr-frame-cli, which integrates two functions: environment data generation and glTF optimization.

[0215] In addition, each system has its own code. After receiving the component configuration data, each system fills the component configuration data of the corresponding component into the code and runs it. Then, after calculation processing, the calculation result is updated to the component to execute the corresponding behavior.

[0216] Through the sub-application processing method of this embodiment, the cost for developers to develop 3D / XR applications in mini-programs is greatly reduced, while also providing users with a better experience.

[0217] In this embodiment, the steps for a developer to develop a 3D / XR applet using the sub-application processing method of this embodiment are as follows:

[0218] Create a small program component and set the third-party renderer "renderer" to "xr-frame" in the "json" configuration;

[0219] like Fig.11 As shown, open the page structure file wxml, and write code in markup language like ordinary applet components; you can also write some javascript code according to your needs, such as Fig.12 As shown;

[0220] Run debugging on a real device.

[0221] For developers, the entire process is almost the same as the process of developing ordinary mini-programs, but because the tags written are specially defined xr tags, Fig.11 wxml file shown and Fig.12 The js file shown can be implemented Fig.13 3D / XR effect shown.

[0222] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0223] Based on the same inventive concept, the embodiment of the present application also provides a sub-application processing device for implementing the sub-application processing method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more sub-application processing device embodiments provided below can refer to the limitations on the sub-application processing method above, and will not be repeated here.

[0224] In one embodiment, Fig.14As shown, a sub-application processing device is provided, including: an acquisition module 1402, an element generation module 1404, a component information generation module 1406 and a page generation module 1408, wherein:

[0225] The acquisition module 1402 is used to acquire a page structure file corresponding to the sub-application, where the page structure file is used to describe the structure of the extended reality interaction page of the sub-application.

[0226] The element generation module 1404 is used to parse the page structure file, obtain the predefined tags in the page structure file, and convert the predefined tags into elements in the extended reality interaction framework.

[0227] The component information generating module 1406 is used to obtain the attribute information corresponding to the predefined tag from the page structure file, and convert the attribute information corresponding to the predefined tag into component information bound to the element.

[0228] The page generation module 1408 is used to generate an extended reality interaction page of the sub-application according to the elements in the extended reality interaction framework and the component information bound to the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

[0229] In this embodiment, by obtaining the page structure file corresponding to the sub-application, the obtained page structure file is used to describe the structure of the extended reality interaction page of the sub-application, so that the page layout of the sub-application to be generated can be known through the page structure file. The page structure file is parsed to obtain each predefined tag in the page structure file. The obtained predefined tags are predefined tags that can be converted into tags used in the extended reality interaction framework, so that each of the predefined tags can be converted into an element in the extended reality interaction framework. The attribute information corresponding to each of the predefined tags is obtained from the page structure file, and the attribute information corresponding to each of the predefined tags is converted into component information bound to the corresponding element, so that the predefined tags and related attribute information in the page structure file can be converted into elements and component information in the extended reality interaction framework, so that the extended reality interaction page of the sub-application can be generated according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements, so that the creation of the extended reality interactive sub-application can be realized through the extended reality interaction framework itself, without the need to introduce third-party processing, making the development of the extended reality interactive sub-application simpler, and effectively reducing the threshold for developers to get started with the development of extended reality interactive sub-applications. In addition, the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, so that the sub-application can provide users with an environment that combines the real and the virtual and the human-computer interaction.

[0230] In one of the embodiments, the attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag and the attribute data corresponding to the tag attribute; the component information includes the component associated with the corresponding element and the component configuration data bound to the component.

[0231] In this embodiment, each of the predefined tags in the page structure file is converted into an element in the extended reality interaction framework, the tag attributes corresponding to the predefined tags are converted into components associated with the elements, and the attribute data corresponding to the tag attributes are converted into component configuration data bound to the components, so that the various parts of the structure describing the extended reality interaction page can be converted to the extended reality interaction framework, so that the extended reality interaction page can be directly generated using the elements, components and component configuration data through the extended reality interaction framework, so that the development of extended reality interactive sub-applications can be realized without the need for additional processing by a third party, thereby lowering the development threshold of extended reality interactive sub-applications.

[0232] In one of the embodiments, the component information generation module 1406 is also used to generate attribute adding instructions for components in the extended reality interaction framework based on the tag attributes corresponding to each of the predefined tags; based on the attribute adding instructions, respectively obtain components matching each of the tag attributes from the component library of the extended reality interaction framework; for each of the tag attributes, establish an association between the component matching the tag attribute and the element converted from the corresponding predefined tag.

[0233] In this embodiment, based on the tag attributes corresponding to each of the predefined tags, attribute adding instructions for components in the extended reality interaction framework are generated, so that based on the attribute adding instructions, components matching each of the tag attributes are respectively obtained from the component library of the extended reality interaction framework, so that the tag attributes in the page structure file can be converted into corresponding components in the extended reality interaction framework, and for each of the tag attributes, an association is established between the component matching the tag attribute and the element converted from the corresponding predefined tag, so that the correspondence between the component and the element is consistent with the correspondence between the predefined tag and the tag attribute, thereby ensuring the accuracy of the converted data.

[0234] In one of the embodiments, the extended reality interaction framework includes a plurality of systems, each of the systems being used to process component configuration data of a corresponding component;

[0235] The page generation module 1408 is also used to determine the systems corresponding to each of the components in the extended reality interaction framework; for each of the components, the component configuration data bound to the component is sent to the system corresponding to the component; and each of the systems is driven to process the received component configuration data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0236] In this embodiment, the extended reality interaction framework includes multiple systems, each of which is used to process the component configuration data of the corresponding component. After converting the predefined tags, the tag attributes corresponding to the predefined tags, and the attribute data corresponding to the tag attributes in the page structure file into the components, the components associated with the elements, and the component configuration data bound to the components in the extended reality interaction framework, the system corresponding to each of the components in the extended reality interaction framework is determined to accurately determine which components need to be sent to which system for processing, so that for each of the components, the component configuration data bound to the targeted component is sent to the system corresponding to the targeted component. Each of the systems is driven to execute the processing of the received component configuration data, so that the corresponding page content is formed through the processing of the corresponding component configuration data by each system, so that the construction of the subroutine of the extended reality interaction class can be directly implemented in the extended reality interaction framework without the help of external tools, which reduces the development cost of the subroutine of the extended reality interaction class and improves the efficiency of the subroutine of the extended reality interaction class.

[0237] In one of the embodiments, the page generation module 1408 is also used to obtain the sub-reference code of each system in the extended reality interaction framework; fill each component configuration data into the sub-reference code of the corresponding system to obtain the component code; drive each system to run its own component code respectively to form the corresponding page content to obtain the extended reality interaction page of the sub-application.

[0238] In this embodiment, the sub-reference code of each of the systems in the extended reality interaction framework is obtained, and each of the component configuration data is filled into the sub-reference code of the corresponding system to obtain the component code, so that the corresponding component code can be automatically generated without the user writing code. Each of the systems is driven to run its own component code respectively to form the corresponding page content to obtain the extended reality interaction page of the sub-application, thereby building a user-defined extended reality interaction type sub-application, which can shorten the development cycle and improve development efficiency. In addition, through the addition of elements and components, and the binding of component configuration data, users who do not have a deep understanding of the development of extended reality interaction type sub-applications can also complete the development of various parts and functions of the sub-application, making the development of extended reality interaction type sub-applications simpler.

[0239] In one embodiment, the plurality of systems include a rendering system, and each of the components includes at least a component corresponding to the rendering system;

[0240] The page generation module 1408 is also used to obtain page rendering data from the component configuration data bound to the component corresponding to the rendering system; send the page rendering data to the rendering system; control the rendering system to perform page rendering according to the page rendering data to form corresponding page content to obtain the extended reality interaction page of the sub-application.

[0241] In this embodiment, page rendering data is obtained from component configuration data bound to the component corresponding to the rendering system, the page rendering data is sent to the rendering system, and the rendering system is controlled to perform page rendering according to the page rendering data to form corresponding page content, so that when developing sub-applications of the extended reality interaction class, there is no need to rely on a third-party rendering engine, and page rendering can be performed directly through the rendering system provided in the extended reality interaction framework, which makes it easy for developers who do not understand the relevant knowledge of third-party rendering engines to develop sub-applications of the extended reality interaction class, thereby lowering the threshold for developing sub-applications of the extended reality interaction class.

[0242] In one embodiment, the multiple systems also include an animation system, and each of the components also includes a component corresponding to the animation system; the page generation module 1408 is also used to obtain animation data from component configuration data bound to the component corresponding to the animation system, and send the animation data to the animation system; control the rendering system to perform page rendering according to the page rendering data, and control the animation system to generate animation content according to the animation data, forming page content including the animation content, so as to obtain an extended reality interaction page of the sub-application.

[0243] In this embodiment, the multiple systems also include an animation system, and each of the components also includes a component corresponding to the animation system. Animation data is obtained from component configuration data bound to the component corresponding to the animation system, and the animation data is sent to the animation system. The rendering system is controlled to perform page rendering according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data to generate dynamic page content and static page content. Therefore, when developing sub-applications of the extended reality interaction class, there is no need to rely on third-party related engines. Page rendering and animation generation can be performed directly through the rendering system and animation system provided in the extended reality interaction framework. Therefore, developers are not required to have relevant knowledge of third-party related engines to develop sub-applications of the extended reality interaction class, thereby lowering the threshold for developing sub-applications of the extended reality interaction class.

[0244] In one of the embodiments, the element generation module 1404 is further used to generate element creation instructions for the extended reality interaction framework according to each of the predefined tags; based on the element creation instructions, obtain elements that match each of the predefined tags from the element library of the extended reality interaction framework.

[0245] In this embodiment, an element creation instruction for the extended reality interaction framework is generated according to each of the predefined tags, so that based on the element creation instruction, elements that match each of the predefined tags are obtained from the element library of the extended reality interaction framework, thereby enabling the predefined tags in the page structure file to be accurately converted into corresponding elements in the extended reality interaction framework.

[0246] In one of the embodiments, the page generation module 1408 is also used to obtain a reference frame code corresponding to the extended reality interaction framework; adjust the reference frame code according to the elements in the extended reality interaction framework and the component information bound to the elements to obtain the page frame code of the sub-application; the page frame code of the sub-application is used to generate an extended reality interaction page of the sub-application.

[0247] In this embodiment, the reference frame code corresponding to the extended reality interaction framework is obtained, and the reference frame code is adjusted according to the elements in the extended reality interaction framework and the component information bound to the elements to obtain the page frame code of the sub-application, so that the sub-application of the user-defined extended reality interaction type can be automatically generated without the user writing code, which can shorten the development cycle and improve development efficiency. In addition, through the addition of elements and components, and the binding of component configuration data, users who do not understand code can also develop sub-applications by themselves, making the development of sub-applications simpler.

[0248] In one of the embodiments, the device also includes an interaction module; the interaction module is used to call a camera in the extended reality interaction page to capture a real scene through the camera; obtain pre-stored three-dimensional model materials, and generate a three-dimensional virtual object based on the three-dimensional model materials; synthesize the real scene and the three-dimensional virtual object, and display the enhanced scene picture obtained by the synthesis on the extended reality interaction page.

[0249] In this embodiment, a camera is called in the extended reality interaction page, and the real scene is captured by the camera to obtain pre-stored three-dimensional model materials, and a three-dimensional virtual object is generated based on the three-dimensional model materials. The real scene and the three-dimensional virtual object are synthesized so that the enhanced scene picture obtained by the synthesis can be displayed in the sub-application, and the scene picture combining the real and virtual can be presented through the sub-application, thereby providing users with a real and virtual environment through the extended reality interaction sub-application.

[0250] In one of the embodiments, the interaction module is used to obtain the posture information of the camera, generate a virtual scene matching the real scene based on the posture information of the camera; merge the three-dimensional virtual scene into the virtual scene, and obtain an enhanced scene picture displayed on the extended reality interaction page.

[0251] In this embodiment, the posture information of the camera is obtained, and a virtual scene matching the real scene is generated based on the posture information of the camera, so as to merge the three-dimensional virtual scene into the virtual scene, and obtain an enhanced scene picture displayed on the extended reality interaction page, so that virtual content can be superimposed on the real scene of the camera in real time, and the virtual content can be seamlessly combined with the real environment to form a visual effect that is indistinguishable from the real thing.

[0252] Each module in the above-mentioned sub-application processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0253] In one embodiment, a computer device is provided, which may be a terminal or a server. Taking a terminal as an example, its internal structure diagram may be as follows: Fig.15As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a sub-application processing method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0254] Those skilled in the art will understand that Fig.15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0255] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0256] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0257] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0258] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0259] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0260] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0261] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A sub-application processing method, characterized in that: The method comprises: Obtaining a page structure file corresponding to the sub-application, wherein the page structure file is used to describe the structure of the extended reality interaction page of the sub-application; Parsing the page structure file to obtain each predefined tag in the page structure file, and converting each predefined tag into an element in the extended reality interaction framework; Acquire the attribute information corresponding to each of the predefined tags from the page structure file, and convert the attribute information corresponding to each of the predefined tags into component information bound to the corresponding element; An extended reality interaction page of a sub-application is generated according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

2. The method according to claim 1, characterized in that The attribute information corresponding to the predefined tag includes the tag attribute corresponding to the predefined tag and the attribute data corresponding to the tag attribute; the component information includes the component associated with the corresponding element and the component configuration data bound to the component.

3. The method according to claim 2, characterized in that The step of converting the attribute information corresponding to each of the predefined tags into component information bound to the corresponding element includes: Based on the tag attributes corresponding to each of the predefined tags, generating a property adding instruction for a component in the extended reality interaction framework; Based on the attribute adding instruction, respectively obtaining components matching each of the tag attributes from the component library of the extended reality interaction framework; For each of the tag attributes, an association is established between the component matching the tag attribute and the element converted from the corresponding predefined tag.

4. The method according to claim 2, characterized in that: The extended reality interaction framework includes a plurality of systems, each of which is used to process component configuration data of a corresponding component; The generating of the extended reality interaction page of the sub-application according to each of the elements in the extended reality interaction framework and the component information bound to each of the elements includes: Determine a system corresponding to each of the components in the extended reality interaction framework; For each of the components, the component configuration data bound to the component is sent to a system corresponding to the component; The systems are driven to process the received component configuration data to form corresponding page contents, so as to obtain an extended reality interaction page of the sub-application.

5. The method according to claim 4, characterized in that The driving system processes the received component configuration data to form corresponding page content to obtain an extended reality interaction page of the sub-application, including: Obtaining respective sub-reference codes of the systems in the extended reality interaction framework; Fill each of the component configuration data into the corresponding system sub-reference code to obtain a component code; The systems are driven to respectively run their respective component codes to form corresponding page contents, so as to obtain the extended reality interaction page of the sub-application.

6. The method according to claim 4, characterized in that The multiple systems include a rendering system, and each of the components includes at least a component corresponding to the rendering system; For each of the components, sending the component configuration data bound to the component to a system corresponding to the component includes: Acquire page rendering data from component configuration data bound to the component corresponding to the rendering system; Sending the page rendering data to the rendering system; The driving each of the systems to execute the received component configuration data to form corresponding page content to obtain the extended reality interaction page of the sub-application includes: The rendering system is controlled to perform page rendering according to the page rendering data to form corresponding page content to obtain an extended reality interaction page of the sub-application.

7. The method according to claim 6, characterized in that The multiple systems further include an animation system, and each of the components further includes a component corresponding to the animation system; the method further includes: Acquire animation data from component configuration data bound to the component corresponding to the animation system, and send the animation data to the animation system; The controlling the rendering system to perform page rendering according to the page rendering data to form corresponding page content to obtain an extended reality interaction page of the sub-application includes: The rendering system is controlled to perform page rendering according to the page rendering data, and the animation system is controlled to generate animation content according to the animation data to form page content including the animation content, so as to obtain an extended reality interaction page of the sub-application.

8. The method according to claim 1, characterized in that The converting each of the predefined tags into an element in the extended reality interaction framework includes: Generating, according to each of the predefined tags, an element creation instruction for the extended reality interaction framework; Based on the element creation instruction, elements matching each of the predefined tags are obtained from an element library of the extended reality interaction framework.

9. The method according to claim 1, characterized in that: The generating an extended reality interaction page of the sub-application according to the element in the extended reality interaction framework and the component information bound to the element includes: Obtaining a reference frame code corresponding to the extended reality interaction frame; According to the elements in the extended reality interaction framework and the component information bound to the elements, the reference framework code is adjusted to obtain the page framework code of the sub-application; the page framework code of the sub-application is used to generate the extended reality interaction page of the sub-application.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Invoke a camera in the extended reality interaction page to capture a real scene through the camera; Acquire pre-stored three-dimensional model materials, and generate a three-dimensional virtual object based on the three-dimensional model materials; The real scene and the three-dimensional virtual object are synthesized, and an enhanced scene picture obtained by the synthesis is displayed on the extended reality interaction page.

11. The method according to claim 10, characterized in that The synthesizing the real scene and the virtual object, and displaying the synthesized enhanced scene picture on the extended reality interaction page, comprises: Acquiring the posture information of the camera, and generating a simulated scene simulating the real scene based on the posture information of the camera; The three-dimensional virtual object is integrated into the simulated scene to obtain an enhanced scene image displayed on the extended reality interaction page.

12. A sub-application processing device, characterized in that: The device comprises: An acquisition module, used to acquire a page structure file corresponding to a sub-application, wherein the page structure file is used to describe the structure of an extended reality interaction page of the sub-application; An element generation module, used to parse the page structure file, obtain predefined tags in the page structure file, and convert the predefined tags into elements in the extended reality interaction framework; A component information generation module, used to obtain the attribute information corresponding to the predefined tag from the page structure file, and convert the attribute information corresponding to the predefined tag into component information bound to the element; A page generation module is used to generate an extended reality interaction page of a sub-application based on the elements in the extended reality interaction framework and the component information bound to the elements; the extended reality interaction page is used to display the interaction between the real scene and the virtual scene, and the sub-application runs on the parent application.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Citation Information

Cited By

  • A system for enabling creators to add games independently

    WO2026226103A1