XR application development systems, methods, devices, and media for terminal devices

Abstract

The objective is to provide XR application development systems, methods, devices, and media for terminal devices. [Solution] In some embodiments, the XR application development system includes a runtime container, a content generator, and an application generation platform. The runtime container is integrated with the content generator and generates application content for the XR application based on the runtime environment provided by the runtime container. Once the application content is generated, the application generation platform generates the XR application based on the application content and the runtime container. The runtime container operates between the application content and the operating system and provides a runtime environment for the application content. By adding a runtime container between the application content and the operating system, strong dependencies between the application content and the operating system are eliminated, enabling cross-platform and cross-device development and execution of XR applications. This simplifies the development and implementation of XR applications and solves problems related to the development and cross-device operation of XR applications.

Claims

1. A runtime container for generating the XR application, configured to provide a runtime environment during the generation and execution of the XR application, A content generator configured to internally integrate a runtime container and generate application content for the XR application based on the runtime environment provided by the runtime container, Includes an application generation platform configured to generate the XR application based on the application content and the runtime container, The runtime container operates between the application content and the operating system of the target device where the XR application resides during execution of the XR application. The aforementioned application content operates within the runtime environment provided by the runtime container. An XR application development system for terminal devices, characterized by the following features.

2. The runtime container comprises a framework layer, a runtime environment layer, and a software library layer. The software library layer includes a plurality of software engines, provides a second type of application programming interface (API) to each of the plurality of software engines, and is used to encapsulate and retrieve a first type of API based on the second type of API. The aforementioned framework layer is By providing the application developer with the aforementioned type of API, the application developer can write the application logic code for the XR application based on the aforementioned type of API. The target event is detected during the generation or execution of the aforementioned XR application. Execute at least one of the above-mentioned API function entities corresponding to the target event, The target API encapsulated within the function entity is provided to the runtime environment layer. The aforementioned target API belongs to the second type of API, The runtime environment layer responds to the target event by calling the corresponding software engine in the software library layer based on the target API. The system according to claim 1, configured as follows.

3. The software library layer is further configured to provide the second type of API to the runtime environment layer, encapsulate the second type of API to obtain the first type of API, and provide the first type of API to the framework layer so that the framework layer can provide the first type of API to the application developer. Alternatively, the software library layer is further configured to provide the second type of API to the runtime environment layer, the runtime environment layer encapsulates the second type of API to obtain the first type of API, and the framework layer is further configured to provide the first type of API to the framework layer in order to provide the first type of API to the application developer. The system according to claim 2.

4. The framework layer, in order to sense the target event during the execution of the XR application, Configured to generate a trigger event to execute the XR application in response to a trigger operation on the XR application's icon or access entry. and / or, in response to an interaction operation during the execution of the XR application, generate an interaction event with the target object of the XR application, which includes a 3D scene, a 2D object, or at least one 3D model in the interaction region. The system according to claim 2, configured as follows.

5. The runtime environment layer responds to the target event based on the target API by calling the corresponding software engine in the software library layer. When the target API includes the second type of API corresponding to the software library layer download engine, the operation of calling the download engine to download the application content for the XR application from the server, When the target API includes the second type API corresponding to the storage engine of the software library layer, an operation to invoke the storage engine to manage the storage of the application content for the XR application, When the target API includes the second type API corresponding to the software library layer's lifecycle management engine, the operation of calling the lifecycle management engine to manage the lifecycle of the XR application, When the target API includes the second type of API corresponding to the 3D rendering engine of the software library layer, the operation of calling the 3D rendering engine to perform 3D rendering of the XR application, When the target API includes the second type of API corresponding to the AI ​​inference engine of the software library layer, the operation of calling the AI ​​inference engine to perform the AI ​​inference process for the XR application, When the target API includes the second type of API corresponding to the 2D UI engine of the software library layer, the operation of calling the 2D UI engine to handle UI interaction between the XR application and the host 2D application, When the target API includes the second type of API corresponding to the event engine of the software library layer, the operation of calling the event engine to process the event callback logic of the XR application, Perform at least one of the following: The system according to claim 2, configured as follows.

6. The runtime environment layer includes at least one runtime virtual machine (VM) to provide the runtime environment for the XR application implemented using different dynamic programming languages, wherein the different runtime virtual machines (VMs) include the runtime VMs implemented using different dynamic programming languages. The system according to any one of claims 2 to 5.

7. The runtime environment layer includes a VM scheduling module configured to schedule, when there are multiple runtime virtual machines (VMs), the target VMs of the multiple runtime virtual machines (VMs) implemented using the same dynamic programming language as the XR application, based on the dynamic programming language used by the XR application, so that the target VMs provide the runtime environment for the XR application. The system according to claim 6, further comprising:

8. The content generator, in response to an XR application generation operation, reads static resources of a 3D scene and adds dynamic information of the 3D scene and application logic code to the static resources of the 3D scene to generate the application content for the XR application, based on the runtime environment provided by the runtime container. The system according to any one of claims 2 to 5, specially configured as such.

9. The aforementioned content generator, A 3D scene effect editing module is configured to respond to the XR application generation operation, load the static resources of the 3D scene, and edit the dynamic information of the 3D scene based on the runtime environment provided by the runtime container so that the static resources of the 3D scene generate 3D scene resource data. A 3D application logic editing module is configured to edit the application logic code for the 3D scene resource data based on the runtime environment provided by the runtime container in order to generate the application content for the XR application, The system according to claim 8, further comprising:

10. The aforementioned 3D scene effect editing module is: In response to the loading operation of the static resources of the 3D scene, the runtime container is started and the static resources of the 3D scene are executed within the runtime environment provided by the runtime container in order to generate a 3D static image. In response to editing operations on the 3D static image, the dynamic information of the 3D scene is edited for the static resources of the 3D scene in order to generate first intermediate resource data. In response to an effect display trigger operation, the runtime container is restarted, and the first intermediate resource data is executed based on the runtime environment provided by the runtime container in order to generate a first intermediate 3D image. If the first intermediate 3D image does not achieve scene effects, the dynamic information of the 3D scene is adjusted to achieve scene effects, and 3D scene resource data is generated. The system according to claim 9, specially configured as such.

11. The aforementioned application logic editing module is In response to a trigger operation for editing the application logic code, the runtime container is started and the 3D scene resource data is executed within the runtime environment provided by the runtime container in order to generate a first intermediate 3D image. In response to the loading operation of the application logic code, second intermediate resource data is generated based on the 3D scene resource data and the application logic code, the runtime container is restarted, and the second intermediate resource data is executed within the runtime environment provided by the runtime container in order to generate a second intermediate 3D image. If the second intermediate 3D image does not achieve the interaction effect, the application logic code is adjusted to achieve the interaction effect and the application content for the XR application is generated. The system according to claim 9, specially configured as such.

12. A runtime container necessary for generating XR applications and providing a runtime environment during the XR application generation process and execution process is developed in advance, and the runtime container is integrated into the content generator. In the content generator integrated into the runtime container, application content for the XR application is generated based on the runtime environment provided by the runtime container. The process involves generating the XR application based on the application content and the runtime container. During the execution process of the XR application, the runtime container operates between the application content and the operating system of the target device hosting the XR application, and the application content operates within the runtime environment provided by the runtime container. A method for developing the XR application for a terminal device.

13. In the content generator integrated into the runtime container, generating the application content of the XR application based on the runtime environment provided by the runtime container is: In response to an XR application generation operation, static resources of the 3D scene are loaded. To obtain 3D scene resource data, dynamic information of the 3D scene is added to the static resources of the 3D scene based on the runtime environment provided by the runtime container. To obtain the application content for the XR application, the application logic code for the 3D scene resource data is edited based on the runtime environment provided by the runtime container. The method according to claim 12, which includes the following:

14. Generating the XR application based on the application content and the runtime container is The runtime container and the application content are integrated into the installation package of the XR application, and the installation package of the XR application is made public. and / or, integrate the runtime container into the installation package of a host 2D application which is a 2D application that needs to embed the XR application, and expose the application content to a resource server corresponding to the host 2D application. The method according to claim 12 or 13.

15. Memory used to store computer programs, A processor connected to the memory and configured to execute the computer program in order to perform the method described in any one of claims 12-14, An electronic device having

16. A computer-readable storage medium for storing a computer program that, when executed by the processor, causes the processor to perform a procedure according to any one of claims 12 to 14.

Images