Method for running interactable virtual object in virtual (3D) space in isolated manner, and device and computer program product

Abstract

The present application relates to a method for running an interactable virtual object in a virtual (3D) space in an isolated manner, the method comprising: in response to a starting instruction of a user in a parent space, loading in the parent space a program package corresponding to an interactable virtual object, wherein the parent space comprises one or more child spaces; parsing the program package to acquire attribute information, space information and script information from the program package; and on the basis of the space information and / or the script information, performing rendering in one of the child spaces to obtain the interactable virtual object, wherein the interactable virtual object can accept one or more interaction operations of the user and perform feedback, and a virtual (3D) space is constructed by means of an augmented reality device and can be browsed by the user in the parent space by means of a display module of the augmented reality device. In the present application, by means of lightweight web development technology, a space application suitable for an augmented reality device can be rapidly, simply and efficiently developed and deployed.

Description

A method, device, and computer program product for isolating and operating interactive virtual objects in a virtual (3D) space

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 202311651289.9 and application date December 4, 2023, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to the technical field of software systems, and in particular to a method, device, and computer program product for isolating and operating interactive virtual objects in a virtual (3D) space. Background Art

[0004] In augmented reality (AR) or virtual reality (VR) scenarios, screen size isn't limited; instead, space can be displayed. Space refers to an area in the real world, which can be a flat surface or a three-dimensional object. Users can access information about the space, such as its size, position, and rotation angle, through programs running within it. In typical AR application development, a spatial scene is used to host the AR application, meaning that the AR application runs within that space. Applications that occupy a specific space are also referred to as spatial apps.

[0005] The key technical point of space APP development is how to run another independent application in a subspace in the parent space (main application) without affecting the operation of the main application. Summary of the Invention

[0006] The purpose of this application is to provide a new method and device for isolating and running interactive virtual objects in a virtual (3D) space based on Web technology.

[0007] According to a first aspect of the present application, a method for isolating and running interactive virtual objects in a virtual (3D) space is provided, comprising: loading a program package corresponding to the interactive virtual object in the parent space in response to a user's startup instruction in the parent space, the parent space including one or more child spaces; parsing the program package to obtain attribute information, spatial information and script information in the program package; rendering an interactive virtual object in one of the child spaces based on the spatial information and / or script information, the interactive virtual object being capable of accepting one or more interactive operations of the user and providing feedback; wherein the virtual (3D) space is constructed by an augmented reality device and can be browsed by a user in the parent space through a display module of the augmented reality device.

[0008] According to a second aspect of the present application, an augmented reality device is provided, comprising a display module, the display module being configured to form a visual 3D space in front of a user's eyes; a memory being configured to store computer instructions; and a processor being configured to execute the computer instructions; wherein when the computer instructions are executed by the processor, the interactive virtual object is displayed in the 3D space presented by the display module, the device comprising: loading an XML program package containing spatial information corresponding to the interactive virtual object in a parent space, the parent space including one or more child spaces; parsing the program package to obtain attribute information, spatial information, and script information in the XML program package containing spatial information; and rendering an interactive virtual object in one of the child spaces according to the spatial information and / or script information, wherein the interactive virtual object can accept one or more interactive operations of the user and provide feedback.

[0009] According to a third aspect of the present application, a computer program product is provided for running interactive virtual objects in an augmented reality device, comprising: an XML document and optional file content, wherein the XML document includes attribute information, spatial information, and script information; when the XML document is executed by the augmented reality device in a parent space, the interactive virtual object can be rendered in a child space of the parent space through the spatial information and / or script information, and the interactive virtual object can accept one or more interactive operations of the user and provide feedback.

[0010] One or more embodiments of this application implement a method, device, and computer program product for isolating and operating interactive virtual objects in a virtual (3D) space. Through lightweight Web development technology, spatial applications suitable for augmented reality devices can be developed quickly, simply, and efficiently. This allows for the easy development and deployment of lightweight spatial applications such as alarm clocks and pets that display and interact with interactive virtual objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG1 is a schematic diagram of an augmented reality device according to one or more embodiments of the present application;

[0012] FIG2 is a schematic diagram of a spatial structure in an augmented reality device according to one or more embodiments of the present application;

[0013] 3 is a flowchart of a method for isolating and operating an interactive virtual object in a virtual (3D) space according to one or more embodiments of the present application;

[0014] FIG4 is a schematic diagram of an interactive virtual object operation interface according to one or more embodiments of the present application;

[0015] FIG5 is a schematic diagram of a computer program product framework according to one or more embodiments of the present application;

[0016] FIG6 is a framework diagram of an augmented reality device according to one or more embodiments of the present application. DETAILED DESCRIPTION

[0017] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation methods, structures, features and effects of the method, device, and computer product for remotely operating interactive virtual objects in a virtual (3D) space proposed in this application.

[0018] In the augmented reality (AR) scenario, space refers to an area in the real world, which can be a plane or a three-dimensional object. Users obtain information about the space through virtual objects running in the space, such as the size of the space, the position of the space, the rotation angle of the space, etc., and in general AR space application development, a scene is used to carry the space. Embeddable space, or subspace, refers to a complete space that contains some objects, and then this space can be embedded in the parent space as a component of the parent space. In this way, you can: 1. Move, rotate, and scale the subspace at will within the parent space; 2. Directly interact with objects in the subspace within the parent space; 3. Combine multiple subspaces within the parent space to form a more complex space.

[0019] Augmented reality scenarios can generally be implemented through augmented reality devices (such as augmented reality glasses, i.e., AR glasses). As shown in FIG1 , an exemplary system architecture 100 may include augmented reality glasses 11 and an optional computing terminal 12 .

[0020] The augmented reality glasses 11 may include one or two display screens 111. The above-mentioned display screens are used to display a virtual interface. In addition, the augmented reality glasses 11 also include a frame 112. In some embodiments, the sensors, processing units, memory and batteries of the augmented reality glasses 11 can be placed inside the frame 112. In some embodiments, one or more components of the sensors, processing units, memory and batteries can also be integrated into another independent accessory (not shown in Figure 1) and connected to the frame 112 via a data cable. In some embodiments, the augmented reality glasses 11 may only have a display function and some sensors, and the computing terminal 12 provides data processing, data storage, power supply and other capabilities.

[0021] The computing terminal 12 may include a touch-sensitive display screen 121. In some embodiments, the augmented reality glasses 11 and the computing terminal 12 may communicate via a wireless connection. In some embodiments, the augmented reality glasses 11 and the computing terminal 12 may also be connected via a data cable (not shown in FIG. 1 ). It should be understood that the number of augmented reality glasses 11 and computing terminals 12 shown in FIG. 1 is merely illustrative. Any suitable number of augmented reality glasses 11 and computing terminals 12 may be used depending on implementation needs.

[0022] The two display screens 111 of the augmented reality glasses 11 can present a virtual (3D) space in front of the user's eyes. The 3D spatial effect is caused by the distance between the human eyes causing each eye to see different perspectives when observing the same object. This difference, called binocular disparity, is the basis for depth perception. AR glasses simulate this disparity to create a sense of three-dimensionality. Augmented reality glasses 11 are typically equipped with binocular cameras to simulate human eyes. These two cameras are mounted on the glasses, typically at a distance similar to that of human eyes. The binocular cameras capture real-time images of the surrounding environment, corresponding to the perspectives of the left and right eyes, respectively. The captured images provide depth information of the scene, helping to more accurately render the 3D effect. The captured left and right eye images are fed into the processing unit built into the augmented reality glasses 11. The processing unit is responsible for image processing and calculations to generate images suitable for each eye. The processing unit adjusts the images based on binocular disparity, ensuring that there is some visual difference between the images seen by the left and right eyes, thereby creating a sense of depth. This usually involves adjusting the image's displacement, angle, or perspective. The adjusted image is projected to the left and right eyes respectively through the display screen or projection technology on the glasses. This allows each eye to see a slightly different image, creating a sense of depth. When the user wears the augmented reality glasses 11, each eye sees a unique image, and through brain processing, these images are combined into a three-dimensional scene. The 3D effect perceived by the user is simulated by the difference between what the two eyes see, which enables virtual objects and real objects to appear deep and realistic in 3D space. In this way, the augmented reality glasses 11 can provide users with a realistic, three-dimensional 3D augmented reality experience.

[0023] Similar to a flat (2D) display, applications can also be run or objects can be realized in a virtual (3D) space. The difference is that the applications or objects running in the virtual (3D) space can be three-dimensional and have three-dimensional spatial coordinates. When a three-dimensional object is displayed in a virtual (3D) space, the three-dimensional object occupies a certain spatial position and spatial size in the virtual (3D) space. According to one or more embodiments of the present application, three-dimensional objects can include basic shapes such as spheres and cubes, and can also include complex objects composed of basic shapes, such as alarm clock shapes, galaxy shapes, animal shapes, etc. Objects that can interact with users are called interactive virtual objects. According to one or more embodiments of the present application, interactive virtual objects can accept user interaction commands, including but not limited to clicking, rotating, zooming, etc. For example, for a virtual pet dog, the user can use the interactive capabilities provided by the augmented reality device to perform interactive actions such as patting the virtual pet dog, and the virtual pet dog can respond to the user's patting action and make corresponding feedback (such as nodding). Interactive virtual objects can be considered as a simple spatial application. In order to ensure that interactive virtual objects are compatible across different platforms, a set of standards is needed to regulate the operation and interaction of interactive virtual objects.

[0024] In the field of augmented reality, degrees of freedom (DoF) are generally used to describe the realism of a space. The higher the DOF, the more realistic the space. Generally speaking, an object's position (Position) and attitude (Rotation) can be used to quantitatively describe its pose in space. Therefore, when describing the movement of an object in space, the object's movement can be decomposed into displacement and attitude. In the XYZ coordinate system, displacement can be expressed as X-axis offset, Y-axis offset, and Z-axis offset, while attitude can be characterized as X-axis rotation angle (pitch angle), Y-axis rotation angle (yaw angle), and Z-axis rotation angle (roll angle). Therefore, for an object in space, if it moves freely in space, it will produce changes in the three axes (X, Y, and Z) caused by movement, as well as angular changes caused by changes in attitude. These degrees of freedom used to describe the changes in the object's position in space are expressed as DoF.

[0025] 0DoF (Zero Degree of Freedom) means that an observer in the current space observes the virtual space from a fixed perspective. 3DoF (Three Degrees of Freedom) means that the observer in the current space can use three degrees of freedom. Typically, these three degrees of freedom refer to the posture of the space. In virtual space, if the camera only changes its posture, this space is said to have three degrees of freedom—3DoF. This means that objects in the virtual space change their positions in space according to programmatic control, while the observer remains at the origin of the space and observes the virtual space from different postures. 6DoF (Six Degrees of Freedom) means that the observer in the current space can use six degrees of freedom: position and rotation combined, for a total of six. When a camera in the virtual space can change both position and rotation, this space is said to have six degrees of freedom—6DoF. The observer (camera) can observe the virtual space from any pose.

[0026] Interactive virtual objects can have one of three degrees of freedom: 0DOF, 3DOF, or 6DOF. The degrees of freedom of an interactive virtual object are generally determined based on the degrees of freedom of the parent space in which it resides. They can be consistent with the degrees of freedom of the parent space, or the parent space can assign appropriate degrees of freedom to them. This allows for the most appropriate interaction method to be set based on the characteristics of different interactive virtual objects. In one embodiment, for a virtual object such as an alarm clock, it can be set to have 0DOF. The alarm clock will always be displayed within the user's observation camera (FOV), thus providing real-time reminders to the user. In one embodiment, for a virtual human assistant, it can be set to have 3DOF. This allows the user to observe and interact with the virtual human without being in sight while focusing on other interface operations, simply by adjusting different postures. In one embodiment, for an interactive virtual object such as a pet, it can be set to have 6DOF. This allows for more realistic interactions with the pet by changing the user's position and posture, enhancing the user experience.

[0027] Since interactive virtual objects are a lightweight spatial application, there may be multiple interactive virtual objects in the running parent space. Therefore, how to ensure that these interactive virtual objects do not affect each other's operation, and how to ensure that the stability of the parent space is not affected after the appearance of multiple interactive virtual objects is one of the key technical issues of the present disclosure.

[0028] As shown in FIG2 , a method for isolating and operating interactive virtual objects in a virtual (3D) space according to one or more embodiments of the present application is shown, including:

[0029] S1. Loading a program package corresponding to an interactive virtual object in a parent space, where the parent space includes one or more child spaces;

[0030] S2. Parse the program package to obtain attribute information, space information, and script information in the program package;

[0031] S3. Render an interactive virtual object in one of the subspaces according to the spatial information and / or script information. The interactive virtual object can accept one or more interactive operations of the user and provide feedback.

[0032] The virtual (3D) space is constructed by an augmented reality device and can be browsed by a user in the parent space through a display module of the augmented reality device.

[0033] In step S1, a program package corresponding to an interactive virtual object is loaded into a parent space, where the parent space includes one or more child spaces. In one embodiment, the program package can be described using XML (eXtensible Markup Language) that contains spatial information. XML is a markup language that provides rules for defining any data. Unlike other programming languages, XML itself cannot directly perform computational operations. In contrast, any programming language or software can be used for structured data management. XML that contains spatial information is XML that contains spatial information on top of the general XML language. It can be referred to as eXtensible Spatial Markup Language (XSML), which stands for extensible spatial markup language. In one or more embodiments of the present application, XSML is a markup language very similar to HTML. It uses some HTML tag definitions, such as head, title, script, style, div, etc., but also adds some tags to represent space, such as replacing body with space and using cube to represent a cube. It is understood that in other embodiments, other markup languages ​​similar to those containing spatial information can also be used to describe interactive virtual objects.

[0034] In one or more embodiments, the parent space can be the system desktop of the operating system allowed by the augmented reality device, and the system desktop can be understood as a space application. The system desktop can display content such as a status bar, an application list, and running application pages. As shown in Figure 3, the system desktop in the augmented reality device is set as a cylindrical space. In this space, the user (USER) can see a series of virtual screens (1, 2, 3) surrounding the space. These virtual screens can display application lists and open application content. At the same time, the user can also see some corresponding independent interactive virtual objects (4, 5, 6). These virtual objects correspond to independent subspaces in the desktop space and do not affect each other during operation. In this application, we also refer to interactive virtual objects as space applets.

[0035] In order to enable the interactive virtual object to run on the system desktop, the user can start the XML package corresponding to the interactive virtual object by clicking an icon, etc., and the system will start loading the XML package of the interactive virtual object. The XML package of the interactive virtual object includes: an XML document containing spatial information and optional file content, wherein the file content includes one or more of a script file, a model file, and a sound effect file. These files can be loaded through the network and locally. In order to increase the loading speed, the resources used can be cached locally when the package file is running, so that they can be read directly from the local area the next time it is loaded. During the resource loading process, md5 can be used to determine whether the package content has changed and whether the cache needs to be updated. If the caching solution is not adopted (that is, the .md5 file is not generated), each time the resource is loaded, it will be loaded from the network instead of using the local cache, so that the loading logic can be kept consistent with the web page.

[0036] In step S2, the program package is parsed to obtain attribute information, spatial information, and script information in the program package. The XML program package includes an XML document and optional file content. The XML document may include attribute information, spatial information, and script information. The optional file content may include one or more of a script file, a model file, and a sound effect file. The script file is linked to the script information. Accordingly, in response to the script information being parsed, the script file is executed on the augmented reality device.

[0037] Script information in XML documents is constructed using languages ​​such as TypeScript and JavaScript. Preferably, TypeScript is used for this purpose. This is because 3D space development, with its more complex object relationships, requires type checking for JavaScript. As a superset of JavaScript, TypeScript provides type checking, enabling developers to detect errors earlier in the development process and improve development efficiency. Furthermore, because TypeScript is a superset of JavaScript, supporting TypeScript also means supporting JavaScript.

[0038] The following table shows two examples of XML packages. Attribute information represents the basic information of the XML package, which may include version number, title, link, style, etc. The version number can be used <version>To indicate the version number of the XML package, such as 1.0, 2.0, etc., to indicate the version number of the XML document used. The title can be used <title>To define, used to define the title of the XML package, its content is a string. Links can be used< / title> <link> Tags to define link information of external resources, for example, <link> Tags reference 3D model files, such as 3D model files in glb format; you can also use <link> The tag introduces SCSS (Spatial CSS, spatial cascading style sheets) and other formats of model files to present richer display effects of interactive virtual objects.

[0039] Spatial information includes style information described by XML documents. Spatial information can be used <space>Tags to define. <space>The tag is used to define the spatial structure. Its subtags include: mesh, cube, sphere, cylinder, cone, plane, text, image, video, audio, etc. <space>The combination of sub-tags in the tag can build a rich interactive virtual object, for example, <space> 3<cube width="10"height="10"depth="10" / > 4< / space> A cube with a side length of 10 is defined in space. In some embodiments <space>The label can be in the default state, that is, <space>The tag does not contain any code, which means that the space is in a vacuum state and does not contain any space elements defined by XSML. However, this does not mean that there is no content in the space. In this case, one or more embodiments of the present application also support <script>标签中的脚本信息描述的样式信息来对相应的空间进行渲染，其中脚本信息描述的样式信息可基于Babylon.js框架。在一个具体的实施例中，可以通过子标签的组合在空间中定义复杂的可交互虚拟物体，例如闹钟、狮子等。

[0040] 在一个或多个实施例中，可以对空间信息中定义的可交互虚拟物体进行更改。可以先识别需要调整的可交互虚拟物体的物体ID，然后通过物体ID的路径进行内部结构的访问，最后可以使用遍历的方法选择该物体内部的子元素，然后使用样式属性对该元素的样式进行调整。在一个支持DOM API空间小程序开发框架的实施例中，基于DOM API，可以新增spatialDocument对象，它表述为空间中的document对象，并通过spatialDocument对象的getSpatialObjectById('foo')来查询到id为foo的3D物体对象。访问这个3D物体的shadowRoot以与其内部结构进行交互。利用querySelectorAll()来选择3D物体内的所有子元素，使用style属性修改所选子元素的样式。除此之外，还可以使用addEventListener()在这些子元素上监听事件。根据本申请的一个或多个实施例，上述函数名也可以调整为其他名称，只需其实现类似的功能即可。

[0041] S3、根据所述空间信息和 / 或脚本信息在其中一个子空间中渲染出可交互虚拟物体，所述可交互虚拟物体可接受用户的一个或多个交互操作并进行反馈。当所述XML文档被处理器解析时，处理器会根据空间信息和 / 或脚本信息，对虚拟物体进行渲染，并呈现在增强现实设备中，即XML文档在父空间执行时，可以通过空间信息和 / 或脚本信息，在父空间的一个子空间中渲染出可交互虚拟物体，可交互虚拟物体可接受用户的一个或多个交互操作并进行反馈。最终的渲染结果可以通过Unity、Unreal、Apple RealityKit等3D引擎执行。在一个优选的实施例中，在渲染可交互虚拟物体时，为可交互虚拟物体创建独立的数据通道，以保证每个可交互物体的渲染是独立的。本方法和Unity引擎的数据通信可以通过Unity SDK提供的接口执行。

[0042] 如图4所示，示出了根据本申请一个或多个实施例的可交互虚拟物体运行界面400的示意图。其中界面400为增强现实设备的系统桌面，可以视为一个父空间。该父空间中包含两个浏览器界面42，以及系统菜单43，以及手势示意图44。该父空间中还包括一个空间小程序41(虚线框范围内)，其运行在浏览器界面42之前，并且可以响应于用户的点击操作，而进行旋转、放大、移动等操作。通过本申请的一个或多个实施例来执行该空间小程序41，用户可以在不影响其他程序运行的情况下，轻松在父空间中打开空间小程序41，并执行空间小程序41的功能，且不影响其他界面元素的运行。

[0043] 根据本申请的一个或多个实施方式，每个子空间具有由父空间设置的空间位置和体积大小，父空间和子空间具备相同的自由度，且自由度选自0DOF、3DOF和6DOF中的其中一种；根据空间位置、体积大小将可交互虚拟物体在子空间中渲染，并按照自由度被用户通过增强现实设备的显示模块浏览。如图4所示的实施例中，由于该实施例中的父空间为6DOF空间，因此在该父空间打开的空间小程序41也支持6DOF，同时该空间小程序41的初始化位置和大小由父空间定义，以避免空间小程序41出现在用户难以交互的位置，或具有过大或过小的尺寸。

[0044] 根据本申请的一个或多个实施方式，子空间的数量由父空间预先设定，子空间的数量优选可以为1-5个。如果子空间数量过多，可能会对处理器的资源要求较高，且过多的子空间可能会对用户造成困扰。如果空间小程序已经占据所有的子空间，则用户将无法打开更多的空间小程序，除非用户将已经运行的空间小程序做关闭处理，以空余出新的子空间。

[0045] 根据本申请的一个或多个实施方式，可交互虚拟物体的程序包支持第三方API(应用程序编程接口，Application Programming Interface)，例如BabyIon.js API，Web API，Node.js API等，通过这些第三方API，可以满足更多复杂的虚拟物体的创建以及复杂交互逻辑的支持。响应于检测到脚本信息中的调用第三方API的指令，调用上述的第三方API。示例性的，Babylon.js是一个使用HTML5和WebGL构建3D游戏的JavaScript框架，在可交互虚拟物体的程序包中，会默认创建一个Babylon.js场景，可以通过全局变量spaceDocument获取到它。Node.js是一个基于Chrome V8引擎的JavaScript运行环境

[0046] 根据本申请的一个或多个实施方式，空间小程序可以支持鼠标、按键、手关节、手势等输入事件，通过输入事件，可以实现一些交互效果，比如点击、拖拽、旋转等。可以响应于脚本信息中的监听事件启动指令，启动输入事件的监听。通过执行所述输入事件，实现对所述可交虚拟物体的点击、拖拽、旋转、缩放、移动中的一种或多种的交互效果。对输入事件的响应，也是可交互虚拟物体的关键特点，相比于传统的平面应用，可交互虚拟物体由于占据3D空间，其可以和用户进行更高维度的互动，从而提升空间小程序的趣味性和互动性。

[0047] 根据本申请的一个或多个实施方式，通过设定父空间和子空间的概念，来确保各个应用之间的安全运行。父空间可以为操作系统的桌面层级所占据的空间；子空间，指的是一个完整的空间，它包含了一些物体，然后可以将这个空间嵌入到父空间中，作为父空间的一个组件。而空间小程序的技术关键点在于，如何将一个空间(空间小程序)嵌入到另一个空间(主应用)中，而不影响到主应用的运行，将这称为空间安全性，更形象的比较就像：在一个浏览器中，不同的网页运行在同一个Web浏览器中互不干扰，这就是浏览器的安全性；在一个窗口操作程序中，不同的窗口运行在同一个屏幕上，窗口和窗口之间互不干扰，这就是窗口操作程序的安全性。

[0048] 如图5所示，示出了根据本申请一个或多个实施例的计算机程序产品框架的示意图。在该框架图中示出了三个包含空间信息的XML程序包(XSML Document)，该包含空间信息的XML程序包由XML文档、Spatial CSS、Type Script组成。通过一个嵌入的Node.js运行时支持BabyIon.js Engine和Web-standard API，然后通过BabyIon.js的RenderLoop方法生成transmute协议，transmute是一个用于将结构化数据转换为可渲染内容的JavaScript库,RenderLoop是一个连续的过程，通过用户手势等将事件传给App，接着App向操作系统传递事件并最终响应事件，响应会通过更新：几何体顶点(Vertex)、材质纹理(Texture)、物体的变换矩阵(Transform)、音频(Audio)以及用户界面布局(GUI Layout)等信息，再将响应的多维度结果传递给用户的过程。在Unity运行时(Runtime)中，通过初始化JSAR(支持开发者使用类似于Web的技术来开发可嵌入空间的空间小程序)提供的Unity预制体(Prefab)获取每个空间小程序在Unity的实例(例如图中5的小程序实例1 / 2 / N)，该实例通过transmute协议上下文(context)生成渲染结果。在渲染时，给每个空间小程序分配独立的数据通道，以保证每个空间小程序的渲染是独立的，每个空间小程序都是隔离运行的。

[0049] 如图6所示，增强现实设备600可以包括处理装置(例如中央处理器、图形处理器等)601、存储器602、输入单元603、输出单元604。其中，处理装置601、存储器602、输入单元603和输出单元604通过总线605彼此相连。在此，根据本申请的实施例的方法可以被实现为计算机程序，并且存储在存储器602中。例如，本申请的一些实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。增强现实设备中的处理装置601通过调用存储器602中存储的上述计算机程序，来具体实现本申请的方法中限定的显示虚拟界面功能。在一些实现方式中，输入单元603可以包括触控设备(例如，目标设备的触敏显示屏幕)、进行手势识别的摄像头。由此，可以通过输入单元603中的触控设备或摄像头感测是否检测到用户对于虚拟界面的操作，进而，响应于确定是，处理装置601可以调用上述计算机程序执行显示应用页面功能。输出单元604可以包括显示屏幕，用于显示虚拟界面。

[0050] 本申请实施例还提供了一种计算机程序产品，用于在增强现实设备中运行可交互虚拟物体，可以是上述增强现实设备中所包含的，也可以是单独存在，而未装配入该增强现实设备中。计算机程序产品包括：XML文档和可选的文件内容，所述XML文档包括属性信息、空间信息和脚本信息；XML文档被增强现实设备在父空间执行时，可以通过空间信息和 / 或脚本信息，在父空间的一个子空间中渲染出可交互虚拟物体，所述可交互虚拟物体可接受用户的一个或多个交互操作并进行反馈。

[0051] 可以以一种或多种程序设计语言或其组合来编写用于执行本申请的一些实施例的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如"C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)——连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。

[0052] 附图中的流程图和框图，图示了按照本申请各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和 / 或流程图中的每个方框、以及框图和 / 或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。

[0053] 本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。

[0054] 以上描述仅为本申请的一些较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本申请的实施例中所涉及的发明范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述发明构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本申请的实施例中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。< / script> < / space> < / space> < / space> < / space> < / space> < / version>

Claims

1. A method for isolating and operating interactive virtual objects in a virtual 3D space, comprising: In response to a user's start-up instruction in a parent space, a program package corresponding to an interactive virtual object is loaded in the parent space, wherein the parent space includes one or more child spaces; Parsing the program package to obtain attribute information, space information and script information in the program package; Rendering an interactive virtual object in one of the subspaces according to the spatial information and / or script information, wherein the interactive virtual object can accept one or more interactive operations of the user and provide feedback; The virtual 3D space is constructed by an augmented reality device and can be browsed by a user in the parent space through a display module of the augmented reality device.

2. The method according to claim 1, wherein the program package is an XML program package containing spatial information, which includes an XML document and optional file content; The XML document includes attribute information, space information and script information, and the optional file content includes one or more of a script file, a model file and a sound effect file; The script file is linked to the script information, and the script information is written in TypeScript language; The method further comprises, In response to the script information being parsed, the script file is executed on the augmented reality device.

3. The method according to claim 1, wherein: The rendering of an interactive virtual object in one of the subspaces according to the space information and / or the script information includes: The spatial information includes style information described by an XML document, and the script information includes style information described by a Babylon.js framework; The space information may be in a default state. When the space information is in the default state, rendering is performed in the subspace only according to the style information described in the script information.

4. The method according to claim 1, wherein: Each subspace has a spatial position and a volume size set by a parent space, and the parent space and the subspace have the same degree of freedom; The interactive virtual object is rendered in the subspace according to the spatial position and volume, and is browsed by the user through a display module of an augmented reality device according to the degree of freedom.

5. According to the method of claim 1, in response to the listening event start instruction in the script information, the listening input event is started, and the input event includes one or more of the mouse, button, hand joint, and gesture input events; by executing the input event, one or more interactive effects of clicking, dragging, rotating, scaling, and moving the interactive virtual object are achieved.

6. According to the method of claim 2, all file contents of the program package can be loaded via the network and locally during runtime. If loaded via the network, the augmented reality device will cache the used resources locally so that they can be read directly from the local during the next loading.

7. The method according to claim 1, in response to detecting an instruction to call a third-party API in the script information, calling the third-party API, wherein the third-party API includes BabyIon.js API, Web API and Node.js API.

8. According to the method of claim 1, when rendering the interactive virtual objects, an independent data channel is created for each interactive virtual object to ensure that the rendering of each interactive object is independent.

9. An augmented reality device, comprising A display module, wherein the display module is used to form a visualized 3D space in front of the user's eyes; A memory, the memory being used to store computer instructions; a processor for executing computer instructions; When the computer instructions are executed by the processor, the interactive virtual object is displayed in the 3D space presented by the display module, including: Loading an XML package containing space information corresponding to an interactive virtual object in a parent space, wherein the parent space includes one or more child spaces; Parsing the program package to obtain attribute information, space information and script information in the XML program package containing space information; An interactive virtual object is rendered in one of the subspaces according to the space information and / or script information, and the interactive virtual object can accept one or more interactive operations of the user and provide feedback.

10. A computer program product for operating an interactive virtual object in an augmented reality device, comprising: An XML document and optional file content, wherein the XML document includes attribute information, space information, and script information; When the XML document is executed by the augmented reality device in the parent space, an interactive virtual object can be rendered in a child space of the parent space through space information and / or script information, and the interactive virtual object can accept one or more interactive operations of the user and provide feedback.

Images