Virtual-real spatial data processing method, virtual home decoration method, virtual-real device, storage medium and program product

Abstract

The virtual-real spatial data processing technical scheme provided by the embodiment of the invention is applied to virtual-real equipment, and the virtual-real equipment reads stored persistent information when determining that initialization virtual rendering needs to be carried out aiming at a currently presented spatial image of an environment (such as a house space), and sends the persistent information to the virtual-real equipment; obtaining anchor point information corresponding to target model identifiers related to the space image from the persistence information, wherein one target model identifier is used for indicating one virtual object model which is previously placed in the space image; and according to the obtained anchor point information, a virtual object model indicated by the target model identifier is restored and displayed in the space image. According to the scheme, the placed virtual content can be restored and displayed without being influenced by closing of virtual and real equipment, exiting of a current space scene or exiting of a corresponding augmented reality application and the like, so that friendly equipment experience is provided for a user, especially in a multi-virtual-content placement scene.

Description

Technical Field

[0001] The present application relates to the field of augmented reality technology, and in particular to a virtual-reality space data processing method, a virtual home decoration method, a virtual-reality device, a storage medium and a program product. Background Art

[0002] With the development of augmented reality technology, various virtual and real devices such as Mixed Reality (MR) devices have been widely used in many fields. Using virtual and real devices, users can place virtual objects in various locations in the real environment space to simulate the scene. However, in related technical solutions, once the user exits the scene or the corresponding application, the previously arranged scene cannot be retained the next time the user enters it, and the user needs to re-place the corresponding virtual objects, which results in a poor user experience. Summary of the invention

[0003] In view of the above problems mentioned in the background technology, the present application provides a virtual-reality space data processing method, a virtual home decoration method, a virtual-reality device, a storage medium and a program product to solve or at least partially solve the above problems.

[0004] In the first embodiment, the present application provides a method for processing virtual and real space data. The method includes:

[0005] When it is determined that it is necessary to initialize virtual rendering of the spatial image of the environment, read the stored persistent information;

[0006] Acquire anchor point information corresponding to a target model identifier associated with the spatial image from the persistent information; wherein one target model identifier is used to indicate a virtual object model previously placed in the spatial image;

[0007] The virtual object model indicated by the target model identifier is displayed in the spatial image according to the anchor point information.

[0008] In the second embodiment, the present application also provides a method for processing virtual and real space data. The method includes:

[0009] Displaying a virtual model interface, wherein a plurality of virtual object models are displayed in the virtual model interface;

[0010] In response to a selection operation triggered on the plurality of virtual object models, superimposing and displaying a selected target virtual object model in a spatial image of a current environment;

[0011] Adding a first anchor point to the target virtual object model based on the position and posture of the target virtual object model in the spatial image;

[0012] The anchor point information of the first anchor point and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object are persistently stored, so that when it is subsequently determined that initial virtual rendering needs to be performed for the spatial image, the target virtual object model can be restored and displayed in the spatial image according to the stored persistent information.

[0013] In a third embodiment, the present application provides a method for virtual home decoration. The method includes:

[0014] Present the spatial image of the current house space;

[0015] When it is determined that the spatial image needs to be initialized for virtual rendering, reading the stored persistent information;

[0016] Acquire anchor point information corresponding to a target model identifier associated with the spatial image from the persistent information; wherein one target model identifier is used to indicate a virtual home decoration model that has been previously placed in the spatial image;

[0017] According to the anchor point information, the virtual home improvement model indicated by the target model identifier is restored and displayed in the spatial image.

[0018] In a fourth embodiment, the present application provides a virtual-reality device. The virtual-reality device includes: a persistent storage module and a processor; wherein the persistent storage module is used to persistently store anchor information, binding relationship information between anchor identifiers and model identifiers; the model identifier is used to indicate a corresponding virtual object model, and the anchor identifier is used to indicate an anchor added to the corresponding virtual object model; the processor is used to implement the steps in the above-mentioned method embodiments of the present application based on the persistent storage module.

[0019] In a fifth embodiment, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the steps in each method embodiment provided by the present application can be implemented.

[0020] In a sixth embodiment, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the processor is enabled to implement the steps in each method embodiment provided by the present application.

[0021] The technical solution provided by the embodiment of the present application is that when the virtual-reality device determines that it is necessary to initialize virtual rendering of the spatial image of the currently presented environment (such as the house space), it will read the stored persistent information and obtain the anchor information corresponding to the target model identifier related to the spatial image from the persistent information; wherein a target model identifier is used to indicate a virtual object model that has been placed in the spatial image before; and then according to the obtained anchor information, the virtual object model indicated by the target model identifier can be restored and displayed in the spatial image. The solution of the present application realizes that even if the virtual-reality device is turned off, the current spatial scene is exited, or the corresponding augmented reality application on the virtual-reality device is exited, when re-entering (such as restarting the virtual-reality device, or re-entering the current spatial scene or starting the augmented reality application), the virtual content previously placed in the spatial image can still be restored and displayed at the original position, which provides users with a more friendly virtual-reality device experience, especially in the scenario where multiple virtual content is placed, the better experience is more prominent. Among them, the virtual object model indicated by the above-mentioned target model identifier is recorded as the target virtual object model. This target virtual object model can be the one that the user has previously selected from multiple virtual object models displayed on the virtual model interface to open the target virtual object model. The virtual-reality device responds to the user's selection operation and is superimposed and displayed in the spatial image. Moreover, the virtual-reality device will also add an anchor point, such as a first anchor point, to the target virtual object model based on the position and posture of the target virtual object model in the spatial image. Furthermore, the anchor point information of the first anchor point and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object will be persistently stored, so that when it is determined that virtual rendering needs to be initialized for the spatial image later, the target virtual object model can be restored and displayed in the spatial image according to the stored persistent information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A flowchart of a data processing method provided in one embodiment of the present application;

[0024] Figure 2a A schematic diagram of a simulated layout of a home decoration scene provided in an embodiment of the present application;

[0025] Figure 2bA schematic diagram of placing virtual objects in a house space before restoring the display in a simulated layout of a home decoration scene provided in an embodiment of the present application;

[0026] Figure 2c A schematic diagram of a simulated layout of a home decoration scene provided by another embodiment of the present application;

[0027] Figure 3 A schematic diagram of the principle of scene simulation arrangement provided in one embodiment of the present application;

[0028] Figure 4 A structural block diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0029] Virtual objects and the real world can be integrated in various virtual and real devices such as MR devices. Generally, when users use virtual and real devices to simulate scenes, the device will first load the virtual object model of the scene after entering the scene. Then, the user selects and operates the corresponding virtual object model and places the virtual object model at a certain position in the real environment space. Since the information generated in the middle (such as the spatial position of the virtual object model, etc.) is stored in the memory, the information in the memory will be destroyed when the scene is exited or the corresponding application or device is closed, which will cause the scene to return to the initial state the next time it is entered, and some of the virtual object modes originally placed will not be presented, resulting in a poor user experience.

[0030] For example, in the home decoration scene, users can use virtual and real devices to place virtual furniture, home appliances and other virtual objects in various locations in the real house space to display the simulated home decoration scene. However, usually, after placing multiple virtual objects in the space, once the user exits the home decoration scene or the corresponding application, the previously arranged scene will not be presented when entering the home decoration scene next time, and the user needs to rearrange it again, which is particularly bad for the user experience of home decoration scenes with multiple virtual objects.

[0031] Although there is a semi-persistent solution in the current virtual and real devices, that is, when the scene (i.e., the immersive space with world tracking turned on) is not exited, the spatial position of each placed virtual object is saved in the memory. As long as the scene is not exited, the placed virtual objects can still be restored when entering or switching to home decoration placement. However, this does not actually achieve true persistence. Once the scene or the corresponding application is exited, the previous spatial information will be completely destroyed, and the previously placed virtual objects will still not be restored when entering again next time.

[0032] Yes, for virtual and real devices, if you want to bring a better experience to users, it should be: after placing virtual objects in different environmental spaces such as bedrooms, kitchens, living rooms, etc., the next time you enter each space, you can see the corresponding virtual objects at the position where they were placed last time.

[0033] In response to the above problems, the present application provides a general solution for position persistence in virtual object placement scenarios in virtual and real devices. The solution can provide a better experience, that is, after placing virtual objects in different real spaces (such as bedrooms, kitchens, living rooms, etc.) through virtual and real devices, even if you exit the space scene or exit the corresponding application (such as augmented reality application) or turn off the virtual and real device, you can restore the previously arranged virtual objects in the corresponding position of the space when you enter again. Among them, when implemented, the solution of the present application is mainly based on the use of an augmented reality (AR) development framework system capability, combined with the persistent information storage capability provided by the virtual and real device for service (application, etc.) data, to bind the virtual object model with the corresponding anchor point and persist it, and in the virtual object model placement and interaction scenarios, it also provides a persistent storage update opportunity. The whole process is imperceptible to the user, which can effectively improve the user experience of placing virtual objects in virtual and real devices. For a detailed description of the augmented reality (AR) development framework system and anchor points, please refer to the following vocabulary description part involved in this application, and no specific details will be given here.

[0034] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0035] Among them, in some processes described in the specification, claims and the above-mentioned figures of this application, multiple operations appearing in a specific order are included, and these operations may not be executed or executed in parallel in the order in which they appear in this article. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit "first" and "second" to different types. The term "or / and" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example: A or / and B, indicating that A can exist alone, A and B can exist at the same time, and B can exist alone; the character " / " in this application generally indicates that the front and back associated objects are an "or" relationship. It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0036] The following is an introduction to the various embodiments provided in this application.

[0037] It should be noted that the information (such as device information used by users, persistent information, etc.) and data (including but not limited to stored data, displayed data, etc.) involved in this application are all information and data authorized by users or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse. In addition, the calling of modules such as the acquisition module (including the shooting module) is also authorized by the user.

[0038] In order to facilitate the understanding of the present application, before introducing the embodiments provided by the present application, the vocabulary involved in the embodiments of the present application is first explained. It can be understood that this explanation is for a clearer understanding of the embodiments of the present application and does not necessarily constitute a limitation on the embodiments of the present application.

[0039] Mixed Reality (MR) is an augmented reality technology that combines the real world and virtual elements. Compared to virtual reality (VR) devices that can only provide a completely virtual experience, MR devices allow users to integrate virtual content with the real world.

[0040] Augmented Reality (AR) development framework: provides environmental understanding and tracking capabilities, such as plane detection, gesture detection, world tracking, etc. It is mainly launched for some operating system platforms, and allows developers to create rich AR applications, thereby using the device's camera, sensors and processors to achieve environmental tracking, plane detection, lighting estimation and other functions.

[0041] World Anchor (also called anchor in world space or space anchor): It is the information defined in the augmented reality AR development framework system for tracking a coordinate point in world space. It represents a relatively fixed position and direction in world space. That is, the anchor is used to fix the position and posture of virtual objects in the real world space, so that the position of these virtual objects in the real world space remains consistent, even if the device is moved or the application is restarted.

[0042] Placement: In a virtual reality device such as an MR device, a virtual object is placed at a location in space, such as the ground, a wall, or any other specified location in the space.

[0043] Persistence: Data persistence is a storage technology that allows data to remain valid after the application is closed or the device is restarted. Data persistence methods are provided in various platform systems.

[0044] The following method embodiments provided in this application are all applied to virtual-reality devices. Virtual-reality devices may be, but are not limited to, MR devices (such as head-mounted MR glasses). There are augmented reality applications on virtual-reality devices. Augmented reality applications are mainly developed and implemented based on the world tracking capability provided by an augmented reality AR development framework. The augmented reality AR development framework can be any one, as long as it can ensure that the augmented reality AR development framework used has world tracking capabilities. And the augmented reality AR development framework has the ability to persist anchor information of anchor points, and can add / delete an anchor point.

[0045] For details on virtual devices, augmented reality applications, and augmented reality AR development frameworks, please refer to the relevant content in other embodiments.

[0046] Figure 1 FIG. 1 is a flow chart of a method for processing virtual and real space data provided by an embodiment of the present application. Figure 1 As shown, the virtual-real space data processing method comprises the following steps:

[0047] 101. When it is determined that it is necessary to initialize virtual rendering of the spatial image of the environment, read the stored persistent information;

[0048] 102. Acquire anchor point information corresponding to a target model identifier related to the spatial image from the persistent information; wherein one target model identifier is used to indicate a virtual object model previously placed in the spatial image;

[0049] 103. Display the virtual object model indicated by the target model identifier in the spatial image according to the anchor point information.

[0050] During the actual use of the virtual-reality device, when it is determined that the preset spatial scanning trigger conditions are met, such as detecting that the user manually triggers the spatial scanning operation or detecting that the virtual-reality device moves to a new position or the environment changes, the virtual device will execute the process of scanning the real spatial environment, and then construct the corresponding spatial image based on the scanning results to ensure that the subsequent virtual content can be accurately superimposed on the corresponding real spatial environment. The presentation of the spatial image also allows the user to see the corresponding real spatial environment through the virtual-reality device.

[0051] Based on this, in the above 101, the spatial image of the environment where the virtual-reality device is located refers to the image constructed by the virtual-reality device after scanning the real spatial environment where the virtual-reality device is currently located. The environment where the virtual-reality device is located can refer to any real spatial environment, such as a house space.

[0052] The initialization virtual rendering mentioned in 101 may refer to: performing the first rendering of the spatial image to overlay the corresponding virtual content on the spatial image. Also, when it is determined that the spatial image meets the preset initialization virtual rendering condition, it is determined that the spatial image needs to be initialized for virtual rendering.

[0053] The spatial image of the environment that meets the initial rendering conditions may include but is not limited to at least one of the following:

[0054] 1) Detecting that the augmented reality application is turned on;

[0055] 2) The spatial image is presented by rescanning the environment without exiting the augmented reality application;

[0056] 3) The spatial image is presented after the virtual-reality device is restarted.

[0057] In the above 1), detecting that the augmented reality application is turned on includes but is not limited to the following situation: the augmented reality application is closed and then restarted without exiting the currently presented spatial image scene.

[0058] In the above 2), when the augmented reality application remains open (i.e., not closed and exited), for example, when a space rescan operation manually triggered by the user is detected or it is determined that the preset space automatic rescan conditions are met, the virtual-reality device will rescan the current environment space to present the corresponding space image to the user. In this case, it can be determined that the space image needs to be initially virtually rendered.

[0059] Among them, the user can manually trigger the rescanning of the space by clicking the corresponding control on the virtual user interface provided by the virtual and real device, or through gestures, voice, etc.

[0060] Furthermore, when the set rescanning cycle (i.e., regular rescanning) is reached, it can be determined that the preset spatial automatic rescanning conditions are met. For example, a timer can be set in the virtual-real device to automatically rescan the environment at intervals, so as to ensure that the device always has the latest environmental data. And / or, rescanning can also be automatically triggered according to certain conditions. For example, when it is detected that the virtual-real device has moved to a new location or the environment has changed, it triggers automatic rescanning of the environment.

[0061] In the above 3), after the virtual and real device is restarted, the spatial image of the current environment presented by the virtual and real device also meets the initial virtual rendering conditions. Among them, after the virtual and real device is restarted, the augmented reality application can be automatically started according to the settings or manually started by the user through gestures, voice, etc., which is not limited here.

[0062] Here, it is necessary to supplement the above 1) to 3) that the development and implementation of augmented reality applications rely on the world tracking capability of the augmented reality AR development framework. Therefore, when the augmented reality application is started, it can be understood that world tracking is turned on accordingly.

[0063] Furthermore, the persistence information described in 101 above includes: the anchor point information stored in the first persistent storage module, and the binding relationship information between the anchor point identifier and the model identifier stored in the second persistent storage module. The model identifier is used to indicate the corresponding virtual object model. The anchor point information includes the anchor point identifier, the anchor point posture, etc. The anchor point identifier is used to indicate a corresponding anchor point (also called a world anchor point, a space anchor point, etc.), and the anchor point is used to fix the corresponding virtual object model at a certain position and posture in the real world space, so that the virtual object model can remain stable in the real world space even when the virtual and real devices move or rotate.

[0064] Among them, the first persistent storage module is used to implement: the augmented reality AR development framework corresponding to the augmented reality application provides the underlying persistent anchor information storage capability at the device system layer. That is, it can be understood that the first persistent storage module is used for the persistent storage of the augmented reality AR development framework, and the persistent storage refers to the persistent storage of the anchor information (such as anchor identification, anchor posture), plane detection information, environmental characteristics and other data in the AR session to the local device or the cloud. Therefore, the above-mentioned first persistent storage module can be used to store the system layer persistent information of the virtual and real device, and the system layer persistent information includes anchor information, etc.; and the first persistent storage module can refer to a storage area in a local persistent storage medium of the virtual and real device, or it can also refer to a storage area in the cloud. And, the second persistent storage module is used for the persistent storage of services provided on the virtual and real device, and the persistent storage of the service refers to: saving the data in applications such as augmented reality applications (such as user information, configuration settings, binding relationship between anchor points and virtual object models, etc.) to a persistent storage medium, so that these data can still be accessed and restored after the application is closed. Therefore, the second persistent storage module may be used to store the service layer persistent information of the virtual-real device, and the service layer persistent information includes the binding relationship information between the anchor identifier and the model identifier (i.e., the binding relationship information between the anchor and the virtual object model). And the second persistent storage module may refer to a storage area in a local persistent storage medium of the virtual-real device.

[0065] In this application, the persistent storage medium in the virtual device may refer to: a storage disk for persistent data storage, such as a disk. Also, the service layer of the virtual device is also called the service logic layer or application layer, which is located above the system layer.

[0066] Because the persistent information to be read involves anchor information, which is a function provided by the augmented reality application, in this application, the execution of reading the stored persistent information is triggered only when the augmented reality application is turned on. Based on this content and in combination with the aforementioned related content, the above 101 "reading the stored persistent information" may include:

[0067] 1011. When the spatial image meets the initialization virtual rendering condition and the augmented reality application on the virtual-reality device is in an open state, trigger the step of executing “reading the stored persistent information”.

[0068] For details on satisfying the initialization virtual rendering conditions described here, please refer to the aforementioned related content.

[0069] Furthermore, the above-mentioned “reading the stored persistent information” may specifically include:

[0070] 10111. Read the persistently stored anchor information;

[0071] 10112. When the persistently stored anchor point information contains a target anchor point identifier related to the spatial image, read the binding relationship between the persistently stored anchor point identifier and the model identifier to determine whether there is a model identifier having a binding relationship with the target anchor point;

[0072] When a model identifier having a binding relationship with the target anchor identifier is obtained from the binding relationship information, it is determined that the persistent information contains a target model identifier related to the spatial image. That is, the target model identifier related to the spatial image is the model identifier having a binding relationship with the target anchor identifier.

[0073] In the above 10111, the corresponding augmented reality AR development framework can be triggered to read the persistently stored anchor information, and during the reading process, it will be determined whether there is a target anchor identifier related to the spatial image of the current environment. Among them, when it exists, the corresponding callback function in the world tracking capability can be used to return the target anchor information (including the target anchor identifier) ​​related to the idle image of the current environment. The target anchor identifier is used to indicate the corresponding virtual object model that the user has previously placed in the spatial image. In this embodiment, the model identifier of a virtual object model refers to: any information that can uniquely identify the virtual object model bound to the virtual object model, such as the model identifier can be a model unique identification number (modelId, model ID) or a model name, etc., which is not specifically limited at this time.

[0074] After the target anchor point identifier (possibly one or more) related to the spatial image of the current environment is read from the persistently stored anchor point information, further, the above 10112 will be triggered to execute the binding relationship information of the persistently stored anchor point identifier and the model identifier, and when reading, it will be determined whether there is a model identifier with a binding relationship with the target anchor point identifier. If there is, the model identifier with a binding relationship with the target anchor point identifier is the target model identifier related to the spatial image of the current environment, so that the virtual object model corresponding to the target model identifier will be loaded from the corresponding virtual object model library, and the initial posture of the loaded virtual object model in the spatial image will be set according to the anchor point posture corresponding to the target anchor point identifier obtained from the persistently stored anchor point information for display.

[0075] Therefore, that is, the above-mentioned step 102 of “obtaining anchor point information corresponding to the target model identifier related to the spatial image from the persistent information” may specifically include:

[0076] 1021. Obtain, from the persistently stored anchor point information, an anchor point posture corresponding to the target anchor point having a binding relationship with the target model identifier.

[0077] Furthermore, in the above 103, “displaying the virtual object model indicated by the target model identifier in the spatial image according to the anchor point information” may include:

[0078] 1031. Render and display the virtual object model indicated by the target model identifier in the spatial image according to the anchor point posture corresponding to the target anchor point identifier.

[0079] In order to facilitate understanding of the technical solution provided by this embodiment, Figure 2a-2b An example scenario is cited to introduce and illustrate the solution of this embodiment.

[0080] See Figure 2a , assuming that the user has previously placed a virtual object model 21 in a spatial image of a house space 200 by wearing a virtual-reality device 100, and then exited the augmented reality application on the virtual-reality device 100 (or exited the spatial image scene of the house space 200 or turned off the virtual-reality device 100), then the anchor information persistently stored in the virtual-reality device 10 will include the anchor information of the anchor Anchor21 corresponding to the virtual object model 21 (such as the anchor identifier anchorId21), and the binding relationship between the anchor identifier anchorId21 and the model identifier modelId21 of the virtual object model 21 will be included in the binding relationship between the anchor identifier anchorId21 and the model identifier modelId21 of the virtual object model 21. Further, see Figure 2b As shown in the figure above, when the user starts the augmented reality application 110 again, it will trigger the reading of the persistently stored anchor information, and will return the anchor information of the anchor Anchor21 related to the spatial image of the currently presented house space 200 (such as including the anchor identifier anchorId21 and the anchor pose Pose21). Furthermore, by reading the binding information of the persistently stored anchor identifier and the model identifier, it will be determined that the returned anchor identifier modelId21 has the corresponding model identifier modelId21. Further, as Figure 2b As shown in the figure below, according to the anchor point pose Pose21 corresponding to the anchor point identifier anchorId21, the virtual object model 21 indicated by the model identifier modelId21 will be rendered and displayed at the corresponding position of the spatial image of this house space 200. In this way, when the user reopens the augmented reality application 110 (or re-enters the spatial image scene of this house space 200 or restarts the virtual-reality device 100), all virtual object models previously placed in this house space 200 will still be displayed at the corresponding original positions, which provides users with a better virtual-reality device home decoration experience.

[0081] The technical solution provided by this embodiment is that when the virtual-reality device determines that it is necessary to initialize virtual rendering of the spatial image of the currently presented environment (such as the house space), it will read the stored persistent information, and when it is determined that the persistent information contains a target model identifier related to the spatial image, it will obtain the anchor information corresponding to the target model identifier from the persistent information; wherein a target model identifier is used to indicate a virtual object model that has been placed in the spatial image before; and then according to the obtained anchor information, the virtual object model indicated by the target model identifier can be restored and displayed in the spatial image. The present application scheme realizes that even if the virtual-reality device is turned off, the current spatial scene is exited, or the corresponding augmented reality application on the virtual-reality device is exited, when re-entering (such as restarting the virtual-reality device, or re-entering the current spatial scene or starting the augmented reality application), the virtual content previously placed in the spatial image can still be restored and displayed at the original position, which provides users with a more friendly virtual-reality device experience, especially in the scenario where multiple virtual content is placed, the better experience is more prominent.

[0082] Further, after the initial virtual rendering of the spatial image of the environment is completed, as described above, Figure 2a and Figure 2b As described, after the previously placed virtual object model 21 is restored and displayed in the spatial image, the user can perform interactive operations such as moving, rotating, zooming in, and reducing the virtual object model (such as the virtual object model 21) currently displayed in the spatial image, and / or can continue to place corresponding virtual object models in the spatial image. Among them, when performing interactive operations on the virtual object model displayed in the spatial image, it is necessary to update the stored anchor point information, the binding relationship information between the anchor point identifier and the model identifier, etc. The specific implementation of the update can refer to the relevant content in other embodiments described below, and will not be described in detail here.

[0083] Based on this, for the placement scenario of the virtual object model, another embodiment of the present application also provides a virtual-real space data processing method. Specifically, the virtual-real space data processing method includes the following steps:

[0084] 201. Displaying a virtual model interface, wherein a plurality of virtual object models are displayed in the virtual model interface;

[0085] 202. In response to a selection operation triggered on the multiple virtual object models, superimpose and display a selected target virtual object model in a spatial image of a current environment;

[0086] 203. Add a first anchor point to the target virtual object model based on the position and posture of the target virtual object model in the spatial image;

[0087] 204. Persistently store the anchor point information of the first anchor point and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object, so that when it is determined that initial virtual rendering needs to be performed for the spatial image later, the target virtual object model can be restored and displayed in the spatial image according to the stored persistent information.

[0088] In the above 201, the displayed virtual model interface is provided by an augmented reality application, and the virtual model interface can be referred to as Figure 2a or Figure 2c The model panel 12 is shown in FIG.

[0089] In the above 202, the corresponding target virtual object model can be selected through interactive methods such as gestures or voice.

[0090] For example, see Figure 2c As shown in the figure above, assuming that the user selects to open the virtual object model 22 displayed on the model panel 12, the virtual object model 22 will be superimposed and displayed in the spatial image of the currently presented house space 200 according to the initial posture set for the virtual object model 22, as shown in FIG. Figure 2c As shown in the figure below.

[0091] In the above 203-204, the anchor information of the first anchor point can be stored in the first persistent storage module, and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object can be stored in the second persistent storage module, so as to achieve corresponding persistent storage. In the present application, the anchor information of an anchor point includes the anchor point identifier, the anchor point posture, etc.

[0092] For detailed description of the first persistent storage module and the second persistent storage module, please refer to the relevant content in other embodiments.

[0093] For example, following the example given in the above step 202, the selected target virtual object model is the virtual object model 22, then the anchor information of the anchor point Anchor22 added to the virtual object model 22 (such as the anchor point identifier anchorId22, the anchor point pose Pose22, etc.) can be stored in the first persistent storage module, and the binding relationship between the anchor point identifier anchorId22 of the anchor point Anchor22 and the model identifier modelId22 of the virtual object model 22 can be stored in the second persistent storage module.

[0094] Regarding the specific implementation of how to restore and display the target virtual object model when it is determined that initial virtual rendering needs to be performed for the spatial image in the subsequent step 204, please refer to the relevant content in other embodiments, such as the content related to the above 101 to 103.

[0095] Furthermore, the user can also perform interactive operations such as moving, rotating, zooming in, and reducing any virtual object model displayed in the spatial image, and when performing interactive operations, the stored anchor point information, the binding relationship information between the anchor point identifier and the model identifier, etc. will be updated. For the specific implementation of the update, please refer to the relevant content described below for the target virtual object model. Therefore, the method provided in this embodiment may also include the following steps:

[0096] 205. In response to an interactive operation on the target virtual model, when it is determined that a preset deletion opportunity is met, deleting the previously persistently stored anchor point information of the first anchor point and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object;

[0097] 206. When the interactive operation ends, add a second anchor point to the target virtual model according to the position of the target virtual model after the interactive operation, and persistently store the anchor point information of the second anchor point and the binding relationship between the anchor point identifier of the second anchor point and the model identifier of the target virtual object.

[0098] In the above 205, the interactive operation performed on the target virtual object model includes but is not limited to moving, rotating, zooming in, zooming out, etc. For example, continuing with the example given in the above step 202, as shown in FIG. Figure 2c The user can move the virtual object module 22 displayed in the space image to a position on the top of the virtual object model 21 (not shown in the figure).

[0099] Furthermore, the preset deletion timing may include but is not limited to: the start of the interactive operation, the removal of the target virtual object model, and the deselection of the target virtual object model. Deselecting a virtual object model means that the user cancels the selection or deactivates the previously selected virtual object model through some interactive method (such as gesture, voice, etc.).

[0100] When the preset deletion timing is met, the augmented reality AR development framework can be triggered to update the anchor point information stored in the first persistent storage module to delete the anchor point information of the first anchor point previously stored persistently for the target virtual model; in addition, the binding relationship information stored in the second persistent storage module will also be updated to delete the binding relationship between the anchor point identifier of the first anchor point previously stored persistently for the target virtual model and the model identifier of the target virtual object.

[0101] In the above 206, when the interactive operation ends, the augmented reality AR development framework can be triggered to add a second anchor point to the target virtual model according to the position of the target virtual model after the interactive operation, and the first persistent storage module will be updated to store the anchor point information of the second anchor point in the first persistent storage module. In addition, the second persistent storage module will also be updated to store the binding relationship between the anchor point identifier of the second anchor point and the model identifier of the target virtual object in the second persistent storage module.

[0102] For the specific implementation of the above 205 to 206, please refer to the following Figure 3 Here, in the scenario of interacting with the target virtual object model, a better (such as the start of interaction, etc.) update timing of persistent information (such as anchor information, information on the binding relationship between the anchor identifier and the model identifier, etc.) is provided. The whole process is imperceptible to the user, which can effectively improve the user experience of interacting with the virtual object model through virtual and real devices.

[0103] It should be noted that: for any content not described in detail in the steps provided in this embodiment, please refer to the relevant content in other embodiments. In addition, this embodiment may include other steps in addition to the above steps, and the other steps and their specific implementations may also refer to the relevant content in other embodiments.

[0104] In summary, combined Figure 3 This application plan is generally introduced and described as follows:

[0105] The present application scheme is applied to virtual-real devices, which may be, but are not limited to, MR devices (such as head-mounted MR glasses). There are augmented reality applications on virtual-real devices. Augmented reality applications are mainly developed and implemented based on the world tracking capabilities provided by an augmented reality AR development framework. Specifically, for example, an augmented reality application may be an application created in a certain engine (such as a game engine) based on the world tracking capabilities provided by an augmented reality AR development framework. In this case, the augmented reality application may also be referred to as an engine application. Thus, in this application, the underlying system augmented reality capabilities (AR capabilities) provided in the virtual-real device rely on the world tracking capabilities of an augmented reality AR development framework. The augmented reality AR development framework can be any framework with world tracking capabilities. After world tracking is turned on, the anchor point information (such as anchor point identification, anchor point posture (position and posture)) of the anchor point in the real world space will be returned in real time. Here, world tracking is turned on, which can be understood as the augmented reality application on the virtual-real device being turned on. The AR development framework corresponding to the AR application has the ability to persist the anchor information of the anchor point, and can add / delete an anchor point, and when adding an anchor point, an anchor point identifier (such as anchorId (anchor point unique identifier)) will be returned to uniquely identify the added anchor point. When exiting the AR application and re-entering it, the corresponding AR development framework will restore and update the anchor point information according to the current environment of the virtual and real device.

[0106] Among them, the position of the anchor point (WorldAnchor) in the real world space is fixed, such as a specific position on the desktop, but because the user's own posture is generally different each time he opens the scene, the value is not fixed. The augmented reality AR development framework will calculate it and call back to the virtual and real device each time. It can be understood here that: in the augmented reality AR development framework, the position of the anchor point is fixed in the real world space, but the user's own position and posture may be different each time he opens the augmented reality application. Therefore, even if the actual position of the anchor point in the real world space has not changed, its value in the device coordinate system will be different. The augmented reality AR development framework will calculate the value of the anchor point position in the virtual and real device coordinate system each time and return it to the virtual and real device.

[0107] The aforementioned augmented reality AR development framework provides the underlying persistent anchor capability at the device system layer. The anchor and the corresponding virtual object model need to be bound together at the device's service layer (application layer). The model identifier of the virtual object model can be the model unique identifier (modelId) (or other unique identifiers, such as model name, etc.). Once a binding relationship occurs, the binding relationship information can be updated to the device's service layer persistent information, that is, the anchor identifier (such as anchorId) and the corresponding model identifier (modelId) are updated to the device's service layer persistent information.

[0108] For example, see Figure 2aIn the virtual reality home decoration scene shown, when a user enters a house space 200 for the first time and wants to use a virtual reality device to perform virtual reality home decoration on the house space 200, the user will first start the virtual reality device 100 he is wearing, and then the virtual reality device 100 will scan the house space 200 of the current real house environment to construct a corresponding spatial image and present it to the user, so that the user can enter the house space 200 scene. Further, the user starts the augmented reality application (not shown in the figure) on the virtual reality device 100 and opens the model panel 12 (a virtual IU interface) provided by the augmented reality application. In the model panel 12, multiple virtual object models are displayed, such as virtual object models belonging to the furniture category, virtual object models belonging to the wall decoration category, and so on. Each virtual object model displayed on the model panel 12 is associated with a model identifier for uniquely identifying the virtual object model. By selecting multiple virtual object models displayed in the model panel 12, the user can place the selected virtual object model in the corresponding position in the spatial image of the house space 200. For example, the user selects to open the virtual object model 21. At this time, the virtual-reality device responds to the selection and opening operation by loading the virtual object model 21 and displaying the virtual object model 21 at the corresponding position in the spatial image of the house space 200 according to the initial position information set for the virtual object model 21. Furthermore, the corresponding augmented reality AR development framework is called at this time to add an anchor point Anchor21 (an anchor point in the house space 200) to the virtual object model 21 based on the current position of the virtual object model 21, and the anchor point information of the anchor point Anchor21 (anchor point identifier anchorId21, anchor point position Pose21, etc.) is persistently stored at the system level, that is, the anchor point information of the anchor point Anchor21 is updated to the system level persistent information for storage. In addition, the augmented reality AR development framework will also return the anchor identifier anchorId21 of the anchor point Anchor21 to the service layer of the virtual and real device (also called the service logic layer or application layer), so as to bind the anchor identifier anchorId21 of the anchor point Anchor21 with the model identifier modelId21 of the virtual object model 21, and perform persistent storage of the service layer for this binding relationship, that is, update the binding relationship between the anchor identifier of the anchor point Anchor21 and the model identifier modelId21 of the virtual object model 21 to the service layer persistent information for storage.

[0109] Figure 2a , a schematic diagram showing that a user places a virtual object model 21 into a house space 200 is shown.

[0110] And, the modules corresponding to the two persistent storages mentioned above are as follows Figure 3The first persistent storage module and the second persistent storage module are shown in the figure. The first persistent storage module is used to store the system layer persistent information of the virtual and real devices, and the system layer persistent information includes anchor information. The second persistent storage module is used to store the service (application) layer persistent information on the virtual and real devices, and the service layer persistent information includes the binding relationship information between the anchor identifier and the model identifier.

[0111] For detailed description of the first persistent storage module and the second storage module, please refer to the relevant content in other embodiments.

[0112] Based on the above content, combined with Figure 3 As shown, the overall main process of this application scheme includes the following steps:

[0113] S1. Enter a scene of placing virtual and real objects in the current environment space, such as entering a scene of placing virtual and real objects in the whole house. This can be understood as: the virtual and real device scans the current environment space and presents the spatial image of the current environment, so that the user can see the current real space environment through the virtual and real device.

[0114] S2. Enable the world tracking provided by the augmented reality (AR) development framework. This can be understood as: enabling the corresponding augmented reality application on the virtual and real devices. For example, see Figure 2b On the virtual main interface 11 provided by the virtual-reality device 100, application icons of various applications on the virtual device are displayed, among which the application icon 110 corresponds to the augmented reality application. The user can operate the application icon 110 through gestures or voice to open the augmented reality application.

[0115] S3. Perform a world tracking callback. It can be understood here that the system-level persistent information of the virtual and real devices (i.e., the system-level persistent information stored in the first persistent storage module, including anchor information) is read; during the reading process, it is determined whether there are anchors related to the current scene (the currently presented spatial image) in the system-level persistent information. If so, the augmented reality AR development framework will call back to return the anchor information of the anchors related to the current scene (including anchor identifiers, anchor poses, etc.). For example, the return result of the callback may include the anchor information of one or more anchors related to the current scene.

[0116] S4. Read the persistent information of the service on the virtual and real devices (ie, the service layer persistent information stored in the second persistent storage module) to initialize the scene.

[0117] Specifically, when reading the persistent information of the service on the virtual and real device (including the binding relationship information between the anchor identifier and the model identifier), it is determined whether the anchor identifier returned by callback when executing the above step S3 has a corresponding model identifier. If yes, the following steps S5 and S6 are executed. Otherwise, if no, the following steps S7 to S9 are executed.

[0118] S5, loading the corresponding virtual object model, that is, according to the model identifier determined in the above step S4, loading the virtual object model indicated by the corresponding model identifier.

[0119] S6, using the posture of the corresponding anchor point to set the initial posture of the virtual object model, so as to render and display it in the current scene. Further, the following step S10 may be performed afterwards.

[0120] S7, select the panel (if Figure 2c The virtual object model on the model panel 12) shown in the figure can be selected to place the selected virtual object model in the current scene.

[0121] For example, see Figure 2c , select the virtual object model 22, and place the virtual object model 22 in the current scene.

[0122] S8. Call the augmented reality AR development framework to add an anchor point (addWorldAnchor) for the virtual object model selected by the user, and update the system layer persistence information of the virtual device and return the anchor point identifier.

[0123] For example, following the example in the above step S7, the augmented reality AR development framework is called to add an anchor point Anchor22 to the virtual object model 22, and the anchor point information of the anchor point Anchor22 is also stored in the first persistent storage module, so as to update the system layer persistent information stored in the second persistent storage module (mainly updating the anchor point information therein). The anchor point identifier anchorId22 of the anchor point Anchor22 is also returned.

[0124] S9. Update the service layer persistence information stored on the virtual-real device according to the anchor point identifier returned by executing the above step S8, that is, associate the returned anchor point identifier with the model identifier of the corresponding virtual object and store them, and further perform the following step S10 for the placed virtual object.

[0125] For example, following the example in the above step S8, the anchor identifier anchorId22 of the anchor point Anchor22 will be associated (i.e. bound) with the model identifier modelId22 of the virtual object model 22 to be stored in the second persistent storage module, thereby updating the service layer persistent information stored in the second persistent storage module (mainly updating the binding relationship information between the anchor identifier and the model identifier).

[0126] It should be noted that the above steps S7 to S9 are understood to be that when the user wants to place a corresponding virtual object model in the current scene, Figure 2a As shown, the model panel 12 provided by the virtual augmented reality application can be opened, so that multiple virtual object models displayed on the model panel 12 can be selected and opened by gestures or voice. At this time, the virtual object model selected by the user will be loaded, and the virtual object model will be rendered and displayed in the current scene according to the initial posture set for the virtual object model, so that the virtual object model selected by the user is placed at a corresponding position in the current scene for display. Furthermore, the corresponding augmented reality AR development framework will be called at this time to add an anchor point to the virtual object model according to the current posture of the placed virtual object model, and the anchor point information of this anchor point will be added and updated to the system layer persistent information of the virtual and real device. At the same time, the anchor point identifier of this anchor point and the model identifier of the corresponding virtual object model will be associated and bound, and the binding relationship will be added and updated to the service layer persistent information of the virtual and real device.

[0127] S10, virtual object model interaction (such as displacement (move), rotation (rotate), etc.). That is, the user can use gestures or voice to perform interactive behaviors such as moving, rotating, zooming in, and reducing the virtual object model placed in the current scene. The interactive behaviors include interactive start behaviors and interactive end behaviors. For the interactive start behavior, the following steps S11a to S12 will be executed, and for the interactive end behavior, the following steps S11b and S8 will be executed.

[0128] S11a-S12, when the interactive behavior starts (i.e., the virtual object starts to move, rotate, etc.), step S12 will be triggered. When executing step S12, the augmented reality AR development framework will be triggered to remove the anchor point information of the anchor point previously added for the virtual object model of the interactive operation from the system layer persistent information; in addition, the binding relationship between the anchor point identifier and the model identifier previously added for the virtual object model will also be removed from the service layer persistent information.

[0129] S11b, after the interaction is completed (i.e., the displacement, rotation, etc. of the virtual object is completed), the process returns to step S8. When the process returns to step S8, the augmented reality AR development framework is triggered to re-add an anchor point to the virtual object model according to the position of the virtual object model after the interaction operation, and the system layer persistent information is updated to add the anchor point information of the added anchor point therein; in addition, the service layer persistent information is also updated to add the binding relationship between the anchor point identifier of the added anchor point and the model identifier of the virtual object model of the interaction operation.

[0130] S13. When the user removes or deselects a virtual object model in the current scene, it also triggers the update of the system layer persistent information to remove the anchor point information corresponding to the virtual object model, and the update of the service layer persistent information to remove the binding relationship information between the anchor point identifier and the model identifier corresponding to the virtual object model.

[0131] Among them, deselecting a virtual object model means that the user cancels the selection or deactivates the previously selected virtual object model through some interactive method (such as gesture, voice, etc.).

[0132] In summary, this application is a complete persistence solution for virtual object placement. The core data of the augmented reality AR development framework and services are stored in a persistent storage module (such as a disk) rather than in memory, so it is not affected by scene and application exit and device restart. No matter when and where the user enters next time, they can still restore the previously placed virtual object models, which brings a better device experience to the user. Moreover, in the scenario of virtual object model interaction (virtual object model posture changes), this application also provides a better (such as the start of interaction, etc.) update time for persistent information (such as anchor point information, binding relationship information between anchor point identifier and model identifier, etc.). The whole process is imperceptible to the user, which can effectively improve the user experience of placing virtual object models through virtual and real devices.

[0133] The solution of this application can be applied to any scene where a virtual object model needs to be placed in a space, such as a virtual home decoration scene. Based on this, for the virtual home decoration scene, this application also provides a virtual home decoration method. Specifically, the method includes the following steps:

[0134] 301. Presenting the spatial image of the current house space;

[0135] 302. When it is determined that the spatial image needs to be initialized for virtual rendering, read the stored persistent information;

[0136] 303. Acquire anchor point information corresponding to a target model identifier related to the spatial image from the persistent information; wherein one target model identifier is used to indicate a virtual home decoration model that has been previously placed in the spatial image;

[0137] 304. According to the anchor point information, restore and display the virtual home decoration model indicated by the target model identifier in the spatial image.

[0138] In the above, the virtual home decoration model includes but is not limited to: virtual furniture models, furniture models, wall installation models, floors, ground decoration models, etc., which are not limited here.

[0139] Furthermore, the method may also include the following steps:

[0140] 305. In response to the interface opening operation, displaying a virtual model interface including a plurality of virtual home improvement models;

[0141] 306. In response to a selection operation triggered on the multiple virtual home decoration models, superimposing and displaying a selected target virtual home decoration model in the spatial image;

[0142] 307. Adding anchor points to the target virtual loading model based on the position and posture of the target virtual home decoration model in the space image;

[0143] 308. Persistently store the anchor information of the anchor point and the binding relationship between the anchor identifier of the anchor point and the model identifier of the target virtual home improvement, so that when it is determined that initial virtual rendering needs to be performed for the spatial image later, the target virtual home improvement model can be restored and displayed in the spatial image according to the stored persistent information.

[0144] This embodiment scheme, in the virtual home decoration scene, realizes that when the scene is exited or the corresponding augmented reality application is exited or the virtual and real device is restarted, no matter when and where the user re-enters the scene next time, all virtual content (such as various virtual home decoration models) previously placed in the scene can be automatically restored to their original positions, which provides users with a real and friendly virtual and real device experience.

[0145] It should be noted that: for any content not described in detail in the steps provided in this embodiment, please refer to the relevant content in other embodiments. In addition, this embodiment may include other steps in addition to the above steps, and the other steps and their specific implementations may also refer to the relevant content in other embodiments.

[0146] The present application also provides device embodiments corresponding to the above method embodiments, which are as follows:

[0147] An embodiment of the present application provides a virtual-reality space data processing device, which is deployed on a virtual-reality device. Specifically, the device includes: a reading module, an acquisition module, and a display module. The reading module is used to read the stored persistent information when it is determined that the spatial image of the environment needs to be initialized for virtual rendering. The acquisition module is used to obtain anchor point information corresponding to a target model identifier related to the spatial image from the persistent information; wherein one of the target model identifiers is used to indicate a virtual object model that has been previously placed in the spatial image. The display module is used to display the virtual object model indicated by the target model identifier in the spatial image according to the anchor point information.

[0148] Furthermore, the above-mentioned reading module is specifically used to: read the persistently stored anchor point information; when the persistently stored anchor point information contains a target anchor point identifier related to the spatial image, read the binding relationship information between the persistently stored anchor point identifier and the model identifier to determine whether there is a model identifier having a binding relationship with the target anchor point identifier; wherein, the target model identifier related to the spatial image is a model identifier having a binding relationship with the target anchor point identifier.

[0149] Furthermore, the above-mentioned acquisition module, when used to obtain the anchor point information corresponding to the target model identifier from the persistent information, can be specifically used to: obtain the anchor point posture corresponding to the target anchor point identifier having a binding relationship with the target model identifier from the persistently stored anchor point information; and, based on the anchor point information, display the virtual object model indicated by the target model identifier in the spatial image, including: displaying the virtual object model indicated by the target model identifier in the spatial image according to the anchor point posture corresponding to the target anchor point identifier.

[0150] Furthermore, the device also includes: a trigger module, which is used to trigger the execution of the step of reading the stored persistent information when the spatial image meets the initialization virtual rendering conditions and the augmented reality application is in an open state; wherein the spatial image meets the initialization virtual rendering conditions and includes at least one of the following: detecting that the augmented reality application is turned on; the spatial image is presented by rescanning the environment when the augmented reality application is not exited; the spatial image is presented after the virtual-reality device is restarted.

[0151] Another embodiment of the present application provides a virtual-reality space data processing device, which is deployed on a virtual-reality device. Specifically, the device includes: a display module, an adding module, and a storage module. Among them, the above-mentioned display module is used to display a virtual model interface, in which a plurality of virtual object models are displayed; and it is also used to respond to a selection operation triggered for the plurality of virtual object models, and to superimpose and display the selected target virtual object model in the spatial image of the current environment. The above-mentioned adding module is used to add a first anchor point to the target virtual object model based on the position of the target virtual object model in the spatial image. The storage module is used to persistently store the anchor point information of the first anchor point, and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object, so that when it is determined that the virtual rendering needs to be initialized for the spatial image later, the target virtual object model can be restored and displayed in the spatial image according to the stored persistent information.

[0152] Furthermore, the above-mentioned storage module is specifically used to: store the anchor point information of the first anchor point to a first persistent storage module; store the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object to a second persistent storage module; wherein the anchor point information includes the anchor point identifier and the anchor point posture; the first persistent storage module is used to store system layer persistent information, and the second persistent storage module is used to store service layer persistent information.

[0153] Furthermore, the device also includes: a deletion module, which is used to respond to the interactive operation on the target virtual model, and when it is determined that the preset deletion timing is met, delete the previously persistently stored anchor information of the first anchor point, and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object. And, the above-mentioned adding module is also used to add a second anchor point to the target virtual model according to the position of the target virtual model after the interactive operation when the interactive operation ends, and persistently store the anchor information of the second anchor point, and the binding relationship between the anchor point identifier of the second anchor point and the model identifier of the target virtual object;

[0154] Wherein, satisfying the preset deletion timing includes: starting the interactive operation, removing the target virtual object model, and deselecting the target virtual object model.

[0155] Another embodiment of the present application provides a virtual home improvement device, which is deployed on a virtual and real device. Specifically, the device includes: a rendering module, a reading module, an acquisition module, and a display module. Among them, the rendering module is used to present the spatial image of the current house space. The reading module is used to read the stored persistent information when it is determined that the spatial image needs to be initialized for virtual rendering. The acquisition module is used to obtain the anchor point information corresponding to the target model identifier related to the spatial image from the persistent information; wherein, one of the target model identifiers is used to indicate a virtual home improvement model that has been placed in the spatial image before. The display module is used to restore and display the virtual home improvement model indicated by the target model identifier in the spatial image according to the anchor point information.

[0156] Furthermore, the above-mentioned display module is also used to: in response to an interface opening operation, display a virtual model interface including multiple virtual home decoration models; in response to a selection operation triggered for the multiple virtual home decoration models, overlay and display the selected target virtual home decoration model in the spatial image. And, the device also includes: an adding module and a storage module. The adding module is used to add an anchor point to the target virtual loading model based on the position of the target virtual home decoration model in the spatial image. The storage module is used to persistently store the anchor point information of the anchor point, and the binding relationship between the anchor point identifier of the anchor point and the model identifier of the target virtual home decoration, so that when it is determined that the virtual rendering needs to be initialized for the spatial image later, the target virtual home decoration model can be restored and displayed in the spatial image according to the stored persistent information.

[0157] It should be noted here that: the devices provided in the above embodiments can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can be found in the corresponding contents in the above method embodiments, which will not be repeated here.

[0158] Figure 4 The schematic diagram of the structure of a virtual reality device provided by the embodiment of the present application is shown. Figure 4As shown, the electronic device includes: a storage device 71 and a processor 72. The storage device 71 includes a memory and a persistent storage module. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk. Specifically, the memory is used to store programs. And, the persistent storage module is used to persistently store anchor point information, binding relationship information between an anchor point identifier and a model identifier; the model identifier is used to indicate a corresponding virtual object model, and the anchor point identifier is used to indicate an anchor point added to the corresponding virtual object model.

[0159] The processor 72 is coupled to the memory, and is used to execute the program stored in the memory, and based on the persistent storage module, to implement the steps or functions in the methods provided in the embodiments of the present application.

[0160] Furthermore, if Figure 4 The above-mentioned persistent storage module includes a first persistent storage module and a second persistent storage module. The first persistent storage module is used to store the system layer persistent information of the virtual and real devices, and the system layer persistent information includes anchor information. The second persistent storage module is used to store the service layer persistent information of the virtual and real devices, and the service layer persistent information includes the binding relationship information between the anchor identifier and the model identifier.

[0161] Going further, Figure 4 The virtual-reality device also includes: a communication component 73, a power component 74, an audio component 75, a display component (not shown in the figure) and other components. Figure 4 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 4 Components shown.

[0162] It should be supplemented here that the virtual-reality device can be but is not limited to an MR device. Specifically, in terms of structure, the virtual-reality device can be an integrated head-mounted device, or it can also be a split head-mounted device. In the case of a split head-mounted device, the virtual-reality device may include a virtual-reality head-mounted device and a processing device that communicates externally with the virtual-reality head-mounted device. The above-mentioned processor 72 can be deployed in the processing device. In addition, the above-mentioned memory and persistent storage module can also be deployed in the processing device. Of course, in some other embodiments, they can also be deployed in the virtual-reality head-mounted device, which is not specifically limited in this embodiment. Preferably, the above-mentioned memory and persistent storage module are deployed in the processing device, which, on the one hand, makes it easier for the processor to retrieve the corresponding required data from the memory and the persistent storage module, and on the other hand, it can reduce the weight of the virtual-reality head-mounted device and enhance the user's wearing experience.

[0163] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computer, the method steps provided in the above embodiments can be implemented.

[0164] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the processor is enabled to implement the method steps or functions provided in the above embodiments.

[0165] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing virtual and real space data, characterized in that: include: When it is determined that the spatial image of the environment needs to be initialized for virtual rendering, the stored persistent information is read; Acquire anchor point information corresponding to a target model identifier associated with the spatial image from the persistent information; wherein one target model identifier is used to indicate a virtual object model previously placed in the spatial image; The virtual object model indicated by the target model identifier is displayed in the spatial image according to the anchor point information.

2. The method according to claim 1, characterized in that: Read stored persistent information, including: Read the persistently stored anchor information; When the persistently stored anchor point information contains a target anchor point identifier related to the spatial image, reading the persistently stored binding relationship information between the anchor point identifier and the model identifier to determine whether there is a model identifier having a binding relationship with the target anchor point identifier; The target model identifier associated with the spatial image is a model identifier having a binding relationship with the target anchor point identifier.

3. The method according to claim 2, characterized in that Acquiring anchor point information corresponding to the target model identifier from the persistent information includes: Acquire, from the persistently stored anchor point information, an anchor point posture corresponding to the target anchor point identifier having a binding relationship with the target model identifier; And, displaying the virtual object model indicated by the target model identifier in the space image according to the anchor point information, comprising: According to the anchor point posture corresponding to the target anchor point identifier, the virtual object model indicated by the target model identifier is displayed in the spatial image.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: When the spatial image meets the initialization virtual rendering condition and the augmented reality application is in an open state, triggering the step of reading the stored persistent information; The spatial image satisfies the initialization virtual rendering condition, which includes at least one of the following: Detecting that the augmented reality application is turned on; The spatial image is presented by rescanning the environment without exiting the augmented reality application; The spatial image is presented after the virtual-reality device is restarted.

5. A method for processing virtual and real space data, characterized in that: include: Displaying a virtual model interface, wherein a plurality of virtual object models are displayed in the virtual model interface; In response to a selection operation triggered on the plurality of virtual object models, superimposing and displaying a selected target virtual object model in a spatial image of a current environment; Adding a first anchor point to the target virtual object model based on the position and posture of the target virtual object model in the spatial image; The anchor point information of the first anchor point and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object are persistently stored, so that when it is subsequently determined that initial virtual rendering needs to be performed for the spatial image, the target virtual object model can be restored and displayed in the spatial image according to the stored persistent information.

6. The method according to claim 5, characterized in that Persistently storing the anchor point information of the first anchor point and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object, including: Storing the anchor point information of the first anchor point in a first persistent storage module; storing a binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object in a second persistent storage module; Among them, the anchor point information includes an anchor point identifier and an anchor point posture; the first persistent storage module is used to store system layer persistent information, and the second persistent storage module is used to store service layer persistent information.

7. The method according to claim 5 or 6, characterized in that: Also includes: In response to an interactive operation on the target virtual model, when it is determined that a preset deletion timing is met, deleting the previously persistently stored anchor point information of the first anchor point and the binding relationship between the anchor point identifier of the first anchor point and the model identifier of the target virtual object; When the interactive operation ends, according to the position of the target virtual model after the interactive operation, a second anchor point is added to the target virtual model, and anchor point information of the second anchor point and a binding relationship between the anchor point identifier of the second anchor point and the model identifier of the target virtual object are persistently stored; Wherein, satisfying the preset deletion timing includes: starting the interactive operation, removing the target virtual object model, and deselecting the target virtual object model.

8. A method for virtual home decoration, characterized in that: include: Present the spatial image of the current house space; When it is determined that the spatial image needs to be initialized for virtual rendering, reading the stored persistent information; Acquire anchor point information corresponding to a target model identifier associated with the spatial image from the persistent information; wherein one target model identifier is used to indicate a virtual home decoration model that has been previously placed in the spatial image; According to the anchor point information, the virtual home improvement model indicated by the target model identifier is restored and displayed in the spatial image.

9. The method according to claim 8, characterized in that: Also includes: In response to the interface opening operation, displaying a virtual model interface including a plurality of virtual home improvement models; In response to a selection operation triggered on the plurality of virtual home improvement models, superimposing and displaying a selected target virtual home improvement model in the spatial image; Based on the position and posture of the target virtual home decoration model in the space image, adding anchor points to the target virtual loading model; The anchor point information of the anchor point and the binding relationship between the anchor point identifier of the anchor point and the model identifier of the target virtual home are persistently stored, so that when it is determined that initial virtual rendering needs to be performed for the spatial image later, the target virtual home model can be restored and displayed in the spatial image according to the stored persistent information.

10. A virtual reality device, characterized in that: include: Persistent storage module and processor; wherein, The persistent storage module is used to persistently store the anchor point information and the binding relationship information between the anchor point identifier and the model identifier; The model identifier is used to indicate a corresponding virtual object model, and the anchor point identifier is used to indicate an anchor point added to the corresponding virtual object model; The processor is used to implement the steps in the method described in any one of claims 1 to 4, claims 5 to 7, or claims 8 to 9 based on the persistent storage module.

11. The virtual reality device according to claim 10, characterized in that: The persistent storage module includes a first persistent storage module and a second persistent storage module; The first persistent storage module is used to store the system layer persistent information of the virtual and real device, wherein the system layer persistent information includes anchor point information; the anchor point information includes anchor point identification and anchor point posture; The second persistent storage module is used to store the service layer persistent information of the virtual and real device, and the service layer persistent information includes the binding relationship information between the anchor point identifier and the model identifier.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program; when the computer program is executed by a computer, it can implement the steps in the method described in any one of claims 1 to 4, or claims 5 to 7, or claims 8 to 9.

13. A computer program product, characterized in that The method comprises a computer program, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 4, claims 5 to 7, or claims 8 to 9.

Images