Virtual environment space generation method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses a virtual environment space generation method and device, electronic equipment and a medium, and relates to the technical field of meta universe, and one specific embodiment of the method comprises the following steps: obtaining a panoramic image and depth information of a target real scene based on the user position of a user in response to a concentration mode starting request triggered by the user; constructing a virtual environment space based on the panoramic image and the depth information; and projecting the virtual environment space to a display interface of user equipment of the user, so that the user performs virtual operation based on the virtual environment space in the concentration mode. When a user carries out operations needing to be focused such as virtual modeling, interference situations such as sudden personnel intrusion in reality can be isolated, and an independent and interference-free focused operation environment is created for the user.

Description

Technical Field

[0001] This application relates to the field of metaverse technology, specifically to a method, apparatus, electronic device, and medium for generating virtual environment space. Background Technology

[0002] Currently, in related technologies, when users perform virtual operations, they do so against a real-world background, achieving a fusion of virtual and real effects.

[0003] However, the virtual-real fusion effect of related technologies depends entirely on the projection of real-time real-world scenes. The real-world scene may be affected by uncontrollable real objects. For example, if a user is working on virtual modeling and someone else suddenly enters the real-world scene, it may interfere with the user's work. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and medium for generating virtual environment space to solve the problems in related technologies where virtual-real fusion relies on real-time real-world scene projection and is susceptible to interference from uncontrollable real objects.

[0005] In a first aspect, embodiments of this application provide a method for generating a virtual environment space. The method includes: responding to a user-triggered request to activate a focus mode, acquiring a panoramic image and depth information of a target real-world scene based on the user's location; constructing a virtual environment space based on the panoramic image and depth information; and projecting the virtual environment space onto the display interface of the user's device, so that the user can perform virtual operations based on the virtual environment space in focus mode.

[0006] In some embodiments, constructing a virtual environment space based on panoramic images and depth information includes: constructing an initial virtual space based on depth information; and rendering the panoramic image as an environment texture into the initial virtual space to obtain the virtual environment space.

[0007] In some embodiments, acquiring a panoramic image of a target real-world scene based on the user's location includes: determining whether a first virtual object exists in the target real-world scene; if a first virtual object exists in the target real-world scene, acquiring a panoramic image of the target real-world scene based on the user's location, and storing the coordinates of the first virtual object in the target real-world scene, wherein the panoramic image does not include the first virtual object.

[0008] In some embodiments, after constructing a virtual environment space based on a panoramic image, the method includes: creating a first virtual object in the virtual environment space based on the coordinates of a first virtual object.

[0009] In some embodiments, after projecting a virtual environment space onto the display interface of a user's device, the method includes: determining an operation direction vector corresponding to the user's gesture operation; determining a target marker in the virtual environment space based on the operation direction vector; creating a second virtual object based on the target marker coordinates in the virtual environment space, and storing the mapping relationship between the operation direction vector, the target marker, and the second virtual object.

[0010] In some embodiments, after storing the operation direction vector, the mapping relationship between the target identifier and the second virtual object, the method includes: in response to a user-triggered focus mode exit request, determining the virtual object direction vector of the second virtual object in the virtual environment space; acquiring at least one scene object in the target real scene; based on the virtual object direction vector, determining a target scene object among the at least one scene object that matches the target identifier, wherein the target identifier is a target identifier associated with the second virtual object determined based on a pre-stored mapping relationship; and creating the second virtual object on the target scene object.

[0011] Secondly, embodiments of this application provide a virtual environment space generation apparatus, the apparatus comprising: The acquisition unit is used to respond to the user's request to activate the focus mode and acquire panoramic images and depth information of the target real-world scene based on the user's location. Construction units are used to build virtual environment spaces based on panoramic images and depth information; The interaction unit is used to project the virtual environment space onto the user's device display interface, so that the user can perform virtual operations based on the virtual environment space in focus mode.

[0012] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor runs the computer program, it performs the method described in any embodiment of the first aspect.

[0013] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described in any embodiment of the first aspect.

[0014] Fifthly, embodiments of this application provide a computer program product including a computer program that, when executed by a processor, performs the method described in any embodiment of the first aspect.

[0015] This application provides a method for generating a virtual environment space. In response to a user-triggered request to activate a focus mode, the method acquires a panoramic image and depth information of the target real-world scene based on the user's location. Based on the panoramic image and depth information, a virtual environment space is constructed. This virtual environment space is then projected onto the user's device display interface, allowing the user to perform virtual operations within the virtual environment space while in focus mode. This method effectively isolates the user from distractions such as unexpected intrusions in the real world when performing focused operations like virtual modeling, creating an independent and undisturbed environment for focused operation.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are merely embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort, and this application can be applied to other similar scenarios based on the provided drawings.

[0018] Figure 1 A flowchart illustrating a method for generating a virtual environment space provided in this application embodiment; Figure 2 A flowchart illustrating the second virtual environment space generation method provided in this application embodiment; Figure 3 A schematic diagram of a specific target real-world scenario provided for an embodiment of this application; Figure 4 A schematic diagram of a specific virtual environment space provided for an embodiment of this application; Figure 5 This is a schematic diagram illustrating a specific user performing virtual operations in a virtual environment space, as provided in an embodiment of this application. Figure 6 A schematic diagram illustrating the creation of a second virtual object in a target virtual scene, provided as an embodiment of this application; Figure 7 A schematic diagram illustrating a specific virtual environment space generation method provided in this application embodiment; Figure 8 A schematic diagram of the structure of a virtual environment space generation device 800 provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. The described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] It should be noted that the terms "system," "device," "unit," and / or "module" used in this application are methods of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they can be replaced by other expressions.

[0021] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0022] Currently, the basic process of using mixed reality devices involves mapping virtual objects onto a real-world scene in real time and establishing interactions or binding relationships with objects in the scene. Users can then interact with, edit, and save the virtual objects to achieve a fusion of virtual and real elements.

[0023] However, in real-world scenarios, interacting with virtual objects may be affected by uncontrollable real-world objects. For example, if a user is working on virtual modeling and someone else suddenly enters the real-world environment, it may disrupt the user's work.

[0024] To address the problems in existing technologies, this application proposes a method for generating virtual environment space. By responding to a user's request to activate a focus mode, the method acquires panoramic images and depth information of the target real-world scene based on the user's location to construct a virtual environment space, which is then projected onto the user's device display interface. This effectively solves the problems of reliance on real-time real-world scene projection and susceptibility to interference from uncontrollable real-world objects in related technologies. When users perform operations requiring focus, such as virtual modeling, this method can isolate them from unexpected intrusions in the real world, creating an independent and undisturbed environment for focused operation. Furthermore, the virtual environment space constructed using panoramic images and depth information retains visual relevance to the real-world scene while ensuring the continuity and stability of the user's virtual operations. This significantly improves the user's focus and experience in mixed reality scenarios, meeting the user's need for high-quality, undisturbed virtual-real interaction in complex real-world environments.

[0025] The following section provides a detailed description of a virtual environment space generation method provided in this application, with reference to the accompanying drawings.

[0026] Figure 1 A flowchart illustrating a virtual environment space generation method provided in an embodiment of this application is shown. Figure 1 As shown, the virtual environment space generation method includes steps 101-103.

[0027] Step 101: In response to the user-triggered request to enable focus mode, obtain panoramic images and depth information of the target real-world scene based on the user's location.

[0028] In the embodiments of this application, this application can first receive the user's active operation command to clarify the user's need to activate the focus mode; then, taking the user's current physical location (i.e., the user's location) as the core reference point, it can simultaneously collect the visual panorama data (panoramic image) and spatial dimension data (depth information) of the real scene to provide data support for the subsequent construction of a virtual environment that matches the real scene, ensuring that the virtual environment not only fits the real visual experience but also conforms to the real spatial scale.

[0029] The focus mode activation request can be a command issued by the user through preset operations on the user device (i.e., mixed reality device, such as MR headset) (such as gesture triggering, voice command, physical button click on the device, etc.) to start the anti-interference virtual operating environment and is the starting condition for triggering execution.

[0030] User location refers to the physical spatial coordinates of the mixed reality device (attached to the user, such as a head-mounted display) when the user activates focus mode. It is usually represented by three-dimensional coordinates determined by the device's built-in positioning module (such as GPS or inertial navigation module) and serves as the reference origin for collecting panoramic images and depth information.

[0031] The target real-world scenario is the real physical environment where the user is currently located and needs to perform virtual operations (such as the user's work area, a part of the family living room, an empty room, etc.). It serves as the real-world prototype for subsequent image and depth information acquisition and the construction of the virtual environment.

[0032] Panoramic images are created by using the camera of a mixed reality device to capture 360-degree surround shots from the user's location. These images are stitched together and fused from multiple frames of real-world scene images taken from different angles to form full-view visual data. This data can completely present the visual information of the surrounding environment (such as wall color, furniture layout, and floor texture) of the target real-world scene.

[0033] Depth information is collected through the depth camera (or sensors such as LiDAR) of a mixed reality device. The distance data of each object's surface point relative to the device's camera in the target real scene is the core data describing the three-dimensional position, outline, and spatial spacing of objects in real space (such as the distance from the table surface to the camera, the vertical distance between the wall and the user, etc.).

[0034] Step 102: Construct a virtual environment space based on panoramic images and depth information.

[0035] In the embodiments of this application, panoramic images (visual data) and depth information (spatial data) can be fused to obtain a virtual environment space.

[0036] Specifically, this application can use the depth information collected step by step as a "spatial skeleton" to determine the three-dimensional scale and boundaries of the virtual environment, ensuring that the virtual environment is consistent with the spatial size and layout of the real scene; then, panoramic images are used as a "visual skin" to cover the spatial skeleton, restoring the visual effect of the real scene; finally, a virtual environment space that "fits the reality in terms of spatial scale and restores the reality in terms of visual presentation" is formed, which not only allows users to have a familiar environmental perception, but also isolates them from the dynamic interference of the subsequent real scene.

[0037] Virtual environment space refers to a dedicated virtual interactive space in focus mode. It is a digital environment built based on panoramic images and depth information of the real scene and independent of the real-time real scene. All virtual operations of the user in focus mode are carried out in this space. Its spatial scale and visual presentation are highly matched with the target real scene, but it is not affected by dynamic interference objects in the real scene (such as intruders or moving furniture).

[0038] Step 103: Project the virtual environment space onto the user's device display interface so that the user can perform virtual operations based on the virtual environment space in focus mode.

[0039] In the embodiments of this application, the completed virtual environment space can be transformed into a visual image that the user can intuitively observe through the hardware capabilities of the mixed reality device; at the same time, the device's projection of the real-time real scene is paused, and only the virtual environment space is displayed, so that the user can get rid of the interference of the real-time real scene and focus on creating, editing, interacting with virtual objects in the virtual environment, achieving a "non-interference + highly immersive" virtual operation experience.

[0040] Projection is the process by which mixed reality devices transform a digitized virtual environment (in data form) into a visual image through the device's built-in display hardware (such as the dual screens and optical components of an MR headset) and present it to the user. Essentially, it is the conversion and output of digital data into a visual image, ensuring that the user can directly observe the virtual environment.

[0041] User equipment is a terminal device that supports mixed reality technology, such as MR headsets and mixed reality glasses. It needs to have hardware capabilities such as a camera (to capture real-world images), a depth sensor (to capture depth information), a display interface (to present a virtual environment), and an interaction module (to capture user operations). It is the execution carrier of this application.

[0042] The display interface is the hardware component in the user device used to present visual images (such as the built-in dual-eye display of an MR headset, or the external display screen of the device). It is the "carrier" that the virtual environment space is ultimately observed by the user. Its display effect needs to conform to the visual habits of the human eye to ensure the immersive experience of the virtual environment.

[0043] Virtual operations are various interactive behaviors that users perform on virtual objects in focus mode, including but not limited to creating (such as building 3D models), editing (such as modifying the shape and attributes of virtual objects), moving (such as adjusting the position of virtual objects in the virtual environment), deleting, and saving virtual objects. These operations are all completed based on the virtual environment space and do not affect the real scene.

[0044] In summary, the method for generating a virtual environment space proposed in this application, in response to a user-triggered request to activate a focus mode, acquires a panoramic image and depth information of the target real-world scene based on the user's location; constructs a virtual environment space based on the panoramic image and depth information; and projects the virtual environment space onto the display interface of the user's device, enabling the user to perform virtual operations based on the virtual environment space in focus mode. This achieves the goal of isolating the user from distractions such as sudden intrusions in the real world when performing focused operations such as virtual modeling, creating an independent and interference-free focus environment for the user.

[0045] based on Figure 1 The embodiment shown, Figure 2 A flowchart of the second method for generating virtual environment space is further shown. Figure 2 based on Figure 1 The illustrated embodiment further defines step 102. Figure 2 In the illustrated embodiment, step 102 includes steps 202 and 203. For example... Figure 2 As shown, the method includes the following steps: Step 201: In response to the user-triggered request to enable focus mode, obtain panoramic images and depth information of the target real-world scene based on the user's location.

[0046] Step 202: Construct an initial virtual space based on depth information.

[0047] Step 203: Render the panoramic image as an environment texture into the initial virtual space to obtain the virtual environment space.

[0048] In this embodiment, the user can choose any suitable scenario (such as a workstation, a private room, or an open space free from interference) and time point according to actual needs, and send a focus mode activation request to this application through preset interaction methods supported by the mixed reality device (such as gesture commands, voice wake-up, physical button pressing, device interface touch control, etc.). This request is the trigger signal for the entire process, used to inform the user that they need to switch to an interference-free virtual operating environment.

[0049] When acquiring panoramic images, this application first scans the current mixed reality scene (i.e., the scene projected by the device when the user triggers a request) to determine whether there are any virtual objects that have been created and displayed (defined as the first virtual object, such as 3D models, virtual documents, interactive components, etc. previously built by the user).

[0050] If a first virtual object is detected, this application records the three-dimensional spatial coordinates of each first virtual object relative to the mixed reality device's coordinate system (i.e., the coordinates of the first virtual object, including X, Y, and Z axis parameters to ensure accurate position reconstruction later), while retaining the attribute data of the first virtual object (such as shape, size, color, interaction permissions, etc.), only pausing its rendering and projection. If no first virtual object is detected, the panoramic image is directly acquired. Figure 3 The diagram shown is a specific real-world scenario provided in this application. Figure 3 This demonstrates the scenario where the first virtual object does not exist in the target real-world scene.

[0051] To avoid the first virtual object obscuring the real-world scene or interfering with image integrity, this application first hides the first virtual object (and temporarily does not project it onto the device's display screen) before capturing panoramic images, ensuring that the final panoramic image contains only real-world scene elements and does not include any virtual objects. Specifically, using the user's current physical location (i.e., the location of the mixed reality device) as the acquisition base point, this application controls the device's camera to activate a 360-degree surround shooting mode. Through camera rotation scanning or multi-camera collaboration, it captures 360-degree (front, back, left, right, top, and bottom) images of the real-world scene around the user. Then, through image stitching and distortion correction algorithms, it generates a panoramic image that completely covers the visual information of the target real-world scene (such as wall texture, furniture layout, floor material, etc.).

[0052] While acquiring panoramic images, this application activates the depth camera (or depth perception module such as LiDAR or ToF sensor) of the mixed reality device to collect distance data of each object surface point in the target real scene relative to the depth camera (such as the vertical distance from the table surface to the camera, the horizontal distance between the wall and the user, etc.). These distance data are integrated to form depth information, which is used to describe the three-dimensional structure, object distribution and boundary range of the real space.

[0053] This application analyzes the maximum coverage area of ​​the real space based on depth information (including all objects and the activity space required for user operation) and generates a minimum bounding cuboid (i.e., the initial virtual space) that can completely enclose this area. This cuboid is the physical skeleton of the virtual environment space, and its size and spatial proportion are strictly consistent with the real scene (e.g., if the distance from the user to the table in reality is 1 meter, the corresponding distance in the virtual environment is also 1 meter), ensuring that the user's spatial perception in the virtual environment is consistent with reality.

[0054] Finally, this application employs "cubmap environment box" technology to process the panoramic image. This involves decomposing the panoramic image into six orthogonal planar textures (corresponding to the front, back, left, right, top, and bottom surfaces of the circumscribed cuboid). A texture mapping algorithm is then used to precisely attach these textures to the corresponding surfaces of the circumscribed cuboid. The device's rendering engine then optimizes lighting and shadow effects, creating an "immersive environment that visually closely matches the real-world scene" within the circumscribed cuboid—essentially a virtual environment space. If a previously hidden first virtual object exists, after its creation, this application can retrieve the stored coordinates of the first virtual object and locate its corresponding position in the virtual environment space (e.g., the position of a table in reality at the corresponding coordinates in the virtual environment). The first virtual object is then re-rendered, ensuring that the user can continue to interact with the existing virtual object without needing to recreate it.

[0055] After obtaining the final virtual environment space, this application can project the virtual environment space onto the user's mixed reality device display (such as the binocular screens of an MR headset). Specifically, as follows... Figure 4 As shown, this application provides a schematic diagram of a specific virtual environment space, referring to... Figure 4 Users are in a constructed virtual environment (symbolized by a cube with panoramic images attached to its surface), where they can focus on interacting with virtual objects. Meanwhile, distracting objects in the target real-world scene (such as the little man on the left) will not appear on the user's mixed reality device screen because they are not included in the panoramic image capture range. This achieves the goal of isolating real-world interference and ensuring that users can focus on their operations.

[0056] Step 204: Project the virtual environment space onto the user's device display interface so that the user can perform virtual operations based on the virtual environment space in focus mode.

[0057] In the embodiments of this application, the constructed virtual environment space can be projected to the user first, allowing the user to interact with virtual objects based on gesture operations in an interference-free environment; by calculating the operation direction vector and identifying target markers, the association creation and mapping relationship storage between virtual objects and scene elements can be realized; finally, when the user exits the focus mode, the virtual objects are accurately restored to the real mixed scene based on the mapping relationship, which not only ensures the continuity of the user's focused operation, but also achieves seamless connection of results.

[0058] In the embodiments of this application, virtual environment space projection and user virtual operation initiation are discussed. For example... Figure 5 As shown, this application provides a specific schematic diagram of a user performing virtual operations in a virtual environment. (Refer to...) Figure 5 This application projects a constructed virtual environment space (consisting of a panoramic view and an outer cuboid, including a reconstructed first virtual object) onto the user's mixed reality device display interface. At this time, the screen seen by the user is entirely a virtual environment space, isolated from the real-time reality scene, allowing the user to focus on interacting with and modeling virtual objects (such as creating new virtual objects, editing the attributes of existing virtual objects, etc.) within this environment.

[0059] When a user performs virtual operations in the virtual environment, this application can capture the three-dimensional coordinates of the user's gestures (such as the spatial position of click or drag actions) interacting with virtual objects in the virtual environment through the gesture recognition module of the mixed reality device. Taking the real-time position of the user's head display as the starting point and the gesture coordinates as the ending point, an operation direction vector (describing the spatial direction of the gesture operation) is calculated and generated.

[0060] Based on the operation direction vector, a region with a radius of 1 meter is selected in the cubmap environment texture of the virtual environment space. A pre-trained image recognition algorithm (such as an object detection model) is called to analyze the region, identify objects of a preset category (such as tables, stools, walls, etc., defined as "target markers"), and record the target marker coordinates in the virtual environment space.

[0061] Once the target object is identified, a built-in program command is triggered (such as "table detected, allow creation of virtual objects in the desktop area"). Based on this command, the user creates a new virtual object (defined as a second virtual object, such as a 3D model, virtual document, etc.) at the location corresponding to the target object.

[0062] At this point, this application can also bind the relationship between the operation direction vector, the target identifier, and the second virtual object, and store it in a database. This mapping relationship is the key basis for the virtual object to be accurately restored to the real scene when exiting the focus mode.

[0063] In this embodiment of the application, the focus mode exit is mapped to the reality of the second virtual object. For example... Figure 6 The diagram illustrates a specific method for creating a second virtual object in a target virtual scene, as provided in this application. After the user completes the virtual operation and triggers a focus mode exit request, this application can begin the real-world scene mapping process for the virtual object.

[0064] Specifically, this application can retrieve the operation direction vector (referred to as the virtual object direction vector) bound to the second virtual object in the database.

[0065] Then, the camera of the mixed reality device is controlled to scan a fan-shaped area along the direction vector of the virtual object, and collect image data and depth data of objects in the real scene in that direction. After classifying and recognizing the objects, a list of scene objects containing object categories, position coordinates, and size parameters is generated (i.e., at least one scene object, such as scanning a table, stool, or lamp).

[0066] The system compares objects in the scene object list with target markers bound in the mapping relationship. If the object category is consistent and the deviation between its position coordinates and the virtual object's direction vector is less than a preset threshold (e.g., a table matching the target marker actually exists in that direction), then the object is determined to be a target scene object. At this point, the application automatically creates a second virtual object at the position corresponding to the target scene object, adjusting its spatial coordinates and orientation to ensure that the virtual object's presentation in the real-world mixed scene remains consistent with its presentation in the virtual environment space.

[0067] In summary, this application constructs a virtual environment that conforms to real-world vision and space by collecting panoramic images and depth information of the target real-world scene based on the user's location and processing the original virtual objects. This virtual environment is then projected onto the user's device display interface, allowing users to create and store mapping relationships of virtual objects based on operation direction vectors and target markers in a virtual environment free from real-world interference. Ultimately, this enables users to activate a focus mode anytime, anywhere, ensuring the continuity and accuracy of virtual operations while isolating real-world interference objects, significantly improving the focus experience and work efficiency of virtual interaction in mixed reality scenarios.

[0068] based on Figures 1 to 2 The illustrated embodiments, such as Figure 7 As shown, this application provides a schematic diagram of a specific method for generating virtual environment space.

[0069] Reference Figure 3Users first operate virtual objects in a mixed reality scene (i.e., the target real scene), then turn on the anti-interference focus mode to enter a virtual environment space constructed by panoramic images; after working, modeling or playing in this virtual environment space and generating new virtual objects (work results), they can exit the anti-interference focus mode and bring the virtual objects of the work results back to the mixed reality scene, achieving a complete experience of seamless connection between interference-free operation and results.

[0070] Specifically, this application first constructs an outer cuboid based on panoramic images and depth information and performs panoramic texture rendering to generate a dedicated virtual environment space for focus mode. This significantly saves virtual scene construction time and resource consumption while eliminating scene interference objects. Second, if virtual objects exist in the original mixed reality scene, their coordinates are hidden and saved when acquiring panoramic images. After the virtual environment space is generated, they are restored to their corresponding positions, ensuring that existing virtual objects do not interfere with environment construction and can be seamlessly integrated into the focus scene. Third, within the virtual space, this application can determine the user's operation direction vector and identify landmarks in the virtual environment space to achieve virtual object interaction or modeling related to the real scene, and bind and save the direction vector, landmarks, and virtual objects. Finally, when exiting focus mode, based on the direction vector corresponding to the virtual object, the application matches the object corresponding to the landmark in the real scene and accurately creates the virtual object at the object's position. This allows the user's interaction or modeling results in focus mode to be directly mapped to the real scene, satisfying the virtual object operation needs of mixed reality scenes, isolating real-world interference during operation, and ensuring no difference in experience after exiting.

[0071] It should be noted that the specific implementation process of each module in the virtual environment space generation system can be found by referring to... Figures 1 to 2 The embodiments shown will not be described again here.

[0072] To achieve the above embodiments, this application also provides a virtual environment space generation device. Figure 8 This is a schematic diagram of the structure of a virtual environment space generation device 800 provided in an embodiment of this application. Figure 8 As shown, the device includes: The acquisition unit 810 is used to respond to a user-triggered focus mode activation request and acquire panoramic images and depth information of the target real-world scene based on the user's location. Construction unit 820 is used to construct a virtual environment space based on panoramic images and depth information; The interaction unit 830 is used to project a virtual environment space onto the display interface of the user's device, so that the user can perform virtual operations based on the virtual environment space in focus mode.

[0073] In some embodiments, the construction unit 820 is configured to: construct an initial virtual space based on depth information; and render a panoramic image as an environment texture into the initial virtual space to obtain a virtual environment space.

[0074] In some embodiments, the acquisition unit 810 is configured to: determine whether a first virtual object exists in the target real scene; if a first virtual object exists in the target real scene, acquire a panoramic image of the target real scene based on the user's location, and store the coordinates of the first virtual object in the target real scene, wherein the panoramic image does not include the first virtual object.

[0075] In some embodiments, the construction unit 820 is configured to: after constructing a virtual environment space based on a panoramic image, create a first virtual object in the virtual environment space based on the coordinates of the first virtual object of the first virtual object.

[0076] In some embodiments, the interaction unit 830 is configured to: after projecting the virtual environment space onto the display interface of the user's device, determine the operation direction vector corresponding to the user's gesture operation; determine the target marker in the virtual environment space based on the operation direction vector; create a second virtual object based on the target marker coordinates in the virtual environment space, and store the mapping relationship between the operation direction vector, the target marker and the second virtual object.

[0077] In some embodiments, the interaction unit 830 is configured to: after storing the mapping relationship between the operation direction vector, the target identifier, and the second virtual object, in response to a user-triggered focus mode exit request, determine the virtual object direction vector of the second virtual object in the virtual environment space; acquire at least one scene object in the target real scene; based on the virtual object direction vector, determine the target scene object that matches the target identifier among the at least one scene object, wherein the target identifier is a target identifier associated with the second virtual object determined based on a pre-stored mapping relationship; and create the second virtual object on the target scene object.

[0078] The methods and apparatus provided in the embodiments of this application have been described above. To implement the functions of the methods provided in the embodiments of this application, the electronic device may include a hardware structure and software modules, and may implement the above functions in the form of a hardware structure, software modules, or a hardware structure plus software modules. One of the above functions may be executed in the form of a hardware structure, software modules, or a hardware structure plus software modules.

[0079] Figure 9This is a block diagram illustrating an electronic device 900 for implementing the above-described virtual environment space generation method, according to an exemplary embodiment. For example, the electronic device 900 may be a mobile phone, computer, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0080] Reference Figure 9 The electronic device 900 may include a communication interface 901, capable of interacting with other devices; a processor 902, connected to the communication interface 901 to enable interaction with other devices, used to execute the methods provided by one or more of the above-described technical solutions when running a computer program; and a memory 903, on which the computer program is stored. Specifically, the specific processing procedure of the processor 902 can refer to the virtual environment space generation method described in the above embodiments of this disclosure.

[0081] Of course, in practical applications, the various components in electronic device 900 are coupled together through bus system 904. It can be understood that bus system 904 is used to realize the connection and communication between these components. In addition to a data bus, bus system 904 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 9 The general designated all buses as Bus System 904.

[0082] The memory 903 in this embodiment is used to store various types of data to support the operation of the electronic device 900. Examples of such data include any computer program used to operate on the electronic device 900.

[0083] The methods disclosed in the embodiments of this application can be applied to processor 902, or implemented by processor 902. Processor 902 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 902 or by instructions in the form of software. The processor 902 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 902 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 903. Processor 902 reads the information in memory 903 and combines its hardware to complete the steps of the aforementioned method.

[0084] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0085] Embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the virtual environment space generation method described in the above embodiments of this disclosure.

[0086] Embodiments of this disclosure also provide a computer program product, including a computer program that is executed by a processor using the virtual environment space generation method described in the above embodiments of this disclosure.

[0087] Embodiments of this disclosure also propose a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory, which, when executed by the processor, cause the electronic device to perform the virtual environment space generation method described in the above embodiments of this disclosure.

[0088] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0089] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0090] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0091] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (control method), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0092] It should be understood that various parts of the embodiments of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0093] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0094] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc.

[0095] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for generating a virtual environment space, characterized in that, The method includes: In response to a user-triggered request to activate focus mode, a panoramic image and depth information of the target real-world scene are acquired based on the user's location. Based on the panoramic image and the depth information, a virtual environment space is constructed; The virtual environment space is projected onto the display interface of the user's device, so that the user can perform virtual operations based on the virtual environment space in the focus mode.

2. The method according to claim 1, characterized in that, The construction of the virtual environment space based on the panoramic image and the depth information includes: Based on the depth information, an initial virtual space is constructed; The panoramic image is used as an environment texture and rendered into the initial virtual space to obtain the virtual environment space.

3. The method according to claim 1, characterized in that, The step of acquiring a panoramic image of the target real-world scene based on the user's location includes: Determine whether the first virtual object exists in the target real-world scene; If a first virtual object exists in the target real-world scene, a panoramic image of the target real-world scene is obtained based on the user's location, and the coordinates of the first virtual object in the target real-world scene are stored. The panoramic image does not include the first virtual object.

4. The method according to claim 3, characterized in that, After constructing a virtual environment space based on the panoramic image, the method includes: Based on the coordinates of the first virtual object, the first virtual object is created in the virtual environment space.

5. The method according to claim 1, characterized in that, After projecting the virtual environment space onto the display interface of the user's device, the method includes: Determine the operation direction vector corresponding to the user's gesture operation; Based on the operation direction vector, the target marker in the virtual environment space is determined; Based on the target marker's coordinates in the virtual environment space, a second virtual object is created, storing the operation direction vector and the mapping relationship between the target marker and the second virtual object.

6. The method according to claim 5, characterized in that, After storing the operation direction vector, the mapping relationship between the target identifier and the second virtual object, the method includes: In response to a user-triggered exit request from focus mode, determine the virtual object orientation vector of the second virtual object in the virtual environment space; Obtain at least one scene object from the target real-world scene; Based on the virtual object's direction vector, a target scene object matching the target identifier is determined among the at least one scene object. The target identifier is a target identifier associated with the second virtual object, determined based on a pre-stored mapping relationship. Create the second virtual object on the target scene object.

7. A virtual environment space generation device, characterized in that, The device includes: The acquisition unit is used to acquire panoramic images and depth information of the target real scene based on the user's location in response to a user-triggered request to activate the focus mode. A construction unit is used to construct a virtual environment space based on the panoramic image and the depth information; An interaction unit is used to project the virtual environment space onto the display interface of the user's user device, so that the user can perform virtual operations based on the virtual environment space in the focus mode.

8. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images