Metaverse platform interaction method, device and equipment based on virtual reality technology

Abstract

The embodiments of the present disclosure disclose a metaverse platform interaction method, apparatus, and device based on virtual reality technology. A specific implementation of the method includes: obtaining a sequence of item image information; for each target item, generating custom virtual item information based on the item image information; rendering each custom virtual item corresponding to each custom virtual item information into a custom virtual scene based on the initial display position information; in response to detecting that any custom virtual item intersects with a preset ray, performing position detection on the custom virtual item to obtain a position detection result; in response to determining that the position detection result meets a preset graspable condition, updating the custom virtual item; in response to receiving a confirmation grasping instruction, controlling the target virtual hand to perform a grasping operation on the updated virtual item. This implementation can increase the speed of virtual scene updates and enhance the user interaction experience.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of virtual reality technology, and more particularly to a metaverse platform interaction method, apparatus, and device based on virtual reality technology. Background Art

[0002] The Metaverse platform is a three-dimensional virtual environment built using 3D modeling, virtual reality, and blockchain technologies. Users can socialize, play games, and transfer items within the 3D scenes presented by the Metaverse platform. However, because virtual items presented in the virtual environment are typically integrated with the underlying virtual scene before being rendered and transmitted to the VR device, the virtual scene is relatively fixed, and users can only interact with a small number of virtual items inherent in the Metaverse platform's virtual scene. To add virtual items, the new virtual items must be integrated into the virtual scene, then rendered and transmitted to the VR device to update the previously displayed virtual scene. This results in slow virtual scene updates and a poor user interaction experience.

[0003] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0004] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] Some embodiments of the present disclosure propose metaverse platform interaction methods, devices, and equipment based on virtual reality technology to solve one or more of the technical problems mentioned in the above background technology section.

[0006] In a first aspect, some embodiments of the present disclosure provide a metaverse platform interaction method based on virtual reality technology, the method comprising: in response to detecting a selection operation of a scene item adding control acting on a metaverse platform interface, obtaining an item image information sequence, wherein each item image information in the above-mentioned item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the above-mentioned item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; for each target item in the preset target item sequence, performing the following steps: constructing a three-dimensional item model based on the item image information corresponding to the above-mentioned target item; generating custom virtual item information based on the above-mentioned three-dimensional item model, wherein the above-mentioned custom virtual item information corresponds to the custom virtual item; and displaying each obtained custom virtual item information based on preset initial display position information. The method includes rendering each custom virtual item into a custom virtual scene, and playing a preset item position adjustment prompt message to prompt the user to replace the custom virtual item; in response to detecting that any custom virtual item among the above-mentioned custom virtual items intersects with a preset ray, performing position detection on the above-mentioned any custom virtual item to obtain a position detection result, wherein the above-mentioned preset ray is a ray emitted by a target virtual hand and is used to select the custom virtual item to be replaced, and the target virtual hand is the virtual hand of a three-dimensional virtual user; in response to determining that the above-mentioned position detection result meets a preset graspable condition, updating the above-mentioned any custom virtual item to obtain an updated virtual item; in response to receiving a confirmation grasping instruction for the above-mentioned updated virtual item, controlling the target virtual hand to perform a grasping operation on the above-mentioned updated virtual item to place the above-mentioned updated virtual item at a target position.

[0007] In a second aspect, some embodiments of the present disclosure provide a metaverse platform interaction device based on virtual reality technology, the device comprising: an acquisition unit, configured to acquire an item image information sequence in response to detecting a selection operation of a scene item adding control acting on a metaverse platform interface, wherein each item image information in the above-mentioned item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the above-mentioned item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; an execution unit, configured to perform the following steps for each target item in the preset target item sequence: construct a three-dimensional item model based on the item image information corresponding to the above-mentioned target item; generate custom virtual item information based on the above-mentioned three-dimensional item model, wherein the above-mentioned custom virtual item information corresponds to the custom virtual item; a rendering and playback unit, configured to display the obtained custom virtual item information based on preset initial display position information. The corresponding custom virtual items are rendered into the custom virtual scene, and preset item position adjustment prompt information is played to prompt the user to replace the custom virtual items; the position detection unit is configured to, in response to detecting that any custom virtual item among the above-mentioned custom virtual items intersects with a preset ray, perform position detection on the above-mentioned any custom virtual item to obtain a position detection result, wherein the above-mentioned preset ray is a ray emitted by the target virtual hand and is used to select the custom virtual item to be replaced, and the target virtual hand is the virtual hand of the three-dimensional virtual user; the update unit is configured to, in response to determining that the above-mentioned position detection result meets the preset graspable condition, update the above-mentioned any custom virtual item to obtain an updated virtual item; the control unit is configured to, in response to receiving a confirmation grasping instruction for the above-mentioned updated virtual item, control the target virtual hand to perform a grasping operation on the above-mentioned updated virtual item to place the above-mentioned updated virtual item at the target position.

[0008] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0009] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0010] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the metaverse platform interaction method based on virtual reality technology of some embodiments of the present disclosure, the virtual scene update speed can be increased, and the user interaction experience can be improved. Specifically, the reason for the slow virtual scene update speed and the poor user interaction experience is that the virtual items presented by the virtual environment are usually integrated with the basic virtual scene, rendered and transmitted to the virtual reality device, making the virtual scene relatively fixed, and the user can only interact with a few inherent virtual items in the virtual scene of the metaverse platform. If you want to add virtual items, you need to integrate the new virtual items into the virtual scene, and then render and transmit them to the virtual reality device to update the previously displayed virtual scene, thereby resulting in a slow virtual scene update speed and a poor user interaction experience. Based on this, the metaverse platform interaction method based on virtual reality technology of some embodiments of the present disclosure first obtains an item image information sequence in response to detecting a selection operation of a scene item addition control in the metaverse platform interface. Each item image information in the item image information sequence corresponds to a target item in a preset target item sequence. Each item image information in the item image information sequence includes a multi-view item image set and a multi-view depth atlas. Therefore, when a user wishes to add a virtual item to a virtual scene, they can first obtain the item image information required to create each custom virtual item. Next, for each target item in the preset target item sequence, the following steps are performed: constructing a three-dimensional item model based on the item image information corresponding to the target item; and generating custom virtual item information based on the three-dimensional item model. The custom virtual item information corresponds to the custom virtual item. Thus, each custom virtual item can be created for subsequent placement in the virtual scene. Then, based on preset initial display position information, each custom virtual item corresponding to each obtained custom virtual item information is rendered into the custom virtual scene, and a preset item position adjustment prompt is played to prompt the user to reposition the custom virtual item. This allows the user to see each custom virtual item to be added in the custom virtual scene. Afterwards, in response to detecting that any of the aforementioned custom virtual items intersects a preset ray, a position detection is performed on the aforementioned custom virtual item to obtain a position detection result. The preset ray is a ray emitted by a target virtual hand, which is used to select the custom virtual item to be relocated. The target virtual hand is the virtual hand of the three-dimensional virtual user. Thus, the custom virtual item to be moved can be selected by emitting a ray. Next, in response to determining that the position detection result satisfies a preset graspability condition, the aforementioned custom virtual item is updated to obtain an updated virtual item. This allows the determination of the custom virtual item available for the user to grasp.Finally, in response to receiving a confirmation grabbing instruction for the updated virtual item, the target virtual hand is controlled to perform a grabbing operation on the updated virtual item, thereby placing the updated virtual item at the target location. Therefore, the virtual reality technology-based metaverse platform interaction method of some embodiments of the present disclosure facilitates real-time updates of the virtual scene by generating customized virtual items within the metaverse platform and, based on this, guiding the user to perform interactive operations such as grabbing and moving the virtual items. This can thereby increase the update speed of the virtual scene and, in turn, enhance the user's interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0012] Figure 1 is a flowchart of some embodiments of the metaverse platform interaction method based on virtual reality technology according to the present disclosure;

[0013] Figure 2 is a schematic structural diagram of some embodiments of a metaverse platform interaction device based on virtual reality technology according to the present disclosure;

[0014] Figure 3 It is a structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0016] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0017] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0018] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0019] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0020] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0021] Figure 1 The process 100 of some embodiments of the virtual reality technology-based metaverse platform interaction method according to the present disclosure is shown. The virtual reality technology-based metaverse platform interaction method includes the following steps:

[0022] Step 101, in response to detecting a selection operation of a scene item adding control in a metaverse platform interface, obtaining an item image information sequence.

[0023] In some embodiments, in response to detecting a selection operation on a scene item addition control in the Metaverse platform interface, an executing entity (e.g., a computing device) of the virtual reality technology-based Metaverse platform interaction method may obtain a sequence of item image information from a preset blockchain network via a wired or wireless connection. Each item image in the sequence may correspond one-to-one to a target item in a preset target item sequence. The target items in the target item sequence may be pre-set real-world items for which three-dimensional models are to be generated. Each item image in the sequence may include a multi-perspective item image set and a multi-perspective depth atlas. The multi-perspective item image set may be a collection of RGB images of the target item captured from different perspectives using an RGB (Red, Green, Blue) camera. The multi-perspective depth atlas may be a collection of depth images of the target item captured from different perspectives using a depth camera. The scene item addition control may be a UI (User Interface) control for adding virtual items to a custom virtual scene. For example, the scene item addition control may be a button control, and the corresponding selection operation may be a click operation. The aforementioned customized virtual scenes can include virtual items that can be created by VR (Virtual Reality) users through the Metaverse platform. VR users can be those who log in to the Metaverse platform using VR devices. Virtual items can be objects corresponding to real-world objects, generated using three-dimensional modeling technology. The aforementioned blockchain network can be used to permanently store all user interaction data and all digital asset generation, modification, and transaction data occurring on the Metaverse platform. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G, WiFi, Bluetooth, WiMAX, Zigbee, UWB (ultra wideband), and other currently known or future developed wireless connection methods.

[0024] Step 102: For each target item in the preset target item sequence, perform the following steps:

[0025] Step 1021: construct a three-dimensional object model based on the object image information corresponding to the target object.

[0026] In some embodiments, the execution entity may construct a three-dimensional object model based on the object image information corresponding to the target object in various ways, wherein the three-dimensional object model may be a three-dimensional model of the target object.

[0027] In practice, the conventional approach to constructing 3D object models is to directly predict the object's 3D structure from a single image or multiple images by training a neural network. However, this conventional approach still presents the following technical problem: because the performance of a neural network is limited by the quantity and quality of training data, insufficient or biased training data can easily lead to inaccurate predictions of the object's 3D structure. Therefore, to address this technical problem, and in light of the solution development team's technical advantages in the field of computer vision, this disclosure has decided to adopt the following solution.

[0028] In some optional implementations of some embodiments, the execution entity may construct a three-dimensional object model based on the object image information corresponding to the target object through the following steps:

[0029] In the first step, instance segmentation is performed on each of the multi-view item images corresponding to the target item to obtain an instance segmentation result set. The instance segmentation results in the instance segmentation result set may correspond one-to-one with the multi-view item images in each of the multi-view item images. The instance segmentation results in the instance segmentation result set may be the result of classifying each pixel in the multi-view item image.

[0030] As an example, the execution entity may perform instance segmentation on each multi-view object image corresponding to the target object using a preset instance segmentation model to obtain a set of instance segmentation results. The instance segmentation model may be a SegFormer semantic segmentation model. It should be noted that the instance segmentation results are stored as instance masks, with the mask region for each object recording the category of the corresponding pixel.

[0031] In the second step, in response to receiving instance selection information from the user for any instance segmentation result, each multi-view depth map corresponding to the target object is completed to obtain a completed depth map. The instance selection information may include information about the pixel region corresponding to the instance selected by the user. For example, the instance selection information may include, but is not limited to, an instance identifier and pixel coordinates. The instance identifier may be the identifier of the object serving as the instance. The completed depth map in each completed depth map may be a multi-view depth map completed with depth values ​​from a sparse region.

[0032] As an example, the execution entity may use a preset depth map completion method to complete the multi-view depth maps corresponding to the target object to obtain completed depth maps. The depth map completion method may be a Deep Completion Network method.

[0033] The third step is to fuse the above-mentioned completed depth maps to obtain global depth information, which can be a TSDF (Truncated Signed Distance Function) voxel volume.

[0034] As an example, the execution entity may perform fusion processing on the above-mentioned completed depth maps by using the TSDF method to obtain global depth information.

[0035] In the fourth step, feature extraction is performed on each multi-view object image corresponding to the target object to obtain a keypoint information set. Each keypoint in the keypoint information set can correspond one-to-one with a keypoint in the multi-view object image. The keypoint information in the keypoint information set can include the coordinates and feature descriptors of the corresponding keypoint. Keypoints can be corner points, edge points, or points in areas with noticeable texture in the image.

[0036] As an example, the execution entity may perform feature extraction processing on each multi-view object image corresponding to the target object using a SIFT (Scale-Invariant Feature Transform) algorithm to obtain a key point information set.

[0037] In the fifth step, the key points corresponding to the obtained key point information set are matched to obtain a key point pair set. The key point pair in the key point pair set can be the initial set of two key points with the smallest distance value on different multi-view object images.

[0038] As an example, the aforementioned execution entity can use the Euclidean distance method to match the key points corresponding to the obtained key point information set to obtain a set of key point pairs. It should be noted that a depth consistency constraint is also introduced in the feature point matching, retaining only matching pairs with a depth difference less than a set threshold, thereby reducing mismatches.

[0039] In the sixth step, based on the global depth information and the key point pair set, a registration point cloud dataset is generated. The registration point cloud dataset may be a point cloud dataset after point cloud alignment.

[0040] As an example, the above-mentioned execution entity can convert each key point information into a point cloud dataset based on the global depth information and the above-mentioned key point pair set, and align the point cloud data in the point cloud dataset through the ICP (Iterative Closest Point) algorithm to obtain a registered point cloud dataset.

[0041] Step 7: Reconstruct a 3D mesh based on the registered point cloud dataset. The 3D mesh can be a continuous surface composed of triangles or polygons.

[0042] As an example, the execution entity may reconstruct a three-dimensional mesh based on the registered point cloud dataset by using a Poisson surface reconstruction algorithm.

[0043] In the eighth step, based on the multi-view object images corresponding to the target object, texture mapping is performed on the three-dimensional grid to obtain a three-dimensional object model. The three-dimensional object model can be a three-dimensional model constructed based on the target object.

[0044] As an example, the execution entity may map the input multi-view object image onto the three-dimensional grid, perform texture mapping, and obtain a three-dimensional object model.

[0045] The above-mentioned 3D object model generation step, as an inventive feature of an embodiment of the present disclosure, addresses the second technical problem mentioned above, namely, the inaccurate predicted 3D structure of the object. Conventional solutions often suffer from inaccurate predicted 3D structures because the performance of neural networks is limited by the quantity and quality of training data. Insufficient or biased training data can easily lead to inaccurate predicted 3D structures. This solution, by addressing this issue, can improve the accuracy of the predicted 3D structure of the object. To achieve this, after each acquisition of a target object image, instance segmentation is first performed on each multi-view object image corresponding to the target object to obtain a set of instance segmentation results. This allows pixel-level classification of the object image, enabling subsequent extraction of segmented regions of the object of interest. Secondly, in response to receiving instance selection information from the user for any instance segmentation result, the multi-view depth maps corresponding to the target object are complemented to obtain complemented depth maps, and the complemented depth maps are fused to obtain global depth information. This generates globally consistent depth information. Finally, feature extraction is performed on each multi-view object image corresponding to the target object to obtain a set of keypoint information. Each key point in the key point information set corresponds to a key point in the multi-view object image. Matching is performed on each key point corresponding to the obtained key point information set to obtain a key point pair set. This allows for the identification of key points that correspond between different viewpoints, facilitating the subsequent alignment of the relative positions of the images from different viewpoints in three-dimensional space. Subsequently, a registration point cloud dataset is generated based on the global depth information and the key point pair set. Based on the image feature point matching and incorporating the global depth information, each key point is converted into point cloud data. Next, a three-dimensional mesh is reconstructed based on the registration point cloud dataset. This yields a three-dimensional mesh structure corresponding to the target object. Finally, texture mapping is performed on the three-dimensional mesh based on the multi-view object images corresponding to the target object to obtain a three-dimensional object model. This allows for the mapping of each pixel on the multi-view object to each vertex in the three-dimensional mesh. This results in a highly accurate three-dimensional object model, reduces the discrepancy between the virtual object corresponding to the three-dimensional object model and the actual object, and further enables users to more accurately perceive the virtual object. This, in turn, enhances the user interaction experience.

[0046] Step 1022: Generate customized virtual item information based on the three-dimensional item model.

[0047] In some embodiments, the execution entity may generate custom virtual item information based on the three-dimensional item model. The custom virtual item information may correspond one-to-one to the custom virtual item. The custom virtual item information may be information about a custom virtual item created by a VR user. For example, the custom virtual item information may include, but is not limited to, at least one of the following: an item identifier and virtual item model data. The item identifier may be a unique identifier for the target item or a unique identifier for the custom virtual item. The virtual item model data may be data for a three-dimensional model of the virtual item. It should be noted that the corresponding target item and the custom virtual item have the same identifier.

[0048] As an example, the execution entity may also determine the item identifier corresponding to the instance selection information and the three-dimensional item model as customized item information.

[0049] It should be supplemented that, in the process of adopting technical solutions to solve the technical problems 1 and 2 mentioned in the above background technology section, the following technical problem often arises: how to generate a multi-perspective item image set including item image information. In response to the above technical problems, conventional solutions are generally: taking images of items from different perspectives according to various pre-set perspectives. However, the above conventional solutions still have the following technical problem 3: since the image shooting is easily affected by external conditions such as lighting, the quality of the images of each item taken is uneven, which further makes it easy for the created virtual items to differ from the actual items, distorting the user's perception, and thus resulting in a poor user interaction experience on the Metaverse platform. Therefore, in the face of the above technical problem 3, combined with the technical advantages of the solution development team itself in the field of computer vision, this disclosure decides to adopt the following solution.

[0050] Optionally, before obtaining the item image information sequence, the execution entity may further perform the following steps of generating a multi-view item image set for each target item:

[0051] The first step is to obtain an object image corresponding to the target object and the current camera pose information. The object image can be an RGB image. The current camera pose information can be the pose information of the RGB camera when the object image was captured.

[0052] As an example, the execution entity may obtain an object image corresponding to a recently captured target object from an RGB camera on a VR device, and obtain current camera pose information from an inertial sensor on the VR device.

[0053] The second step is to perform quality inspection on the above-mentioned object image to obtain a quality inspection result, wherein the above-mentioned quality inspection result can indicate whether the object image is clear.

[0054] As an example, the execution entity may first determine the grayscale variance value corresponding to the item image using a grayscale variance detection algorithm. Then, in response to determining that the grayscale variance value is less than a preset variance threshold, a preset image unclear flag is determined as the quality inspection result. The preset variance threshold may be a preset upper limit of the grayscale variance value. The image unclear flag may indicate that the corresponding item image is unclear. Finally, in response to determining that the grayscale variance value is not less than the preset variance threshold, a preset image clear flag is determined as the quality inspection result. The image clear flag may indicate that the corresponding item image is relatively clear.

[0055] In a third step, in response to determining that the quality inspection result satisfies a preset inspection result condition, the viewing angle coverage of the target item is determined based on the historical item image sequence corresponding to the target item. The historical item image sequence may be an ordered collection of item images captured from different viewing angles of the target item. The preset inspection result condition may be that the quality inspection result indicates that the item image is relatively clear. The viewing angle coverage may be the degree to which the area where the target item is located is covered by the captured item images from various viewing angles. It should be noted that the historical item image sequence includes the acquired item images.

[0056] As an example, the execution entity may determine the view coverage of the target object based on the historical item image sequence corresponding to the target object through a view coverage evaluation interface. The view coverage evaluation interface may encapsulate a view coverage evaluation function. The view coverage evaluation function may be used, using the Unity engine, to determine the mask of the portion of the target object's area that is captured and the mask of the portion of the area that is not captured, and to estimate the view coverage of the target object based on the historical item image sequence corresponding to the target object.

[0057] In step 4, in response to determining that the viewing angle coverage is greater than or equal to a preset viewing angle coverage threshold, the historical item image sequence corresponding to the target item is determined as a multi-view item image set. The preset viewing angle coverage threshold may be a lower limit of the viewing angle coverage.

[0058] Optionally, the above execution entity may further perform the following steps:

[0059] Step 1: In response to determining that the viewing angle coverage is less than the preset viewing angle coverage threshold, determine uncovered area information, wherein the uncovered area information may be information of a mask of a portion of the area where the target object is located that is not captured.

[0060] As an example, the execution entity may obtain, from the viewing angle coverage evaluation interface, a mask of a partial area where the target object is located but not captured as the uncovered area information.

[0061] Step 2: Generate optimized shooting path information based on the current camera pose information and the uncovered area information. The optimized shooting path information may correspond one-to-one to the shooting path. The shooting path may be a path for the VR user to travel along to photograph the target object. The optimized shooting path information may be information about a sequence of path point coordinates corresponding to the shooting path. The path point coordinate sequence may be a sequence of coordinates of points along the VR user's path.

[0062] As an example, the execution entity may first use the camera pose of the RGB camera as the starting point of the path. Then, the edge point of the region that meets the preset edge position condition is selected from the edge points of the region corresponding to the uncovered area information as the end point of the path. The preset edge position condition may be that the distance value between the edge point of the region and the starting point of the path is the minimum value among the distance values. The distance value among the distance values ​​may be the straight-line distance between the edge point of the partial area where the target object is located that is not captured and the starting point of the path. Finally, the optimized shooting path information is generated based on the starting point of the path and the end point of the path through a preset path planning algorithm. The path planning algorithm may be an A* algorithm.

[0063] In practice, in simple scenarios (where the number of objects in the image is less than a certain value), the aforementioned execution entity can use a greedy algorithm to quickly guide the user to shoot; in complex scenarios, it can be combined with the A* algorithm for precise optimization to ensure the shortest path and full coverage.

[0064] Step 3: Rendering the shooting path corresponding to the optimized shooting path information into a 3D shooting scene to guide the user to take another image of the target object, and then executing the step of generating the multi-view object image set again. The 3D shooting scene may be the 3D scene displayed on the VR device screen when photographing the target object.

[0065] As an example, the execution entity may render the shooting path corresponding to the optimized shooting path information into a three-dimensional shooting scene using augmented reality technology. A VR user may walk along the shooting path and capture images of each target object.

[0066] The multi-perspective item image set generation step, as an inventive feature of the embodiments of the present disclosure, addresses the aforementioned technical issue #3: "Poor user interaction experience on the Metaverse platform." Conventional solutions suffer from a poor user interaction experience on the Metaverse platform because image capture is easily affected by external conditions such as lighting, resulting in varying quality of each captured item image. This further easily causes the created virtual item to differ from the actual item, leading to distorted user perception. By addressing this issue, the present solution can improve the user interaction experience on the Metaverse platform. To achieve this, each time an item image of a target item is acquired, the image quality is first checked. Then, if the image quality is satisfactory, the view coverage of the target item is determined based on the acquired historical item image sequences corresponding to each view. This allows determination of whether the target item is fully covered by each item image. If the target item is fully covered by each item image, the corresponding historical item image sequence can be included in the multi-perspective item image set. If the target item is not fully covered by each item image, a shooting path is planned for the user in real time based on the current camera pose information and uncovered area information. Finally, the shooting path corresponding to the optimized shooting path information is rendered into the 3D shooting scene, guiding the user to retake an image of the target object and executing the multi-view object image generation step again. This generates high-quality object images from different perspectives, reducing the discrepancy between the created virtual object and the actual object, further enabling the user to have a more accurate perception of the virtual object. This improves the user interaction experience on the Metaverse platform.

[0067] Step 103 , based on the preset initial display position information, rendering each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene, and playing preset item position adjustment prompt information to prompt the user to reposition the custom virtual item.

[0068] In some embodiments, the execution entity may render the custom virtual items corresponding to the obtained custom virtual item information into a custom virtual scene in various ways based on the preset initial display position information, and play the preset item position adjustment prompt information to prompt the user to reposition the custom virtual items. The initial display position information may include the initial center point horizontal coordinate and the initial center point vertical coordinate. The initial center point horizontal coordinate may be the horizontal coordinate of the center point of the virtual item display area. The initial center point horizontal coordinate may be the vertical coordinate of the center point of the virtual item display area. The virtual item display area may be a rectangular area for displaying virtual items. The preset item position adjustment prompt information may be a pre-set voice text for prompting the user to adjust the item position. For example, the preset item position adjustment prompt information may be "Please select the item to be moved and move it to the appropriate position."

[0069] In some optional implementations of some embodiments, the execution entity may render each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene based on the preset initial display position information through the following steps:

[0070] In the first step, for each piece of custom virtual item information, perform the following steps:

[0071] Step 1: Determine the serial number corresponding to the above-mentioned customized virtual item information as the target serial number.

[0072] Step 2: Generate position translation information based on the target number and a preset step size. The preset step size may be the preset pixel distance between the center points of two custom virtual item display areas. The position translation information may be the pixel distance required for the horizontal translation. First, the difference between the target number and 1 is determined as the translation factor. Then, the product of the translation factor and the preset step size is determined as the position translation information.

[0073] Step 3: Determine the target display location information based on the position translation information and the initial display location information. The target display location information may be information about the location where the customized virtual item is to be displayed. First, the target location abscissa is determined as the sum of the initial center point's horizontal coordinate included in the initial display location information and the pixel distance corresponding to the position translation information. Then, the initial center point's vertical coordinate included in the initial display location information is used as the target location's vertical coordinate, and the target location's horizontal coordinate and vertical coordinate are determined as the target display location information.

[0074] Step 4: Based on the target display position information, the custom virtual item corresponding to the custom virtual item information is rendered into the custom virtual scene.

[0075] As an example, the execution entity may render the custom virtual item corresponding to the custom virtual item information into a virtual item display area in the custom virtual scene with the coordinates corresponding to the target display position information as the center point.

[0076] In practice, the aforementioned execution entity can play the preset object position adjustment prompt information through the VR device's built-in voice player. Furthermore, the aforementioned custom virtual scene, including the basic virtual environment, such as a room or an outdoor environment, can be pre-built using Blender software and the Unity engine. VR users can then add virtual objects to this pre-built basic virtual environment.

[0077] Step 104 : In response to detecting that any customized virtual item among the customized virtual items intersects with a preset ray, performing position detection on the customized virtual item to obtain a position detection result.

[0078] In some embodiments, the execution entity may, in response to detecting that any of the custom virtual items intersects a preset ray, perform position detection on the custom virtual item in various ways to obtain a position detection result. The preset ray may be a ray emitted by a target virtual hand and used to select the custom virtual item to be relocated. The target virtual hand may be the virtual hand of a three-dimensional virtual user. The three-dimensional virtual user may be a digital human corresponding to the VR user. The position detection result may include information on whether the custom virtual item at the corresponding location can be grabbed.

[0079] In some optional implementations of some embodiments, the execution entity may perform position detection on any of the above-mentioned customized virtual items through the following steps to obtain a position detection result:

[0080] The first step is to obtain the layer information corresponding to any of the above-mentioned custom virtual items. The above-mentioned layer information may be information about the layer where the custom virtual item is located. For example, the above-mentioned layer information may include, but is not limited to, a layer identifier. The layer identifier may be a unique identifier for the layer.

[0081] As an example, the execution entity may obtain the attribution layer information corresponding to any of the above-mentioned customized virtual items from the blockchain network.

[0082] The second step is to match the preset layer information of the items to be captured with the aforementioned layer information to obtain a layer matching result. The preset layer information of the items to be captured may be information indicating the layer where the custom virtual items to be captured are located. The layer matching result may indicate whether the layer where any custom virtual items intersecting the preset ray are located is the preset layer where the custom virtual items to be captured are located.

[0083] As an example, in response to determining that the preset layer information of the item to be captured is the same as the above-mentioned attribution layer information, the above-mentioned execution entity may determine a preset layer match success flag as a layer match result. The above-mentioned preset layer match success flag may indicate that the layer where any custom virtual item intersecting with the above-mentioned preset ray is located is the layer where the preset custom virtual item to be captured is located. Then, in response to determining that the preset layer information of the item to be captured is different from the above-mentioned attribution layer information, the above-mentioned preset layer match failure flag may be determined as a layer match result. The above-mentioned preset layer match failure flag may indicate that the layer where any custom virtual item intersecting with the above-mentioned preset ray is located is not the layer where the preset custom virtual item to be captured is located.

[0084] In a third step, in response to determining that the layer matching result satisfies a preset matching condition, the intersection distance corresponding to the arbitrary custom virtual item is determined. The preset matching condition may be that the layer matching result indicates that the layer where the arbitrary custom virtual item intersects with the preset ray is located is the layer where the preset custom virtual item to be captured is located. The intersection distance is the distance between the intersection point between the arbitrary custom virtual item and the preset ray and the virtual hand emitting the ray.

[0085] As an example, the execution entity may determine the distance between the center point of the virtual hand and the intersection point as the intersection distance corresponding to any custom virtual item.

[0086] In step 4, in response to determining that the intersection distance satisfies a preset distance condition, a preset grabbable indicator is determined as the position detection result. The preset distance condition may be that the intersection distance is the minimum of the intersection distances corresponding to the virtual items intersecting the preset ray. The preset grabbable indicator may indicate that the custom virtual item at the corresponding location is grabbable.

[0087] Step 105 : In response to determining that the position detection result satisfies the preset grabbable condition, any customized virtual item is updated to obtain an updated virtual item.

[0088] In some embodiments, the execution entity may, in response to determining that the position detection result satisfies a preset grabbability condition, update any of the custom virtual items to obtain an updated virtual item. The preset grabbability condition may be that the position detection result indicates that the custom virtual item at the corresponding location is grabbable. The updated virtual item may be a custom virtual item with a luminous effect.

[0089] As an example, the execution entity may set the self-luminous property of any of the custom virtual items to be on, so as to update the any of the custom virtual items and obtain an updated virtual item.

[0090] Step 106 : In response to receiving the confirmation grabbing instruction for the updated virtual item, controlling the target virtual hand to perform a grabbing operation on the updated virtual item, so as to place the updated virtual item at a target location.

[0091] In some embodiments, the execution subject may, in response to receiving a confirmation grab instruction for the updated virtual item, control the target virtual hand to perform a grab operation on the updated virtual item, thereby placing the updated virtual item at a target location. The confirmation grab instruction may be an instruction for instructing the grab operation to be performed on the updated virtual item. The target location may be a location in the virtual scene where the VR user desires to place the updated virtual item.

[0092] As an example, the above-mentioned execution entity can receive a confirmation grabbing instruction sent by the VR user through the trigger button on the handle, disable the physical properties of the above-mentioned updated virtual object such as gravity and collision, and set the above-mentioned updated virtual object as a child object of the above-mentioned target virtual hand. Through inverse kinematics technology, the target virtual hand adjusts its position according to the shape of the object and performs a grabbing action. The position of the virtual object will be updated in real time following the above-mentioned target virtual hand.

[0093] Optionally, the above execution entity may further perform the following steps:

[0094] In the first step, in response to receiving user interaction data, the target virtual hand is controlled to move the updated virtual item to the target location. The user interaction data may include, but is not limited to, user posture data and gesture interaction data. The user posture data may be data on the VR user's position and posture captured by an inertial sensor. The gesture interaction data may be images of the VR user's hand movements captured by a camera.

[0095] As an example, after the above-mentioned execution entity receives the user interaction data, it can use the VNect posture estimation algorithm to bind the posture and action corresponding to the above-mentioned user interaction data to the skeleton of the three-dimensional virtual user, and recognize the continuous posture action data and gesture interaction data to obtain coherent human body skeleton posture data, and render it in real time in the virtual environment.

[0096] In a second step, in response to receiving a confirmation placement instruction for the updated virtual item, the target virtual hand is controlled to release the updated virtual item to place the updated virtual item at the target location. The confirmation placement instruction may be an instruction for releasing the grip of the updated virtual item.

[0097] For example, the execution entity receives a confirmation placement instruction for the updated virtual item, activates the physical properties of the updated virtual item, changes the parent object corresponding to the updated virtual item from the target virtual hand to an empty object, and synchronously renders the user's release action to the target virtual hand to perform the release operation. As a result, the updated virtual item can be placed at the target location.

[0098] Optionally, the Metaverse platform interface further includes a virtual scene selection control. The virtual scene selection control may be a button control for selecting a virtual scene. The execution subject may further perform the following steps:

[0099] The first step is to obtain a virtual scene list information set in response to detecting a selection operation on a virtual scene selection control in the metaverse platform interface. Each virtual scene list information in the virtual scene list information set may correspond one-to-one to a virtual scene in a preset virtual scene set. The virtual scene selection control may be a button control for selecting a virtual scene. The virtual scene list information in the virtual scene list information set may include a virtual scene identifier and a scene link. The virtual scene identifier may be a unique identifier for the virtual scene. The scene link may be a network address for loading the corresponding virtual scene.

[0100] As an example, the above-mentioned execution entity can obtain a pre-stored virtual scene list information set from the blockchain network.

[0101] In the second step, each virtual scene list information in the virtual scene list information set is filled into a preset virtual scene list display panel, so that the user can select a virtual scene to enter. The preset virtual scene list display panel can be used to display the identification of each virtual scene.

[0102] As an example, the execution entity may fill each virtual scene list information in the virtual scene list information set into a preset virtual scene list display panel in the form of a list element. The user may click a list item to enter the virtual scene corresponding to the list item.

[0103] The third step is to load the virtual scene corresponding to any virtual scene list information in response to detecting a selection operation acting on any virtual scene list information.

[0104] As an example, the execution entity may load the corresponding virtual scene according to the scene link included in any selected virtual scene list information.

[0105] Optionally, the above execution entity may further perform the following steps:

[0106] In the first step, in response to determining that the virtual scene corresponding to any of the virtual scene list information satisfies a preset scene type condition, preset item transfer operation prompt information is played, allowing the user to select a virtual item to be transferred. The preset scene type condition may be that the virtual scene is a virtual store type. The virtual store type may indicate that the corresponding virtual scene is a virtual store. A virtual store may be a store constructed using virtual reality technology that displays and transfers virtual items. The preset item transfer operation prompt information may be a pre-set voice text that prompts the user on how to perform an item transfer operation. The virtual item to be transferred may be a virtual item that can be transferred.

[0107] In practice, the above-mentioned execution entity can play the above-mentioned preset item transfer operation prompt information through the voice player built into the VR device.

[0108] In the second step, in response to detecting a selection operation on any virtual item to be transferred, item details corresponding to the virtual item to be transferred are obtained. The virtual item to be transferred may be a virtual item displayed in the virtual scene corresponding to the virtual scene list information. The item details may be descriptive information of the virtual item to be transferred obtained from the blockchain network. The item details may include, but are not limited to, virtual item identification, item source information, functional information, and value attributes (e.g., price). The item source information may be information about the item owner. When the item source information is 0, the virtual item is owned by the virtual store. When the item source information is XXX (e.g., Wang XX), the virtual item is owned by XXX.

[0109] The third step is to determine the ownership type of the virtual item based on the item details information, wherein the ownership type of the virtual item can indicate whether the virtual item is owned by the virtual store.

[0110] As an example, the execution entity may determine the first ownership type as the virtual item ownership type in response to determining that the item source information included in the item details is 0, and determine the second ownership type as the virtual item ownership type in response to determining that the item source information included in the item details is not 0. The first ownership type may indicate that the virtual item is owned by the virtual store. The second ownership type may indicate that the virtual item is owned by an individual.

[0111] In a fourth step, in response to determining that the virtual item's ownership type satisfies a preset ownership type condition, the item details and a confirm transfer button are displayed in a preset scene interaction panel. The preset ownership type condition may be that the virtual item's ownership type indicates that the corresponding virtual item is owned by the virtual store. The scene interaction panel may be an interface for user interaction with the Metaverse platform system. The confirm transfer button may be a button for the user to confirm whether to execute the transfer operation for the selected virtual item to be transferred.

[0112] In step 5, in response to detecting a selection operation on the confirm transfer button control, various form item controls are displayed in the scene interaction panel to receive value transfer information entered by the user. The various form item controls may include, but are not limited to, input box controls, single-select controls, and multiple-select controls. The value transfer information may be information about the value required to perform the item transfer operation.

[0113] Step 6: Upon receiving the value transfer information corresponding to any of the aforementioned virtual items to be transferred, the value transfer information is transmitted to the pre-defined blockchain network for execution of the virtual item transfer operation. It should be noted that within the blockchain network, the smart contract initiates conditional verification of the virtual item transfer operation. Upon successful verification, ownership is transferred, the transfer record is updated, and both parties involved in the transfer are notified.

[0114] Optionally, the above execution entity may further perform the following steps:

[0115] In the first step, in response to determining that the virtual item ownership type does not meet the preset ownership type condition, the item details and a session creation control are displayed in the scene interaction panel. The session creation control may be a button control that allows a session to be established between two virtual users.

[0116] In the second step, in response to detecting a selection operation on the aforementioned session establishment control, a session window is displayed for communicating item transfer information between virtual users. The session window may include an input box control, a session display panel, an item exchange control, an item gift control, and an item value exchange control. The input box control allows virtual users to input information. The session display panel displays the content of the session between two virtual users. The item exchange control can be used to initiate a virtual item exchange request. The item gift control can be used to initiate an instruction to gift a virtual item. The item value exchange control can be used to input value transfer information, and its function is equivalent to the aforementioned confirm transfer button control.

[0117] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the metaverse platform interaction method based on virtual reality technology of some embodiments of the present disclosure, the virtual scene update speed can be increased, and the user interaction experience can be improved. Specifically, the reason for the slow virtual scene update speed and the poor user interaction experience is that the virtual items presented by the virtual environment are usually integrated with the basic virtual scene, rendered and transmitted to the virtual reality device, making the virtual scene relatively fixed, and the user can only interact with a few inherent virtual items in the virtual scene of the metaverse platform. If you want to add virtual items, you need to integrate the new virtual items into the virtual scene, and then render and transmit them to the virtual reality device to update the previously displayed virtual scene, thereby resulting in a slow virtual scene update speed and a poor user interaction experience. Based on this, the metaverse platform interaction method based on virtual reality technology of some embodiments of the present disclosure first obtains an item image information sequence in response to detecting a selection operation of a scene item addition control in the metaverse platform interface. Each item image information in the item image information sequence corresponds to a target item in a preset target item sequence. Each item image information in the item image information sequence includes a multi-view item image set and a multi-view depth atlas. Therefore, when a user wishes to add a virtual item to a virtual scene, they can first obtain the item image information required to create each custom virtual item. Next, for each target item in the preset target item sequence, the following steps are performed: constructing a three-dimensional item model based on the item image information corresponding to the target item; and generating custom virtual item information based on the three-dimensional item model. The custom virtual item information corresponds to the custom virtual item. Thus, each custom virtual item can be created for subsequent placement in the virtual scene. Then, based on preset initial display position information, each custom virtual item corresponding to each obtained custom virtual item information is rendered into the custom virtual scene, and a preset item position adjustment prompt is played to prompt the user to reposition the custom virtual item. This allows the user to see each custom virtual item to be added in the custom virtual scene. Afterwards, in response to detecting that any of the aforementioned custom virtual items intersects a preset ray, a position detection is performed on the aforementioned custom virtual item to obtain a position detection result. The preset ray is a ray emitted by a target virtual hand, which is used to select the custom virtual item to be relocated. The target virtual hand is the virtual hand of the three-dimensional virtual user. Thus, the custom virtual item to be moved can be selected by emitting a ray. Next, in response to determining that the position detection result satisfies a preset graspability condition, the aforementioned custom virtual item is updated to obtain an updated virtual item. This allows the determination of the custom virtual item available for the user to grasp.Finally, in response to receiving a confirmation grabbing instruction for the updated virtual item, the target virtual hand is controlled to perform a grabbing operation on the updated virtual item, thereby placing the updated virtual item at the target location. Therefore, the virtual reality technology-based metaverse platform interaction method of some embodiments of the present disclosure facilitates real-time updates of the virtual scene by generating customized virtual items within the metaverse platform and, based on this, guiding the user to perform interactive operations such as grabbing and moving the virtual items. This can thereby increase the update speed of the virtual scene and, in turn, enhance the user's interactive experience.

[0118] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a metaverse platform interaction device based on virtual reality technology. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the metaverse platform interaction device 200 based on virtual reality technology can be specifically applied to various electronic devices.

[0119] like Figure 2As shown, some embodiments of the metaverse platform interaction device 200 based on virtual reality technology include: an acquisition unit 201, an execution unit 202, a rendering and playback unit 203, a position detection unit 204, an updating unit 205, and a control unit 206. The acquisition unit 201 is configured to, in response to detecting a selection operation of a scene item addition control in the metaverse platform interface, acquire an item image information sequence, wherein each item image information in the item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; the execution unit 202 is configured to, for each target item in the preset target item sequence, perform the following steps: construct a three-dimensional item model based on the item image information corresponding to the target item; generate custom virtual item information based on the three-dimensional item model, wherein the custom virtual item information corresponds to the custom virtual item; the rendering and playback unit 203 is configured to render each custom virtual item corresponding to each custom virtual item information obtained to the custom virtual object based on the preset initial display position information. The scene is configured to play a preset item position adjustment prompt message to prompt the user to re-place the custom virtual item; the position detection unit 204 is configured to, in response to detecting that any of the above-mentioned custom virtual items intersects with a preset ray, perform position detection on the above-mentioned any custom virtual item to obtain a position detection result, wherein the above-mentioned preset ray is a ray emitted by the target virtual hand and used to select the custom virtual item to be re-placed, and the target virtual hand is the virtual hand of the three-dimensional virtual user; the updating unit 205 is configured to, in response to determining that the above-mentioned position detection result meets the preset graspable condition, update the above-mentioned any custom virtual item to obtain an updated virtual item; the control unit 206 is configured to, in response to receiving a confirmation grasping instruction for the above-mentioned updated virtual item, control the target virtual hand to perform a grasping operation on the above-mentioned updated virtual item to place the above-mentioned updated virtual item at the target position.

[0120] It is understandable that the units recorded in the virtual reality technology-based metaverse platform interactive device 200 and the reference Figure 1 Therefore, the operations, features, and beneficial effects described above for the method are also applicable to the metaverse platform interaction device 200 based on virtual reality technology and the units contained therein, and will not be repeated here.

[0121] Further references Figure 3 , which shows a structural diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0122] like Figure 3 As shown, electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 302 or programs loaded from a storage device 308 into a random access memory (RAM) 303. RAM 303 also stores various programs and data required for the operation of electronic device 300. Processing device 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to bus 304.

[0123] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0124] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0125] It should be noted that in some embodiments of the present disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. Furthermore, in some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wire, optical cable, RF (radio frequency), or any suitable combination thereof.

[0126] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0127] The above-mentioned computer-readable medium may be included in the above-mentioned device; or it may exist independently and not be assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to detecting a selection operation of the scene item adding control acting on the metaverse platform interface, obtains an item image information sequence, wherein each item image information in the above-mentioned item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the above-mentioned item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; for each target item in the preset target item sequence, performs the following steps: based on the item image information corresponding to the above-mentioned target item, constructs a three-dimensional item model; based on the above-mentioned three-dimensional item model, generates custom virtual item information, wherein the above-mentioned custom virtual item information corresponds to the custom virtual item; based on the preset initial display position information, displays each custom virtual item obtained Each custom virtual item corresponding to the information is rendered into the custom virtual scene, and a preset item position adjustment prompt information is played to prompt the user to replace the custom virtual item; in response to detecting that any custom virtual item among the above-mentioned custom virtual items intersects with a preset ray, a position detection is performed on the above-mentioned any custom virtual item to obtain a position detection result, wherein the above-mentioned preset ray is a ray emitted by a target virtual hand and is used to select the custom virtual item to be replaced, and the target virtual hand is the virtual hand of a three-dimensional virtual user; in response to determining that the above-mentioned position detection result meets a preset graspable condition, the above-mentioned any custom virtual item is updated to obtain an updated virtual item; in response to receiving a confirmation grasping instruction for the above-mentioned updated virtual item, the target virtual hand is controlled to perform a grasping operation on the above-mentioned updated virtual item to place the above-mentioned updated virtual item at a target position.

[0128] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] The units described in some embodiments of the present disclosure may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as follows: a processor comprising: an acquisition unit, an execution unit, a rendering and playback unit, a position detection unit, an update unit, and a control unit. The names of these units do not, in some cases, limit the units themselves. For example, the acquisition unit may also be described as a "unit for acquiring an object image information sequence."

[0131] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0132] The above descriptions are merely some preferred embodiments of the present disclosure and illustrate the underlying technical principles. Those skilled in the art should understand that the scope of the invention encompassed by the embodiments of the present disclosure is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned inventive concept. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A metaverse platform interaction method based on virtual reality technology, comprising: In response to detecting a selection operation of a scene item addition control in a metaverse platform interface, obtaining an item image information sequence, wherein each item image information in the item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; For each target item in the preset target item sequence, perform the following steps: constructing a three-dimensional object model based on the object image information corresponding to the target object; generating custom virtual item information based on the three-dimensional item model, wherein the custom virtual item information corresponds to the custom virtual item; Based on the preset initial display position information, rendering each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene, and playing the preset item position adjustment prompt information to prompt the user to reposition the custom virtual item; In response to detecting that any of the customized virtual items intersects a preset ray, performing position detection on the customized virtual item to obtain a position detection result, wherein the position detection result is information indicating whether the customized virtual item at the corresponding position can be grabbed, the preset ray being a ray emitted by a target virtual hand and used to select the customized virtual item to be relocated, and the target virtual hand being a virtual hand of a three-dimensional virtual user; In response to determining that the position detection result satisfies a preset grabbable condition, updating the arbitrary customized virtual item to obtain an updated virtual item; In response to receiving a confirmation grabbing instruction for the updated virtual item, controlling the target virtual hand to perform a grabbing operation on the updated virtual item so as to place the updated virtual item at a target location; wherein, constructing a three-dimensional object model based on the object image information corresponding to the target object includes: performing instance segmentation processing on each multi-view object image corresponding to the target object to obtain an instance segmentation result set; in response to receiving instance selection information of a user for any instance segmentation result, performing completion processing on each multi-view depth map corresponding to the target object to obtain each completed depth map; performing fusion processing on each completed depth map to obtain global depth information; performing feature extraction processing on each multi-view object image corresponding to the target object to obtain a key point information set; performing matching processing on each key point corresponding to the obtained key point information set to obtain a key point pair set, wherein the key point pair in the key point pair set is the initial set of two key points with the smallest distance value on different multi-view object images; generating a registration point cloud dataset based on the global depth information and the key point pair set; reconstructing a three-dimensional mesh based on the registration point cloud dataset; performing texture mapping processing on the three-dimensional mesh based on each multi-view object image corresponding to the target object to obtain a three-dimensional object model; The performing position detection on the arbitrary custom virtual item to obtain a position detection result includes: obtaining the belonging layer information corresponding to the arbitrary custom virtual item; matching the preset layer information of the item to be grabbed with the belonging layer information to obtain a layer matching result; in response to determining that the layer matching result satisfies a preset matching condition, determining the intersection distance corresponding to the arbitrary custom virtual item; in response to determining that the intersection distance satisfies a preset distance condition, determining a preset grabbable identifier as the position detection result.

2. The method for interacting with the metaverse platform based on virtual reality technology according to claim 1, wherein: The metaverse platform interface further includes a virtual scene selection control; and the method further includes: In response to detecting a selection operation on a virtual scene selection control in the metaverse platform interface, obtaining a virtual scene list information set, wherein each virtual scene list information in the virtual scene list information set corresponds to a virtual scene in a preset virtual scene set; Filling each virtual scene list information in the virtual scene list information set into a preset virtual scene list display panel, so that the user can select a virtual scene to enter; In response to detecting a selection operation acting on any virtual scene list information, a virtual scene corresponding to the any virtual scene list information is loaded.

3. The method for interacting with the metaverse platform based on virtual reality technology according to claim 2, wherein: The method further comprises: In response to determining that the virtual scene corresponding to the arbitrary virtual scene list information meets the preset scene type condition, playing preset item transfer operation prompt information for the user to select the virtual item to be transferred; In response to detecting a selection operation on any virtual item to be circulated, obtaining item detail information corresponding to the any virtual item to be circulated, wherein the any virtual item to be circulated is the virtual item to be circulated displayed in the virtual scene corresponding to the any virtual scene list information; Determining the type of the virtual item based on the item details information; In response to determining that the virtual item ownership type meets a preset ownership type condition, displaying the item details information and a confirmation transfer button control in a preset scene interaction panel; In response to detecting a selection operation on the confirm transfer button control, displaying various form item controls in the scene interaction panel to receive value transfer information input by the user; In response to receiving the value transfer information corresponding to any virtual item to be circulated, the value transfer information is sent to a preset blockchain network for executing a virtual item circulation operation.

4. The method for interacting with the metaverse platform based on virtual reality technology according to claim 3, wherein: The method further comprises: In response to determining that the virtual item ownership type does not meet the preset ownership type condition, displaying the item details information and establishing a session control in the scene interaction panel; In response to detecting a selection operation acting on the establishing session control, a session window is displayed for communicating item flow information between virtual users, wherein the session window includes an input box control, a session display panel, an item exchange control, an item gift control, and an item value exchange control.

5. The method for interacting with the metaverse platform based on virtual reality technology according to claim 1, wherein: The step of rendering each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene based on the preset initial display position information includes: For each custom virtual item information in each custom virtual item information, perform the following steps: Determining the serial number corresponding to the customized virtual item information as the target serial number; Generate position translation information based on the target sequence number and the preset step size; determining target display position information based on the position translation information and the initial display position information; Based on the target display position information, the custom virtual item corresponding to the custom virtual item information is rendered into a custom virtual scene.

6. The method for interacting with the metaverse platform based on virtual reality technology according to claim 1, wherein: The method further comprises: In response to receiving the user interaction data, controlling the target virtual hand to move the updated virtual object to the target position; In response to receiving a confirmation placement instruction for the updated virtual item, controlling the target virtual hand to perform a release operation on the updated virtual item to place the updated virtual item at the target location.

7. A metaverse platform interactive device based on virtual reality technology, comprising: an acquisition unit configured to, in response to detecting a selection operation of a scene item addition control in a metaverse platform interface, acquire an item image information sequence, wherein each item image information in the item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; The execution unit is configured to execute the following steps for each target item in a preset target item sequence: constructing a three-dimensional object model based on the object image information corresponding to the target object; generating custom virtual item information based on the three-dimensional item model, wherein the custom virtual item information corresponds to the custom virtual item; a rendering and playing unit configured to render each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene based on the preset initial display position information, and play preset item position adjustment prompt information to prompt the user to reposition the custom virtual item; a position detection unit configured to, in response to detecting that any of the customized virtual items intersects a preset ray, perform position detection on the any customized virtual item to obtain a position detection result, wherein the position detection result is information indicating whether the customized virtual item at the corresponding position can be grabbed, the preset ray being a ray emitted by a target virtual hand and used to select the customized virtual item to be relocated, the target virtual hand being a virtual hand of a three-dimensional virtual user; an updating unit configured to, in response to determining that the position detection result satisfies a preset grabbable condition, update the arbitrary customized virtual item to obtain an updated virtual item; a control unit configured to, in response to receiving a confirmation grabbing instruction for the updated virtual item, control the target virtual hand to perform a grabbing operation on the updated virtual item so as to place the updated virtual item at a target location; wherein, constructing a three-dimensional object model based on the object image information corresponding to the target object includes: performing instance segmentation processing on each multi-view object image corresponding to the target object to obtain an instance segmentation result set; in response to receiving instance selection information of a user for any instance segmentation result, performing completion processing on each multi-view depth map corresponding to the target object to obtain each completed depth map; performing fusion processing on each completed depth map to obtain global depth information; performing feature extraction processing on each multi-view object image corresponding to the target object to obtain a key point information set; performing matching processing on each key point corresponding to the obtained key point information set to obtain a key point pair set, wherein the key point pair in the key point pair set is the initial set of two key points with the smallest distance value on different multi-view object images; generating a registration point cloud dataset based on the global depth information and the key point pair set; reconstructing a three-dimensional mesh based on the registration point cloud dataset; performing texture mapping processing on the three-dimensional mesh based on each multi-view object image corresponding to the target object to obtain a three-dimensional object model; The performing position detection on the arbitrary custom virtual item to obtain a position detection result includes: obtaining the belonging layer information corresponding to the arbitrary custom virtual item; matching the preset layer information of the item to be grabbed with the belonging layer information to obtain a layer matching result; in response to determining that the layer matching result satisfies a preset matching condition, determining the intersection distance corresponding to the arbitrary custom virtual item; in response to determining that the intersection distance satisfies a preset distance condition, determining a preset grabbable identifier as the position detection result.

8. A device for interacting with a Metaverse platform, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

Images