Virtual reality technology-based meta-universe platform interaction method, apparatus and device

Abstract

The embodiment of the invention discloses a meta-universe platform interaction method, device and equipment based on a virtual reality technology. A specific embodiment of the method comprises the following steps: acquiring an article image information sequence; for each target article, generating custom virtual article information based on the article image information; on the basis of the initial display position information, rendering each self-defined virtual item corresponding to each piece of self-defined virtual item information into a self-defined virtual scene; in response to the detection that any user-defined virtual article intersects with a preset ray, performing position detection on any user-defined virtual article to obtain a position detection result; in response to determining that the position detection result meets a preset grabable condition, performing updating processing on any user-defined virtual article; and in response to a received grabbing confirmation instruction, controlling the target virtual hand to execute grabbing operation on the updated virtual article. According to the embodiment, the virtual scene updating speed can be increased, and the user interaction experience is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of virtual reality technology, and in particular to a metaverse platform interaction method, device, and equipment based on virtual reality technology. Background Art

[0002] The Metaverse platform is a three-dimensional scene virtual environment built using three-dimensional modeling, virtual reality and blockchain technology. Users can socialize, play games and transfer items in the three-dimensional scene presented by the Metaverse platform. However, since the virtual items presented by the virtual environment are usually integrated with the basic virtual scene, rendered and transmitted to the virtual reality device, the virtual scene is relatively fixed, and users can only interact with a few inherent virtual items in the virtual scene of the Metaverse platform. If you want to add virtual items, you need to integrate the virtual items to be added into the virtual scene, and then render and transmit them to the virtual reality device to update the previously displayed virtual scene, which results in a slow update speed of the virtual scene and a poor user interaction experience.

[0003] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0004] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0005] Some embodiments of the present disclosure propose a metaverse platform interaction method, apparatus, and device based on virtual reality technology to solve one or more of the technical problems mentioned in the above background technology section.

[0006] In a first aspect, some embodiments of the present disclosure provide a method for interacting with a metaverse platform based on virtual reality technology, the method comprising: in response to detecting a selection operation of a scene item adding control acting on a metaverse platform interface, obtaining an item image information sequence, wherein each item image information in the item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; for each target item in the preset target item sequence, performing the following steps: based on the item image information corresponding to the target item, constructing a three-dimensional item model; based on the three-dimensional item model, generating custom virtual item information, wherein the custom virtual item information corresponds to the custom virtual item; based on preset initial display position information, displaying each custom virtual item information to the target item; The method comprises the steps of: rendering each custom virtual item into a custom virtual scene, and playing a preset item position adjustment prompt information to prompt the user to re-place the custom virtual item; in response to detecting that any custom virtual item among the above-mentioned custom virtual items intersects with a preset ray, performing position detection on the above-mentioned any custom virtual item to obtain a position detection result, wherein the above-mentioned preset ray is a ray emitted by a target virtual hand and used to select the custom virtual item to be re-placed, and the target virtual hand is a virtual hand of a three-dimensional virtual user; in response to determining that the above-mentioned position detection result meets a preset graspable condition, updating the above-mentioned any custom virtual item to obtain an updated virtual item; in response to receiving a confirmation grasping instruction for the above-mentioned updated virtual item, controlling the target virtual hand to perform a grasping operation on the above-mentioned updated virtual item to place the above-mentioned updated virtual item at a target position.

[0007] In a second aspect, some embodiments of the present disclosure provide a metaverse platform interaction device based on virtual reality technology, the device comprising: an acquisition unit, configured to acquire an item image information sequence in response to detecting a selection operation of a scene item adding control acting on a metaverse platform interface, wherein each item image information in the above item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the above item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; an execution unit, configured to perform the following steps for each target item in the preset target item sequence: construct a three-dimensional item model based on the item image information corresponding to the above target item; generate custom virtual item information based on the above three-dimensional item model, wherein the above custom virtual item information corresponds to the custom virtual item; a rendering and playback unit, configured to display the obtained custom virtual item information based on preset initial display position information. The corresponding custom virtual items are rendered into the custom virtual scene, and the preset item position adjustment prompt information is played to prompt the user to re-place the custom virtual items; the position detection unit is configured to, in response to detecting that any custom virtual item among the above-mentioned custom virtual items intersects with the preset ray, perform position detection on the above-mentioned any custom virtual item to obtain a position detection result, wherein the above-mentioned preset ray is a ray emitted by the target virtual hand and used to select the custom virtual item to be re-placed, and the target virtual hand is the virtual hand of the three-dimensional virtual user; the update unit is configured to, in response to determining that the above-mentioned position detection result meets the preset graspable condition, perform update processing on the above-mentioned any custom virtual item to obtain an updated virtual item; the control unit is configured to, in response to receiving a confirmation grasping instruction for the above-mentioned updated virtual item, control the target virtual hand to perform a grasping operation on the above-mentioned updated virtual item to place the above-mentioned updated virtual item at the target position.

[0008] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0009] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0010] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the Metaverse platform interaction method based on virtual reality technology of some embodiments of the present disclosure, the virtual scene update speed can be improved, and the user interaction experience can be improved. Specifically, the reason for the slow virtual scene update speed and the poor user interaction experience is that the virtual objects presented by the virtual environment are usually integrated with the basic virtual scene, rendered and transmitted to the virtual reality device, so that the virtual scene is relatively fixed, and the user can only interact with a few inherent virtual objects in the virtual scene of the Metaverse platform. If you want to add virtual objects, you need to integrate the virtual objects to be added into the virtual scene, and then render and transmit them to the virtual reality device to update the previously displayed virtual scene, thereby resulting in a slow virtual scene update speed and a poor user interaction experience. Based on this, the Metaverse platform interaction method based on virtual reality technology of some embodiments of the present disclosure, first, in response to detecting a selection operation of adding a scene object control acting on the Metaverse platform interface, obtains an object image information sequence. Wherein, each item image information in the above-mentioned item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the above-mentioned item image information sequence includes a multi-view item image set and a multi-view depth atlas. Thus, when a user wants to add a virtual item to a virtual scene, the item image information required to create each custom virtual item can be obtained first. Secondly, for each target item in the preset target item sequence, the following steps are performed: based on the item image information corresponding to the above-mentioned target item, a three-dimensional item model is constructed; based on the above-mentioned three-dimensional item model, custom virtual item information is generated. Wherein, the above-mentioned custom virtual item information corresponds to the custom virtual item. Thus, each custom virtual item can be created so as to be subsequently placed in the virtual scene. Then, based on the preset initial display position information, each custom virtual item corresponding to each custom virtual item information obtained is rendered into the custom virtual scene, and the preset item position adjustment prompt information is played to prompt the user to re-place the custom virtual item. Thus, the user can see each custom virtual item to be added in the custom virtual scene. Afterwards, in response to detecting that any of the above-mentioned custom virtual items intersects with the preset ray, the position detection of the above-mentioned custom virtual items is performed to obtain a position detection result. The above-mentioned preset ray is a ray emitted by the target virtual hand for selecting the custom virtual item to be relocated, and the target virtual hand is the virtual hand of the three-dimensional virtual user. Thus, the custom virtual item to be moved can be selected by emitting a ray. Then, in response to determining that the above-mentioned position detection result meets the preset graspable condition, the above-mentioned custom virtual item is updated to obtain an updated virtual item. Thus, the custom virtual item available for the user to grasp can be determined.Finally, in response to receiving a confirmation grabbing instruction for the updated virtual item, the target virtual hand is controlled to perform a grabbing operation on the updated virtual item so as to place the updated virtual item at the target location. Therefore, the metaverse platform interaction method based on virtual reality technology in some embodiments of the present disclosure can facilitate real-time updating of virtual scenes by generating various customized virtual items in the metaverse platform and guiding users to perform interactive operations such as grabbing and moving virtual items on this basis. Thereby, the update speed of the virtual scene can be increased. Furthermore, the user's interactive experience can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0012] Figure 1 is a flowchart of some embodiments of the metaverse platform interaction method based on virtual reality technology according to the present disclosure; Figure 2 It is a structural schematic diagram of some embodiments of the Metaverse platform interaction device based on virtual reality technology according to the present disclosure; Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0013] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0014] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0015] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0016] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0017] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0018] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0019] Figure 1 The process 100 of some embodiments of the Metaverse platform interaction method based on virtual reality technology according to the present disclosure is shown. The Metaverse platform interaction method based on virtual reality technology includes the following steps: Step 101, in response to detecting a selection operation acting on a scene object adding control in a Metaverse platform interface, obtaining an object image information sequence.

[0020] In some embodiments, the execution subject (e.g., computing device) of the virtual reality technology-based metaverse platform interaction method may obtain an item image information sequence from a preset blockchain network through a wired connection or a wireless connection in response to detecting a selection operation of a scene item addition control in a metaverse platform interface. Wherein, each item image information in the above-mentioned item image information sequence may correspond one-to-one to a target item in a preset target item sequence. The target item in the above-mentioned target item sequence may be an item in the real world of a preset three-dimensional model to be generated. Each item image information in the above-mentioned item image information sequence may include a multi-perspective item image set and a multi-perspective depth atlas. The above-mentioned multi-perspective item image set may be a collection of RGB images of different perspectives taken by an RGB (Red Green Blue) camera for the target item. The above-mentioned multi-perspective depth atlas may be a collection of depth images of different perspectives taken by a depth camera for the target item. The above-mentioned scene item addition control may be a UI (User Interface) control for adding virtual items to a custom virtual scene. For example, the above-mentioned scene item addition control may be a button control, and the corresponding selection operation may be a click operation. The above-mentioned custom virtual scene may be a virtual scene including virtual items that can be created by VR (Virtual Reality) users through the Metaverse platform. The above-mentioned VR users may be users who log in to the Metaverse platform using VR devices. Virtual items may be items generated by three-dimensional modeling technology corresponding to items in the real world. The above-mentioned blockchain network can be used to permanently store all user interaction data and all digital asset generation, modification, and transaction data occurring in the Metaverse platform. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0021] Step 102: for each target item in the preset target item sequence, perform the following steps: Step 1021: construct a three-dimensional object model based on the object image information corresponding to the target object.

[0022] In some embodiments, the execution subject may construct a three-dimensional object model in various ways based on the object image information corresponding to the target object, wherein the three-dimensional object model may be a three-dimensional model of the target object.

[0023] In practice, when constructing a three-dimensional object model, the conventional solution is generally to directly predict the three-dimensional structure of the object from a single image or multiple images by training a neural network. However, the above conventional solution still has the following technical problem 2: Since the performance of the neural network is limited by the quantity and quality of the training data, when the training data is insufficient or biased, it is easy to cause the predicted three-dimensional structure of the object to be inaccurate. Therefore, in the face of the above technical problem 2, combined with the technical advantages of the solution development team in the field of computer vision, the present disclosure decides to adopt the following solution.

[0024] In some optional implementations of some embodiments, the execution subject may construct a three-dimensional object model based on the object image information corresponding to the target object through the following steps: In the first step, instance segmentation processing is performed on each multi-view object image corresponding to the target object to obtain an instance segmentation result set. The instance segmentation results in the instance segmentation result set may correspond to the multi-view object images in each multi-view object image. The instance segmentation results in the instance segmentation result set may be the result of classifying each pixel in the corresponding multi-view object image.

[0025] As an example, the execution subject may perform instance segmentation processing on each multi-view object image corresponding to the target object through a preset instance segmentation model to obtain an instance segmentation result set. The instance segmentation model may be a SegFormer semantic segmentation model. It should be noted that the instance segmentation result is saved as an instance mask, and the mask area of ​​each object records the category of the corresponding pixel.

[0026] In the second step, in response to receiving the instance selection information of the user for any instance segmentation result, the multi-view depth maps corresponding to the above-mentioned target object are completed to obtain each completed depth map. Among them, the instance selection information can be the information of the pixel area corresponding to the instance selected by the user. For example, the above-mentioned instance selection information can include but is not limited to the instance identifier and pixel coordinates. The above-mentioned instance identifier can be the identifier of the object as the instance. The completed depth map in the above-mentioned completed depth maps can be a multi-view depth map after the depth value of the sparse area is completed.

[0027] As an example, the execution subject may use a preset depth map completion method to complete each multi-view depth map corresponding to the target object to obtain each completed depth map. The depth map completion method may be a Deep Completion Network method.

[0028] The third step is to fuse the above-mentioned completed depth maps to obtain global depth information, wherein the above-mentioned global depth information can be a TSDF (Truncated Signed Distance Function) voxel volume.

[0029] As an example, the execution subject may perform fusion processing on the above-mentioned completed depth maps through the TSDF method to obtain global depth information.

[0030] In the fourth step, feature extraction is performed on each multi-view object image corresponding to the target object to obtain a key point information set. Each key point information in the key point information set may correspond to a key point in the multi-view object image. The key point information in the key point information set may be information on the coordinates and feature descriptors of the corresponding key point. The key point may be a corner point, an edge point, or a point in an area with obvious texture in the image.

[0031] As an example, the execution subject may perform feature extraction processing on each multi-view object image corresponding to the target object through a SIFT (Scale-Invariant Feature Transform) algorithm to obtain a key point information set.

[0032] In the fifth step, each key point corresponding to the obtained key point information set is matched to obtain a key point pair set, wherein the key point pair in the key point pair set can be the initial set of two key points with the smallest distance value on different multi-view object images.

[0033] As an example, the above execution subject can match each key point corresponding to the key point information set obtained by the Euclidean distance method to obtain a key point pair set. It should be noted that a depth consistency constraint is also introduced in the feature point matching, and only matching pairs with a depth difference less than a set threshold are retained, thereby reducing mismatches.

[0034] Step 6: Generate a registration point cloud dataset based on the global depth information and the key point pair set. The registration point cloud dataset may be a point cloud dataset after point cloud alignment.

[0035] As an example, the above-mentioned execution entity can convert each key point information into a point cloud dataset according to the global depth information and the above-mentioned key point pair set, and align the point cloud data in the point cloud dataset through the ICP (Iterative Closest Point) algorithm to obtain a registered point cloud dataset.

[0036] Step 7: Reconstruct a 3D mesh based on the registered point cloud dataset, wherein the 3D mesh may be a continuous surface composed of triangles or polygons.

[0037] As an example, the execution subject may reconstruct a three-dimensional mesh based on the registered point cloud dataset by using a Poisson surface reconstruction algorithm.

[0038] In the eighth step, based on each multi-view object image corresponding to the target object, the three-dimensional grid is texture mapped to obtain a three-dimensional object model, wherein the three-dimensional object model can be a three-dimensional model constructed according to the target object.

[0039] As an example, the execution entity may map the input multi-view object image onto the three-dimensional grid, perform texture mapping, and obtain a three-dimensional object model.

[0040] The above-mentioned three-dimensional object model generation step, as an inventive point of an embodiment of the present disclosure, solves the above-mentioned technical problem 2, "the predicted three-dimensional structure of the object is not accurate enough". The reason why the three-dimensional structure of the object predicted in the conventional solution is not accurate enough is that the performance of the neural network is limited by the quantity and quality of the training data. When the training data is insufficient or there is a deviation, it is easy to cause the predicted three-dimensional structure of the object to be not accurate enough. If this solution solves the above-mentioned problem, it can achieve the effect of improving the accuracy of the predicted three-dimensional structure of the object. In order to achieve this effect, each time the object image of the target object is obtained, first, the instance segmentation processing is performed on each multi-view object image corresponding to the target object to obtain an instance segmentation result set. In this way, the object image can be pixel-classified so as to subsequently extract the segmented area of ​​the object of interest to the user. Secondly, in response to receiving the instance selection information of the user for any instance segmentation result, the multi-view depth maps corresponding to the target object are complemented to obtain each complement depth map, and the complement depth maps are fused to obtain global depth information. In this way, globally consistent depth information can be generated. Then, feature extraction processing is performed on each multi-view object image corresponding to the target object to obtain a key point information set. Among them, each key point information in the above key point information set corresponds to a key point in the multi-view object image. Matching processing is performed on each key point corresponding to the obtained key point information set to obtain a key point pair set. Thus, key points with corresponding relationships between different viewpoints can be obtained, which is convenient for subsequent use to align the relative positions of different viewpoint images in three-dimensional space. Afterwards, a registration point cloud data set is generated based on the above global depth information and the above key point pair set. Thus, each key point can be converted into point cloud data based on the matching of feature points of the image and combined with the global depth information. Then, a three-dimensional grid is reconstructed based on the above registration point cloud data set. Thus, a three-dimensional grid structure corresponding to the target object can be obtained. Finally, based on each multi-view object image corresponding to the above target object, texture mapping processing is performed on the above three-dimensional grid to obtain a three-dimensional object model. Thus, each pixel on the multi-view object can be matched with each vertex of the three-dimensional grid. Thus, a three-dimensional object model with higher precision can be obtained, and the difference between the virtual object corresponding to the three-dimensional object model and the actual object can be reduced, and further the user can have a more accurate perception of the virtual object. Thus, the user interaction experience can be improved.

[0041] Step 1022: Generate customized virtual item information based on the three-dimensional item model.

[0042] In some embodiments, the execution entity may generate custom virtual item information based on the three-dimensional item model. The custom virtual item information may correspond to the custom virtual item one-to-one. The custom virtual item information may be information about a custom virtual item created by a VR user. For example, the custom virtual item information may include but is not limited to at least one of the following: an item identifier, a virtual item model data. The item identifier may be a unique identifier of a target item or a unique identifier of a custom virtual item. The virtual item model data may be data of a three-dimensional model of a virtual item. It should be noted that the target item and the custom virtual item in a corresponding relationship have the same identifier.

[0043] As an example, the execution entity may also determine the item identifier corresponding to the instance selection information and the three-dimensional item model as the custom item information.

[0044] It should be supplemented that in the process of adopting technical solutions to solve the technical problems 1 and 2 mentioned in the above background technology section, the following technical problem is often accompanied: how to generate a multi-perspective object image set including the object image information. In response to the above technical problems, the conventional solution is generally: according to the pre-set perspectives, the image of the object is captured from different perspectives. However, the above conventional solution still has the following technical problem 3: Since the image is easily affected by external conditions such as lighting, the quality of the images of each object captured is uneven, which further makes it easy for the created virtual objects to be different from the actual objects, and the user perception is distorted, thus resulting in a poor user interaction experience on the Metaverse platform. Therefore, in the face of the above technical problem 3, combined with the technical advantages of the solution development team itself in the field of computer vision, the present disclosure decides to adopt the following solution.

[0045] Optionally, before acquiring the object image information sequence, the execution subject may also perform the following steps of generating a multi-view object image set for each target object: The first step is to obtain the object image and current camera pose information corresponding to the target object. The object image may be an RGB image. The current camera pose information may be the pose information of the RGB camera when taking the object image.

[0046] As an example, the execution subject may obtain an object image corresponding to a target object that has been recently photographed from an RGB camera on a VR device, and obtain current camera position information from an inertial sensor on the VR device.

[0047] The second step is to perform quality inspection on the above-mentioned object image to obtain a quality inspection result, wherein the above-mentioned quality inspection result can indicate whether the object image is clear.

[0048] As an example, the execution subject may first determine the grayscale variance value corresponding to the above-mentioned object image through a grayscale variance detection algorithm. Then, in response to determining that the above-mentioned grayscale variance value is less than a preset variance threshold, a preset image unclear mark is determined as the quality detection result. Among them, the above-mentioned preset variance threshold may be a preset upper limit value of the grayscale variance value. The above-mentioned image unclear mark may indicate that the quality of the corresponding object image is unclear. Finally, in response to determining that the above-mentioned grayscale variance value is not less than the above-mentioned preset variance threshold, a preset image clear mark is determined as the quality detection result. Among them, the above-mentioned image clear mark may indicate that the corresponding object image is relatively clear.

[0049] The third step is to determine the viewing angle coverage rate corresponding to the target object based on the historical object image sequence corresponding to the target object in response to determining that the quality inspection result satisfies the preset inspection result condition. The historical object image sequence may be an ordered set of object images taken at different viewing angles for the target object. The preset inspection result condition may be that the quality inspection result indicates that the object image is clear. The viewing angle coverage rate may be the degree to which the area where the target object is located is covered by the object images taken at various viewing angles. It should be noted that the historical object image sequence includes the object images obtained above.

[0050] As an example, the execution subject may determine the viewing angle coverage corresponding to the target object based on the historical object image sequence corresponding to the target object through the viewing angle coverage evaluation interface. The viewing angle coverage evaluation interface may encapsulate a viewing angle coverage evaluation function. The viewing angle coverage evaluation function may be used to determine the mask of the partial area that is photographed and the mask of the partial area that is not photographed of the area where the target object is located through the Unity engine according to the historical object image sequence corresponding to the target object, and estimate the viewing angle coverage corresponding to the target object.

[0051] In step 4, in response to determining that the viewing angle coverage is greater than or equal to a preset viewing angle coverage threshold, the historical item image sequence corresponding to the target item is determined as a multi-view item image set. The preset viewing angle coverage threshold may be a lower limit of the viewing angle coverage.

[0052] Optionally, the above execution entity may further perform the following steps: Step 1: in response to determining that the viewing angle coverage is less than the preset viewing angle coverage threshold, determine uncovered area information, wherein the uncovered area information may be information of a mask of a partial area where the target object is located that is not captured.

[0053] As an example, the execution subject may obtain, from the viewing angle coverage evaluation interface, a mask of a partial area where the target object is located but is not captured as the uncovered area information.

[0054] Step 2: Generate optimized shooting path information based on the current camera pose information and the uncovered area information. The optimized shooting path information may correspond to the shooting path one by one. The shooting path may be a path for the VR user to travel to shoot the target object. The optimized shooting path information may be information of a path point coordinate sequence corresponding to the shooting path. The path point coordinate sequence may be a sequence of coordinates of points during the VR user's travel.

[0055] As an example, the execution entity may first use the camera pose of the RGB camera as the starting point of the path. Then, select the edge point of the region that meets the preset edge position condition from the edge points of the region corresponding to the uncovered region information as the end point of the path. The preset edge position condition may be that the distance value between the edge point of the region and the starting point of the path is the minimum value among the distance values. The distance value among the distance values ​​may be the straight-line distance between the edge point on the partial area where the target object is located that is not photographed and the starting point of the path. Finally, through the preset path planning algorithm, the optimized shooting path information is generated according to the starting point of the path and the end point of the path. The path planning algorithm may be an A* algorithm.

[0056] In practice, in simple scenarios (when the number of objects in the image is less than a certain value), the above-mentioned execution entity can use a greedy algorithm to quickly guide the user to shoot; in complex scenarios, it can be combined with the A* algorithm for precise optimization to ensure the shortest path and full coverage.

[0057] Step 3: Render the shooting path corresponding to the above-mentioned optimized shooting path information into a three-dimensional shooting scene, so as to guide the user to shoot the object image again for the target object, and execute the above-mentioned multi-view object image set generation step again. The above-mentioned three-dimensional shooting scene can be the three-dimensional scene displayed on the screen of the VR device when shooting the above-mentioned target object.

[0058] As an example, the execution subject may render the shooting path corresponding to the optimized shooting path information into a three-dimensional shooting scene through augmented reality technology. A VR user may travel along the shooting path and shoot images of each object for the target object.

[0059] The above-mentioned multi-view object image set generation step, as an inventive point of an embodiment of the present disclosure, solves the above-mentioned technical problem three "the user interaction experience of the Metaverse platform is poor". The reason why the conventional solution has a poor user interaction experience of the Metaverse platform is that the quality of each object image taken is uneven due to the influence of external conditions such as lighting when shooting images, which further makes it easy for the created virtual objects to differ from the actual objects and the user's perception is distorted. If this solution solves the above-mentioned problem, it can achieve the effect of improving the user interaction experience of the Metaverse platform. In order to achieve this effect, each time the object image of the target object is obtained, first, the object image is quality tested. Then, when the image quality is good, the view coverage rate corresponding to the target object is determined according to the historical object image sequence corresponding to each view angle that has been obtained. In this way, it can be determined whether the target object can be completely covered by each object image. Afterwards, when the target object can be completely covered by each object image, the historical object image sequence corresponding to the target object can be determined as a multi-view object image set. If the target object is not completely covered by each object image, the shooting path is planned for the user in real time according to the current camera posture information and the uncovered area information. Finally, the shooting path corresponding to the above-mentioned optimized shooting path information is rendered into the three-dimensional shooting scene to guide the user to shoot the object image again for the target object, and to perform the above-mentioned multi-view object image set generation step again. In this way, high-quality object images with different perspectives can be generated, reducing the difference between the created virtual objects and the actual objects, and further enabling the user to have a more accurate perception of the virtual objects. Thus, the user interaction experience of the Metaverse platform can be improved.

[0060] Step 103 , based on the preset initial display position information, rendering each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene, and playing preset item position adjustment prompt information to prompt the user to re-place the custom virtual item.

[0061] In some embodiments, the execution subject may render each custom virtual item corresponding to each custom virtual item information obtained into a custom virtual scene in various ways based on the preset initial display position information, and play the preset item position adjustment prompt information to prompt the user to re-place the custom virtual item. The initial display position information may include the initial center point horizontal coordinate and the initial center point vertical coordinate. The initial center point horizontal coordinate may be the horizontal coordinate of the center point of the virtual item display area. The initial center point horizontal coordinate may be the vertical coordinate of the center point of the virtual item display area. The virtual item display area may be a rectangular area for displaying virtual items. The preset item position adjustment prompt information may be a pre-set voice text for prompting the user to adjust the item position. For example, the preset item position adjustment prompt information may be "Please select the item to be moved and move it to the appropriate position."

[0062] In some optional implementations of some embodiments, the execution subject may render each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene through the following steps based on the preset initial display position information: In the first step, for each piece of custom virtual item information in each piece of custom virtual item information, perform the following steps: Step 1: determine the serial number corresponding to the above-mentioned custom virtual item information as the target serial number.

[0063] Step 2: Generate position translation information based on the target serial number and the preset step length. The preset step length may be the pixel distance between the center points of two preset custom virtual item display areas. The position translation information may be information about the pixel distance that needs to be translated horizontally. First, the difference between the target serial number and 1 is determined as the translation multiple. Then, the product of the translation multiple and the preset step length is determined as the position translation information.

[0064] Step 3: Determine the target display position information based on the position translation information and the initial display position information. The target display position information may be information about the position of the custom virtual item to be displayed. First, the sum of the initial center point horizontal coordinate included in the initial display position information and the pixel distance corresponding to the position translation information is determined as the target position horizontal coordinate. Then, the initial center point vertical coordinate included in the initial display position information is used as the target position vertical coordinate, and the target position horizontal coordinate and the target position vertical coordinate are determined as the target display position information.

[0065] Step 4: Based on the target display position information, the custom virtual item corresponding to the custom virtual item information is rendered into the custom virtual scene.

[0066] As an example, the execution entity may render the custom virtual item corresponding to the custom virtual item information to a virtual item display area in the custom virtual scene with the coordinates corresponding to the target display position information as the center point.

[0067] In practice, the execution subject can play the preset object position adjustment prompt information through the voice player on the VR device. In addition, the basic virtual environment such as room and outdoor included in the custom virtual scene can be pre-built by Blender software and Unity engine. VR users can add virtual objects on top of the built basic virtual environment.

[0068] Step 104 , in response to detecting that any customized virtual item among the customized virtual items intersects with a preset ray, performing position detection on the any customized virtual item to obtain a position detection result.

[0069] In some embodiments, the execution subject may detect the position of any of the above-mentioned custom virtual items in response to detecting that any of the above-mentioned custom virtual items intersects with a preset ray, and obtain a position detection result by various means. Among them, the preset ray may be a ray emitted by a target virtual hand for selecting a custom virtual item to be relocated. The target virtual hand may be a virtual hand of a three-dimensional virtual user. The three-dimensional virtual user may be a digital person corresponding to the VR user. The position detection result may be information on whether the custom virtual item at the corresponding position can be grabbed.

[0070] In some optional implementations of some embodiments, the execution subject may perform position detection on any of the above-mentioned custom virtual items through the following steps to obtain a position detection result: The first step is to obtain the layer information corresponding to any of the above-mentioned custom virtual items. The above-mentioned layer information may be the information of the layer where the custom virtual item is located. For example, the above-mentioned layer information may include but is not limited to the layer identifier. The layer identifier may be the unique identifier of the layer.

[0071] As an example, the above-mentioned execution entity can obtain the belonging layer information corresponding to any of the above-mentioned customized virtual items from the blockchain network.

[0072] The second step is to match the preset layer information of the items to be captured with the above-mentioned belonging layer information to obtain a layer matching result. The above-mentioned preset layer information of the items to be captured can be the information of the identification of the layer where the custom virtual items to be captured are located. The above-mentioned layer matching result can indicate whether the layer where any custom virtual items intersecting with the above-mentioned preset ray are located is the layer where the custom virtual items to be captured are located.

[0073] As an example, in response to determining that the preset layer information of the item to be captured is the same as the above-mentioned belonging layer information, the above-mentioned execution subject may determine the preset layer match success flag as the layer match result. Among them, the above-mentioned preset layer match success flag may indicate that the layer where any custom virtual item intersecting with the above-mentioned preset ray is located is the layer where the preset custom virtual item to be captured is located. Then, in response to determining that the preset layer information of the item to be captured is not the same as the above-mentioned belonging layer information, the preset layer match failure flag may be determined as the layer match result. Among them, the above-mentioned preset layer match failure flag may indicate that the layer where any custom virtual item intersecting with the above-mentioned preset ray is located is not the layer where the preset custom virtual item to be captured is located.

[0074] In the third step, in response to determining that the layer matching result satisfies the preset matching condition, the intersection distance corresponding to the arbitrary custom virtual item is determined. The preset matching condition may be that the layer matching result can indicate that the layer where the arbitrary custom virtual item intersects with the preset ray is located is the layer where the preset custom virtual item to be captured is located. The intersection distance is the distance between the intersection between the arbitrary custom virtual item and the preset ray and the virtual hand that emits the ray.

[0075] As an example, the execution entity may determine the distance between the center point of the virtual hand and the intersection point as the intersection distance corresponding to any custom virtual item.

[0076] In step 4, in response to determining that the intersection distance satisfies a preset distance condition, a preset grabbable identifier is determined as the position detection result. The preset distance condition may be that the intersection distance is the minimum value of the intersection distances corresponding to the virtual items intersecting with the preset ray. The preset grabbable identifier may indicate that the custom virtual item at the corresponding position can be grabbed.

[0077] Step 105 , in response to determining that the position detection result satisfies the preset graspable condition, updating any customized virtual item to obtain an updated virtual item.

[0078] In some embodiments, the execution subject may update any of the above-mentioned custom virtual items in response to determining that the above-mentioned position detection result satisfies the preset graspable condition to obtain an updated virtual item. The above-mentioned preset graspable condition may be that the position detection result indicates that the custom virtual item at the corresponding position can be grasped. The above-mentioned updated virtual item may be a custom virtual item with a luminous effect.

[0079] As an example, the execution subject may set the self-luminous property of any of the custom virtual items to be turned on, so as to update the any of the custom virtual items and obtain an updated virtual item.

[0080] Step 106 , in response to receiving a confirmation grabbing instruction for the updated virtual item, controlling the target virtual hand to perform a grabbing operation on the updated virtual item, so as to place the updated virtual item at a target position.

[0081] In some embodiments, the execution subject may, in response to receiving a confirmation grabbing instruction for the updated virtual item, control the target virtual hand to perform a grabbing operation on the updated virtual item so as to place the updated virtual item at a target location. The confirmation grabbing instruction may be an instruction for instructing to perform a grabbing operation on the updated virtual item. The target location may be a location in the virtual scene where the VR user wants to place the updated virtual item.

[0082] As an example, the above-mentioned execution entity can receive a confirmation grabbing instruction sent by a VR user through the trigger button on the handle, disable the physical properties of the above-mentioned updated virtual object such as gravity and collision, and set the above-mentioned updated virtual object as a child object of the above-mentioned target virtual hand. Through inverse kinematics technology, the target virtual hand adjusts its position according to the shape of the object and performs a grabbing action. The position of the virtual object will be updated in real time following the above-mentioned target virtual hand.

[0083] Optionally, the above execution entity may further perform the following steps: The first step is to control the target virtual hand to move the updated virtual item to the target position in response to receiving the user interaction data. The user interaction data may include but is not limited to user posture data and gesture interaction data. The user posture data may be data on the position and posture of the VR user collected by an inertial sensor. The gesture interaction data may be an image of the VR user's hand movement collected by a camera.

[0084] As an example, after receiving the user interaction data, the above-mentioned execution entity can use the VNect posture estimation algorithm to bind the posture and action corresponding to the above-mentioned user interaction data to the skeleton of the three-dimensional virtual user, and recognize the continuous posture action data and gesture interaction data to obtain coherent human body skeleton posture data, and perform real-time rendering in the virtual environment.

[0085] The second step is, in response to receiving a confirmation placement instruction for the updated virtual item, controlling the target virtual hand to perform a release operation on the updated virtual item to place the updated virtual item at the target location. The confirmation placement instruction may be an instruction for instructing to release the grip of the updated virtual item.

[0086] As an example, the execution subject receives a confirmation placement instruction for the updated virtual item, enables the physical properties of the updated virtual item, changes the parent object corresponding to the updated virtual item from the target virtual hand to an empty object, and synchronously renders the user's release action to the target virtual hand to perform the release operation. Thus, the updated virtual item can be placed at the target position.

[0087] Optionally, the Metaverse platform interface further includes a virtual scene selection control. The virtual scene selection control may be a button control for selecting a virtual scene. The execution subject may further perform the following steps: The first step is to obtain a virtual scene list information set in response to detecting a selection operation acting on a virtual scene selection control in the above-mentioned metaverse platform interface. Among them, each virtual scene list information in the above-mentioned virtual scene list information set can correspond one-to-one to a virtual scene in a preset virtual scene set. The above-mentioned virtual scene selection control can be a button control for selecting a virtual scene. The virtual scene list information in the above-mentioned virtual scene list information set may include a virtual scene identifier and a scene link. The above-mentioned virtual scene identifier can be a unique identifier of a virtual scene. The above-mentioned scene link can be a network address for loading the corresponding virtual scene.

[0088] As an example, the above-mentioned execution entity can obtain a pre-stored virtual scene list information set from the blockchain network.

[0089] The second step is to fill each virtual scene list information in the virtual scene list information set into the preset virtual scene list display panel, so that the user can select the virtual scene he wants to enter. The preset virtual scene list display panel can be used to display the identification of each virtual scene.

[0090] As an example, the execution subject may fill each virtual scene list information in the virtual scene list information set into a preset virtual scene list display panel in the form of a list through a list element. The user may enter the virtual scene corresponding to the list item by clicking on the list item.

[0091] The third step is to load the virtual scene corresponding to any virtual scene list information in response to detecting a selection operation acting on any virtual scene list information.

[0092] As an example, the execution subject may load the corresponding virtual scene according to the scene link included in any selected virtual scene list information.

[0093] Optionally, the above execution entity may further perform the following steps: In the first step, in response to determining that the virtual scene corresponding to any of the virtual scene list information satisfies the preset scene type condition, the preset item circulation operation prompt information is played to allow the user to select the virtual item to be circulated. Among them, the preset scene type condition may be that the virtual scene is a virtual store type. The virtual store type may indicate that the corresponding virtual scene is a virtual store. The virtual store may be a store built through virtual reality technology for displaying and circulating virtual items. The preset item circulation operation prompt information may be a pre-set voice text for prompting the user how to perform the item circulation operation. The virtual item to be circulated may be a virtual item that can perform the circulation operation.

[0094] In practice, the above-mentioned execution entity can play the above-mentioned preset item flow operation prompt information through the voice player built into the VR device.

[0095] The second step is to obtain the item details information corresponding to any virtual item to be circulated in response to detecting a selection operation acting on any virtual item to be circulated. Among them, the above-mentioned any virtual item to be circulated can be a virtual item to be circulated displayed in a virtual scene corresponding to the above-mentioned any virtual scene list information. The above-mentioned item details information can be the description information of any virtual item to be circulated obtained from the blockchain network. The above-mentioned item details information may include but is not limited to virtual item identification, item source information, functional information, and value attributes (such as price). The item source information can be the information of the owner of the item. When the item source information is 0, it indicates that the virtual item is owned by the virtual store. When the item source information is XXX (for example, Wang XX), it indicates that the virtual item is owned by XXX.

[0096] The third step is to determine the ownership type of the virtual item based on the item details information, wherein the ownership type of the virtual item can indicate whether the virtual item is owned by the virtual store.

[0097] As an example, the execution subject may determine the first attribution type as the virtual item attribution type in response to determining that the item source information included in the item details information is 0, and determine the second attribution type as the virtual item attribution type in response to determining that the item source information included in the item details information is not 0. The first attribution type may indicate that the virtual item is owned by the virtual store. The second attribution type may indicate that the virtual item is owned by an individual.

[0098] In the fourth step, in response to determining that the virtual item ownership type satisfies the preset ownership type condition, the item details information and the confirmation transfer button control are displayed in the preset scene interaction panel. Among them, the preset ownership type condition may be: the virtual item ownership type indicates that the corresponding virtual item is owned by the virtual store. The scene interaction panel may be an interface for the Metaverse platform system to interact with the user. The confirmation transfer button control may be a button control for the user to confirm whether to perform the transfer operation on the selected virtual item to be transferred.

[0099] In step 5, in response to detecting the selection operation on the above-mentioned confirmation transfer button control, various form item controls are displayed in the scene interaction panel to receive the value transfer information input by the user. Among them, the above-mentioned various form item controls may include but are not limited to input box controls, single-choice controls, and multiple-choice controls. The above-mentioned value transfer information may be information on the value required to perform the item transfer operation.

[0100] In the sixth step, in response to receiving the value transfer information corresponding to any of the above virtual items to be circulated, the above value transfer information is sent to the preset blockchain network for executing the virtual item transfer operation. It should be noted that in the blockchain network, the smart contract initiates the conditional verification of the virtual item transfer operation, and after the verification is passed, the ownership is transferred, the transfer record is updated, and the two parties involved in the transfer are notified.

[0101] Optionally, the above execution entity may further perform the following steps: In the first step, in response to determining that the virtual item attribution type does not meet the preset attribution type condition, the item details and a session creation control are displayed in the scene interaction panel, wherein the session creation control may be a button control that allows a session to be established between two virtual users.

[0102] In the second step, in response to detecting the selection operation acting on the above-mentioned establishment session control, a session window is displayed for virtual users to communicate item flow information. Among them, the above-mentioned session window may include an input box control, a session display panel, an item exchange control, an item gift control, and an item value exchange control. The above-mentioned input box control can be used for virtual users to input information. The above-mentioned session display panel is used to display the content of the session between two virtual users. The above-mentioned item exchange control can be used to initiate a virtual item exchange request. The above-mentioned item gift control can be used to initiate an instruction to gift a virtual item. The above-mentioned item value exchange control can be used to input value flow information, and its function is equivalent to the above-mentioned confirmation flow button control.

[0103] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the Metaverse platform interaction method based on virtual reality technology of some embodiments of the present disclosure, the virtual scene update speed can be improved, and the user interaction experience can be improved. Specifically, the reason for the slow virtual scene update speed and the poor user interaction experience is that the virtual objects presented by the virtual environment are usually integrated with the basic virtual scene, rendered and transmitted to the virtual reality device, so that the virtual scene is relatively fixed, and the user can only interact with a few inherent virtual objects in the virtual scene of the Metaverse platform. If you want to add virtual objects, you need to integrate the virtual objects to be added into the virtual scene, and then render and transmit them to the virtual reality device to update the previously displayed virtual scene, thereby resulting in a slow virtual scene update speed and a poor user interaction experience. Based on this, the Metaverse platform interaction method based on virtual reality technology of some embodiments of the present disclosure, first, in response to detecting a selection operation of adding a scene object control acting on the Metaverse platform interface, obtains an object image information sequence. Wherein, each item image information in the above-mentioned item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the above-mentioned item image information sequence includes a multi-view item image set and a multi-view depth atlas. Thus, when a user wants to add a virtual item to a virtual scene, the item image information required to create each custom virtual item can be obtained first. Secondly, for each target item in the preset target item sequence, the following steps are performed: based on the item image information corresponding to the above-mentioned target item, a three-dimensional item model is constructed; based on the above-mentioned three-dimensional item model, custom virtual item information is generated. Wherein, the above-mentioned custom virtual item information corresponds to the custom virtual item. Thus, each custom virtual item can be created so as to be subsequently placed in the virtual scene. Then, based on the preset initial display position information, each custom virtual item corresponding to each custom virtual item information obtained is rendered into the custom virtual scene, and the preset item position adjustment prompt information is played to prompt the user to re-place the custom virtual item. Thus, the user can see each custom virtual item to be added in the custom virtual scene. Afterwards, in response to detecting that any of the above-mentioned custom virtual items intersects with the preset ray, the position detection of the above-mentioned custom virtual items is performed to obtain a position detection result. The above-mentioned preset ray is a ray emitted by the target virtual hand for selecting the custom virtual item to be relocated, and the target virtual hand is the virtual hand of the three-dimensional virtual user. Thus, the custom virtual item to be moved can be selected by emitting a ray. Then, in response to determining that the above-mentioned position detection result meets the preset graspable condition, the above-mentioned custom virtual item is updated to obtain an updated virtual item. Thus, the custom virtual item available for the user to grasp can be determined.Finally, in response to receiving a confirmation grabbing instruction for the updated virtual item, the target virtual hand is controlled to perform a grabbing operation on the updated virtual item so as to place the updated virtual item at the target location. Therefore, the metaverse platform interaction method based on virtual reality technology in some embodiments of the present disclosure can facilitate real-time updating of virtual scenes by generating various customized virtual items in the metaverse platform and guiding users to perform interactive operations such as grabbing and moving virtual items on this basis. Thereby, the update speed of the virtual scene can be increased. Furthermore, the user's interactive experience can be improved.

[0104] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a metaverse platform interaction device based on virtual reality technology. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the metaverse platform interaction device 200 based on virtual reality technology can be specifically applied to various electronic devices.

[0105] like Figure 2As shown, in some embodiments, the Metaverse platform interaction device 200 based on virtual reality technology includes: an acquisition unit 201, an execution unit 202, a rendering and playing unit 203, a position detection unit 204, an updating unit 205 and a control unit 206. The acquisition unit 201 is configured to acquire an item image information sequence in response to detecting a selection operation of a scene item adding control in the Metaverse platform interface, wherein each item image information in the item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the item image information sequence includes a multi-view item image set and a multi-view depth atlas; the execution unit 202 is configured to perform the following steps for each target item in the preset target item sequence: construct a three-dimensional item model based on the item image information corresponding to the target item; generate custom virtual item information based on the three-dimensional item model, wherein the custom virtual item information corresponds to the custom virtual item; the rendering and playing unit 203 is configured to render each custom virtual item corresponding to each custom virtual item information obtained to the custom virtual object based on the preset initial display position information. The scene is configured to play a preset item position adjustment prompt information to prompt the user to re-place the custom virtual item; the position detection unit 204 is configured to, in response to detecting that any of the above-mentioned custom virtual items intersects with the preset ray, perform position detection on the above-mentioned any custom virtual item to obtain a position detection result, wherein the above-mentioned preset ray is a ray emitted by the target virtual hand and used to select the custom virtual item to be re-placed, and the target virtual hand is the virtual hand of the three-dimensional virtual user; the updating unit 205 is configured to, in response to determining that the above-mentioned position detection result meets the preset graspable condition, perform update processing on the above-mentioned any custom virtual item to obtain an updated virtual item; the control unit 206 is configured to, in response to receiving a confirmation grasping instruction for the above-mentioned updated virtual item, control the target virtual hand to perform a grasping operation on the above-mentioned updated virtual item to place the above-mentioned updated virtual item at the target position.

[0106] It is understandable that the units recorded in the virtual reality technology-based metaverse platform interactive device 200 and the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the Metaverse platform interaction device 200 based on virtual reality technology and the units contained therein, and will not be repeated here.

[0107] Further references Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0108] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 to a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0109] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0110] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0111] It should be noted that the computer-readable medium mentioned above in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0112] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0113] The above-mentioned computer-readable medium may be included in the above-mentioned device; or it may exist independently and not be assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to detecting a selection operation of a scene item adding control acting on the metaverse platform interface, obtains an item image information sequence, wherein each item image information in the above-mentioned item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the above-mentioned item image information sequence includes a multi-perspective item image set and a multi-perspective depth atlas; for each target item in the preset target item sequence, perform the following steps: based on the item image information corresponding to the above-mentioned target item, construct a three-dimensional item model; based on the above-mentioned three-dimensional item model, generate custom virtual item information, wherein the above-mentioned custom virtual item information corresponds to the custom virtual item; based on the preset initial display position information, display each custom virtual item obtained The method comprises the steps of: rendering each custom virtual item corresponding to the information into a custom virtual scene, and playing a preset item position adjustment prompt information to prompt the user to replace the custom virtual item; in response to detecting that any custom virtual item among the above-mentioned custom virtual items intersects with a preset ray, performing position detection on the above-mentioned any custom virtual item to obtain a position detection result, wherein the above-mentioned preset ray is a ray emitted by a target virtual hand and is used to select the custom virtual item to be replaced, and the target virtual hand is a virtual hand of a three-dimensional virtual user; in response to determining that the above-mentioned position detection result meets a preset graspable condition, updating the above-mentioned any custom virtual item to obtain an updated virtual item; in response to receiving a confirmation grasping instruction for the above-mentioned updated virtual item, controlling the target virtual hand to perform a grasping operation on the above-mentioned updated virtual item to place the above-mentioned updated virtual item at a target position.

[0114] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0116] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes: an acquisition unit, an execution unit, a rendering and playback unit, a position detection unit, an update unit and a control unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the acquisition unit may also be described as a "unit for acquiring an object image information sequence".

[0117] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0118] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A metaverse platform interaction method based on virtual reality technology, comprising: In response to detecting a selection operation of a scene item adding control in a metaverse platform interface, obtaining an item image information sequence, wherein each item image information in the item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the item image information sequence includes a multi-view item image set and a multi-view depth atlas; For each target item in the preset target item sequence, perform the following steps: constructing a three-dimensional object model based on the object image information corresponding to the target object; Based on the three-dimensional item model, generating custom virtual item information, wherein the custom virtual item information corresponds to the custom virtual item; Based on the preset initial display position information, rendering each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene, and playing the preset item position adjustment prompt information to prompt the user to re-place the custom virtual item; In response to detecting that any of the customized virtual items intersects with a preset ray, performing position detection on the any customized virtual item to obtain a position detection result, wherein the preset ray is a ray emitted by a target virtual hand and used to select the customized virtual item to be relocated, and the target virtual hand is a virtual hand of a three-dimensional virtual user; In response to determining that the position detection result satisfies a preset graspable condition, updating the arbitrary customized virtual item to obtain an updated virtual item; In response to receiving a confirmation grabbing instruction for the updated virtual item, controlling the target virtual hand to perform a grabbing operation on the updated virtual item so as to place the updated virtual item at a target position.

2. The method for interacting with the metaverse platform based on virtual reality technology according to claim 1, wherein: The metaverse platform interface also includes a virtual scene selection control; and the method also includes: In response to detecting a selection operation acting on a virtual scene selection control in the metaverse platform interface, obtaining a virtual scene list information set, wherein each virtual scene list information in the virtual scene list information set corresponds to a virtual scene in a preset virtual scene set; Filling each virtual scene list information in the virtual scene list information set into a preset virtual scene list display panel, so that the user can select a virtual scene to enter; In response to detecting a selection operation acting on any virtual scene list information, a virtual scene corresponding to the any virtual scene list information is loaded.

3. The method for interacting with the metaverse platform based on virtual reality technology according to claim 2, wherein: The method further comprises: In response to determining that the virtual scene corresponding to the arbitrary virtual scene list information meets the preset scene type condition, playing preset item circulation operation prompt information for the user to select the virtual item to be circulated; In response to detecting a selection operation on any virtual item to be circulated, obtaining item detail information corresponding to the any virtual item to be circulated, wherein the any virtual item to be circulated is the virtual item to be circulated displayed in the virtual scene corresponding to the any virtual scene list information; Determine the type of the virtual item based on the item details information; In response to determining that the virtual item attribution type meets a preset attribution type condition, displaying the item detail information and a confirmation transfer button control in a preset scene interaction panel; In response to detecting a selection operation acting on the confirm transfer button control, displaying various form item controls in the scene interaction panel to receive value transfer information input by the user; In response to receiving the value transfer information corresponding to any virtual item to be circulated, the value transfer information is sent to a preset blockchain network for executing a virtual item circulation operation.

4. The method for interacting with the metaverse platform based on virtual reality technology according to claim 3, wherein: The method further comprises: In response to determining that the virtual item attribution type does not meet the preset attribution type condition, displaying the item detail information and establishing a session control in the scene interaction panel; In response to detecting a selection operation acting on the establishing session control, a session window is displayed for communicating item flow information between virtual users, wherein the session window includes an input box control, a session display panel, an item exchange control, an item donation control, and an item value exchange control.

5. The method for interacting with the metaverse platform based on virtual reality technology according to claim 1, wherein: The rendering of each custom virtual item corresponding to each custom virtual item information obtained into the custom virtual scene based on the preset initial display position information includes: For each custom virtual item information in each custom virtual item information, perform the following steps: Determine the serial number corresponding to the custom virtual item information as the target serial number; Generate position translation information based on the target sequence number and the preset step size; Determining target display position information based on the position translation information and the initial display position information; Based on the target display position information, the custom virtual item corresponding to the custom virtual item information is rendered into a custom virtual scene.

6. The method for interacting with the metaverse platform based on virtual reality technology according to claim 1, wherein: The performing position detection on the arbitrary customized virtual item to obtain a position detection result includes: Obtaining the belonging layer information corresponding to the arbitrary customized virtual item; Matching the preset layer information of the object to be captured and the belonging layer information to obtain a layer matching result; In response to determining that the layer matching result satisfies a preset matching condition, determining an intersection distance corresponding to the arbitrary customized virtual item; In response to determining that the intersection distance satisfies a preset distance condition, a preset graspable mark is determined as a position detection result.

7. The method for interacting with the metaverse platform based on virtual reality technology according to claim 1, wherein: The method further comprises: In response to receiving the user interaction data, controlling the target virtual hand to move the updated virtual object to the target position; In response to receiving a confirmation placement instruction for the updated virtual item, controlling the target virtual hand to perform a release operation on the updated virtual item to place the updated virtual item at the target position.

8. A metaverse platform interactive device based on virtual reality technology, comprising: an acquisition unit, configured to acquire an item image information sequence in response to detecting a selection operation of a scene item adding control in a metaverse platform interface, wherein each item image information in the item image information sequence corresponds to a target item in a preset target item sequence, and each item image information in the item image information sequence includes a multi-view item image set and a multi-view depth atlas set; The execution unit is configured to execute the following steps for each target item in a preset target item sequence: constructing a three-dimensional object model based on the object image information corresponding to the target object; Based on the three-dimensional item model, generating custom virtual item information, wherein the custom virtual item information corresponds to the custom virtual item; A rendering and playing unit is configured to render each custom virtual item corresponding to each custom virtual item information obtained into a custom virtual scene based on the preset initial display position information, and play preset item position adjustment prompt information to prompt the user to reposition the custom virtual item; a position detection unit configured to, in response to detecting that any of the customized virtual items intersects with a preset ray, perform position detection on the any customized virtual item to obtain a position detection result, wherein the preset ray is a ray emitted by a target virtual hand and used to select the customized virtual item to be relocated, and the target virtual hand is a virtual hand of a three-dimensional virtual user; an updating unit, configured to, in response to determining that the position detection result satisfies a preset graspable condition, update the arbitrary customized virtual item to obtain an updated virtual item; The control unit is configured to, in response to receiving a confirmation grabbing instruction for the updated virtual item, control the target virtual hand to perform a grabbing operation on the updated virtual item so as to place the updated virtual item at a target position.

9. A device for interacting with a Metaverse platform, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

Images