Method and system for three dimensional (3D) construction of an object in a virtual environment

Abstract

The present disclosure relates to methods and systems for three-dimensional (3D) construction of an object in a virtual environment. For 3D construction, firstly, a two dimensional (2D) multi-media stream segment of the object is received, and then a 3D measurement information and a set of physical properties, of the object, is obtained based on the 2D multi-media stream segment. Then, a 3D object is obtained based on the 3D measurement information and the set of physical properties. Thereafter, at least one of a scaling operation and an alignment operation is performed on the 3D object in the virtual environment for providing, to a user, an interaction of the 3D object with one or more virtual objects.

Description

METHOD AND SYSTEM FOR THREE DIMENSIONAL (3D) CONSTRUCTION OF AN OBJECT IN A VIRTUAL ENVIRONMENT

[0001] The present disclosure relates to a field of virtual spaces and interaction of an avatar with objects. More particularly, the present disclosure relates to three-dimensional (3D) construction of object(s) in the virtual environment.

[0002] The following description of the related art is intended to provide background information pertaining to the field of the disclosure. This section may include certain aspects of the art that may be related to various features of the present disclosure. However, it should be appreciated that this section is used only to enhance the understanding of the reader with respect to the present disclosure, and not as admissions of the prior art.

[0003] A virtual environment refers to a digital simulation of a real world having various virtual spaces, such as malls, clubs, gaming zones, restaurants, bar, etc. created by computer technology, where users can interact, engage, and experience immersive activities. This can include virtual reality (VR), augmented reality (AR), metaverses, online gaming platforms, social media sites, and other digital spaces. The usage of virtual environments has been increasing exponentially, as they offer a wide range of benefits, such as enhanced collaboration, improved learning experiences, and endless entertainment opportunities. With the advancement of technology and the rise of remote work, virtual events, and social distancing measures, the adoption of virtual environments has accelerated, transforming the way people live, work, and play. As a result, virtual environments have become an integral part of modern life, with millions of users worldwide, and their increasing usage is expected to continue shaping the future of human interaction, entertainment, and innovation.

[0004] Further, in virtual environments where avatars represent users and interact within various virtual spaces, and objects within such virtual spaces, there exists a problem of incapability of providing an immersive experience to the user during interaction of the avatar with the objects. The users are not able to experience or interact with the virtual objects as they would be able to interact / experience in a real environment based on a physical property or characteristics of the object this in turn leads to missing immersive user experience. Further, the conventional solutions are unable to provide accurate and efficient scaling or alignment of a 3D object representation (such as an avatar) in a virtual environment. Further, the conventional solutions are unable to efficiently analyse a 360-degree depth measurement, and surface measurements which may further lead to improper 3D object representation.

[0005] Hence, there exists a need in the art to provide an enhanced solution for three-dimensional (3D) construction of an object in a virtual environment to ensure proper interaction among the objects and accurate construction of the objects in the virtual space and provide an immersive user experience.

[0006] This section is provided to introduce certain aspects of the present disclosure in a simplified form that are further described below in the detailed description. This summary is not intended to identify the key features or the scope of the claimed subject matter.

[0007] An aspect of the present disclosure may relate to a method for three-dimensional (3D) construction of an object in a virtual environment. The method comprises receiving a two dimensional (2D) multi-media stream segment of the object. Then the method comprises obtaining a 3D measurement information and a set of physical properties of the object, based on the 2D multi-media stream segment. Next, the method comprises obtaining a 3D object based on the 3D measurement information and the set of physical properties. Thereafter, the method comprises performing at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, an interaction of the 3D object with one or more virtual objects.

[0008] An aspect of the present disclosure may relate to an electronic device for three-dimensional (3D) construction of an object in a virtual environment, the electronic device comprises memory, and at least one processor connected with each other. The processor is configured to receive a two dimensional (2D) multi-media stream segment of the object. The processor is further configured to obtain a 3D measurement information and a set of physical properties of the object, based on the 2D multi-media stream segment. The processor is further configured to obtain a 3D object based on the 3D measurement information and the set of physical properties. The processor is further configured to perform at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, an interaction of the 3D object with one or more virtual objects.

[0009] An aspect of the present disclosure may relate to a non-transitory computer readable storage medium storing instructions for three-dimensional (3D) construction of an object in a virtual environment. The instructions include executable code which, when executed by a processor of an electronic device may cause the processor to receive a two dimensional (2D) multi-media stream segment of the object. Further, the execution of the instruction may cause the processor to obtain a 3D measurement information and a set of physical properties, of the object, based on the 2D multi-media stream segment. Further, the execution of the instruction may cause the processor to obtain a 3D object based on the 3D measurement information and the set of physical properties. Further, the execution of the instruction may cause the processor to perform at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, a multi-sensory feedback of an interaction of the 3D object.

[0010] The accompanying drawings, which are incorporated herein, and constitute a part of this disclosure, illustrate exemplary embodiments of the disclosed methods and systems in which like reference numerals refer to the same parts throughout the different drawings. Components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Also, the embodiments shown in the figures are not to be construed as limiting the disclosure, but the possible variants of the method and system according to the disclosure are illustrated herein to highlight the advantages of the disclosure. It will be appreciated by those skilled in the art that disclosure of such drawings includes disclosure of electrical components or circuitry commonly used to implement such components.

[0011] FIG. 1 illustrates an exemplary block diagram of a system for three-dimensional (3D) construction of an object in a virtual environment, in accordance with exemplary embodiments of the present disclosure.

[0012] FIG. 2 illustrates an exemplary block diagram of another system for three-dimensional (3D) construction of the object in the virtual environment, in accordance with exemplary embodiments of the present disclosure.

[0013] FIG. 3 illustrates an exemplary block diagram depicting exemplary sub-modules of the system for implementing a process for three-dimensional (3D) construction of the object in the virtual environment, in accordance with exemplary implementations of the present disclosure.

[0014] FIG. 4 illustrates a flow diagram of a method for three-dimensional (3D) construction of the object in the virtual environment, in accordance with exemplary embodiments of the present disclosure.

[0015] FIG. 5 illustrates a flow diagram of an exemplary method for three-dimensional (3D) construction of the object in the virtual environment, in accordance with exemplary embodiments of the present disclosure.

[0016] FIG. 6 illustrates an Activity Management Unit (AMU), in accordance with exemplary embodiments of the present disclosure.

[0017] FIG. 7 illustrates a Flat Plan Surface Mapping, in accordance with exemplary embodiments of the present disclosure.

[0018] FIG. 8 illustrates an effect estimator, in accordance with exemplary embodiments of the present disclosure.

[0019] FIG. 9 illustrates a mesh generator, in accordance with exemplary embodiments of the present disclosure.

[0020] FIG. 10 illustrates mesh generators and a Calibrating and Overlapping Unit, in accordance with exemplary embodiments of the present disclosure.

[0021] FIG. 11 illustrates an Avatar Construction Unit, in accordance with exemplary embodiments of the present disclosure.

[0022] FIG. 12 illustrates a Feedback Generation and Partial Scale Modification Unit, in accordance with exemplary embodiments of the present disclosure.

[0023] FIG. 13 illustrates a flow diagram depicting an exemplary use case of three-dimensional (3D) construction of the object in the virtual environment, in accordance with exemplary embodiments of the present disclosure.

[0024] The foregoing shall be more apparent from the following more detailed description of the disclosure.

[0025] Some of the objects of the present disclosure, which at least one embodiment disclosed herein satisfies are listed herein below.

[0026] It is an object of the present disclosure to provide a method and a system for three-dimensional (3D) construction of an object in a virtual environment.

[0027] It is an object of the present disclosure to provide a solution for a complete 360-degree depth measurement and surface measurements for correct alignment and / or scaling of the object within the virtual environment.

[0028] It is an object of the present disclosure to provide a solution which provides scaling and / or aligning of the 3D geometrical user body representation within the virtual environment for interaction with the virtual objects, and vice versa.

[0029] It is an object of the present disclosure to provide a solution which provides interaction based on physical property or characteristic of the objects in the virtual space.

[0030] It is an object of the present disclosure to provide a solution for providing multi-sensory feedback generated in a virtual environment to user(s) in real environment based on object characteristics.

[0031] It is an object of the present disclosure to provide a solution for modifying object characteristic in a virtual environment based on various parameters such as actions or expressions of the user / avatar / user body representation etc.

[0032] In the following description, for the purposes of explanation, various specific details are set forth in order to provide a thorough understanding of embodiments of the present disclosure. It will be apparent, however, that embodiments of the present disclosure may be practiced without these specific details. Several features described hereafter may each be used independently of one another or with any combination of other features. An individual feature may not address any of the problems disclosed above or might address only some of the problems disclosed above.

[0033] The ensuing description provides exemplary embodiments only, and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the ensuing description of the exemplary embodiments will provide those skilled in the art with an enabling description for implementing an exemplary embodiment. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosure as set forth.

[0034] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, circuits, systems, processes, and other components may be shown as components in block diagram form in order not to obscure the embodiments in unnecessary detail.

[0035] It should be noted that the terms "first", "second", "primary", "secondary", "target" and the like, herein do not necessarily denote any order, ranking, quantity, or importance, but rather may be used to distinguish one element from another. A person skilled in the art would appreciate utilisation of these terms based on the context and non-limitation to any reference to any order, ranking, quantity, or importance.

[0036] Also, it is noted that individual embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations may be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed but could have additional steps not included in a figure.

[0037] The word "exemplary" and / or "demonstrative" is used herein to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. In addition, any aspect or design described herein as "exemplary" and / or "demonstrative" is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art. Furthermore, to the extent that the terms "includes," "has," "contains," and other similar words are used in either the detailed description or the claims, such terms are intended to be inclusive in a manner similar to the term "comprising" as an open transition word without precluding any additional or other elements.

[0038] As used herein, a "processing unit" or "processor" or "operating processor" includes one or more processors, wherein processor refers to any logic circuitry for processing instructions. A processor may be a general-purpose processor, a special purpose processor, a conventional processor, a digital signal processor, a plurality of microprocessors, one or more microprocessors in association with a Digital Signal Processing (DSP) core, a controller, a microcontroller, Application Specific Integrated Circuits, Field Programmable Gate Array circuits, any other type of integrated circuits, etc. The processor may perform signal coding data processing, input / output processing, and / or any other functionality that enables the working of the system according to the present disclosure. More specifically, the processor or processing unit is a hardware processor.

[0039] As used herein, "a user equipment", "a user device", "a smart-user-device", "a smart-device", "an electronic device", "a mobile device", "a handheld device", "a wireless communication device", "a mobile communication device", "a communication device" may be any electrical, electronic and / or computing device or equipment, capable of implementing the features of the present disclosure. The user equipment / device may include, but is not limited to, a mobile phone, smart phone, laptop, a general-purpose computer, desktop, personal digital assistant, tablet computer, wearable device or any other computing device which is capable of implementing the features of the present disclosure. Also, the user device may contain at least one input means configured to receive an input from unit(s) which are required to implement the features of the present disclosure.

[0040] As used herein, "storage unit" or "memory unit" refers to a machine or computer-readable medium including any mechanism for storing information in a form readable by a computer or similar machine. For example, a computer-readable medium includes read-only memory ("ROM"), random access memory ("RAM"), magnetic disk storage media, optical storage media, flash memory devices or other types of machine-accessible storage media. The storage unit stores at least the data that may be required by one or more units of the system to perform their respective functions.

[0041] As used herein "interface" or "user interface" refers to a shared boundary across which two or more separate components of a system exchange information or data. The interface may also be referred to a set of rules or protocols that define communication or interaction of one or more modules or one or more units with each other, which also includes the methods, functions, or procedures that may be called.

[0042] All modules, units, components used herein, unless explicitly excluded herein, may be software modules or hardware processors, the processors being a general-purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASIC), Field Programmable Gate Array circuits (FPGA), any other type of integrated circuits, etc.

[0043] Furthermore, as used herein, a "processing unit" or "processor" or "operating processor" may include one or a plurality of processors. At this time, one or a plurality of processors may be a general-purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and / or an AI-dedicated processor such as a neural processing unit (NPU).

[0044] One or more of the plurality of modules may be implemented through an AI model. A function associated with AI may be performed through the non-volatile memory, the volatile memory, and the processor.

[0045] The one or the plurality of processors control the processing of the input data in accordance with a predefined operating rule or artificial intelligence (AI) model stored in the non-volatile memory and the volatile memory. The predefined operating rule or artificial intelligence model is provided through training or learning.

[0046] Here, being provided through learning means that, by applying a learning algorithm(s) to a plurality of learning data, a predefined operating rule or AI model of a desired characteristic is made. The learning may be performed in a device itself in which AI according to an embodiment is performed, and / or may be implemented through a separate server / system.

[0047] The AI model may consist of a plurality of neural network layers, such as long short-term memory (LSTM) layers. Each layer has a plurality of weight values and performs a layer operation through calculation of a previous layer and an operation of a plurality of weights. Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GAN), and deep Q-networks.

[0048] The learning algorithm is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0049] Also, as used herein, a virtual environment may refer to a digital or a virtual space provided via a networked application that allows a user to interact with one or more virtual objects in such virtual environment. The virtual environment may be created for example by combining various technologies such as Artificial Intelligence (AI), Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), etc. to allow people to access the virtual world. For instance, AR technologies can integrate virtual objects into the real world. Similarly, VR technology allows users to experience 3D virtual environments or 3D reconstructions using 3D computer modelling. The virtual environment may also refer to virtual worlds in which users represented by avatars interact, usually in 3D and may be focused on social and economic connection.

[0050] As used herein, an object may refer to a visual representation of a computer-generated entity within the virtual environment and the object appears to exist in the virtual environment. The virtual objects may be three-dimensional objects having depth. The virtual objects may be different kind of objects such as a user representation in terms of avatar, or commonly known objects such as shoes, apparels, etc.

[0051] As used herein, an avatar may refer to a visual representation of a character or a user which is controlled by the user. The avatar more specifically is a 3D-representation of the character / user. The avatar may be customizable and may be able to perform a variety of functions in the virtual environment.

[0052] As used herein, "construction" or "3D construction" of objects may refer to a process involving the formation, refinement, scaling, alignment, and finalisation of the objects in the virtual environment. In the present disclosure there may be various implementations for construction of the objects and the "construction" or "3D construction" depending on a use case may include one or more or all such processes, techniques, and configurations for formation, refinement, scaling, alignment, and finalisation of the objects, as may be appreciated by a person skilled in the art.

[0053] As used herein, "scaling" may refer to techniques used for altering the size and shape of the objects within the virtual environment. For example, the scaling may include increasing or decreasing size of the object based on required parameters. It may be understood that depending on use cases the conventional techniques used for scaling may also be applied while implementing the solutions provided by the present disclosure.

[0054] As used herein, "alignment" may refer to techniques used for management of positioning of certain features of the objects in the virtual environment. For example, the alignment may include the alignment of a mesh configuration of the objects or one or more portions of the objects, wherein the mesh configuration represents a structural built that for defining shapes with height, width and depth uses reference points in X, Y and / or Z axes. It may be noted that depending on the use cases the conventional techniques used for alignment may also be applied while implementing the solutions provided by the present disclosure.

[0055] As disclosed in the background section above, the current known solutions have several shortcomings. The present disclosure aims to overcome the above-mentioned and other existing problems in this field of technology by providing a solution for three-dimensional (3D) construction of an object in a virtual environment. As provided by the present solution, firstly, a 2D multi-media stream segment of an object is received, and a 3D measurement information along with a set of physical properties of the object are then captured based on the received 2D multi-media stream segment. Then, based on the captured information, a 3D object is created in the virtual environment and thereafter depending on use cases scaling operations and alignment operations may be performed on the 3D object for better creation of the 3D object. This efficient creation of the 3D object in the virtual environment further helps in providing to a user, a multi-sensory feedback of an interaction of the 3D object with one or more other objects present in the virtual environment. Additionally, a user response from the user may be received in response to the multi-sensory feedback. This user response may also be considered to modify the 3D object in the virtual environment.

[0056] Hereinafter, exemplary embodiments of the present discourse will be described with reference to the accompanying drawings.

[0057] Referring to FIG. 1, an exemplary block diagram of a system 100 for three-dimensional (3D) construction of the object in the virtual environment, in accordance with exemplary embodiments of the present disclosure is shown. The system 100 comprises at least one memory 102, and at least one processor 104. Also, all of the components / units of the system 100 are assumed to be connected to each other unless otherwise indicated below. Also, in FIG. 1 only a few units are shown, however, the system 100 may comprise multiple such units, or the system 100 may comprise any such number of said units, as required to implement the features of the present disclosure. Further, in an implementation, the system 100 may reside in and / or connected to and / or in communication with a user device (may also be referred herein as a user equipment or a UE) to implement the features of the present disclosure. In an implementation, the system 100 may reside in a server.

[0058] In operation, for three-dimensional (3D) construction of the object in the virtual environment, the processor 104 is configured to receive a two dimensional (2D) multi-media stream segment. The 2D multi-media stream segment is associated with the object. In a preferred implementation the object is a user, therefore in such implementation a 2D multi-media stream segment of the user is received. It is pertinent to note that the object here is not limited to the "user" and any other object such as an animal, a vehicle or any object as appreciated by a person skilled in the art may be considered.

[0059] In one exemplary implementation of the present disclosure, the 2D multi-media stream segment may be received based on a predefined time frame. The predefined time frame may be determined based on one or more activities related to the object. More specifically, the predefined time frame may a time period during which the one or more activities related to the object are being performed. For example, a 2D multi-media stream segment may be a portion of a video (e.g., at least one frame), wherein such portion includes a video of an activity of a user, such as the user wearing a shoe.

[0060] Further, the processor 104 may be configured to capture a 3D measurement information and a set of physical properties of the object. The capturing of 3D measurement information and the set of physical properties of the object may be based on the 2D multi-media stream segment.

[0061] In some implementations of the present disclosure, the 3D measurement information may be captured based on a plurality of depth maps that may be generated based on a depth estimation. The depth estimation may include but not limited to a trough and valley estimation, a shadow estimation, a blur estimation, a shading estimation, a movement estimation, a triangulation mapping estimation, or a combination thereof.

[0062] Furthermore, the 3D measurement information includes but not limited to at least one of a 3D flat plane mapping, a surface mesh, a direction ratio, a direction cosine, a distance formula, a midpoint formula, and a section formula. Also, the set of physical properties includes but not limited to a roughness value, a softness value, a hardness value, a temperature value, or a combination thereof.

[0063] Then, based on the 3D measurement information and the set of physical properties that were captured, the processor 104 is configured to create a 3D object.

[0064] Continuing further, on creation of the 3D object, the processor 104 in an implementation is configured to perform a scaling operation and / or an alignment operation on the 3D object in the virtual environment. The scaling operation and / or the alignment operation is performed on the 3D object using at least one of an overall surface mesh and a volume mesh. The overall surface mesh and / or the volume mesh is generated based at least on at least one of a combination of two or more depth maps from the plurality of depth maps and an overlapping of two or more depth maps from the plurality of depth maps.

[0065] The creation of the 3D object facilitates for providing, to a user, a multi-sensory feedback of an interaction of the 3D object with one or more virtual objects in the virtual environment. Also, the interaction of the 3D object with the one or more virtual objects may be based on the set of physical properties, e.g., the roughness value, the softness value, the hardness value, and / or the temperature value related to the object. Moreover, the interaction may be based on one or more physical properties of either or both of the object and the one or more virtual objects. For example, in a virtual environment an interaction between an avatar (i.e., a 3D object) of a user (i.e., an object) with a motor bike (i.e., a virtual object) may be provided based on one or more physical properties related to the user and / or the motor bike.

[0066] In exemplary implementations of the present disclosure, the performing of the scaling operation and / or the alignment operation may be based on one or more virtual objects in the virtual environment. The 3D object may be adjusted based on the performing the at least one of the scaling operation and the alignment operation. Considering the abovementioned example where an interaction between the avatar and the motor bike is to be provided in the virtual environment, in this implementation, the avatar may be scaled and / or aligned based on one or more properties of the motor bike. Further based on such scaling and / or adjustment, the avatar (i.e., the 3D object) is then adjusted in the virtual environment to create a suitable avatar. It is pertinent to note that the present disclosure is not limited to the adjustment of the 3D object and a person skilled in the art would appreciate that depending on use cases the one or more virtual objects may also be adjusted based on one or more properties of the 3D object.

[0067] Also, in some implementations of the present disclosure, the multi-sensory feedback may include but not limited to sound-based feedback, touch based feedback, visual feedback, emotion based feedback, action based feedback, vibration based feedback, or a combination thereof.

[0068] More specifically, in one implementation, the processor 104 is configured to provide to the user, the multi-sensory feedback of the interaction of the 3D object with one or more virtual objects in the virtual environment. Then, in such implementations, the processor 104 is configured to receive from the user, a user response to the provided multi-sensory feedback. Then the processor 104 is configured to create a modified 3D object in the virtual environment based on the user response to the provided multi-sensory feedback. More specifically, for modifying the 3D object in the virtual environment, the processor 104 may be configured to provide to the user, the multi-sensory feedback of the interaction based on at least one of a set of characteristics of the one or more virtual objects, and the set of physical properties. The set of characteristics of the one or more virtual objects include but not limited to a set of physical characteristics of the one or more virtual objects, and wherein the set of physical characteristics comprises at least one of a roughness characteristics, a softness characteristics, a hardness characteristics, and a temperature characteristics. Thereafter, the processor 104 is configured to receive, from the user, the user response to the provided multi-sensory feedback. Then, based on the multi-sensory feedback and / or the user response, the processor 104 is configured to modify a set of characteristics of the 3D object and / or the set of characteristics of the one or more virtual objects. Considering the abovementioned example where an interaction between the avatar and the motor bike is provided in the virtual environment, in this implementation, once the avatar interacts with the motor bike say the avatar rides the motor bike at 40 kmph speed, a multi-sensory feedback is provided to the user. The multi-sensory feedback may be provided, for example, as an output in form of a sound of engine of the motor bike, a vibration corresponding to the speed of the motor bike, and a happy feeling of the avatar. Further, the user based on the multi-sensory feedback may provide a user feedback such as an information indicating, the emotion of the user is happy and relaxed, generally in a real environment when the user drives the motor bike at 40kmph speed. Thereafter, based on the user feedback, the avatar is modified in the virtual environment to depict the emotion and expression as happy and relaxed in place of only the happy emotion.

[0069] Furthermore, the process of three-dimensional (3D) construction of the object in the virtual environment is further explained below in the description of FIG. 2 - FIG. 4.

[0070] Referring to FIG. 2, an exemplary block diagram of a system 200 for three-dimensional (3D) construction of the object in the virtual environment, in accordance with exemplary embodiments of the present disclosure is shown. Further, the system 200, in an implementation, comprises the exemplary modules to implement one or more features of the present disclosure. These exemplary modules as shown in FIG. 2, in an implementation, may be implemented by the processor 104 of the system 100.

[0071] Referring to FIG. 3, an exemplary block diagram 300 depicting exemplary sub-modules of the system 200 for implementing the process for three-dimensional (3D) construction of the object in the virtual environment, in accordance with exemplary embodiments of the present disclosure is shown. These exemplary sub-modules as shown in FIG. 3, in an implementation, may be implemented by the processing unit 104 of the system 100.

[0072] It may be noted that the FIG. 2 and the FIG. 3 may be referred together and are explained in conjunction with each other in the foregoing description.

[0073] As shown in FIG. 2, the system 200 comprises an activity manager and scale analyser unit (AMSU) 202, a 3D Effect Estimator and Surface Mapping Unit (3D EE-SMU) 204, a Volume and Depth Mesh Generator Unit (VDMGU) 206, a Geometrical Scaling and Alignment Unit (GSAU) 208, an Interactive Feedback and Re-alignment Unit (IFRU) 210, and a Database 212. Each of these modules may be explained in detail with reference to one or more figures in the forthcoming description. Further, for three-dimensional (3D) construction of the object in the virtual environment, other associated software components may also be used, wherein these other associated software components may be used in conjunction with the system 100 and / or the system 200.

[0074] Further, in accordance with the present disclosure, it is to be acknowledged that the functionality described for the various components / units can be implemented interchangeably. While specific embodiments or implementations may disclose a particular functionality of these units for clarity, it is recognized that various configurations and combinations thereof are within the scope of the disclosure. The functionality of specific units as disclosed in the disclosure should not be construed as limiting the scope of the present disclosure. Consequently, alternative arrangements and substitutions of units, provided they achieve the intended functionality described herein, are considered to be encompassed within the scope of the present disclosure.

[0075] The activity manager and scale analyser unit (AMSU) 202 may be a component used for tracking an activity in a multimedia stream i.e., the 2D multi-media stream segment. Also, the AMSU 202 may be used for determination of a scale of object(s) in a video frame of the multimedia stream with respect to a scale of the object(s) in real environment. The AMSU 202 may comprise a real-time multi-media stream generation unit (MMS GU) 302, an activity management unit (AMU) 304, and a scale analyser unit (SAU) 306. The activity management unit 304 may utilize an artificial intelligence model and / or a model trained using machine learning techniques for detection of activity(activities) of the objects e.g., user's body activity, in a multi-media stream, and the activity management unit 304 may also be able to specify a type of activity being performed. Further, the MMS GU 302 may be responsible for generation of the real-time stream with respect to various particular user scenarios using device cameras, augmented reality (AR) / virtual reality (VR) / mixed reality (MR), and / or such other techniques. Further, the AMU 304 and the SAU 306 may analyse the activity and the scale of objects in the multimedia stream.

[0076] The 3D Effect Estimator and Surface Mapping Unit (3D EE-SMU) 204 is a component used for determining a mapping of the object with a direction of a flat surface which may be used for capturing minute details and features that may be required for construction / reconstruction of 3D objects. The 3D EE-SMU 204 may comprise a feature extraction unit (FEU) 308, and a depth analyser unit (DAU) 310. The DAU 310 may comprise a flat plane surface mapping unit 312, an AI model 314, and an effect estimator 316. The FEU 308 may be responsible for extraction of features from the multi-media streams, the features may be needed for reconstruction of 3D objects. The flat plane surface mapping unit 312 may be used for 3D planar mapping of the object with a flat plane surface to identify minute troughs and valleys of the surface. Further, the effect estimator 316 may be used to capture the effects such as shade, shadow, blur, and / or motion etc. to compute the depth of the object.

[0077] The Volume and Depth Mesh Generator Unit (VDMGU) 206 may be a component used for generating a surface mesh and volume mesh which may be responsible for accurate depth cue mapping and for generating more realistic 3D object. The VDMGU 206 may comprise a surface mesh generator 318, a depth mesh generator 320, and a calibrating and overlapping unit 322 comprising a volume mesh generator 324. The surface mesh generator 318 and the depth mesh generator 320 may generate the volume mesh to cover the overall minute depth of object, the surface and its build. The calibrating and overlapping unit 322 may comprise the volume mesh generator 324 which is used to overlap and calibrate different planar mesh for building and obtaining the actual volume mesh for the visual representation of the object.

[0078] The Geometrical Scaling and Alignment Unit (GSAU) 208 may be a component used for geometrical scaling and alignment of the object for example, the 3D object body representation within the virtual environment for interaction with other objects. The GSAU 206 may comprise a Texture Construction Unit (TCU), and colour, contour and contrast unit 326, a 3D object construction unit (Avatar construction unit: ACU) 328, and an alignment and scaling unit 330. The TCU, colour, contour and contrast unit 326, and the 3D object construction unit 328 may be used to paint the 3D mesh for the visual representation of the object such as an avatar. The alignment and scaling unit 330 ensures the alignment and scaling of the objects, and mesh are according to the desired scale.

[0079] The Interactive Feedback and Re-alignment Unit (IFRU) 210 may be a component used for providing interaction such as modifying certain characteristics of the object based on user actions / expressions in cases where the type of interaction / feedback may be based on the characteristics of the object. The IFRU 210 may comprise a user experience unit 332, a monitoring unit 334, a feedback generation unit 336, and a scale modification unit 338. The user experience unit 332 and the monitoring unit 334 enables an immersive experience of the 3D object with the virtual object(s) in the virtual environment. The feedback generation unit 336, and the scale modification unit 338 may be responsible for generation of feedback based on the integrated interactive user experience and then may utilise such feedback for further training of the AI based models and improvements in 3D construction of the object.

[0080] The Database 212 may be configured to store data associated with the object, 3D object, the 2D multi-media stream segment, predefined time frame, the 3D measurement information, the set of physical properties of the object, depth maps, depth estimation, other virtual object(s), multi-sensory feedback of the interaction of the 3D object, the set of characteristics of the 3D object, the overall surface mesh, the volume mesh, etc.

[0081] Referring to FIG. 4 a flow diagram of a method 400 for three-dimensional (3D) construction of the object in the virtual environment is illustrated in accordance with exemplary implementations of the present disclosure. In an implementation the method 400 may be performed by the system 100. Further, in an implementation the method 400 may be performed by the system 200. Further, in an implementation, the method 400 may be performed by the system 100 in conjunction with the system 200. The method 400 as depicted in FIG. 4 may start at step 402.

[0082] Referring to FIG. 5, a flow diagram of a method 500 for three-dimensional (3D) construction of the object in the virtual environment is illustrated in accordance with exemplary implementations of the present disclosure. In an implementation the method 500 may be performed by the system 100. Further, in an implementation the method 500 may be performed by the system 200. Further, in an implementation, the method 500 may be performed by the system 100 in conjunction with the system 200. The method 500 as depicted in FIG. 5 starts at step 502.

[0083] It may be noted that the FIG. 4 and FIG. 5 are explained in conjunction with each other in the foregoing description for explanation of the solutions provided by the present disclosure.

[0084] Referring to FIG. 4, the method for three-dimensional (3D) construction of the object in the virtual environment may include operations 404 to 410. The method for three-dimensional (3D) construction of the object in the virtual environment is not limited to that shown in FIG. 4. In one or more embodiments, additional operations not shown in FIG. 4 may be included or some operations may be omitted.

[0085] In operation, at step 404, the method 400 may involve receiving a two dimensional (2D) multi-media stream segment of the object. In the exemplary implementation where the method 400 may be performed by the system 100, then such step may be performed by the processor 104. In the exemplary implementations where the method 400 may be performed by the system 200, then such step may be performed by the Activity Manager and Scale Analyzer Unit (AMSU) 202 or by the MMS GU 302. The 2D multi-media stream segment of the object may refer to a multimedia file containing a stream showing a performance of an activity by the object such as a user itself or an animal or any other object as appreciated by a person skilled in the art. The multimedia file containing a stream showing a performance of an activity by the object may be obtained from, but are not limited to, VR device, mobile phone, cameras, etc. It is pertinent to note that a person skilled in the art would appreciate that in the present disclosure the object for which the 3D object is created is referred as a user at many instances of the present disclosure, however the same is for illustrative purposes and the present disclosure is not limited thereto. The 2D multi-media stream segment may in an example refer to a video illustrating an action performed by the user. For example, the 2D multi-media stream segment may feature birds chirping, fan running, and a user activity such as wearing shoes / accessory / clothes, etc. Then from such 2D multi-media stream segment, noise may be eliminated and the portion relevant for the user activity may be extracted. Further, as shown in FIG. 5, the 2D multi-media stream may be provided to the AMU 204 and then the SAU 206 to analyse the activity being performed within the stream and analyse certain scaling requirements which helps in calculation of the scaling requirements through a scale parameter calculator 307.

[0086] Referring to FIG. 6, for analysing the activity being performed within the 2D multi-media stream, an AI model for activity detection of the object say user's body may be implemented which may enable specifying the type of activity being performed. In an example, a multimedia segment length (SL), and a user input activity (IA) (for example, a hand movement), may be used for creating a function f(a) to calculate different activities in the 2D multimedia stream. Accordingly based on a priority of an activity as per the user input, an aggregated activity chunk may be generated for depicting the type of activity being performed and the order of the activity being performed. The f(a) may be calculated based on Pa(f(a), IA, SL), and aggregated activity chunk= f(Pa(f(a), IA, SL), SL). For example, the different activities in the 2D multimedia stream may include birds chirping, fan running, user activity (wearing shoes, bangles, caps, shirts, etc.). The AI model can detect the user's body activity among the different activities in the 2D multimedia stream, and aggregated activity chunks may be generated. The aggregated activity chunks may be stored in storage (e.g., cloud storage).

[0087] For analysing the scale, the scale analyser unit 306 may utilise a generic object identifier which utilises the 2D multimedia stream segment for identifying generic objects such as switches, fans, bulbs, etc. Then, in an implementation, using an inclination angle calculation and scaling unit (ICSU), inclination angle of generic object is calculated corresponding to the front face camera and the inclination angle is initialised i.e., set to zero. Then, a generic height mapper and / or a generic scale calculator may be implemented for calculating a height of the generic object and for calculating the scaling requirements. Also, a human body identifier and a background depth detector may be utilised for identifying the body of the object say user and calculate the depth of the background from the user. Then, using a masked region-based convolutional neural network (R-CNN model) and a scale generator, scaling requirements are analysed corresponding to the depth of the background. Thereafter, an aggregated scale requirements are calculated which may be very close to reality. The aggregated scale requirements may be calculated by an object aggregated scale calculator based on generic and object body scale parameters.

[0088] In one exemplary implementation of the present disclosure, the 2D multi-media stream segment may be received based on a predefined time frame. The predefined time frame may be determined based on one or more activities related to the object. The predefined time frame may refer to a time period during which the one or more activities related to the object is being performed. For example, a required video segment may be where a user is tying a shoe (e.g., in the last 40 sec segment), such required video segment may be received based on the relevant action for tying the shoe such as only for the last 40 second segment.

[0089] Then, based on the 2D multi-media stream segment, at step 406, the method 400 may involve capturing a 3D measurement information and a set of physical properties of the object. In the exemplary implementation where the method 400 may be performed by the system 100, then such step may be performed by the processor104. In the exemplary implementations where the method 400 may be performed by the system 200, then such step may be performed by the 3D Effect Estimator & Surface Mapping Unit (3D EE-SMU) 204 or by the Feature extraction unit (FEU) 308. The 3D measurement information may refer to data for three-dimensional measurements associated with shape and size of the objects, such as the length, width, and the height of the object.

[0090] In an example, for capturing the features of the 2D multi-media stream segment for 3D reconstruction the edges, faces, vertices, curved surfaces, lateral surfaces, and volume of the object is required to be analysed. Using the feature extraction unit 308 of the 3D EE-SMU 204 features from single frame and / or multi-frames of the 2D) multi-media stream segment may be extracted. From such extraction, depth cues comprising linear perspective, atmospheric scattering, and shape from shading may be extracted. Depth cues comprising binocular disparity, motion parallax, an image blur, silhouette, and structure from motion may also be extracted. The flat plane surface mapping unit 312 may be used for estimation of physical features and N plane mapping along different directions, for 360-degree planar mapping to estimate the minute mapping for better reconstruction. Also, the effect estimator 316 may be used for computing depth estimation on the basis of various effects such as light direction change, multiple shadow generation, focus and structure motion analysis and runtime effect change for generating a 3D physical boundary.

[0091] In one of the exemplary implementations of the present disclosure, the 3D measurement information may include a 3D flat plane mapping, a surface mesh, a direction ratio, a direction cosine, a distance formula, a midpoint formula, a section formula, or a combination thereof. The 3D flat plane mapping may refer to a mapping of the object with the flat surface and / or a ground surface which may be used as a base for orientation of the objects in the virtual environment. The surface mesh may refer to a mesh structure used for formation of different objects within the virtual environment. The direction ratio may refer to a ratio of coordinates which may be used for positioning of the objects within the virtual space. The direction cosine may refer to a cosine of the angles between a vector and the coordinate axes. The distance formula may comprise information associated with the coordinates of a first point and the coordinates of the second point which may be used for calculating the distance between the objects in the virtual space. The midpoint formula may comprise information associated with middle points of line segment, which may be used for getting information associated with midpoints of the objects The section formula may comprise the information associated with an intersecting line dividing two or more lines which may be used for calculating an intersection of one object and another object in the virtual space.

[0092] Referring to FIG. 7, in an example, for flat plane mapping, N directional planes may be used to capture the minute details related to planes taking the planar measurement of the object say the user captured based on the following formula: Pdir L ,1 <=L<= n. Then, on calculation of the basic layout via each plane using the formula, the cross-sectional and total result may be computed using the following formula: Result R = R(Pdir 1) + R(Pdir 2) + R(Pdir 3) + ....... + R(Pdir n). Based on the N directional planes, and flat plane mapping, tentative boundary of the 3D object may be generated on the basis of flat plane scalar mapping. If there are missing details, a point estimation model may be used to estimate points to generate missing details (edges, faces, vertices, curved surfaces, lateral surfaces).

[0093] In an exemplary implementation of the present disclosure, the set of physical properties may include a roughness value, a softness value, a hardness value, a temperature value, or a combination thereof. In another exemplary implementation of the present disclosure, the interaction of the 3D object with the one or more virtual objects may be based on the set of physical properties. The roughness value may comprise an information associated with a rough texture of the object. The softness value may comprise an information associated with a soft texture of the object. The hardness value may comprise information associated with hard texture of the object. The temperature value may comprise information associated with a temperature of the object within the virtual environment.

[0094] Referring to FIG. 8, in an exemplary implementation of the present disclosure, the effect estimator 316 may capture the effects to compute the depth underlying user object (shade, shadow, blur, motion). The effect estimator 316 may obtain shape from shading. For example, the effect estimator 316 may obtain brightness variation of the image and estimate depth using brightness gradients and perform depth map reconstruction. The effect estimator 316 may obtain depth from shadows. For example, the effect estimator 316 may perform shadow detection using ML (Machine Learning) or threshold and perform shadow analysis to compute shadow position or deformation, and perform depth estimation using Geometric principles and known light source pose. The effect estimator 316 may obtain depth from focus. For example, the effect estimator 316 may perform blur analysis to obtain blur gradients and estimate depth using blur gradients. The effect estimator 316 may obtain structure from motion. For example, the effect estimator 316 may detect and match features across images. The features may include Brightness / Intensity Gradients, Surface Normal, shadow Regions, shadow Boundaries, blur amount, focus measure. The effect estimator 316 may perform camera Pose Estimation and use triangulation to reconstruct the 3D structure. The effect estimator 316 may output Multiple Depth maps and / or Basic structure grid.

[0095] In some implementations of the present disclosure, the 3D measurement information may be captured based on a plurality of depth maps that may be generated based on a depth estimation. In some exemplary implementations, the depth estimation may be a trough and valley estimation, a shadow estimation, a blur estimation, a shading estimation, a movement estimation, a triangulation mapping estimation, or a combination thereof. The trough and valley estimation may refer to an estimation for the 3D measurement information for areas of protrusions and depressions. The shadow estimation may refer to an estimation for the 3D measurement information for shadow of different objects within the virtual environment and may also include analysis of position and deformation using geometric principles and

[0096] position of known light sources. The shadow estimation may be done using machine learning techniques or threshold-based technique. The blur estimation may refer to an estimation of an amount of blurring of different objects by applying blur gradient that may be required for objects in the background. The shading estimation may refer to an estimation associated with the type and color of shades of the objects within the virtual environment for example brightness variation estimation. The movement estimation may refer to an estimation for a movement of the object in a particular direction and may be done by camera pose estimation and / or detecting and matching features across frames. In other examples, the features required to be extracted may include brightness / intensity gradients, normal surface, shadow regions, shadow boundaries, blur amount, focus measure. The triangulation mapping estimation may refer to an estimation of the 3D positioning of distinctive map points of the objects within the virtual environment and may be used for identifying the projections of the objects. By using such 3D measurement information, the formation of the objects and placement of such objects may be done within the virtual environment. Such 3D measurement information is required to be calculated for creating an immersive experience of the user within the virtual environment.

[0097] Once the 3D measurement information and the set of physical properties are captured, then at step 408, the method 400 may involve creating a 3D object based on the 3D measurement information and the set of physical properties that were captured. In the exemplary implementation where the method 400 may be performed by the system 100, then such step may be performed by the processor 104. In the exemplary implementations where the method 400 may be performed by the system 200, then such step may be performed by the Volume & Depth Mesh Generator Unit (VDMGU) 206. Further, as shown in FIG. 5 in the exemplary implementations where the method 400 may be performed by the system 200, then such step may be performed by the surface mesh generator 318, the depth mesh generator 320, and the volume mesh generator 324. The 3D object may refer to an object or a visual representation of an object having 3D characteristics and may refer to different objects within the virtual environment such as the user representation (avatar), apparels and accessories worn by the avatar, and other interactive or non-interactive objects that may be present within the virtual environment.

[0098] Referring to FIG. 9, in an example, based on the received 2D multi-media stream (e.g., segmented activity stream, scaling parameters) and the captured 3D measurement information such as the N-dimensional surface maps, effect estimation, and the depth maps, the mesh generators (both the surface mesh generator 318 and the depth mesh generator 320) may create a mesh that represent the surface and depth of the object. The calibration and overlapping unit (COU) 322 may be responsible for creating a mesh that fills the interior space of the object.

[0099] Referring to FIG. 10, the mesh generators may obtain output from the effect estimator 316. For example, the mesh generators may obtain depth maps and flat surface mapping points from the effect estimator 316. The depth mesh generator 320 may generate depth mesh after overlapping and calibrating multiple depth maps generated by the effect estimator 316. The surface mesh generator 318 may perform point cloud generation using the flat surface mapping points generated by the effect estimator 316. The surface mesh generator 318 may use Marching Cube algorithms to generate the surface mesh based on cloud point generation. The Marching Cube algorithms are used to extract iso-surfaces based on constant values from three-dimensional grid data. The surface mesh generator 318 may generate final surface mesh after smoothing and optimization. The calibration and overlapping unit (COU) 322 may perform volume point insertion to for improving or constructing a volume mesh (3D mesh) or Delaunay tetrahedralization. The calibration and overlapping unit (COU) 322 may perform global smoothing. Global Smoothing is a method of smoothing that takes into account the overall structure or data, and is used to remove noise and improve quality in mesh processing. The calibration and overlapping unit (COU) 322 may generate final volume mesh from depth mesh and surface mesh.

[0100] Thereafter, at step 410, the method 400 may involve performing at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, a multi-sensory feedback of an interaction of the 3D object. In the exemplary implementation where the method 400 may be performed by the system 100, then such step may be performed by the processor 104. In the exemplary implementations where the method 400 may be performed by the system 200, then such step may be performed by the Geometrical Scaling & Alignment Unit (GSAU) 208, and the Interactive Feedback & Re-alignment Unit (IFRU) 210 using their respective sub-components.

[0101] Referring to FIG. 11, the GSAU 206 may include a Texture Construction Unit (TCU) 326, the 3D object construction unit (Avatar construction unit: ACU) 328, and an alignment and scaling unit (ASU) 330. The GSAU 206 may obtain the final volume mesh from the calibration and overlapping unit (COU) 322 and generate the 3D object (e.g., avatar). The ACU 328 may generate Global texture from the Volume mesh to calibrate with Color Texture. The TCU 326 may create sub texture components for RGB images from multi-media stream. The ACU 328 may perform texture map optimization using the sub texture components. The ACU 328 may generate a refined global texture map after overlapping color sub texture components and global texture. The ACU 328 may create Avatar 1100 with the final volume mesh after alignment and scaling and merging with the texture components.

[0102] In exemplary implementations of the present disclosure, the performing of the scaling operation and / or the alignment operation may be based on one or more virtual objects in the virtual environment. The 3D object may be adjusted based on the performing the at least one of the scaling operation and the alignment operation. The GSAU 206 may scale and align the final volume mesh (3D object) as per real-time. It is pertinent to note that the present disclosure is not limited to the adjustment of the 3D object and a person skilled in the art would appreciate that depending on use cases the one or more virtual objects may also be adjusted based on one or more properties of the 3D object. Also, the method 400 may also involve generating at least one of an overall surface mesh and a volume mesh based at least on at least one of a combination of two or more depth maps from the plurality of depth maps and an overlapping of two or more depth maps from the plurality of depth maps. Also, in some implementation of the present disclosure, the scaling operation and / or the alignment operation on the 3D object may be performed using the overall surface mesh and / or the volume mesh.

[0103] In some exemplary implementations of the present disclosure, the multi-sensory feedback may be sound based feedback, touch based feedback, visual feedback, emotion-based feedback, action based feedback, vibration based feedback, or a combination thereof.

[0104] Thereafter, in one exemplary implementation, the method 400 may also involve providing to the user, the multi-sensory feedback of the interaction of the 3D object with the one or more virtual objects in the virtual environment. Then, in such implementations, responding to such multi-sensory feedback, and from the user, a user response may be received in response to the provided multi-sensory feedback. Then the processor 104 may be configured to create a modified 3D object in the virtual environment based on the user response to the provided multi-sensory feedback.

[0105] More specifically, for modifying the 3D object in the virtual environment, the processor 104 may be configured to provide to the user, the multi-sensory feedback of the interaction based on at least one of a set of characteristics of the one or more virtual objects, and the set of physical properties. Then the processor 104 may be configured to receive, from the user, the user response to the provided multi-sensory feedback. Then, based on the multi-sensory feedback and / or the user response, the processor 104 may be configured to modify a set of characteristics of the 3D object and / or the set of characteristics of the one or more virtual objects. This step is illustrated as step 504 in FIG. 5. Further, at this step, the multi-sensory feedback may be generated and then provided to the user. It may be noted that the method may be terminated after this step in one example. In an example, the user response for the provided feedback may be checked as illustrated in step 506 in FIG. 5. In case the user response is positive feedback, then the method may be terminated. In other cases, where the user response is not positive and portrays certain errors in configuration, then based on such observations in the user response, the modification may be made to the 3D object as also illustrated by step 508 in the FIG. 5.

[0106] Referring to FIG. 12, in certain exemplary implementations of the present disclosure, the processor 104 may monitor the metaverse space and user space in real time. The processor 104 may obtain the set of characteristics of the one or more virtual objects and integration feedback based on the one or more virtual objects as monitoring parameters. The set of characteristics of the one or more virtual objects may be a set of physical characteristics of the one or more virtual objects. The set of physical characteristics may include roughness characteristics, softness characteristics, hardness characteristics, temperature characteristics, or any combination thereof. The roughness value associated with the rough texture of the virtual object(s) may be used to modify the texture. The softness value associated with the soft texture of the virtual object(s) may be used to modify the texture. The hardness value associated with hard texture of the virtual object(s) may be used to modify the texture. The temperature value associated with a temperature of the virtual object(s) within the virtual environment may be used for modifying the temperature. The integration feedback may include feel, comfort level, working functionality, fit etc. According to one embodiment of the present disclosure, feedback may be generated in a virtual environment and communicated to a user in real time via sound, feel, visual and / or haptics (vibratory). The processor 104 may capture user's feedback based on user's voice, physical movements and voice. When the user's feedback is negative feedback, the processor 104 may modify the virtual object or parameters related to the virtual object. For example, the processor 104 may modify a behavior or characteristic or scale, alignment of the virtual object.

[0107] Thereafter, the method 400 and the method 500 terminates at step 412 and step 510 respectively.

[0108] Referring to FIG. 13, a flow diagram illustrating an exemplary use case 1300 of three-dimensional (3D) construction of the object in the virtual environment, is provided, in accordance with exemplary implementations of the present disclosure. As shown in the FIG. 13, the exemplary use case illustration may initiate when a user enters in a virtual environment associated with a shoe store. In such shoe store, the user may select a particular shoe and may want to know how the shoe would look on their avatar. On selection of the shoe, the user may initiate an action of wearing the shoe, for which in the background the action of the user wearing the shoe may be recorded by a camera setup or received from a storage unit and the same may be then used for providing the 2D multimedia stream. Then as depicted in step 1302, the user may try to wear the shoe. However, due to improper synchronisation between the shoe and the avatar, the visual representation of the shoe may appear as a proper fit, however, may not be an appropriate fit due to which the user satisfaction may not be achieved. The implementation of the present disclosure would lead to perfection of the avatar i.e., the 3D object and the 3D object is reconstructed based on the past user finger / hand experience and the multi-sensory feedback from the user. Due to implementation of the present disclosure, the synchronisation between the avatar and the shoe is achieved. Now, when the user places such order in the virtual environment as depicted by step 1304, due to reconstruction of the 3D object and / or the shoe, there will be higher chances of achieving an improved fit of the shoe, thereby leading to improved user satisfaction. Then, at step 1306, when the user tries wearing the shoe physically, the user satisfaction will be improved as depicted in step 1308 of the FIG. 13.

[0109] Yet an aspect of the present disclosure may relate to a non-transitory computer readable storage medium storing instructions for three-dimensional (3D) construction of an object in a virtual environment. The instructions include executable code which, when executed by a processor 104 of a system 100 causes the processor 104 to receive a two dimensional (2D) multi-media stream segment of the object. Further, the execution of the instruction causes the processor 104 to capture a 3D measurement information and a set of physical properties, of the object, based on the 2D multi-media stream segment. Further, the execution of the instruction causes the processor 104 to create a 3D object based on the 3D measurement information and the set of physical properties. Further, the execution of the instruction causes the processor 104 to perform at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, a multi-sensory feedback of an interaction of the 3D object.

[0110] As is evident from the above, the present disclosure provides a technically advanced solution for three-dimensional (3D) construction of an object in a virtual environment. The present solution provides scaling / aligning operations on the three-dimensional representation of the object (such as the user representation) and also utilizes different parameters such as scale differentiation, movements, blurring, shading, shadow, troughs, and valleys to compute overall surface and volume mesh helping in overall construction of the objects. The construction of the objects using the overall surface and volume mesh is varied based on combining and overlapping of multiple depth maps, which results in generating an immersive 3D object. This in turn improves the overall interaction experience with virtual objects leading to increased tendency of realistic objects and realistic experience, which also captures and integrates feedback provided to the user. The present solution further provides technical advantages in terms of real time immersive user experience and provides a proper 3D mapping of objects in virtual environments. Further, the present solution enables high performance while interacting with virtual objects thereby reducing delays and lags. Also, the present disclosure provides high fault tolerance and risk estimation as the 3D object constructed based on the technical features as disclosed in the present disclosure is almost same as real-time user object. Further, the present solution enables Horizontal, Vertical and Depth based Plane mapping to compute major valleys and troughs and enables estimation of different kinds of effects such as shadows, shading, blur, motions, and triangulation.

[0111] According to one embodiment of the present disclosure, a method for three-dimensional (3D) construction of an object in a virtual environment may comprise receiving a two dimensional (2D) multi-media stream segment of the object. The method may comprise obtaining a 3D measurement information and a set of physical properties of the object, based on the 2D multi-media stream segment. The method may comprise obtaining a 3D object based on the 3D measurement information and the set of physical properties. The method may comprise performing at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, an interaction of the 3D object with one or more virtual objects.

[0112] In an exemplary aspect of the present disclosure, the performing the at least one of the scaling operation and the alignment operation may be based on one or more virtual objects in the virtual environment, and wherein the 3D object is adjusted based on the performing the at least one of the scaling operation and the alignment operation.

[0113] In an exemplary aspect of the present disclosure, the 2D multi-media stream segment may be received based on a predefined time frame, wherein the predefined time frame is determined based on one or more activities related to the object.

[0114] In an exemplary aspect of the present disclosure, the 3D measurement information may comprise at least one of a 3D flat plane mapping, a surface mesh, a direction ratio, a direction cosine, a distance formula, a midpoint formula, or a section formula.

[0115] In an exemplary aspect of the present disclosure, the set of physical properties comprises at least one of a roughness value, a softness value, a hardness value, or a temperature value.

[0116] In an exemplary aspect of the present disclosure, the interaction of the 3D object with the one or more virtual objects may be based on the set of physical properties.

[0117] In an exemplary aspect of the present disclosure, the method may comprise providing, to the user, the multi-sensory feedback of the interaction of the 3D object with one or more virtual objects in the virtual environment. The method may comprise receiving, from the user, a user response to the provided multi-sensory feedback. The method may comprise modifying the 3D object in the virtual environment based on the user response.

[0118] In an exemplary aspect of the present disclosure, the modifying the 3D object in the virtual environment may comprise providing, to the user, the multi-sensory feedback of the interaction based on at least one of a set of characteristics of the one or more virtual objects, and the set of physical properties. The method may comprise receiving, from the user, the user response to the provided multi-sensory feedback. The method may comprise modifying at least one of a set of characteristics of the 3D object and the set of characteristics of the one or more virtual objects, based on at least one of the multi-sensory feedback and the user response.

[0119] In an exemplary aspect of the present disclosure, the multi-sensory feedback may comprise at least one of a sound-based feedback, a touch based feedback, a visual feedback, an emotion based feedback, an action based feedback, or a vibration based feedback.

[0120] In an exemplary aspect of the present disclosure, the set of characteristics of the one or more virtual objects may comprise at least a set of physical characteristics of the one or more virtual objects, and wherein the set of physical characteristics may comprise at least one of a roughness characteristics, a softness characteristics, a hardness characteristics, or a temperature characteristics.

[0121] In an exemplary aspect of the present disclosure, the 3D measurement information may be captured based on a plurality of depth maps, wherein the plurality of depth maps may be generated based at least on a depth estimation.

[0122] In an exemplary aspect of the present disclosure, the depth estimation may comprise at least one of a trough and valley estimation, a shadow estimation, a blur estimation, a shading estimation, a movement estimation, and a triangulation mapping estimation.

[0123] In an exemplary aspect of the present disclosure, the method may comprise generating at least one of an overall surface mesh and a volume mesh based on at least one of a combination of two or more depth maps from the plurality of depth maps and an overlapping of two or more depth maps from the plurality of depth maps.

[0124] In an exemplary aspect of the present disclosure, the at least one of the scaling operation and the alignment operation on the 3D object may be performed using at least one of the overall surface mesh and the volume mesh.

[0125] An aspect of the present disclosure may relate to an electronic device for three-dimensional (3D) construction of an object in a virtual environment, the electronic device may comprise memory, and at least one processor connected with each other. The processor may be configured to receive a two dimensional (2D) multi-media stream segment of the object. The processor may be configured to obtain a 3D measurement information and a set of physical properties of the object, based on the 2D multi-media stream segment. The processor may be configured to obtain a 3D object based on the 3D measurement information and the set of physical properties. The processor may be configured to perform at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, an interaction of the 3D object with one or more virtual objects.

[0126] An aspect of the present disclosure may relate to a non-transitory computer readable storage medium storing instructions for three-dimensional (3D) construction of an object in a virtual environment. The instructions may include executable code which, when executed by a processor of an electronic device causes the processor to receive a two dimensional (2D) multi-media stream segment of the object. Further, the execution of the instruction may cause the processor to capture a 3D measurement information and a set of physical properties, of the object, based on the 2D multi-media stream segment. Further, the execution of the instruction causes the processor to create a 3D object based on the 3D measurement information and the set of physical properties. Further, the execution of the instruction may cause the processor to perform at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, a multi-sensory feedback of an interaction of the 3D object.

[0127] According to one embodiment of the present disclosure, a method for three-dimensional (3D) construction of an object in a virtual environment may comprise receiving a 2D multi-media stream segment of the user (e.g. selected based on performing of an activity like trying a shoe in a time frame segment selected); capturing 3D measurement information (e.g. 3D Flat plane mapping, surface mesh, direction ratio, direction cosine, distance formula, midpoint formula, and section formula) of the user body; capturing physical property of the object for a virtual object representation; and scaling / aligning the three-dimensional geometric user body representation (e.g., foot, hands, face, neck etc.) within the virtual environment for interaction with the virtual objects (e.g., bangles, amputated mechanics, shoes etc.). The method may include providing interaction based on physical property object (e.g. roughness, softness or hardness, temperature what else can be); providing integration feedback type based on the object characteristic (e.g. shoes, how it feels like, dog, how it feels like, or virtual pet etc.), wherein multi-sensory feedback is sound, feel, visual and / or vibration. The providing interaction includes modifying object characteristic (object behavior) based on user actions / expressions (e.g., voice, facial).

[0128] According to one embodiment of the present disclosure, the multi-media stream segment is selected based on time frame determined by performing of an activity and detection of multiple movements.

[0129] According to one embodiment of the present disclosure, the scaling / aligning of three-dimensional geometric user body representation is generated using overall surface and volume mesh, wherein the maximum and minimum surface volume threshold is varied based on combining and overlapping of multiple depth maps.

[0130] According to one embodiment of the present disclosure, the integration feedback is generated based on the object characteristic. The method may include modifying virtual object based on user actions / expressions.

[0131] In one embodiment of the present disclosure, almost accurate scale is determined for virtual object reconstruction based on detected activity, 3D measurement and construction of the user body avatar is done as per the minute details considering the 360 degree planar construction, integration feedback type is generated based on physical property object and the object characteristics, and modification, re-alignment and rescaling of virtual object is done as per the integration feedback. In one embodiment of the present disclosure, integrated feedback is generated based on real-time and virtual monitoring, thereby modifying virtual objects to provide a more immersive user experience.

[0132] While considerable emphasis has been placed herein on the disclosed implementations, it will be appreciated that many implementations can be made and that many changes can be made to the implementations without departing from the principles of the present disclosure. These and other changes in the implementations of the present disclosure will be apparent to those skilled in the art, whereby it is to be understood that the foregoing descriptive matter to be implemented is illustrative and non-limiting.

Claims

1.A method for three dimensional (3D) construction of an object in a virtual environment, the method comprising:receiving a two dimensional (2D) multi-media stream segment of the object;obtaining a 3D measurement information and a set of physical properties of the object, based on the 2D multi-media stream segment;obtaining a 3D object based on the 3D measurement information and the set of physical properties; andperforming at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, an interaction of the 3D object with one or more virtual objects.2.The method of claim 1, wherein the performing the at least one of the scaling operation and the alignment operation is based on the one or more virtual objects in the virtual environment, and wherein the 3D object is adjusted based on the performing the at least one of the scaling operation and the alignment operation.3.The method of claim 1 or claim 2, wherein the 2D multi-media stream segment is received based on a predefined time frame, wherein the predefined time frame is determined based on one or more activities related to the object.4.The method of any one of claims 1 to 3, wherein the 3D measurement information comprises at least one of a 3D flat plane mapping, a surface mesh, a direction ratio, a direction cosine, a distance formula, a midpoint formula, or a section formula.5.The method of any one of claims 1 to 4, wherein the set of physical properties comprises at least one of a roughness value, a softness value, a hardness value, or a temperature value.6.The method of any one of claims 1 to 5, wherein the interaction of the 3D object with the one or more virtual objects is based on the set of physical properties.7.The method of any one of claims 1 to 6, the method further comprising:providing, to the user, a multi-sensory feedback of the interaction of the 3D object with the one or more virtual objects in the virtual environment;receiving, from the user, a user response to the provided multi-sensory feedback; andmodifying the 3D object in the virtual environment based on the user response.8.The method as claimed in claim 7, wherein the modifying the 3D object in the virtual environment comprises:providing, to the user, the multi-sensory feedback of the interaction based on at least one of a set of characteristics of the one or more virtual objects, and the set of physical properties,receiving, from the user, the user response to the provided multi-sensory feedback,modifying at least one of a set of characteristics of the 3D object and the set of characteristics of the one or more virtual objects, based on at least one of the multi-sensory feedback and the user response.9.The method as claimed in claim 7, wherein the multi-sensory feedback comprises at least one of a sound based feedback, a touch based feedback, a visual feedback, an emotion based feedback, an action based feedback, or a vibration based feedback.10.The method as claimed in claim 8, wherein the set of characteristics of the one or more virtual objects comprises at least a set of physical characteristics of the one or more virtual objects, and wherein the set of physical characteristics comprises at least one of a roughness characteristics, a softness characteristics, a hardness characteristics, or a temperature characteristics.11.The method of any one of claims 1 to 10, wherein the 3D measurement information is captured based on a plurality of depth maps, wherein the plurality of depth maps are generated based on a depth estimation.12.The method as claimed in claim 11, wherein the depth estimation comprises at least one of a trough and valley estimation, a shadow estimation, a blur estimation, a shading estimation, a movement estimation, or a triangulation mapping estimation.13.The method as claimed in claim 11, further comprising:generating at least one of an overall surface mesh and a volume mesh based on at least one of a combination of two or more depth maps from the plurality of depth maps and an overlapping of two or more depth maps from the plurality of depth maps.14.The method as claimed in claim 13, wherein the at least one of the scaling operation and the alignment operation on the 3D object is performed using at least one of the overall surface mesh or the volume mesh.15.An electronic device for three dimensional (3D) construction of an object in a virtual environment, the electronic device comprising:memory (102); andat least one processor (104) connected the memory (102), the processor (104) configured to:receive a two dimensional (2D) multi-media stream segment of the object;obtain a 3D measurement information and a set of physical properties, of the object, based on the 2D multi-media stream segment;obtain a 3D object based on the 3D measurement information and the set of physical properties; andperform at least one of a scaling operation and an alignment operation on the 3D object in the virtual environment for providing, to a user, an interaction of the 3D object with one or more virtual objects.

Images