To generate interactive and immersive virtual and augmented reality environments that correspond to digital twins of real-world life elements.

Abstract

A method and system for generating a digital twin of a live event in an interactive virtual environment. The system includes a server configured to receive a video data stream and an audio data stream. The server operates a computing module configured to analyze the video data stream and its corresponding metadata, determine the spatial depth between images in the video data stream, align objects in each image to a virtual space coordinate matrix, align a 3D audio field to a virtual space coordinate matrix, and generate a digital twin of the live event based on the images, the corresponding metadata, the spatial depth, a predetermined set of coordinates, and the coordinate matrix. The server is configured to generate multiple digital representations of the live event from multiple directions, including at least one 360-degree digital representation and multiple unidirectional digital representations.

Claims

1. A system for generating a digital twin in an interactive virtual reality environment, wherein the digital twin is compatible with live events, and the system is Multiple video capturing devices connected to a network, Multiple video capturing devices, each for capturing video of the live event, with each captured video having corresponding metadata. Multiple audio recording devices for recording the audio of the aforementioned live event, A server configured to receive data from each video capturing device and each audio recording device. Equipped with, The server operates the computing module, and the module, Analyzing the captured video and the corresponding metadata, Determining the spatial depth between images captured by different video capturing devices, Aligning each captured video to a coordinate matrix in a virtual space, To align the aforementioned audio with the coordinate matrix of the virtual space, To generate a digital twin of at least a portion of the live event based on the captured video, the corresponding metadata, the spatial depth, a predetermined set of coordinates, and the coordinate matrix. It is configured to do the following: The server is configured to generate multiple digital representations of the live event from multiple directions, each digital representation being at least one 360-degree digital representation and multiple unidirectional digital representations, with each digital representation corresponding to a viewing angle within the digital twin in the interactive virtual reality environment. A system in which, during operation, the digital representation and the audio can be transmitted to the devices of users participating in the interactive virtual reality environment of the live event.

2. The system according to claim 1, further configured to receive data from the user's device, wherein the data corresponds to the user's interaction with the virtual reality environment.

3. The system according to claim 2, wherein the computing module is further configured to calculate thresholds for determining which real-life elements are suitable to be in the digital representation of the digital twin in the virtual reality environment.

4. The system according to claim 1, 2, or 3, wherein the server is further configured to transmit the digital representation to the device in real time or with a delay after the live event.

5. The system according to any one of claims 1 to 4, wherein the digital representation, audio recording, corresponding metadata, and corresponding coordinate matrix data are parsed by the system and stored in a database for at least partial on-demand redistribution.

6. The system according to any one of claims 1 to 5, wherein the digital representation, audio recording, corresponding metadata, and corresponding coordinate matrix data are stored in a distributed manner using blockchain technology.

7. The system according to claim 6, wherein a non-fungible token is assigned to at least a portion of the digital twin, which includes the digital representation, the audio recording, the corresponding metadata, and the corresponding coordinate matrix data.

8. The system according to claim 7, wherein the system includes a payment processor adapted for identifying non-fungible tokens purchased by the user and assigning the corresponding non-fungible token rights to the user.

9. The system according to any one of claims 1 to 8, wherein the system assigns non-fungible tokens to the digital twin of the live event.

10. The system according to any one of claims 1 to 9, wherein the server uses artificial intelligence to generate digital representations.

11. A method for generating an interactive virtual reality environment in the metaverse, wherein the environment includes a digital twin of real-life elements, comprising at least one of visual, audio, spatial, and tactical elements, the environment is connected to distributed ledger technology, and the method is The digital twin of the aforementioned real-life elements, (a) capturing video and generating the visual elements, recording audio and generating the audio elements, determining tactical characteristics and generating the tactical elements, and measuring distance and spatial coordinates and generating the spatial elements, (b) Recording corresponding metadata in one of the visual element, the audio element, the spatial element, and the tactical element. It is generated from at least one of the following, The aforementioned digital twin is stored on a server, Assigning a token to at least one of the visual element, the audio element, the spatial element, and the tactical element, Connecting the aforementioned server to a metaverse hosting server, By generating the virtual reality environment corresponding to the digital twin, the digital twin is integrated into the metaverse, To provide the user device with access to the virtual reality environment corresponding to the digital twin. Methods that include...

12. The method according to claim 11, further comprising storing the digital twin on a server in a distributed manner using blockchain technology.

13. The method according to claim 11 or 12, further comprising communicating token data with the user device.

14. The method according to claim 13, wherein the token is a non-fungible token.

15. The method according to any one of claims 11 to 14, wherein measuring distance and spatial coordinates includes determining at least one of the shape of the visual element, the distance between a first visual element and a second visual element, and the velocity associated with the moving visual element.

16. The method according to any one of claims 11 to 15, further comprising connecting a payment processor adapted to identify the purchased tokens and the corresponding rights of the purchased tokens.

17. The method according to any one of claims 11 to 16, wherein providing access to the virtual reality environment includes enabling the user to send the user's digital self to any location within the virtual reality environment.

18. The method according to any one of claims 11 to 17, further comprising generating a portion of the metaverse based on the virtual reality environment of the digital twin.

19. The method according to any one of claims 11 to 18, further comprising generating multiple virtual reality environments corresponding to multiple digital twins.

20. The method according to any one of claims 11 to 19, further comprising integrating multiple digital twins into the virtual reality environment.

21. The method according to any one of claims 11 to 20, further comprising selecting real-world elements for digitization within the digital twin.

22. A method operated by a system for generating a 3D graphical representation in a virtual reality environment, wherein the 3D graphical representation corresponds to elements of real life, the system is connected to a network, and the system includes a video capturing device, a corresponding metadata recording device, an audio recording device, and a server configured to receive data from the video capturing device, the audio recording device, and the metadata recording device, wherein the server operates a computing module configured to analyze the captured video, the recorded audio, and the recorded corresponding metadata, and to digitize the objects shown in the images of the captured video to create the 3D graphical representation of the elements of real life, and the method is, a. Based on the objects shown in the images of the captured video, based on determining the distance between the objects shown in the images of the captured video, and based on the analyzed corresponding metadata, b. Linking audio tracks to elements in the 3D graphical representation, c. Storing the 3D graphical representation in the server, d. Connecting the aforementioned server to a metaverse hosting server, e. Integrating the 3D graphical representation into the metaverse by generating the virtual reality environment corresponding to the 3D graphical representation, f. Providing access to the virtual reality environment to the user's device. Methods that include...

23. A system for generating a digital twin in an interactive virtual reality environment, wherein the digital twin corresponds to elements of real life, the system is connected to a network, and the system is The system comprises a video capturing device, a corresponding metadata recording device, an audio recording device, and a server configured to receive data from the video capturing device, the audio recording device, and the metadata recording device, wherein the server operates a computing module configured to analyze the captured video, the recorded audio, and the recorded corresponding metadata; determine the shapes of objects in the captured video and the distances between the objects in the captured video; and determine the coordinate matrix of a 3D virtual space containing a digital twin of the elements of real life. system.

24. A system for integrating an interactive virtual reality environment corresponding to a digital twin of real-life elements into the metaverse, wherein the system is connected to a network, and the system The system comprises a video capturing device, a corresponding metadata recording device, an audio recording device, and a server configured to receive data from the video capturing device, the audio recording device, and the metadata recording device, wherein the server operates a computing module configured to analyze captured video, recorded audio, and recorded corresponding metadata; determine the spatial depth between the real-world life elements captured by the video capturing device, recorded by the audio device, or recorded by the metadata device; approximate the shapes of the real-world life elements; determine real-world life elements suitable for digitization; and generate digital twins of at least a portion of the life elements, and the server is configured to send data to a metaverse hosting server for integrating the interactive virtual reality environment into the metaverse. system.

25. A system for generating a digital twin in an interactive virtual reality environment, wherein the digital twin is compatible with live events, and the system is Multiple video data streams received over a network, each video stream corresponding to a predetermined set of coordinates in the physical space of the live event and having corresponding metadata, Multiple audio data streams corresponding to the 3D audio field of the aforementioned live event, A server configured to receive the video data stream and the audio data stream. Equipped with, The server operates the computing module, and the module, Analyzing the video data stream and the corresponding metadata, Determining the spatial depth between images in the aforementioned video data stream, Aligning the objects in each image with the coordinate matrix of the virtual space, Aligning the 3D audio field with the coordinate matrix of the virtual space, To generate a digital twin of at least a portion of the live event based on the aforementioned image, the corresponding metadata, the spatial depth, the predetermined set of coordinates, and the coordinate matrix. It is configured to do the following: The aforementioned server, To generate multiple digital representations of the live event from multiple directions, including at least one 360-degree digital representation and multiple unidirectional digital representations, wherein each digital representation corresponds to the viewing angle within the digital twin in the interactive virtual reality environment. Transmitting the aforementioned digital representation and audio to a user device via a network. A system configured to perform the following actions.

26. The system according to claim 25, wherein the system is further configured to receive data from the user device, and the data corresponds to the user's interaction with the virtual reality environment.

27. The system according to claim 26, wherein the computing module is located within the graphical representation of the digital twin in the virtual reality environment and is further configured to calculate thresholds for determining a real-life element suitable for approximating a digital representation of that real-life element.

28. The system according to claim 25, 26, or 27, wherein the digital representation is transmitted to the device in real time or with a delay after the live event.

29. The system according to any one of claims 25 to 28, wherein the digital representation, the audio, the corresponding metadata, and the corresponding coordinate matrix data are parsed by the system and stored in a database for at least partial on-demand redistribution.

30. The system according to any one of claims 25 to 29, wherein the digital representation, the audio, the corresponding metadata, and the corresponding coordinate matrix data are stored in a distributed manner by blockchain technology.

31. The system according to claim 30, wherein a non-fungible token is assigned to at least a portion of the digital twin, which includes the digital representation, the audio recording, the corresponding metadata, and the corresponding coordinate matrix data.

32. The system according to claim 31, wherein the system includes a payment processor adapted for identifying non-fungible tokens purchased by the user and assigning the corresponding non-fungible token rights to the user.

33. The system according to any one of claims 25 to 32, wherein the system assigns non-fungible tokens to the digital twin of the live event.

34. The system according to any one of claims 25 to 33, wherein the server uses artificial intelligence to generate a digital representation.

Images