Tracking in-game assets using NFTs to track activity across multiple platforms

Abstract

Data processing / GUI (400) of NFT blockchain data (202) to "tell the story" of ownership or to simplify and highlight the "cool" features of a computer game related NFT (200). Machine learning (ML) may be used to boil down complex data (602) to what people need or want to understand. The ownership timeline (800) presented in the GUI can be interactive. Describe the types of metadata to encapsulate in the NFT.

Description

Technical Field

[0001] This application generally relates to tracking in-game assets using non-fungible tokens (NFTs) that track operation histories (impressions) across multiple platforms.

Background Art

[0002] Non-fungible tokens (NFTs) are the digital world's version of physical collectibles, such as, but not limited to, artworks. NFTs are digital files on a blockchain, and while it is nearly impossible to forge proof of ownership of an NFT due to the use of blockchain technology, they prove the ownership of the underlying digital asset in the same way a sales receipt proves the ownership of a physical painting. Similar to prints and paintings, the ownership of an NFT does not necessarily include the copyright of the original work, and the copyright can be retained by the creator. While digital assets can be viewed by anyone, only the person identified by the NFT can sell the ownership of the asset, and that ownership is recorded on the blockchain. In this way, digital assets can be bought and sold like physical collectibles through NFT transactions.

Summary of the Invention

[0003] As understood herein, in some applications, such as computer simulations like computer games, players and spectators may be interested in who has previously owned an NFT related to that game in a clean and intuitive representation. Such NFTs make it easier to track the operation histories of gamers and spectators.

[0004] Accordingly, the system includes at least one computer medium having instructions executable by at least one processor to input at least one non-fungible token (NFT) representing at least one digital asset related to at least one computer simulation, rather than a transient signal, into at least a first machine learning (ML) model. The instructions are executable to receive information from the NFT indicative of characteristics of the NFT's duration from the ML model and present that information on at least one computer display.

[0005] In an exemplary embodiment, the instructions can be executable to receive information from the NFT indicative of characteristics of the NFT's duration from the ML model. The information from the NFT may be derived from some, but not all, of the metadata associated with the NFT.

[0006] In some implementations, the instructions can be executable to present information from the NFT on at least one computer display. The information may include at least one user interface (UI) capable of presenting a list of at least some of the owners of the NFT. In some examples, the UI presents the respective periods during which each owner held the NFT. In a non-limiting embodiment, the UI presents the respective games and / or game scenes associated with the acquisition of the NFT by each owner. If desired, the UI can present each event within the game associated with the acquisition of the NFT.

[0007] In some embodiments, the instructions can be executable to present a secondary UI that surfaces elements of information related to a selection from the UI in response to the selection.

[0008] As an example, this instruction may be executable to present information from an NFT on at least one computer display in a UI that presents a timeline each containing spikes representing important events of the NFT. In such an example, the instruction may be executable to receive a selection of the first spike within the timeline and, in response to the selection, present a second UI that presents information associated with the first spike.

[0009] In another aspect, the method includes inputting at least one training set of data including metadata of a non-fungible token (NFT) associated with a computer simulation asset and interesting elements of the ground truth therein into at least one machine learning (ML) model. The method includes training the ML model using the training set. After learning, the method includes inputting at least one NFT including the metadata into the ML model and presenting the metadata output by the ML model for the user to hear, visualize, or feel important events during the lifespan of the NFT.

[0010] If necessary, user feedback can be added to the ML model to continuously improve the model over time. User feedback includes viewable operation history, focus events, click events, etc.

[0011] In another aspect, the assembly includes at least one display and at least one processor programmed with an instruction to present on the display information associated with at least one non-fungible token (NFT) derived to represent at least one digital asset in a computer game.

[0012] The details of the present application can be best understood with reference to the accompanying drawings, both as to its structure and operation, in which like reference numerals indicate like parts.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0014] The present disclosure generally relates to a computer ecosystem including features of a consumer electronics (CE) device network not limited to a computer game network. The systems herein may include a server component and a client component connectable via a network so that data can be exchanged between the client component and the server component. The client component may include one or more computing devices including gaming consoles such as Sony PlayStation®, gaming consoles manufactured by manufacturers such as Microsoft and Nintendo, virtual reality (VR) headsets, augmented reality (AR) headsets, portable televisions (such as smart TVs, Internet-enabled TVs, etc.), portable computers such as laptops and tablet computers, mobile devices such as smartphones, and additional examples described later. These client devices may operate in various operating environments. For example, some client computers may employ a Linux® operating system, a Microsoft operating system, a Unix® operating system, an operating system of Apple or Google. These operating environments can be used to execute one or more browsing programs such as browsers made by Microsoft, Google, Mozilla, and other browser programs that can access websites hosted by Internet servers described later. Also, the operating environment according to the present principle can be used to execute one or more computer game programs.

[0015] The server and / or gateway may include one or more processors that execute instructions configuring the server to receive and transmit data via a network such as the Internet. Alternatively, the client and server can also be connected via a local intranet or a virtual private network. The server or controller can be instantiated by a gaming console such as the Sony PlayStation (registered trademark), a personal computer, or the like.

[0016] Information may be exchanged between the client and the server via the network. For this purpose and for security, the server, and / or the client may include a firewall, a load balancer, temporary storage, a proxy, and other network infrastructure for reliability and security. One or more servers can form a device that implements a method of providing a secure community such as an online social website to network members.

[0017] The processor may be a single-chip or multi-chip processor that can execute logic by means of various lines such as address lines, data lines, control lines, registers, shift registers, etc.

[0018] The components included in one embodiment can be used in other embodiments in appropriate combinations. For example, any of the various components described herein and / or shown in the drawings can be integrated, substituted, or excluded from other embodiments.

[0019] A "system having at least one of A, B, and C" (similarly "a system having at least one of A, B, and C", "a system having at least one of A, B, and C") includes a system having only A, only B, only C, A and B, A and C, B and C, and / or A, B, and C.

[0020] Referring specifically to FIG. 1 here, an example of the system 10 is shown. This system 10 may include one or more of the above-described exemplary devices and will be further described below in accordance with this principle. The first of the exemplary devices included in the system 10 is a consumer electronics (CE) device such as an audio-video device (AVD) 12, such as an Internet-enabled TV with a TV tuner (equivalently, a set-top box that controls the TV). The AVD 12 may alternatively be a computer-controlled Internet-enabled (“smart”) telephone, a tablet computer, a notebook computer, an HMD, a wearable computer device, a computer-controlled Internet-enabled music player, a computer-controlled Internet-enabled headset, a computer-controlled Internet-enabled embedded device such as an embedded skin device. In any case, it should be understood that the AVD 12 is configured to implement this principle (e.g., communicate with other CE devices to implement this principle, execute the logic described herein, and perform other functions and / or operations described herein).

[0021] Therefore, in order to implement such a principle, the AVD12 can be established by some or all of the components shown in FIG. 1. For example, the AVD12 can be implemented by a high-definition or ultra-high-definition "4K" or higher flat screen, and may include one or more displays 14 that are touch-enabled to receive user input signals via touch on the display. The AVD12 may also include one or more speakers 16 for outputting sound according to this principle, and at least one additional input device 18 such as a voice receiver / microphone for inputting audible commands to the AVD12 to control the AVD12. The exemplary AVD12 may also include one or more network interfaces 20 for communicating via at least one network 22 such as the Internet, WAN, LAN, etc. under the control of one or more processors 24. Therefore, the interface 20 may be, but is not limited to, a Wi-Fi transceiver, such as a mesh network transceiver, which is an example of a wireless computer network interface. It should be understood that the processor 24 controls the AVD12 to implement this principle, including other elements of the AVD12 described herein, such as controlling the display 14 to present images and receive inputs therefrom. Further, it should be noted that the network interface 20 may be a wired or wireless modem or router, or other suitable interfaces such as a wireless phone transceiver, the aforementioned Wi-Fi transceiver, etc.

[0022] In addition to the above, the AVD12 may include one or more input ports and / or output ports 26, such as a high-quality multimedia interface (HDMI (registered trademark)) port and a USB port for physically connecting to other CE devices, and / or a headphone port for connecting headphones to the AVD12 to provide audio to the user via the headphones from the AVD12. For example, the input port 26 can be connected, either wired or wirelessly, to a cable or satellite source 26a of audio-video content. Thus, the source 26a may be a separate or integrated set-top box, or a satellite receiver. Alternatively, the source 26a may be a game console or a disc player containing content. When implemented as a game console, the source 26a may include some or all of the components described below in relation to the CE device 48.

[0023] The AVD12 can further include one or more computer memories 28, such as disk-based or solid-state storage that is not a transient signal, and in some cases, can be embodied in the AVD's chassis as a stand-alone device, or as an internal or external personal video recording device (PVR) or video disc player for playing AV programs, or as a removable memory media or a server described below. Also, in some embodiments, the AVD12 can include a position or location receiver, such as a cellular phone receiver, a GPS receiver, and / or an altimeter 30, configured to receive geographical location information from a satellite or a cellular phone base station, provide the information to the processor 24, and / or determine the altitude at which the AVD12 is disposed in conjunction with the processor 24, but is not limited thereto. The component 30 may also be implemented by an inertial measurement unit (IMU), typically a combination of an accelerometer, a gyroscope, and a magnetometer, or by an event-based sensor, to determine the location and orientation of the AVD12 in three dimensions.

[0024] Continuing the description of the AVD12, in some embodiments, the AVD12 may include one or more cameras 32, such as an infrared camera, a digital camera such as a webcam, an event-based sensor, and / or a camera integrated with the AVD12, controllable by the processor 24, and capable of collecting photos / images and / or videos according to this principle. Also, the AVD12 may include a Bluetooth (registered trademark) transceiver 34 and another near-field communication (NFC) element 36 for communicating with other devices that use Bluetooth (registered trademark) and / or NFC technology respectively. An example of the NFC element may be a radio frequency identification (RFID) element.

[0025] Furthermore, the AVD12 may include one or more auxiliary sensors 38 that provide inputs to the processor 24 (such as motion sensors such as an accelerometer, a gyroscope, a cyclometer, or a magnetic sensor, an infrared (IR) sensor, an optical sensor, a speed and / or cadence sensor, an event-based sensor, a gesture sensor (e.g., for sensing gesture commands)). The AVD12 may include an OTA television broadcast port 40 for receiving OTA television broadcasts that provide inputs to the processor 24. In addition to the above, it should be noted that the AVD12 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 42, such as an IR data association (IRDA) device. A battery (not shown) may be provided to supply power to the AVD12, or it may be a kinetic energy harvester that converts kinetic energy into power for charging the battery and / or supplying power to the AVD12. It may include a graphics processing unit (GPU) 44 and a field programmable gate array 46. One or more tactile generators 47 may be provided to generate tactile signals that can be sensed by a person holding or in contact with the device.

[0026] Continuing to refer to FIG. 1, in addition to the AVD 12, the system 10 may include one or more other CE device types. In one example, the first CE device 48 may be a computer game console that can be used to transmit the audio and video of a computer game to the AVD 12 via commands sent directly to the AVD 12 and / or via the server described below, while the second CE device 50 may include components similar to those of the first CE device 48. In the illustrated example, the second CE device 50 may be configured as a computer game controller operated by a player or a head-mounted display (HMD) worn by the player. It should be understood that although only two CE devices are shown in the illustrated example, fewer or more devices may be used. The devices herein may implement some or all of the components shown for the AVD 12. Any of the components shown in the following figures may incorporate some or all of the components shown for the AVD 12 in the case of the AVD 12.

[0027] Referring now to at least one of the servers 52 described above, it includes at least one server processor 54, at least one tangible computer-readable storage medium 56 such as disk-based or solid-state storage, and at least one network interface 58 that enables communication with other devices in FIG. 1 via the network 22 under the control of the server processor 54 and can actually facilitate communication between the server and the client device in accordance with this principle. Note that the network interface 58 may be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interfaces such as a wireless phone transceiver.

[0028] Thus, in some embodiments, server 52 may be an Internet server or an entire server “farm”. For example, in an exemplary embodiment for a network game application, it may include and execute “cloud” functionality such that devices of system 10 can access a “cloud” environment via server 52. Alternatively, server 52 may be implemented on one or more game machines or other computers in the same room as or near the other devices shown in FIG. 1.

[0029] The components shown in the following figures may include some or all of the components shown in FIG. 1. The user interfaces (UIs) described herein can be integrated, extended, mixed, or combined among the UIs.

[0030] The present principles may employ various machine learning models including deep learning models. Machine learning models consistent with the present principles can use various algorithms trained in ways including supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning, and other learning modalities. Examples of such algorithms can be implemented by computer circuitry and include one or more neural networks such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and types of RNNs known as long short-term memory (LSTM) networks. Support vector machines (SVMs) and Bayesian networks may also be considered an example of a machine learning model.

[0031] Thus, as understood herein, performing machine learning may include accessing training data and then training a model on the training data such that the model can process further data to make inferences. Thus, an artificial neural network / artificial intelligence model trained by machine learning may be configured to make inferences with respect to an input layer, an output layer, and an appropriate output, and may include a plurality of hidden layers therebetween that are weighted.

[0032] FIG. 2 shows a data structure 200 configured as included in blockchain 202. The data structure 200 in the illustrated embodiment is typically configured as a non-fungible token (NFT) related to or derived from digital assets 204 such as images generated or composed by an artist, audio recordings, game events, or other digitally embodied assets. In an exemplary implementation, the digital asset 204 may be from a computer simulation such as a computer game and may represent other features of the computer game such as game characters, weapons, plots, or events.

[0033] In some cases, the digital asset 204 may be encoded as part of the data structure 200 (hereinafter simply referred to as "NFT200") for inclusion in the blockchain 200, or may be stored separately from the NFT200 itself, in which case the NFT200 may include a pointer 206 to the network address 208 of the digital asset 204.

[0034] NFT200 typically includes metadata 210 indicating ownership of the NFT200 and thus ownership of the digital asset 204. The metadata may include an indication of the current owner of the NFT200 and, if necessary, past owners, the price or other means paid for acquisition of ownership, the conditions of ownership (e.g., whether copyright accompanies ownership), the period of ownership, whether ownership can be transferred during a temporary period of ownership, etc.

[0035] Figure 3 shows a UI that can be presented on a display 300, such as any display in this specification, at 302 to prompt the user to make a choice as to whether they want to view (or hear) a story about a digital asset, in this case an image 304 of a weapon ("Boss Knife") and who owns it. For example, the image 304 might be an image of a weapon used by a famous streamer to kill the character of another famous streamer in a computer game.

[0036] The digital asset is associated with an NFT. The NFT may contain metadata about the digital asset, as will be further described with reference to FIG. 2, and the metadata can be mined to present the user with a clear and interesting story about the asset. The user can select the "Yes" selector 306 to view the story of the asset.

[0037] Figure 4 shows a UI that presents information about an NFT asset in a first presentation layout. In this layout, a list of NFT owners 400, the period 402 for which each owner has owned the NFT, the game 404 and game scene 406 associated with each owner's purchase of the NFT, the in-game event(s) 408 associated with the acquisition of the NFT, and the amount 410 paid by each owner for the NFT are presented. Note that the game 404, scene 406, and in-game event(s) 408 may be inferred to be the game, scene, and event contemporaneous with the NFT purchase agreement at the time of NFT acquisition, or may be specified by the purchaser at the time of purchase, or may be learned by machine learning (ML) based on the importance of event 408 according to learning based on a training set of ground truth important events in various games.

[0038] Figure 5 shows additional interactive features along with the presentation of Figure 4, where at 412, it shows that the user has selected to learn more about the circumstances of player B who was killed using the digital asset. Figure 5 shows accessing the metadata of the game and scene of Figure 4 and calling from storage (e.g., cloud storage) a recording of the event in question, typically from a recorded play session of the streamed game. Recording 500 is presented as shown in the figure, showing player B being killed while brandishing the asset. If necessary, a list 502 of players who used the asset may be presented along with the results at the same action points as shown in the animation scene of 500.

[0039] Rather than presenting all the metadata of the NFTs associated with the digital assets, it can be ensured that the presentation to the user does not become overly complex. Thus, this principle understands that it is necessary to create a clear and concise story that people are interested in, taking into account the complexity of the knowledge required for people to understand the presentation. For this reason, machine learning (ML) may be employed to extract metadata from NFTs associated with computer game assets.

[0040] As shown in Figure 6. First, at block 600, for example, in response to user input to learn more about an asset, NFT metadata associated with a selected digital asset such as a computer game asset is accessed. The metadata is input into the ML model at block 602. The ML model outputs the elements of the metadata that the model has learned are more important than other elements, and this output is displayed at block 604.

[0041] Figure 7 shows that a training set 700 of NFT metadata and, for example, the "interesting" or "important" elements of the ground truth therein indicated by an expert, is input into the ML model 702 to train the ML model 702 of FIG. 6. Examples of such "interesting" or "important" metadata elements are presented, for example, in FIGS. 4, 8, and 9.

[0042] Figure 8 shows another representation of interesting features of digital assets associated with an NFT derived from NFT metadata. The representation of FIG. 8 includes a timeline 800 along the X-axis where time progresses from left to right. There are spikes 802 on the timeline, each representing an important event during the lifespan of the NFT. For example, the first spike in FIG. 8 indicates the date (and time if necessary) when the NFT was created, and the second spike indicates that expert A, who acquired the NFT, killed the "boss" character using the underlying digital asset. The third spike indicates the date (and time if desired) when the owner sold the NFT, and the fourth spike indicates that expert B acquired the NFT after losing a game. Events that occur between the third and fourth spikes during the lifespan of the NFT, such as an intervening sale, are omitted as not being considered important by the ML model. Note that the context of the purchase, such as whether the acquirer won or lost the game event, may be embedded in the metadata of the NFT. Thus, other participants such as the physical and / or virtual location of the player, the time, and the game characters the player fought against may be encapsulated in the NFT metadata captured on the blockchain. When accessing the metadata, access the blockchain to read the metadata for the reasons described above and display the elements of the metadata in a way that visualizes the timeline controllable by the user. For example, the timeline 800 may be dropped onto the profiles of browsing users who owned the NFT at various times to view the "spikes" of "coolness".

[0043] 804 in Figure 8 indicates that the browsing user has clicked on the spike. In this case, the fourth spike indicating that the character associated with Expert B has lost is an event associated with the NFT. Figure 9 shows an example of a UI where the character 900 associated with Expert B is losing to the villain 904 of the boss while waving the digital asset 902 related to the NFT. The UI in Figure 9 may further show, at 906, the date, time, game name, and scene number of the game in which the depicted action was performed.

[0044] The presentation of NFT metadata can be done auditorily and / or visually and / or tactilely using at least one appropriately configured computer display.

[0045] The NFT data structure can be linked to a game play data transmission and processing system that is used when reporting metadata about game play, such as the start and end of an activity, the mechanics the player is using, the user's position on the game map, etc. This data is sent from the game to our back-end server, and the back-end server uses that data to operate various functions within the computer simulation ecosystem. In this way, the log information of the NFT can access and record who was present at an event associated with or generated by the NFT, the corresponding activity metadata, etc. This information can be accessed from the NFT and may be sold to those who were present during the game play or when the NFT was newly created.

[0046] In addition to or instead of purchasing NFTs, players can obtain them by performing in-game tasks such as winning tournaments, slaying or defeating opponents. Therefore, the importance of victory can be frozen in time, for example, to record the professional debut of a winner who defeated the current champion. The information contained in or indicated by the NFT may include a snapshot of the metaverse containing game statistics indicating important features of the game, such as the side that pressed through to defeat the champion or a comeback victory from behind. Since this information related to the recorded event can be dynamic, it may appear as the game progresses for the purpose of creating new NFTs. The importance of an event can be judged by factors such as how many people achieved the task and what social value is attributed to that task.

[0047] All records of the metaverse participating in creating new NFTs are maintained in an internal database, and only events that remain in memory can be written to the blockchain, which includes pointers indicating where the full records are stored.

[0048] The cross-platform use of NFTs and the benefits it brings may also be promoted. For example, NFTs created on one game platform can be used on another by using common file formats such as.jpg or image files.

[0049] Regarding special information related to the assets underlying NFTs, such as game cars or game swords, it can be transferred to a cloud server to reformat asset attributes (a car being dented, a sword being chipped) that may affect the performance of the assets in the game from one game format (such as PlayStation) to another (such as Xbox). Alternatively, the attributes may be encoded in the NFT by a pointer indicating the location of the network where the attributes may be accessed.

[0050] Using NFTs according to this principle, it is possible to track the "operation history". The assets generated by the user are used to track the "operation history" within our community being transported, i.e., how the assets are used, circulated, and associated with events. The operation history may include interactions with the user, and as described above, NFTs may also be newly created from events attached to objects such as in-game assets or a great victory using a specific weapon.

[0051] In this specification, specific embodiments are shown and described in detail, but it should be understood that the subject matter encompassed by the present invention is limited only by the claims.

Claims

1. At least one non-fungible token (NFT) representing at least one digital asset related to at least one computer simulation, rather than a transient signal, is input into at least a first machine learning (ML) model, information from the NFT indicating characteristics of the NFT's duration of existence is received from the ML model, and at least one computer medium comprising instructions executable by at least one processor to present the information on at least one computer display. A system comprising at least one computer medium comprising instructions executable by at least one processor to present the information on at least one computer display.

2. The system of claim 1, comprising the at least one processor.

3. The system of claim 1, wherein the instructions are executable to receive from the ML model information from the NFT indicating characteristics of the NFT's duration of existence, the information being derived from a portion, but not all, of the metadata associated with the NFT.

4. The system of claim 1, wherein the instructions are executable to present, on at least one computer display, information comprising at least one user interface (UI) that presents a list of owners of at least a portion of the NFT, the information being from the NFT.

5. The system of claim 4, wherein the UI indicates the respective periods during which each owner owned the NFT.

6. The system of claim 4, wherein the UI presents each game and / or game scene associated with the acquisition of the NFT by the respective owner.

7. The system of claim 4, wherein the UI presents each event in the game associated with the acquisition of the NFT.

8. The system of claim 3, wherein the instructions are executable to present a secondary UI that surfaces elements of the information related to the selection in response to a selection from the UI.

9. The instructions are executable to present, on at least one computer display, information comprising at least one user interface (UI) that presents a timeline including spikes each representing a significant event of the NFT, the information being from the NFT.

10. The system of claim 9, wherein the command is executable to receive a selection of a first spike within the timeline and, in response to the selection, present a second UI that presents information associated with the first spike. **Claim 11** A method executed by a system including at least one processor, comprising: the processor inputs at least one training set of data including metadata of a non-fungible token (NFT) associated with a computer simulation asset and interesting elements of the ground truth therein into at least one machine learning (ML) model; the processor trains the ML model using the training set; after learning, the processor inputs at least one NFT including metadata into the ML model; the processor presents metadata output by the ML model such that a user can query important events during the lifespan of the NFT, visualize the events, or feel the events. **Claim 12** The method of claim 11, wherein the interesting elements of the ground truth include the name of at least one owner of the NFT, the name of at least one computer game, and activities in the computer game. **Claim 13** The method of claim 11, wherein the metadata is presented in at least one user interface (UI) that presents a list of at least several owners of the NFT. **Claim 14** The method of claim 13, wherein the UI indicates the respective periods during which each owner owned the NFT. **Claim 15** The method of claim 13, wherein the UI presents each game and / or game scene associated with the acquisition of the NFT by each owner. **Claim 16** The method of claim 13, wherein the UI presents each event in the game associated with the acquisition of the NFT. **Claim 17** The method of claim 13, comprising the processor presenting a secondary UI that surfaces elements of information related to the selection in response to a selection from the UI. The method according to claim 11, wherein the processor presents the metadata to at least one user interface (UI) that presents a timeline including spikes each representing a significant event of the NFT on at least one computer display. The method according to claim 18, wherein the processor receives a selection of a first spike of the timeline, and the processor presents a secondary UI that presents information associated with the first spike in response to the selection.

Images