Information processing method, information processing system, and program

Abstract

This information processing method involves: acquiring first content (S301); determining whether first validity information indicating that the acquired first content is valid as content to be input into a trained model has been stored as a digital watermark in the first content (S302); if it is determined that the first validity information has been stored in the first content, acquiring second content generated by means of the trained model by inputting the first content into the trained model (S303, S304); and outputting the acquired second content (S312).

Description

Information processing method, information processing system, and program

[0001] The present invention relates to an information processing method, an information processing system, and a program.

[0002] A machine learning model (so-called AI (Artificial Intelligence) model) is capable of receiving various pieces of information as input and generating and outputting content having characteristics based on the input information.

[0003] For example, there is a server device that generates an illustrated background image by inputting a prompt to an AI image generation unit that has been trained using a Diffusion Model, which is a machine learning model (see Patent Document 1).

[0004] Patent No. 7398723

[0005] When a huge amount of information is input into a machine learning model, the model generates and outputs information based on each piece of input information, which can result in the power consumption of the information processing system that runs the machine learning model becoming enormous.

[0006] Therefore, the present invention provides an information processing method and the like that contributes to reducing the power consumption of an information processing system that runs a machine learning model.

[0007] An information processing method according to one aspect of the present invention is an information processing method that acquires a first content, determines whether first legitimacy information indicating that the acquired first content is legitimate content to be input into a trained model is stored in the first content as an electronic watermark, and if it is determined that the first legitimacy information is stored in the first content, inputs the first content into the trained model to acquire second content generated by the trained model, and outputs the acquired second content.

[0008] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, and a recording medium.

[0009] The present invention contributes to reducing the power consumption of an information processing system that runs a machine learning model.

[0010] 1 is a schematic diagram showing the overall configuration of an information processing system in an embodiment. FIG. 1 is a block diagram showing the functional configuration of a ledger server in an embodiment. FIG. 2 is a block diagram showing the functional configuration of a control system in an embodiment. FIG. 3 is a block diagram showing the functional configuration of a generation system in an embodiment. FIG. 4 is a block diagram showing the functional configuration of a training system in an embodiment. FIG. 5 is a sequence diagram showing a storage process for storing validity information in content in an embodiment. FIG. 6 is an explanatory diagram showing a first example of NFT metadata in an embodiment. FIG. 7 is a flow diagram showing a training process for a machine learning model in an embodiment. FIG. 8 is an explanatory diagram showing a first example of training content in an embodiment. FIG. 9 is an explanatory diagram showing a second example of training content in an embodiment. FIG. 10 is a sequence diagram showing a generation process for generating content in an embodiment. FIG. 11 is an explanatory diagram showing an example of original content in an embodiment. FIG. 12 is an explanatory diagram showing an example of generated content in an embodiment. FIG. 13 is an explanatory diagram showing correspondence information in an embodiment. FIG. 14 is an explanatory diagram showing a second example of NFT metadata in an embodiment. FIG. 15 is a sequence diagram showing a purchase process for generated content in an embodiment. FIG. 16 is an explanatory diagram showing an example of purchase transaction data in an embodiment. FIG. 17 is an explanatory diagram showing a first example of transfer transaction data in an embodiment. FIG. 18 is an explanatory diagram showing a second example of transfer transaction data in an embodiment. FIG. 19 is a sequence diagram showing a modified example of a purchase process for generated content in an embodiment. FIG. 10 is an explanatory diagram showing a third example of transfer transaction data in an embodiment. FIG. 11 is an explanatory diagram showing the data structure of a blockchain, which is an example of a distributed ledger. FIG. 12 is an explanatory diagram showing the data structure of transaction data. FIG. 13 is an explanatory diagram showing transaction data related to the execution of a smart contract. FIG. 14 is an explanatory diagram showing processing related to the execution of a smart contract. FIG. 15 is an explanatory diagram showing the structures of an NFT and metadata.

[0011] (Findings that Form the Basis of the Present Invention) The present inventors have found the following problems with the technology relating to the generation of information using a machine learning model, which is described in the "Background Art" section.

[0012] By inputting various pieces of information, the machine learning model is able to generate and output content having characteristics based on the input information.

[0013] When a huge amount of information is input into a machine learning model, the model generates and outputs information based on each piece of input information. In this case, the processing load on the information processing system that runs the machine learning model can become very large, resulting in enormous power consumption. In addition, the storage capacity required to store the generated information can also become enormous.

[0014] Furthermore, when electronic content (also simply referred to as content) is used as information to be input to a machine learning model, invalid content (also simply referred to as invalid content) may be input to the machine learning model. Note that content may include image content (including still image content or video content), audio content, or text content, and one piece of content may include two or more types of content (for example, image content and audio content).

[0015] For example, a person other than the owner or creator (also referred to as the owner, etc.) of the content may input the content into a machine learning model without the permission of the owner, etc.

[0016] In addition, the expression "content is input into a machine learning model" includes cases where the content is input into a machine learning model as training data for the machine learning model, as well as cases where the content is input into a machine learning model as input data for generating content by the machine learning model.

[0017] When unauthorized content is input to a machine learning model as training data for the machine learning model, the machine learning model may generate content that has the characteristics of the input unauthorized content. Also, when unauthorized content is input to a machine learning model as input data for generating content, the machine learning model may generate content based on the input unauthorized content.

[0018] If unauthorized content is input into a machine learning model and the content generated by the model is then freely available, the legitimate rights of the owner of the content input into the machine learning model may not be protected. In this case, the power consumed to train the machine learning model using unauthorized content or to generate an image based on the input unauthorized content may be deemed to have been wasted.

[0019] Therefore, the present invention provides an information processing method and the like that contributes to reducing the power consumption of an information processing system that runs a machine learning model.

[0020] The present invention can also contribute to reducing the processing load of an information processing system that runs a machine learning model, or reducing the required storage capacity of a storage device.

[0021] Below, examples of inventions that can be obtained from the disclosure of this specification will be given, and the effects and the like that can be obtained from these inventions will be explained.

[0022] (1) An information processing method comprising: acquiring a first content; determining whether first legitimacy information indicating that the acquired first content is legitimate content to be input into a trained model is stored in the first content as an electronic watermark; and, if it is determined that the first legitimacy information is stored in the first content, inputting the first content into the trained model to acquire second content generated by the trained model; and outputting the acquired second content.

[0023] According to the above aspect, it is possible to determine whether the first content is legitimate using a digital watermark, and then output second content generated by the trained model by inputting the legitimate first content into the trained model. This may reduce the output of second content generated by inputting illegitimate content into the trained model compared to when the above determination is not made. This may reduce the power consumption of an information processing system that operates a machine learning model. In this way, the above information processing method may contribute to reducing the power consumption of an information processing system that operates a machine learning model.

[0024] (2) When the second content is acquired, the information processing method further controls so that second legitimacy information is stored in the second content as an electronic watermark, the second legitimacy information being information indicating that the second content is legitimate content output by the trained model, and when the second content is output, the second content with the second legitimacy information stored therein is output.

[0025] According to the above aspect, since the validity information is stored as a digital watermark in the content generated by the trained model, the validity information stored as a digital watermark can indicate that the second content to be output is valid. This may prevent invalid content from being used as content output from the information processing system, and as a result, valid content may be appropriately used as content output from the information processing system. Therefore, the above information processing method may more appropriately contribute to reducing the power consumption of an information processing system that operates a machine learning model.

[0026] (3) An information processing method described in (1) or (2), in which a first NFT (Non-Fungible Token) that corresponds one-to-one to the first content and is linked to the owner of the first content as the owner of the first NFT is stored in a distributed ledger, and when the second content is acquired, a second NFT that corresponds one-to-one to the second content and is linked to the operator of the information processing system that executes the information processing method as the owner of the second NFT is further stored in the distributed ledger.

[0027] According to the above aspect, NFTs that correspond one-to-one to each piece of content (i.e., content including a first content and a second content) and that are linked to the owner of the content as the owner of the NFT are stored in a distributed ledger, so that information indicating the owner of the content can be appropriately managed without being substantially tampered with. Therefore, the above information processing method can more appropriately contribute to reducing the power consumption of an information processing system that operates a machine learning model.

[0028] (4) An information processing method described in (3), further comprising storing in the distributed ledger purchase transaction data indicating that a user purchases the second content, the purchase transaction data further indicating an amount of value information equivalent to the consideration for the purchase of the second content, storing in the distributed ledger first transfer transaction data transferring a first portion of the value information from the user to the owner of the first content, and storing in the distributed ledger second transfer transaction data transferring a second portion of the value information excluding the first portion from the user to the operator.

[0029] According to the above aspect, when a user purchases second content, the price of the second content can be distributed and transferred to the owner of the first content and the operator of the information processing system. Since the first content used as input data for generating the second content and the information processing system having the model can be said to have contributed to the generation of the second content using the trained model, it is appropriate to distribute and transfer the price to the provider of the first content and the operator of the information processing system having the model. Therefore, the above information processing method can more appropriately contribute to reducing the power consumption of an information processing system that operates a machine learning model.

[0030] (5) An information processing method described in (4), wherein the first part is a larger portion of the value information the greater the contribution of the first content to the generation of the second content, and the second part is the portion of the value information minus the first part.

[0031] According to the above aspect, a larger amount of value information is transferred to a provider of the first content who has made a greater contribution to the creation of the second content, so that an amount of value information corresponding to the degree of contribution to the creation of the second content can be appropriately transferred to the provider of the first content. Thus, the above information processing method can more appropriately contribute to reducing the power consumption of an information processing system that operates a machine learning model.

[0032] (6) The information processing method according to (5), wherein the magnitude of the contribution of the first content to the generation of the second content increases as the similarity between the second content and the first content increases.

[0033] According to the above aspect, the contribution of the first content to the generation of the second content can be easily calculated based on the similarity between the first content and the second content. Therefore, the above information processing method can more easily and appropriately contribute to reducing power consumption of an information processing system that operates a machine learning model.

[0034] (7) An information processing method described in any of (1) to (6), in which, when it is determined that the first legitimacy information is not stored in the first content, output of the second content generated by the trained model is prohibited.

[0035] According to the above aspect, after determining that the first content is legitimate using a digital watermark, output of the second content generated by the trained model by inputting illegitimate content into the trained model is prohibited. This can further reduce the power consumption of the information processing system that operates the machine learning model. In this way, the above information processing method can contribute to further reducing the power consumption of the information processing system that operates the machine learning model.

[0036] (8) An information processing method described in any of (1) to (7), wherein the trained model is a machine learning model trained to generate and output fifth content based on fourth content input to the machine learning model using third content in which third legitimacy information indicating that the third content is legitimate content to be input to the trained model is stored as an electronic watermark, and the fifth content is content to which features related to the training performed by the machine learning model have been added to the fourth content.

[0037] According to the above aspect, by inputting legitimate first content into a trained model trained using legitimate third content, second content generated by the trained model can be output. This may prevent the use of a trained model trained using illegitimate content and may prevent the output of content generated by the trained model based on illegitimate content. In this way, the above information processing method may more appropriately contribute to reducing the power consumption of an information processing system that operates a machine learning model.

[0038] (9) The information processing method according to (1), wherein the trained model is a neural network model.

[0039] According to the above aspect, by using a neural network model as a trained model, it is possible to more easily contribute to reducing the power consumption of an information processing system that operates a machine learning model.

[0040] (10) An information processing method comprising: acquiring a third content; determining whether third legitimacy information indicating that the acquired third content is legitimate content to be input to a machine learning model is stored in the third content as an electronic watermark; and, if it is determined that the third legitimacy information is stored in the third content, training the machine learning model to generate and output fifth content based on fourth content input to the machine learning model; the fifth content being content to which features based on the training performed by the machine learning model have been added to the fourth content.

[0041] According to the above aspect, by training a machine learning model using legitimate third content, it is possible to construct a trained model that outputs fifth content in which training-related features are added to fourth content. As a result, the above information processing method can contribute to reducing power consumption of an information processing system that operates the machine learning model.

[0042] (11) An information processing system comprising: an acquisition unit that acquires a first content; a judgment unit that determines whether first legitimacy information indicating that the acquired first content is legitimate content to be input into a trained model is stored in the first content as an electronic watermark; a control unit that acquires second content generated by the trained model by inputting the first content into the trained model when it is determined that the first legitimacy information is stored in the first content; and an output unit that outputs the acquired second content.

[0043] According to the above aspect, the same effects as those of the above information processing method are achieved.

[0044] (12) An information processing system comprising: an acquisition unit that acquires third content; a judgment unit that determines whether third legitimacy information indicating that the acquired third content is legitimate content to be input to a machine learning model is stored in the third content as an electronic watermark; and a training unit that trains the machine learning model to generate and output fifth content based on fourth content input to the machine learning model when it is determined that the third legitimacy information is stored in the third content, wherein the fifth content is content to which features based on the training performed by the machine learning model have been added to the fourth content.

[0045] According to the above aspect, the same effects as those of the above information processing method are achieved.

[0046] (13) A program that causes a computer to execute the information processing method described in (1).

[0047] According to the above aspect, the same effects as those of the above information processing method are achieved.

[0048] (14) A program that causes a computer to execute the information processing method described in (10).

[0049] According to the above aspect, the same effects as those of the above information processing method are achieved.

[0050] These comprehensive or specific aspects may be realized as a system, an apparatus, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, an integrated circuit, a computer program, or a recording medium.

[0051] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0052] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

[0053] Embodiment In this embodiment, an information processing method and an information processing system that contribute to reducing the power consumption of an information processing system that runs a machine learning model will be described.

[0054] 1 is a schematic diagram showing the overall configuration of an information processing system 1 according to the present embodiment. The information processing system 1 is an example of an information processing system that contributes to reducing the power consumption of an information processing system that runs a machine learning model.

[0055] Information processing system 1 controls so that content that is legitimate as content to be input to a machine learning model (also referred to as legitimate content) is input to the machine learning model, and also controls so that content that is not legitimate as content to be input to a machine learning model (also simply referred to as invalid content) is not input to the machine learning model. Note that the expression "content is input to a machine learning model" includes cases where the content is input to a machine learning model as training data for the machine learning model, and also includes cases where the content is input to a machine learning model as input data for generating content by the machine learning model.

[0056] 1, the information processing system 1 includes a ledger system 10, a control system 20, and a generation system 30. The information processing system 1 may be connected to a storage device 5 and terminals T1, T2, and T3. The information processing system 1 may further include the storage device 5 or the terminals T1, T2, or T3. Each of the above devices is connected to a network N and is capable of communicating via the network N.

[0057] The ledger system 10 is an information processing system that stores information using a distributed ledger. The distributed ledger of the ledger system 10 stores information (specifically, the history of NFT generation and transfer) related to non-fungible tokens (NFTs) that are associated one-to-one with legitimate content. The NFTs are NFTs that trace the generation and transfer of legitimate content (in other words, the creation and change of owners) on the distributed ledger.

[0058] The ledger system 10 can execute processes using smart contracts by using a distributed ledger. The ledger system 10 can generate NFTs and realize the transfer of NFTs using smart contract processes.

[0059] The ledger system 10 includes ledger servers 11, 12, and 13 (also referred to as ledger servers 11, etc.) as a group of servers that hold a distributed ledger. When at least one of the ledger servers 11, etc. receives transaction data, the transaction data is shared by all of the ledger servers 11, etc. and stored in the distributed ledger. Note that the number of ledger servers included in the server group is not limited to three, and may be two or more than three.

[0060] The ledger server 11 is a computer server that holds and manages a distributed ledger. The ledger server 11 holds the distributed ledger and updates it while synchronizing with other ledger servers (specifically, ledger servers 12 and 13).

[0061] The ledger servers 12 and 13 are similar to the ledger server 11 and operate independently of the ledger server 11 .

[0062] The storage device 5 is a storage device that stores data. The storage device 5 can be accessed (specifically, read or written) from the ledger system 10 or the terminals T1, T2, or T3 via the network N. There may be one or more storage devices 5. The storage device 5 can store various information including content or metadata.

[0063] The control system 20 is an information processing system that manages or controls content input to a machine learning model. The control system 20 stores legitimacy information in legitimate content as a digital watermark. The control system 20 also controls the storage of NFTs in the ledger system 10 that correspond one-to-one to the legitimate content. The specific processing content of the control system 20 will be described in detail later. The control system 20 may be configured by a single device or by multiple devices connected via a network N.

[0064] The generation system 30 is an information processing system that generates content using a machine learning model. The generation system 30 possesses a trained machine learning model (also referred to as a trained model) and generates new content by inputting legitimate content into the trained model. When the generated content is purchased, the generation system 30 can distribute tokens obtained as consideration for the purchase to the content provider and the operator of the generation system 30. The specific processing content of the generation system 30 will be described in detail later. The generation system 30 may be configured as a single device or may be configured as multiple devices connected via a network N.

[0065] Terminal T1 is an information processing device used by user U1 who owns content (also referred to as training content) used to train the machine learning model owned by generation system 30. Terminal T1 includes at least a processor, memory, a user interface, and a communication interface, and can accept information input using the user interface or communication interface, as well as generate, display, output audio, or transmit and receive information. Terminal T1 may be, for example, a personal computer, a tablet, or a smartphone. Terminal T1 stores training data (specifically, training content) for model 304 in its memory and can provide it to control system 20, etc. Specific processing details of terminal T1 will be described in detail later.

[0066] Terminal T2 is an information processing device used by user U2 who owns content (also referred to as original content) that is input to the generation system 30 as input data for content generation by the generation system 30. The configuration of terminal T2 is similar to the configuration of terminal T1. Terminal T2 stores the original content in its memory and can provide it to the control system 20, etc. The specific processing content of terminal T2 will be explained in detail later.

[0067] Terminal T3 is an information processing device used by user U3 who purchases content (also referred to as generated content) generated by generation system 30. The configuration of terminal T3 is similar to the configuration of terminal T1. The specific processing content of terminal T3 will be described in detail later.

[0068] FIG. 2 is a block diagram showing the functional configuration of the ledger server 11 in this embodiment.

[0069] 2, the ledger server 11 includes, as functional units, a communication unit 101, a ledger processing unit 102, an execution unit 103, and a storage unit 104. At least some of the functional units included in the ledger server 11 are realized by a processor (e.g., a CPU (Central Processing Unit)) included in the ledger server 11 executing a program using a memory.

[0070] The communication unit 101 is a communication interface communicatively connected to the network N. The communication unit 101 may be a communication interface for a wired communication standard (e.g., Ethernet (registered trademark) or the like), or may be a communication interface for a wireless communication standard (e.g., Wi-Fi (registered trademark) or the like, or a mobile communication system (3G (Generation), 4G, 5G, or the like)). The communication unit 101 is used when the functional units included in the ledger server 11 communicate with other devices.

[0071] The ledger processing unit 102 performs processing related to the distributed ledger 111 and transaction data. Specifically, when the ledger processing unit 102 receives transaction data from the generation system 30 or the like, it verifies the digital signature included in the received transaction data and controls the storage of successfully verified transaction data in the distributed ledger 111 held by the storage unit 104. When storing transaction data in the distributed ledger 111, the ledger processing unit 102 generates a block including the transaction data to be stored, and can control the storage of the generated block in the distributed ledger 111 when consensus is reached with the ledger processing units 102 of the other ledger servers, ledger servers 12 and 13.

[0072] The execution unit 103 executes information processing. The execution unit 103 can execute information processing by, for example, executing a smart contract using the distributed ledger 111. Note that when the execution unit 103 does not use a smart contract, it executes information processing according to normal program code.

[0073] The execution unit 103 executes a process of generating an NFT as the information processing. The NFT is associated one-to-one with legitimate content and includes, as metadata, related information about the legitimate content associated with the NFT. The related information includes at least information indicating the owner of the legitimate content associated with the NFT.

[0074] Furthermore, the execution unit 103 executes a process of transferring NFT as the information processing.

[0075] The storage unit 104 is a storage device that stores information. The storage unit 104 stores the distributed ledger 111. The storage unit 104 is realized by a non-volatile storage device (such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive)).

[0076] The distributed ledger 111 stores data having a structure in which blocks, each containing one or more transaction data, are linked in a chain. The one or more transaction data stored in the distributed ledger 111 may include transaction data containing the contract code of a smart contract, transaction data containing instructions for executing a smart contract, or transaction data containing other information. Specific data stored in the distributed ledger 111 will be described in detail later.

[0077] FIG. 3 is a block diagram showing the functional configuration of the control system 20 according to this embodiment.

[0078] 3, the control system 20 includes, as functional units, a communication unit 201, a generation unit 202, and a watermark storage unit 203. At least some of the functional units included in the control system 20 are realized by a processor (e.g., a CPU) included in the control system 20 executing a program using a memory.

[0079] The communication unit 201 is a communication interface communicatively connected to the network N. The communication unit 201 may be a communication interface for a wired communication standard (e.g., Ethernet (registered trademark) or the like), or may be a communication interface for a wireless communication standard (e.g., Wi-Fi (registered trademark) or the like, or a mobile communication system (3G, 4G, 5G, or the like)). The communication unit 201 is used when the functional units included in the control system 20 communicate with other devices.

[0080] The generation unit 202 generates information (also referred to as an NFT generation request) requesting the generation of an NFT that corresponds one-to-one to the legitimate content. The generation unit 202 transmits the generated NFT generation request to the ledger system 10 via the communication unit 201.

[0081] The watermark storage unit 203 stores a digital watermark in legitimate content. Specifically, the watermark storage unit 203 stores, for the legitimate content, information (also referred to as legitimacy information) indicating that the content to be input to the model 304, which is a trained model, is legitimate. Specifically, the watermark storage unit 203 stores the legitimacy information (corresponding to third legitimacy information) as a digital watermark in content (corresponding to training content or third content) to be input to the machine learning model as training data for the model 304. Furthermore, the watermark storage unit 203 stores the legitimacy information (corresponding to first legitimacy information) as a digital watermark in content (corresponding to original content or first content) to be input to the model 304 as input data for generating content by the model 304.

[0082] Next, the generation system 30 will be described.

[0083] FIG. 4 is a block diagram showing the functional configuration of the generation system 30 according to this embodiment.

[0084] 4, the generation system 30 includes, as functional units, an acquisition unit 301, a control unit 302, and an output unit 305. At least some of the functional units included in the generation system 30 are realized by a processor (e.g., a CPU) included in the generation system 30 executing a program using a memory.

[0085] The acquisition unit 301 acquires content (corresponding to first content). The content acquired by the acquisition unit 301 is content to be used as input data for generating content by the model 304. The acquisition unit 301 provides the acquired content to the control unit 302.

[0086] If the content acquired by the acquisition unit 301 is legitimate content to be input to the model 304, which is a trained model, the first legitimacy information is stored as a digital watermark in the content acquired by the acquisition unit 301. In addition, in the above case, an NFT (corresponding to the first NFT) that has a one-to-one correspondence with the content acquired by the acquisition unit 301 is stored in the distributed ledger 111.

[0087] The control unit 302 uses the content provided by the acquisition unit 301 to acquire content (corresponding to generated content or second content) generated by the model 304. The control unit 302 includes a determination unit 303 and a model 304.

[0088] The determining unit 303 determines whether the first validity information is stored as a digital watermark in the content provided by the acquiring unit 301 .

[0089] The model 304 is a trained machine learning model. The model 304 is, for example, a neural network model, and the architecture of the model 304 may be, for example, a Generative Adversarial Network (GAN) or a Diffusion model. The model 304 is a trained machine learning model that has been trained to generate and output content (corresponding to fifth content) based on content (corresponding to fourth content) input to the model 304, using content (corresponding to third content) in which legitimacy information (corresponding to third legitimacy information) indicating that the content input to the model 304 is legitimate is stored as a digital watermark. The generated fifth content is content in which features based on the training performed by the model 304 are added to the fourth content input to the model 304. A model 404 (described below) trained by the training system 40 may be used as the model 304, but is not limited to this.

[0090] When the determination unit 303 determines that the first validity information is stored in the content provided from the acquisition unit 301, the control unit 302 inputs the content to the model 304 to acquire the second content generated by the model 304. On the other hand, when the control unit 302 determines that the first validity information is not stored in the content provided from the acquisition unit 301, the control unit 302 does not input the content to the model 304, and as a result, the content generated by the model 304 is not output (in other words, the output of the content generated by the model 304 is prohibited).

[0091] When the control unit 302 acquires content generated by the model 304, the control unit 302 may further perform control so that second validity information is stored in the content as a digital watermark. Here, the second validity information is information indicating that the content is valid content output by the model 304.

[0092] Furthermore, when the control unit 302 acquires content generated by the model 304, it further controls to store a second NFT that corresponds one-to-one to the content in the distributed ledger 111. The second NFT is linked to the operator of the generation system 30 as the owner of the second NFT.

[0093] The control unit 302 also controls the purchase processing of generated content. The purchase processing is processing performed when a user purchases generated content generated by the model 304. The purchase processing includes processing of storing, in the distributed ledger 111, purchase transaction data indicating that the user will purchase the generated content, and transfer transaction data indicating that value information will be transferred as consideration for the purchase. The value information is, for example, a token, which is value information managed by the distributed ledger 111, and this case will be described as an example, but is not limited to this and may be virtual currency or the like. Details will be explained later.

[0094] The output unit 305 outputs the content acquired by the control unit 302 (i.e., the content generated using the model 304). Specifically, the output unit 305 can output the content by transmitting the content to an external device via a communication line or by storing the content in a storage device that is removable from the generation system 30. The content output by the output unit 305 stores the second validity information.

[0095] Next, a description will be given of the training system 40. The training system 40 is an information processing system that trains a machine learning model.

[0096] FIG. 5 is a block diagram showing the functional configuration of a training system 40 according to this embodiment.

[0097] 5, the training system 40 includes, as functional units, an acquisition unit 401 and a control unit 402. At least a part of the functional units included in the training system 40 is realized by a processor (e.g., a CPU) included in the training system 40 executing a program using a memory.

[0098] The acquisition unit 401 acquires content (corresponding to training content or third content). The content acquired by the acquisition unit 401 is content to be used as training data for the model 404. The acquisition unit 401 provides the acquired content to the control unit 402.

[0099] If the content acquired by the acquisition unit 401 is legitimate content to be input to the model 404, which is a machine learning model, the third legitimacy information is stored as a digital watermark in the content acquired by the acquisition unit 401. In addition, in the above case, an NFT (corresponding to the third NFT) that has a one-to-one correspondence with the content acquired by the acquisition unit 401 is stored in the distributed ledger 111.

[0100] The control unit 402 trains the model 404 using the content provided by the acquisition unit 401. The control unit 402 includes a determination unit 403, a model 404, and a training unit 405.

[0101] The judgment unit 403 judges whether third legitimacy information indicating that the content provided from the acquisition unit 401 is legitimate content to be input into the machine learning model (in other words, model 404) is stored as an electronic watermark in the content provided from the acquisition unit 401.

[0102] The model 404 is a machine learning model trained by the training system 40. The model 404 is trained by the training unit 405 using training content in which third legitimacy information indicating that the content input to the model 404 is legitimate is stored as a digital watermark. The training content will be described later. The model 404 is, for example, a neural network model, and the architecture of the model 404 may be, for example, a GAN or a diffusion model. The model 404 is trained by the training unit 405, whereby coefficients (weights) included in the neural network model are adjusted.

[0103] When the determination unit 403 determines that the third validity information is stored in the content provided from the acquisition unit 401, the training unit 405 trains the model 404 so as to generate and output fifth content based on the fourth content input to the model 404. The fifth content is content to which features based on the training performed by the model 404 have been added to the fourth content.

[0104] The training system 40 may be provided in the generation system 30 as one function of the generation system 30. In this case, the function of the acquisition unit 401 may be provided in the acquisition unit 301, and the function of the control unit 402 may be provided in the control unit 302.

[0105] Next, a process for storing validity information in content will be described. The content in which validity information is stored may be training content or original content.

[0106] Fig. 6 is a sequence diagram showing a storage process for storing validity information in content in this embodiment. Fig. 7 is an explanatory diagram showing a first example of metadata of an NFT in this embodiment.

[0107] Here, a storage process for storing validity information (corresponding to the third validity information) in training content held by terminal T1, which is an example of content, will be described as an example. Note that the storage process for storing validity information (the first validity information) in original content held by terminal T2, which is another example of content, is similar. In this case, terminal T1 shown in FIG. 6 is replaced with terminal T2.

[0108] In step S101, the terminal T1 transmits content to the control system 20. The terminal T1 may transmit content to the control system 20 in response to receiving a content acquisition request from the control system 20, or may transmit content to the control system 20 voluntarily (in other words, without receiving an acquisition request from the control system 20). The control system 20 receives the transmitted content.

[0109] In step S102, the generation unit 202 of the control system 20 generates generation request information (also called a generation request) requesting the generation of an NFT and transmits it to the ledger system 10. The generation request is information requesting the generation of an NFT that corresponds one-to-one to the content received in step S101. The ledger system 10 receives the transmitted generation request.

[0110] In step S103, in response to receiving the generation request in step S102, the ledger system 10 generates transaction data including an NFT and stores it in the distributed ledger 111. Transaction data including an NFT can also be said to be transaction data indicating the generation of an NFT. An NFT is generated by storing transaction data including an NFT in the distributed ledger 111. When generating an NFT, the ledger system 10 generates a token ID (also referred to as an NFT-ID), which is identification information for the NFT, and assigns it to the NFT. An example of metadata for the generated NFT is shown in FIG. 7. The metadata shown in FIG. 7 is metadata for an NFT with a token ID of "001," and "U1" (i.e., user U1) is shown as information indicating the owner. Note that the metadata may be stored in the storage device 5.

[0111] In step S104, the ledger system 10 transmits the NFT-ID generated in step S103 to the control system 20. The control system 20 receives the transmitted NFT-ID.

[0112] In step S105, the watermark storage unit 203 of the control system 20 stores the validity information as a digital watermark in the content received in step S101. The validity information includes the NFT-ID received in step S104.

[0113] In step S106, the watermark storage unit 203 of the control system 20 transmits the content in which the validity information has been stored in step S105 to the terminal T1. The terminal T1 receives the transmitted content. Note that the control system 20 may transmit the content in response to the reception of the content in step S101.

[0114] In step S107, terminal T1 stores the content received in step S106 in its storage device. Terminal T1 may also store the content received in step S106 in storage device 5. The content stored in storage device 5 can be accessed by other devices via network N.

[0115] The generation request transmitted in step S102 may be transaction data (also referred to as generation request transaction data). In this case, the ledger system 10, upon receiving the transmitted generation request transaction data, stores the generation request transaction data in the distributed ledger 111. In this case, the processing of steps S103 to S104 may be executed by a smart contract based on the storage of the generation request transaction data in the distributed ledger 111 in step S102. In this case, the generation request transaction data includes an instruction to execute a smart contract that executes the above processing, and the execution unit 103 can execute the above processing in accordance with the instruction in response to the storage of the generation request transaction data in the distributed ledger 111 (step S102).

[0116] Next, a training process for training a machine learning model will be described. The training process is executed by a training system 40.

[0117] Fig. 8 is a flow diagram showing the training process of the machine learning model in this embodiment. Fig. 9A and Fig. 9B are explanatory diagrams showing examples of training content in this embodiment.

[0118] 8, the acquisition unit 401 acquires content (corresponding to training content). The number of pieces of content acquired by the acquisition unit 401 may be, for example, several thousand to several tens of thousands. The acquisition unit 401 provides the acquired content to the control unit 402.

[0119] The content acquired by the acquisition unit 401 may be content belonging to a content group having specific characteristics. The content acquired by the acquisition unit 401 may be, for example, a photographic image showing a scene under specific weather conditions (for example, clear skies, rainfall, snowfall, or thunder), an illustrated image depicting the scene, a photographic image showing a specific subject (a person with specific attributes, a specific animal, an object with specific functions, or the like), an illustrated image depicting a specific subject, or an illustrated image drawn with a specific touch (style, color, technique).

[0120] 9A and 9B show examples of training content acquired by the acquisition unit 401 when the specific feature of the content is "snowing weather conditions."

[0121] Fig. 9A is a photographic image of a snowy scene, showing falling snow and buildings covered in snow, while Fig. 9B is a photographic image of a snowy scene, showing falling snow and houses, cars, garages, and the like covered in snow.

[0122] In step S202, the determination unit 403 acquires content in which the third validity information is stored as a digital watermark from the content acquired in step S201. Specifically, the determination unit 403 determines whether or not the third validity information is stored as a digital watermark in each of the content acquired in step S201, and acquires the content in which it is determined that the third validity information is stored as a digital watermark.

[0123] In step S203, the training unit 405 uses the content determined in step S202 to have the third validity information stored as a digital watermark to perform a training process for the model 404. The model 404, which is a machine learning model that has undergone the training process in step S203, may also be referred to as a trained model. The model 404 may be used as the model 304.

[0124] Next, a generation process in which the generation system 30 generates content using the model 304 will be described.

[0125] Fig. 10 is a sequence diagram showing a generation process for generating content in this embodiment. Fig. 11 is an explanatory diagram showing an example of original content in this embodiment. Fig. 12 is an explanatory diagram showing an example of generated content in this embodiment. Fig. 13 is an explanatory diagram showing correspondence information in this embodiment. Fig. 14 is an explanatory diagram showing metadata of NFT in this embodiment.

[0126] In step S301 shown in Fig. 10, the acquisition unit 301 acquires original content (corresponding to first content). The owner of the original content acquired by the acquisition unit 301 is, for example, user U1 of terminal T1, and this case will be described as an example. The number of pieces of content acquired by the acquisition unit 301 is any number equal to or greater than one. The acquisition unit 301 provides the acquired content to the control unit 302.

[0127] An example of original content is shown in Fig. 11. The original content shown in Fig. 11 is an illustration image depicting a building under fine weather conditions.

[0128] In step S302, the determination unit 303 of the generation system 30 determines whether the first validity information is stored as a digital watermark in the original content acquired in step S301. If it is determined that the first validity information is stored as a digital watermark in the original content (Yes in step S302), the process proceeds to step S303. Note that if it is determined that the first validity information is not stored as a digital watermark in the original content, the process shown in this sequence diagram ends without generating content. For convenience of explanation, the illustration of the case where it is determined that the validity information is not stored as a digital watermark in the original content is omitted.

[0129] In step S303, the control unit 302 of the generation system 30 inputs the original content acquired in step S301 into the model 304.

[0130] In step S304, the control unit 302 of the generation system 30 acquires the content (corresponding to the generated content or the second content) generated by the model 304 as a result of inputting the original content to the model 304 in step S303.

[0131] An example of generated content is shown in Figure 12. The generated content shown in Figure 12 is an illustration of a building similar to the building in Figure 11 under snowfall weather conditions, and is content in which the feature "snowfall weather conditions" based on the training performed by model 304 has been added to the original content shown in Figure 11.

[0132] In step S305, the control unit 302 of the generation system 30 associates the identification information of the original content acquired in step S301, the identification information of the generated content acquired in step S304, and the identification information of the model 304, and stores them as correspondence information.

[0133] An example of the correspondence information is shown in Fig. 13. In the correspondence information shown in Fig. 13, "SC001" as the identification information of the original content, "GC001" as the identification information of the generated content, and "M304" as the identification information of the model (i.e., model 304) are shown in association with each other.

[0134] In step S306, the control unit 302 of the generation system 30 transmits the generated content acquired in step S304 and the correspondence information stored in step S305 to the control system 20. The control system 20 receives the transmitted generated content and correspondence information.

[0135] In step S307, the generation unit 202 of the control system 20 generates generation request information (also called a generation request) requesting the generation of an NFT and transmits it to the ledger system 10. The generation request is information requesting the generation of an NFT that corresponds one-to-one to the generated content received in step S306. The generation request includes at least correspondence information. The ledger system 10 receives the transmitted generation request.

[0136] In step S308, in response to receiving the creation request in step S307, the ledger system 10 generates transaction data including an NFT and stores it in the distributed ledger 111. Transaction data including an NFT can also be said to be transaction data indicating the creation of an NFT. An NFT is generated by storing transaction data including an NFT in the distributed ledger 111. When generating an NFT, the ledger system 10 generates a token ID (also referred to as an NFT-ID), which is identification information for the NFT, and assigns it to the NFT. The generated NFT has corresponding information included in the creation request received in step S307 as metadata. Note that the metadata may be stored in the storage device 5.

[0137] An example of metadata is shown in Fig. 14. The metadata shown in Fig. 14 is metadata for an NFT with a token ID of "001", and includes the corresponding information shown in Fig. 13 (specifically, "SC001" as identification information of the original content, "GC001" as identification information of the generated content, and "M304" as identification information of the model), and "U1" as information indicating the owner (i.e., user U1).

[0138] In step S309, the ledger system 10 transmits the NFT-ID generated in step S308 to the control system 20. The control system 20 receives the transmitted NFT-ID.

[0139] In step S310, the watermark storage unit 203 of the control system 20 stores the second validity information as a digital watermark in the generated content received in step S306. The second validity information includes the NFT-ID received in step S309.

[0140] In step S311, the watermark storage unit 203 of the control system 20 transmits the generated content in which the second validity information was stored in step S310 to the generation system 30. The generation system 30 receives the transmitted generated content. Note that the watermark storage unit 203 may transmit the generated content in response to receiving the content in step S306.

[0141] In step S312, the output unit 305 of the generation system 30 outputs the generated content received in step S311. Outputting the generated content may include, for example, transmitting the generated content to the terminal T1 that provided the original content. In this case, transmitting the generated content may be executed in response to receiving the original content in step S301. Outputting the generated content may also be transmitting the generated content to another device. Outputting the generated content may also be, for example, displaying the generated content on a display device of the generation system 30 or a display device of another device.

[0142] The generation request transmitted in step S307 may be transaction data (also referred to as generation request transaction data). In this case, the ledger system 10, upon receiving the transmitted generation request transaction data, stores the generation request transaction data in the distributed ledger 111. In this case, the processing of steps S308 to S309 may be executed by a smart contract based on the storage of the generation request transaction data in the distributed ledger 111 in step S307. In this case, the generation request transaction data includes an instruction to execute a smart contract that executes the above processing, and the execution unit 103 can execute the above processing in accordance with the instruction in response to the storage of the generation request transaction data in the distributed ledger 111 (step S307).

[0143] Next, a purchase process that is executed when generated content is purchased will be described. Note that the word "purchase" may also be expressed as "transfer."

[0144] Fig. 15 is a sequence diagram showing the process of purchasing generated content in this embodiment. Fig. 16 is an explanatory diagram showing an example of purchase transaction data in this embodiment. Fig. 17 and Fig. 18 are explanatory diagrams showing examples of transfer transaction data in this embodiment.

[0145] Here, we will explain an example of a purchase process that is executed when user U3 purchases generated content generated by the generation system 30. The purchase process includes a process of transferring ownership of the generated content and a process of providing compensation for the purchase. The compensation for the purchase is provided by user U3 to user U2, who is the provider of the original content, and to the administrator of the model.

[0146] It is assumed that FIG. 15 is executed in a state where the generating system 30 has acquired and holds the generated content.

[0147] Here, an example will be described in which user U3 views the generated content once and then purchases the generated content. If user U3 purchases the generated content without ever viewing it, steps S401 and S402 below may be omitted.

[0148] In step S401, the generation system 30 transmits the generated content to the terminal T3. The generation system 30 may transmit the generated content to the terminal T3 in response to receiving a request to acquire the generated content from the terminal T3, or may transmit the generated content to the terminal T3 spontaneously (in other words, without receiving a request to acquire the generated content from the terminal T3). The terminal T3 receives the transmitted generated content.

[0149] In step S402, terminal T3 presents the generated content received in step S401. For example, if the generated content is image content, terminal T3 displays the image content on a display screen, and if the generated content is audio content, terminal T3 outputs the audio content as audio through a speaker. It is assumed that the presented generated content will be perceived by a user. It is assumed that, by perceiving the generated content, the user will want to purchase the generated content (in other words, become the owner of the generated content).

[0150] The following describes the process when user U3 wishes to purchase the generated content. When user U3 wishes to purchase the generated content, he or she performs an operation on terminal T3 to start the process of purchasing the generated content. This operation is, for example, an operation on an operation button that is displayed on terminal T3 together with the presentation of the generated content. When this operation is performed, terminal T3 executes the following process.

[0151] In step S403, terminal T3 generates transaction data (also referred to as purchase transaction data) indicating that user U3 will purchase the generated content, and transmits the generated transaction data to the generation system 30. The generation system 30 receives the transmitted purchase transaction data.

[0152] An example of purchase transaction data generated by terminal T3 is shown in FIG.

[0153] The purchase transaction data shown in Figure 16 includes at least identification information of the generated content, information indicating the owner of the generated content, information indicating the purchaser of the generated content, and the amount of tokens indicating the purchase price (see Figure 16).

[0154] For example, the identification information of the generated content is "GC001", the information indicating the owner of the generated content is "GS30" (i.e., generation system 30), the information indicating the purchaser of the generated content is "U3" (i.e., user U3), and the amount of tokens indicating the purchase price is set to "100" (i.e., 100 tokens).

[0155] In step S404, the generation system 30 controls the storage of the purchase transaction data received in step S403 in the distributed ledger 111 of the ledger system 10. Specifically, the generation system 30 transmits the purchase transaction data received in step S403 to the ledger system 10. The ledger system 10 receives the transmitted purchase transaction data.

[0156] In step S405, the ledger system 10 stores the purchase transaction data received in step S404 in the distributed ledger 111. By storing the purchase transaction data in the distributed ledger 111, the ownership of the NFT corresponding to the generated content is transferred from the operator, who is the current owner of the generated content, to user U3, who is the purchaser of the generated content.

[0157] In step S406, the generation system 30 identifies the original content that was input as input data to the model 304 when the generated content to be purchased was generated. The generation system 30 can identify the original content by referencing the metadata of the generated content to be purchased and acquiring the "identification information of the original content" included in the metadata.

[0158] In step S407, the generation system 30 calculates the contribution of each of the original content and the model 304 to the generated content. For example, the generation system 30 calculates a similarity indicating the degree of similarity between the generated content and the original content, and obtains the calculated similarity as the contribution of the original content to the generated content. In this case, the generation system 30 can obtain the value obtained by subtracting the contribution of the original content from the total (100%) as the contribution of the model 304 to the generated content (see Equation 1 below).

[0159] Contribution of model 304 = 100% - contribution of original content (Equation 1)

[0160] For example, if the generation system 30 calculates that the similarity between the generated content and the original content is 70%, the contribution of the original content to the generated content can be determined to be 70%, and the contribution of the model 304 to the generated content can be determined to be 30%.

[0161] In step S408, the generation system 30 controls the storage of transaction data (also referred to as first transfer transaction data) indicating the transfer of tokens from user U3, the purchaser of the generated content, to user U2, the provider of the original content, in the distributed ledger 111 of the ledger system 10. Specifically, the generation system 30 generates the first transfer transaction data and transmits it to the ledger system 10. The ledger system 10 receives the transmitted first transfer transaction data. The transferred tokens are a portion (also referred to as the first portion) of the tokens paid by user U3 to the operator of the generation system 30.

[0162] An example of the first transfer transaction data generated by the generation system 30 in step S408 is shown in FIG.

[0163] 17 includes a token transfer source, a token transfer destination, and the amount of tokens to be transferred. Specifically, the token transfer source is "U3" (i.e., user U3), the token transfer destination is "U2" (i.e., user U2), and the amount of tokens to be transferred is "70" (i.e., 70 tokens).

[0164] In step S409, the ledger system 10 stores the transfer transaction data received in step S408 in the distributed ledger 111.

[0165] In step S410, the generation system 30 controls the storage of transaction data (also referred to as second transfer transaction data) in the distributed ledger 111 of the ledger system 10, indicating that tokens will be transferred from user U3, the purchaser of the generated content, to the operator of the generation system 30. Specifically, the generation system 30 generates the second transfer transaction data and transmits it to the ledger system 10. The ledger system 10 receives the transmitted second transfer transaction data. The transferred tokens are a portion (also referred to as the second portion) of the tokens paid by user U3 to the operator of the generation system 30, excluding the first portion.

[0166] An example of the second transfer transaction data generated by the generation system 30 in step S410 is shown in FIG.

[0167] 18 includes a token transfer source, a token transfer destination, and the amount of tokens to be transferred. Specifically, the token transfer source is "U3" (i.e., user U3), the token transfer destination is "GS30" (i.e., generation system 30), and the amount of tokens to be transferred is "30" (i.e., 30 tokens).

[0168] In step S411, the ledger system 10 stores the transfer transaction data received in step S410 in the distributed ledger 111.

[0169] The generation system 30 may determine the first portion of the tokens to be a larger portion of the tokens paid by user U3 to the operator of the generation system 30, the larger the contribution of the original content to the generation of the generated content. In this case, the generation system 30 may determine the second portion of the tokens to be the total amount of tokens paid by user U3 to the operator of the generation system 30 minus the first portion. The generation system 30 may determine the contribution of the original content to the generation of the generated content to be a larger value, the larger the similarity between the generated content and the original content.

[0170] 19 is a sequence diagram showing a modified example of the process for purchasing generated content in this embodiment. The process shown in FIG. 19 is a modified example of the process enclosed by the dashed line in FIG.

[0171] Here, another example of a purchase process executed when user U3 purchases generated content generated by generation system 30 will be described. The purchase process includes a process of transferring ownership of the generated content and a process of providing compensation for the purchase. The compensation for the purchase is provided by user U3 to user U2, who is the provider of the original content, the administrator of model 304, and user U1, who is the provider of the training content.

[0172] In step S407A, the generation system 30 calculates the contribution of each of the original content, the model 304, and the training content to the generated content. For example, the generation system 30 calculates a similarity indicating the degree of similarity between the generated content and the original content. The generation system 30 can then obtain the calculated similarity as the contribution of the original content to the generated content (see Equation 1).

[0173] Furthermore, the generation system 30 can obtain the contribution of the model 304 and the training content to the generated content by subtracting the contribution of the original content from the total (100%) (see Equation 2 below).

[0174] Contribution A of model 304 and training content = 100% - contribution of original content (Equation 2)

[0175] In this case, the breakdown of the contribution of the model 304 and the contribution of the training content can be determined, for example, by a predetermined ratio (for example, contribution of the model 304: contribution of the training content = 5:5, or 2:3, etc.) (see (Equation 3) and (Equation 4) below).

[0176] Contribution of model 304 = A × contribution ratio of model 304 (Equation 3)

[0177] Contribution of training content = A × contribution ratio of training content (Equation 4)

[0178] For example, if the generation system 30 calculates the similarity between the generated content and the original content to be 70%, and the breakdown of the contribution of the model 304 and the contribution of the training content is 5:5, the contribution of the original content to the generated content can be determined to be 70%, the contribution of the model 304 to the generated content to be 15%, and the contribution of the training content to the generated content to be 15%.

[0179] In addition, if there are multiple providers of training content, contributions proportional to the number of training content pieces provided may be allocated to the multiple providers. For example, if 10,000 pieces of training content pieces are used to train model 304 and the contribution of the training content pieces to the generated content is 15%, the contribution of a provider who provided 1,000 pieces of training content pieces may be set to 1.5%, and the contribution of a provider who provided 9,000 pieces of training content pieces may be set to 13.5%.

[0180] Steps S408 to S411 are the same as the processing steps of the same names in Fig. 15. However, the value calculated in step S407A is used as the amount of tokens included in the second transfer transaction data generated in step S410.

[0181] In step S421, the generation system 30 generates transaction data (also referred to as third transfer transaction data) indicating the transfer of tokens to the provider of the training content, and transmits the transaction data to the ledger system 10. The ledger system 10 receives the transmitted transfer transaction data.

[0182] An example of the third transfer transaction data generated by the generation system 30 in step S421 is shown in FIG.

[0183] The third transfer transaction data shown in Figure 20 includes the source of the token transfer, the destination of the token transfer, and the amount of tokens to be transferred. Specifically, the source of the token transfer is "U3" (i.e., user U3), the destination of the token transfer is "U1" (i.e., user U1), and the amount of tokens to be transferred is "15" (i.e., 15 tokens) (see Figure 20).

[0184] In step S422, the ledger system 10 stores the transfer transaction data received in step S421 in the distributed ledger 111.

[0185] If there are multiple providers of training content, transfer transaction data for transferring tokens to each of the multiple providers can be generated in step S421. In this case, the amount of tokens transferred to the multiple providers can be determined according to the number of training content pieces provided by the provider (e.g., proportional to the number of training content pieces provided by the provider), or can be set to a fixed value regardless of the number of training content pieces provided by the provider.

[0186] (Explanation of the Distributed Ledger System) The ledger system 10 (also called the distributed ledger system) described above will be explained in detail below.

[0187] A distributed ledger system is a system that stores and maintains information using P2P (peer-to-peer) network technology connected to multiple nodes. A node is an information processing device that performs predetermined processing by using a processor (e.g., a CPU) to execute a program using memory.

[0188] In a distributed ledger system, multiple nodes autonomously hold copies of information and keep them synchronized. This allows the system to properly store information while substantially preventing tampering, without using a privileged node (e.g., a centralized server or a server in a client-server model).

[0189] Furthermore, a device attempting to access the distributed ledger only needs to access one of the multiple nodes included in the distributed ledger system; in other words, there is no need to access a small number of devices, such as a centralized server. This avoids the concentration of communication or processing load on the centralized server, which can occur in a centralized system. This has the advantage that nodes do not require particularly high specifications for their resources (CPU, memory, etc.), and that communication lines connecting the nodes do not require particularly large communication capacities. This allows the distributed ledger system to be configured using general (or general-purpose) nodes or communication lines, which can contribute to reducing the required computer or communication resources and the costs required for nodes and communication lines.

[0190] Furthermore, a distributed ledger system can store information with high fault tolerance and can also access information with high fault tolerance. Generally, the multiple nodes included in a distributed ledger system are physically or network-distributedly located. If all of the multiple nodes included in a distributed ledger system stop, the distributed ledger system will stop. However, since it is rare for all of the multiple nodes that are physically or network-distributedly located to stop, the distributed ledger system rarely stops. This can be said to be an advantage over a centralized system, where if a centralized server stops, information cannot be stored or accessed.

[0191] With reference to Figures 21 to 25, we will explain the data structure of a distributed ledger, the execution of smart contracts, and the data structure of NFTs.

[0192] FIG. 21 is an explanatory diagram showing the data structure of a blockchain, which is an example of a distributed ledger.

[0193] A blockchain is a chain of blocks, which are units of record. Each block contains multiple transaction data and the hash value of the previous block.

[0194] FIG. 21 shows blocks B1, B2, and B3 included in the blockchain.

[0195] For example, block B2 contains the hash value of the previous block B1, which is calculated by performing a hash algorithm on the contents of block B1.

[0196] Furthermore, block B3 includes, as the hash value of block B2, a hash value calculated from multiple transaction data included in block B2 and the hash value of block B1.

[0197] In this way, a blockchain is structured so that blocks containing the contents of the previous block as a hash value are connected in a chain, which effectively prevents tampering with the recorded transaction data.

[0198] If past transaction data is changed (in other words, tampered with), the hash value of the block containing that transaction data will be different from the value before the change. In that case, to make the block containing the changed transaction data appear correct, all blocks after that block in the distributed ledger stored on multiple servers would have to be recreated, which is extremely difficult in reality. This feature makes it virtually impossible to tamper with transaction data contained in the blockchain.

[0199] When a node stores transaction data in a blockchain, it generates a block containing the transaction data to be stored and attempts to reach consensus on the generated block by executing processing based on a consensus algorithm with other nodes. Then, when consensus is reached, the node controls the storage of the block in the blockchain. This allows multiple nodes operating in an autonomous and decentralized manner to connect legitimate blocks to the blockchain. As a consensus algorithm, PBFT (Practical Byzantine Fault Tolerance), PoW (Proof of Work), PoS (Proof of Stake), or the like may be used. When Hyperledger Fabric is used as an example of a distributed ledger technology, a consensus algorithm does not need to be executed.

[0200] FIG. 22 is an explanatory diagram showing the data structure of transaction data.

[0201] The transaction data shown in Figure 22 includes a transaction body BP1 and a digital signature BP2 (also simply referred to as a signature). The transaction body BP1 is the data body included in the transaction data. The digital signature BP2 is generated by encrypting the hash value of the transaction body BP1 with the signature key (in other words, the private key) of the creator of the transaction data.

[0202] A node that receives transaction data can verify that the transaction body BP1 is legitimate (in other words, that it has not been tampered with) using the digital signature BP2 included in the transaction data. This makes it virtually impossible to tamper with the data included in the transaction body BP1. Furthermore, by storing successfully verified transaction data in the blockchain, the legitimacy of the transaction data stored in the blockchain can be maintained.

[0203] As described above, transaction data included in the blockchain is stored in a chain using the hash values ​​of the transaction data and the hash values ​​of the blocks. This allows the transaction data included in the blockchain to be stored and maintained substantially without being tampered with. This is an advantage over databases or distributed databases that simply store a collection of data.

[0204] 23 and 24 are explanatory diagrams showing transaction data related to the execution of a smart contract, and processing related to the execution of a smart contract, respectively.

[0205] A series of processes related to the execution of a smart contract using a distributed ledger will be described with reference to Figures 23 and 24.

[0206] In step SB1, the node stores transaction data B11, including contract code B12 that describes the processing of the smart contract, in the distributed ledger B10. For example, the node acquires transaction data B11 by receiving the transaction data B11 from an information processing device via communication or by the node itself generating the transaction data B11, and stores the acquired transaction data B11 in the distributed ledger B10. Step SB1 is performed before executing the smart contract.

[0207] In step SB2, the node stores transaction data B15, including instructions B16 for executing the smart contract, in the distributed ledger B10. For example, the node receives transaction data B15 from an information processing device via communication and stores the received transaction data B15 in the distributed ledger B10.

[0208] In step SB3, in response to the transaction data B15 including the instruction B16 being stored in the distributed ledger B10 in step SB2, the node reads the contract code B12 from the distributed ledger B10 and executes processing based on the contract code B12. The results of the processing may be included in the transaction data and stored in the distributed ledger B10.

[0209] Through the above series of processes, when the distributed ledger system receives transaction data B15 including instructions B16 for executing a smart contract, it automatically (i.e., without manual intervention) executes the processing in accordance with the instructions B16, enabling highly efficient (i.e., high speed or short processing time). Achieving highly efficient processing has the effect of reducing power consumption. Furthermore, since no manual intervention is required, it is possible to prevent human tampering with information, fraud, or human error. Furthermore, since the results of the processing thus executed are stored in the blockchain, it is virtually impossible to tamper with the results of the processing.

[0210] FIG. 25 is an explanatory diagram showing the structure of an NFT and metadata. An NFT is a token stored in a distributed ledger and is a unique token (in other words, a non-fungible token). NFTs are standardized, for example, as ERC (Ethereum Request for Comments) 721, but are not limited to this. NFTs may conform to a different standard or may be non-standard (for example, proprietary to an organization). While ERC 721 is a standard for unique tokens, the NFTs described herein do not necessarily have to be unique tokens.

[0211] 25 shows transaction data B21 stored in the distributed ledger. The transaction data B21 stores an NFT. The NFT includes a token ID (i.e., identification information that can uniquely identify the NFT).

[0212] The NFT has metadata. The metadata may be located in a location accessible via a network (e.g., storage device B22). A token URI indicating the location of the metadata is calculated using the NFT's token ID and a predetermined base URI.

[0213] The information managed as an NFT may be included in the transaction data B21 or in the metadata. Including the information managed as an NFT in the metadata has the advantage of reducing the amount of information included in the transaction data B21 (in other words, the information included in the blockchain). In this case, it can be said that the metadata contains the actual information managed as an NFT. When an image is managed as an NFT, a URL indicating the image data of the image may be managed as an NFT.

[0214] In the above-described embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that realizes the information processing device of the above-described embodiments is the following program.

[0215] In other words, this program causes a computer to execute an information processing method that acquires a first content, determines whether first legitimacy information indicating that the acquired first content is legitimate content to be input into a trained model is stored in the first content as an electronic watermark, and if it is determined that the first legitimacy information is stored in the first content, acquires second content generated by the trained model by inputting the first content into the trained model, and outputs the acquired second content.

[0216] Although the information processing method according to one or more aspects has been described based on the embodiments, the present invention is not limited to these embodiments. As long as it does not deviate from the spirit of the present invention, various modifications conceivable by those skilled in the art to the present embodiments, or forms constructed by combining components of different embodiments, may also be included within the scope of one or more aspects.

[0217] The present invention can be used in a system that promotes effective use of resources.

[0218] 1 Information processing system 5, B22 Storage device 10 Ledger system 11, 12, 13 Ledger server 20 Control system 30 Generation system 40 Training system 101, 201 Communication unit 102 Ledger processing unit 103 Execution unit 104 Storage unit 111, B10 Distributed ledger 202 Generation unit 203 Watermark storage unit 301, 401 Acquisition unit 302, 402 Control unit 303, 403 Determination unit 304, 404 Model 305 Output unit 405 Training unit B1, B2, B3 Block B11, B15, B21 Transaction data B12 Contract code B16 Instruction BP1 Transaction body BP2 Digital signature N Network T1, T2, T3 Terminal U1, U2, U3 User

Claims

1. An information processing method comprising: acquiring a first content; determining whether or not first legitimacy information indicating that the acquired first content is legitimate content to be input into a trained model is stored in the first content as an electronic watermark; if it is determined that the first legitimacy information is stored in the first content, acquiring a second content generated by the trained model by inputting the first content into the trained model; and outputting the acquired second content.

2. The information processing method of claim 1, further comprising: when the second content is acquired, controlling so that second legitimacy information is stored in the second content as an electronic watermark; the second legitimacy information is information indicating that the second content is legitimate content output by the trained model; and when outputting the second content, outputting the second content in which the second legitimacy information is stored.

3. The information processing method described in claim 1 or 2, wherein a first NFT (Non-Fungible Token) that corresponds one-to-one to the first content and is linked to the owner of the first content as the owner of the first NFT is stored in the distributed ledger, and when the second content is acquired, a second NFT that corresponds one-to-one to the second content and is linked to the operator of an information processing system that executes the information processing method as the owner of the second NFT is further stored in the distributed ledger.

4. The information processing method of claim 3, further comprising: storing in the distributed ledger purchase transaction data indicating that a user purchases the second content, the purchase transaction data further indicating an amount of value information equivalent to the purchase price of the second content; storing in the distributed ledger first transfer transaction data transferring a first portion of the value information from the user to the owner of the first content; and storing in the distributed ledger second transfer transaction data transferring a second portion of the value information excluding the first portion from the user to the operator.

5. The information processing method of claim 4, wherein the first portion is a larger portion of the value information the greater the contribution of the first content to the generation of the second content, and the second portion is a portion of the value information obtained by subtracting the first portion from the entire value information.

6. The information processing method according to claim 5, wherein the magnitude of the contribution of the first content in generating the second content increases as the degree of similarity between the second content and the first content increases.

7. An information processing method as described in claim 1 or 2, further comprising the step of prohibiting output of a second content generated by the trained model when it is determined that the first legitimacy information is not stored in the first content.

8. The information processing method of claim 1 or 2, wherein the trained model is a machine learning model trained to generate and output a fifth content based on a fourth content input to the machine learning model using a third content in which third legitimacy information indicating that the third content is legitimate content to be input to the trained model is stored as an electronic watermark, and the fifth content is content in which features related to the training performed by the machine learning model have been added to the fourth content.

9. The information processing method according to claim 1, wherein the trained model is a neural network model.

10. An information processing method comprising: acquiring a third content; determining whether third legitimacy information indicating that the acquired third content is legitimate content to be input to a machine learning model is stored in the third content as a digital watermark; and if it is determined that the third legitimacy information is stored in the third content, training the machine learning model to generate and output a fifth content based on a fourth content input to the machine learning model; the fifth content is content in which features based on the training performed by the machine learning model have been added to the fourth content.

11. An information processing system comprising: an acquisition unit that acquires a first content; a judgment unit that judges whether first legitimacy information indicating that the acquired first content is legitimate content to be input to a trained model is stored in the first content as an electronic watermark; a control unit that acquires a second content generated by the trained model by inputting the first content into the trained model when it is determined that the first legitimacy information is stored in the first content; and an output unit that outputs the acquired second content.

12. An information processing system comprising: an acquisition unit that acquires a third content; a judgment unit that judges whether third legitimacy information indicating that the acquired third content is legitimate content to be input to a machine learning model is stored in the third content as an electronic watermark; and a training unit that trains the machine learning model to generate and output a fifth content based on a fourth content input to the machine learning model when it is judged that the third legitimacy information is stored in the third content, wherein the fifth content is content to which features based on the training performed by the machine learning model have been added to the fourth content.

13. A program for causing a computer to execute the information processing method according to claim 1.

14. A program for causing a computer to execute the information processing method according to claim 10.

Images