Composite Image Management System, Composite Image Management Method, and Composite Image Management Program

Abstract

Provided are a composite video generated by inserting an avatar, and a composite video management system, method, and program for appropriately managing an information output mode for the composite video. 【Solution means】The composite video management system includes an insertion target video generation unit 3 that generates an insertion target video for an avatar based on video material input from a video material input unit 1, a person video input unit 4 that inputs a person video serving as the source of the avatar, an avatar generation unit 5 that generates an avatar from the person video, an avatar insertion unit 7 that inserts the generated avatar into the insertion target video, a video information generation unit 8 that generates composite video information which is information regarding the composite video, a composite video update information generation unit 9 that generates update information for the composite video information, a composite video information acquisition unit 14 that acquires the composite video information from a distributed ledger of a storage destination, a composite video output unit 15 that outputs a composite video with the composite video information added, an update information acquisition unit 16 that acquires the update information from the distributed ledger, and an update information output unit 17 that outputs the acquired update information.

Description

Technical Field

[0001] The present invention relates to a technique for managing a composite video generated by inserting an avatar generated based on a person video into an insertion target video composed of one or more video materials.

Background Art

[0002] In recent years, with the improvement of processing capabilities in electronic computers such as computers, a large number of computer graphics of human figures, so-called avatars, that reflect the characteristics of real people have been widely used. For example, an avatar is used as one's own icon for use in SNS (Social Networking Service), or one's own avatar is used as the character of the protagonist in an online game or the like. In addition, services have been proposed in which some of the characters appearing in content such as still images and moving images are replaced with one's own avatar for viewing and the like.

[0003] By using an avatar that reflects one's own characteristics in this way, for example, when a protagonist consisting of an avatar expressing the characteristics of a user in a game battles an enemy character, an effect of improving the user's sense of immersion in the game world occurs. By using an avatar that abstractly represents the user himself / herself as an icon indicating the user in SNS, it is expected that an effect such as promoting communication between users in a virtual space in the same sense as the real world will occur.

[0004] Both Patent Documents 1 and 2 disclose techniques for using avatars that simulate the actual appearances of the player himself / herself or co-players in a computer game that expresses a virtual space using a head-mounted display.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] However, when generating a video in which some of the characters appearing in contents such as still images and videos are replaced with one's own avatar, the handling of such a video becomes a problem. With the improvement of image processing technology in recent years, as the quality of such a synthesized video improves, the possibility that the generated synthesized video is not recognized as a result of a synthesis process different from reality is increasing. Due to such misunderstandings, confusion such as so-called fake news occurs, so it is necessary to take measures so that it can be easily understood that the synthesized video is a creative work different from the original. For example, when taking measures such as performing image processing that impairs the sense of reality, there is a problem that the quality of the synthesized video deteriorates.

[0007] In recent years, not only do individuals enjoy the generated synthesized videos, but it has also become common to widely disclose them on SNS and the like. Furthermore, it is also assumed that the generated synthesized videos will be used by a third party, whether for a fee or free of charge. However, for example, when the presence or absence and content of the license for using the video such as a still image or video that is the insertion target of the avatar that is the source of the synthesized video are not clear, there is a problem that a situation may occur in which the rights of the legitimate rights holder are inadvertently infringed by a service that discloses the synthesized video on SNS or the like or a third party that uses the synthesized video. Regarding technologies for solving these problems, no disclosure is made in either Patent Document 1 or 2.

[0008] The present invention has been made in view of the above problems, and an object thereof is to provide a technology for appropriately managing an output mode of a synthesized video generated by inserting an avatar generated based on a person video into an insertion target video composed of one or more video materials and information related to the synthesized video. [Means for Solving the Problems]

[0009] To achieve the above object, a composite video management system according to claim 1 is a composite video management system that manages the output mode of a composite video generated by inserting an avatar generated based on a predetermined person video into a partial area of a video to be inserted, comprising: an insertion target video generation means for generating the insertion target video based on one or more video materials; an avatar generation means for generating the avatar based on the person video; an avatar insertion means for generating a composite video by inserting the avatar generated by the avatar generation means into all or part of the video materials in the insertion target video; a composite video information generation means for generating composite video information including at least one of generation information which is information regarding the generation mode of the composite video, right information which is information regarding the right relationship of the composite video, and usage information which is information regarding the usage conditions of the composite video; a composite data generation means for generating composite data by integrating all or part of the composite video information generated by the composite video information generation means and the composite video; and a composite data output means for outputting the composite data generated by the composite data generation means. Disclosure condition information generation means for generating disclosure condition information regarding disclosure conditions of one or more pieces of information included in the synthesized video information, which is determined according to attribute information regarding the attribute of the output destination of the synthesized data; determination means for determining whether the attribute information regarding the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when generating the synthesized data regarding the predetermined output destination; and information extraction means for extracting information determined by the determination means to satisfy the disclosure conditions from among one or more pieces of information included in the synthesized video information. The synthesized data generation means generates synthesized data integrated with the synthesized video with the information extracted by the information extraction means as part of the synthesized video information. characterized in that

[0011] Also, to achieve the above object, in the composite video management system according to claim 2 it further comprises: an update information generation means for generating update information which is information for updating the content of the composite video information; a disclosure condition information generation means for generating disclosure condition information which is information regarding the disclosure conditions of one or more pieces of information included in the update information determined according to attribute information which is information regarding the attributes of the output destination; a determination means for determining whether the attribute information regarding the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when outputting the update information to the predetermined output destination; and an information extraction means for extracting information determined by the determination means to satisfy the disclosure conditions from one or more pieces of information included in the update information. The composite data output means is characterized by outputting the information extracted by the information extraction means separately from the composite data.

[0012] Also, to achieve the above object, in the composite video management system according to claim 3The composite video management method according to the present invention manages the output mode of a composite video generated by inserting an avatar generated based on a predetermined person video into a partial area of an insertion target video, and includes an insertion target video generation step of generating the insertion target video based on one or more video materials, an avatar generation step of generating the avatar based on the person video, an avatar insertion step of generating a composite video by inserting the avatar generated in the avatar generation step into all or part of the video materials in the insertion target video, a composite video information generation step of generating composite video information including at least one of generation information which is information regarding the generation mode of the composite video, right information which is information regarding the right relationship of the composite video, and use information which is information regarding the use conditions of the composite video, A synthesized data generation step of generating synthesized data in which all or part of the synthesized video information generated in the synthesized video information generation step and the synthesized video are integrated; outputting all or part of the composite video information generated in the composite video information generation step and the composite video The integrated synthesized data is in an output step, An update information generation step of generating update information which is information for updating the content of the synthesized video information; a disclosure condition information generation step of determining a condition for disclosing one or more pieces of information included in the update information determined according to attribute information regarding the attribute of the output destination of the synthesized data, and a disclosure timing when the disclosure condition is satisfied; a determination step of determining whether the attribute information regarding the predetermined output destination satisfies the condition for disclosing the one or more pieces of information in the disclosure condition information when outputting the update information to the predetermined output destination; and an information extraction step of extracting information determined to satisfy the disclosure condition in the determination step from among one or more pieces of information included in the update information. In the output step, separately from all or part of the synthesized video information and the synthesized video, the information extracted in the information extraction step from among one or more pieces of information included in the update information is output at the timing determined by the disclosure condition information. which is characterized in that.

[0014] Also, to achieve the above object, the composite video management program according to claim 4 causes a computer to manage the output mode of a composite video generated by inserting an avatar generated based on a predetermined person video into a partial area of an insertion target video, and includes an insertion target video generation function for generating the insertion target video based on one or more video materials for the computer, an avatar generation function for generating the avatar based on the person video, an avatar insertion function for generating a composite video by inserting the avatar generated by the avatar generation function into all or part of the video materials in the insertion target video, a composite video information generation function for generating composite video information including at least one of generation information which is information regarding the generation mode of the composite video, right information which is information regarding the right relationship of the composite video, and use information which is information regarding the use conditions of the composite video, a composite data generation function for generating composite data integrating all or part of the composite video information generated by the composite video information generation function and the composite video, and a composite data output function for outputting the composite data generated by the composite data generation function, An update information generation function that generates update information which is information for updating the content of the composite video information, a disclosure condition information generation function that generates disclosure condition information which is information regarding disclosure conditions of one or more pieces of information included in the update information determined according to attribute information which is information regarding the attributes of the output destination, a determination function that determines whether or not the attribute information regarding the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when outputting the update information to the predetermined output destination, and an information extraction function that extracts information determined to satisfy the disclosure conditions by the determination function from among one or more pieces of information included in the update information are executed, and in the composite data output function, information extracted by the information extraction function is output separately from the composite datacharacterized by causing

Advantages of the Invention

[0015] According to the present invention, it is possible to appropriately manage a composite video generated by inserting an avatar generated based on a person video into an insertion target video composed of one or more video materials, and an output mode of information regarding the composite video.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Modes for Carrying Out the Invention

[0017] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following embodiments, examples considered to be the most appropriate as embodiments of the present invention are described. Of course, the content of the present invention should not be construed as being limited to the specific examples shown in the present embodiments. Any configuration having the same operation and effect, even if it is other than the specific configuration shown in the embodiments, is of course included in the technical scope of the present invention.

[0018] (Embodiment 1) First, the composite video management system according to Embodiment 1 will be described. As shown in FIG. 1, the composite video management system according to Embodiment 1 includes a video material input unit 1 for inputting video materials used as materials for composite videos, a material information generation unit 2 for generating video material information which is information about the input video materials, an insertion target video generation unit 3 for generating an insertion target video which is a video to which an avatar is to be inserted based on one or more video materials input via the video material input unit 1, a person video input unit 4 for inputting a person video which is the source of the avatar to be inserted into the insertion target video, an avatar generation unit 5 for generating an avatar from the person video input via the person video input unit 4, an avatar information generation unit 6 for generating information about the avatar generated by the avatar generation unit 5, an avatar insertion unit 7 for inserting the avatar generated by the avatar generation unit 5 into the insertion target video, a video information generation unit 8 for generating composite video information which is information about the composite video based on the video material information and the avatar information, a composite video update information generation unit 9 for generating update information for the composite video information, a token generation unit 10 for generating a composite video token which is a non-fungible token having a one-to-one correspondence with the composite video, a transaction generation unit 11 for generating a transaction which is information stored in a block in a distributed ledger (described later) having a one-to-one correspondence with the composite video token and including the composite video information and the update information, an electronic signature generation unit 12 for generating an electronic signature which is data for proving that the generation and output of the information included in the transaction are in accordance with the intention of the holder of the composite video token, an output unit 13 for outputting the transaction and the electronic signature corresponding to the transaction to the distributed ledger having a correspondence with the composite video token, a composite video information acquisition unit 14 for acquiring the composite video information from the distributed ledger, a composite video output unit 15 for outputting a composite video with the composite video information added, an update information acquisition unit 16 for acquiring the update information from the distributed ledger, and an update information output unit 17 for outputting the acquired update information.

[0019] The pixel material input unit 1 is for inputting pixel materials which are image data used for generating a composite video. The composite video is composed of an insertion target video and an avatar inserted into all or part of the insertion target video. The pixel materials input via the pixel material input unit 1 are directly used for generating the insertion target video. The insertion target video is composed of one or more pixel materials. As a simple configuration, a single pixel material may be used as the insertion target video as it is, or a single pixel material may be used as the insertion target video after performing image processing such as enlargement, reduction, rotation, and partial modification. Also, when the insertion target video is composed of a plurality of pixel materials, after performing image processing on each pixel material as necessary, each pixel material is arranged at a predetermined position, or a plurality of pixel materials are superimposed, etc., to form the insertion target video. The form of the pixel material may be either a 2D video or a 3D video, and may also be either a moving image or a still image. Also, the content of the pixel material may be realistic (for example, something photographed using an imaging device such as a camera of a real scene, etc.), or may be pictorial (for example, an illustration created by an illustrator). In the following description of Embodiment 1, the case where the pixel material is generated using an imaging device is assumed.

[0020] The material information generation unit 2 is for generating video material information which is information regarding the video material input via the video material input unit 1. Specifically, the material information generation unit 2 includes, for example, at least one or more of the generation information which is information regarding the generation mode of the video material, the right information which is information regarding the right relationship of the video material, and the usage information which is information regarding the usage conditions when using the video material in the completed video. However, it may also include information other than these. The generation information includes, for example, when the video material is realistically generated using an imaging device, information regarding the photographer, shooting location, shooting equipment, shooting date, shooting environment (temperature, weather, etc.), content of the processing from the original video, etc. When the video material is generated pictorially like an illustration, or even if it is realistic but generated without using an imaging device, it is information including one or more of the information regarding the creator, creation equipment (painting materials, computer used for creation, software, etc.), creation date, and specific matters of the original video if there is a reference original video. However, it may also include information other than these. The right information includes, for example, one or more of the information regarding the creator, the right holder of intellectual property rights such as copyright, the duration of the intellectual property rights, the licensee, the content of the usage rights held by the licensee, etc. However, it may also include information other than these. The usage information includes, for example, the usage conditions of the composite video using the video material (how far to allow the public range of the composite video, how far to allow the usage of the composite video (video distribution, output to paper media such as glossy paper or postcards of still images, etc.), how far to allow commercial use, etc.), the usage conditions of the video material in the generation of the composite video (how far to allow the processing when performing processing, which other video materials to allow to be used in combination in the same composite video, etc.), the range of allowable attributes for each of the person who generates the composite video using the video material and the person who uses the generated composite video (gender, age, nationality, creed, identification information, eligibility to use this system (for example, paid user or free user), etc.), one or more of the information regarding the usage fee of the video material, etc. However, it may also include information other than these.As a specific mode of generating material information by the material information generation unit 2, it may be configured to artificially input data constituting the content of each piece of information when inputting material video. For example, regarding information such as the shooting date and shooting equipment, it may be generated based on the information included in the data of the material video. Furthermore, when the video material is publicly available video data, it may be generated based on the public information referring to the video material.

[0021] The insertion target video generation unit 3 is for generating an insertion target video based on one or more video materials. The insertion target video is the original video of the composite video. More specifically, it has a relationship such that the composite video is the one obtained by performing avatar insertion processing on the insertion target video. The insertion target video generation unit 3 performs necessary processing on one or more video materials, and when using a single video material, it directly uses it as the insertion target video. When using a plurality of video materials, it generates the insertion target video by performing processing such as arranging each at a predetermined position. As a specific configuration of the insertion target video generation unit 3, for example, it can be configured to have an image processing function using the prior art, and the configuration of the generated insertion target video can also be a moving image / still image, color video / black-and-white video, 2D video / 3D video, etc. according to the form of the completed video.

[0022] The human image input unit 4 is for inputting a human image which is a material used for generating an avatar that constitutes a composite image. Specifically, the human image input unit 4 has a function of inputting a human image which is an image related to the whole or a part of a person who is the source of the avatar to be inserted into the video to be inserted. As a specific configuration, it may be configured to input an image from the outside, or may be configured to include an imaging mechanism for acquiring an image. As a specific form of the human image, it may be a full-body image of the target person, or may be a partial image such as a face image. Also, it may be either a still image or a moving image, and may be either a 2D image or a 3D image. In the first embodiment, an explanation will be given using an image composed of a 2D and still image related to the face of the target person as the human image. However, it goes without saying that human images other than such a configuration can also be used in the composite image generation system according to the first embodiment.

[0023] The avatar generation unit 5 is for generating an avatar to be inserted into and replace all or part of the video materials constituting the video to be inserted, based on the person video input through the person video input unit 4. Specifically, as the configuration of the avatar, the avatar generation unit 5 may be configured from skeleton information (bones), surface information (skin), and weight information that defines the relationship between the two. However, it is also preferable to adopt a configuration consisting only of surface information including information on the three-dimensional shape and color tone on the surface. As an even simpler configuration, it may be configured by the image data itself. Also, as the avatar generated by the avatar generation unit 5, an avatar corresponding to a full-body image of a person may be generated, but an avatar consisting of only a part of a person, for example, only a part of the face, may also be generated. In the first embodiment, an avatar consisting only of the head above the neck is generated. The avatar generation unit 5 extracts feature points set according to the features (eyes, eyebrows, nose, mouth, ears, hairstyle, etc.) on the surface and the positions and shapes of internal features (joints, etc.) of the person video based on the person video input through the person video input unit 4, and reflects the positional relationship between the extracted feature points in the avatar, thereby generating a realistic avatar that reflects the physical characteristics of the model. However, when adopting a simpler configuration, for example, an avatar may be generated that uses the person video as it is or has a configuration that only performs the minimum necessary corrections such as adjusting the size and orientation of the person video.

[0024] The avatar information generation unit 6 is for producing avatar information, which is information about the avatar generated by the avatar generation unit 5. Specifically, the avatar information generation unit 6 includes, for example, at least one or more pieces of information among generation information, which is information about the generation mode of the avatar, rights information, which is information about the rights relationship of the avatar, and usage information, which is information about the usage conditions when using the avatar in the completed video. However, it may also include information other than these. The generation information includes, for example, at least one or more pieces of information among information about the photographer, shooting location, shooting equipment, shooting date, shooting environment, etc. of the person video that is the basis of the avatar, the content of the processing of the person video during avatar generation, the computer and software that performed the processing, etc., information about the creation date and creator of the avatar. However, it may also include information other than these. The rights information includes one or more pieces of information among information about the author of the avatar and the person video that is the basis of the avatar, the rights holder of intellectual property rights such as copyright, the duration of the intellectual property rights, the licensee, the content of the license rights held by the licensee, etc. However, it may also include information other than these. The usage information includes, for example, usage conditions of the composite video using the avatar (how far to allow the public scope of the composite video, how far to allow the usage of the composite video (such as video distribution, output to paper media such as glossy paper or postcards of still images, etc.), how far to allow commercial use, etc.), usage conditions of the avatar in composite video generation (how far to allow processing when performing processing, conditions of video materials that can be used in the same composite video, etc.), the range of acceptable attributes for each of the person who generates the composite video using the avatar and the person who uses the generated composite video (gender, age, nationality, creed, identification information, the qualification for using this system (for example, paid user or free user), etc.). However, it may also include information other than these.

[0025] The avatar insertion unit 7 is for inserting the generated avatar into all or part of the video material in the video to be inserted to generate a composite video. Specifically, the avatar insertion unit 7 has a function of completing the composite video by inserting the avatar into the video to be inserted through a process of replacing all or part of the video material area in the video to be inserted with the avatar generated by the avatar generation unit 5. When the video to be inserted is a still image, the avatar insertion unit 7 acquires information regarding the size and direction of the replacement target in the video to be inserted, appropriately changes the size and direction of the avatar based on this information, and then inserts the avatar into the video to be inserted in a manner of replacing the replacement target. When the video to be inserted is a moving image, the avatar insertion unit 7 acquires information regarding the motion content of the replacement target in the video to be inserted, for example, by detecting the positional variation of feature points, appropriately changes the size, direction, and motion of the avatar based on this information, and then inserts the avatar into the video to be inserted in a manner of replacing the replacement target. As the replacement target of the avatar, it is usually desirable to use the video related to the person in the video to be inserted, but it is not necessary to be limited to this. For example, when the content video of a locomotive is arranged, it may be configured to insert an avatar consisting of a part of the face image in front of the locomotive.

[0026] The video information generation unit 8 is for generating composite video information which is information regarding the composite video. Specifically, the video information generation unit 8 includes a composite video information generation unit 19 that generates composite video information including information regarding the generation mode, usage conditions, etc. of the composite video, and a disclosure condition information generation unit 20 that generates disclosure condition information which is information regarding the disclosure conditions of the composite video information generated by the composite video information generation unit 19.

[0027] The composite video information generation unit 19 is for generating information including at least one of generation information which is information regarding the generation mode of the composite video, right information which is information regarding the right relationship of the composite video, and usage information which is information regarding the usage conditions of the composite video, as composite video information which is information regarding the composite video. Here, the generation information includes, for example, generation information regarding the video material / avatar that is the source of the composite video, in addition to information regarding the processor who performed the avatar insertion process during the generation of the composite video, the equipment (computer, software, etc.) used in the process, information regarding the date when the avatar insertion process was carried out, and information regarding the authenticity of the composite video (information regarding whether it is a synthesized video and a creative work (= the video content itself does not actually exist), whether the individual video materials / avatars constituting the composite video are videos of actually existing things, etc.), but it may also include information other than these. The right information includes, for example, information regarding the creator, the right holder of intellectual property rights such as copyright, the duration of the intellectual property rights, the licensee, and the content of the license rights held by the licensee, for each of the composite video and the video materials / avatars used in the composite video, but it may also include information other than these. The usage information includes, for example, the usage conditions of the composite video (to what extent the public range of the composite video is allowed, to what extent the usage of the composite video (video distribution, output to paper media such as glossy paper or postcards of still images, etc.) is allowed, to what extent commercial use is allowed, etc.), the range of attributes that can be allowed for the person using the generated composite video (including cases where all usage conditions are allowed and cases where only certain usage conditions are allowed. Gender, age, nationality, creed, identification information, the qualification for using this system (for example, paid user or free user), etc.), and information regarding the usage fee of the composite video, etc., but it may also include information other than these. Regarding the usage information, when the usage conditions of the composite video are limited to the range that satisfies the usage conditions of the video materials / avatars, a condition determination means for determining whether the usage conditions of the composite video are within the range of the usage conditions of the video materials / avatars may be provided during the generation of the usage information, and the configuration may be such that the usage conditions outside the range are not included in the usage information.In addition, when changing the usage conditions according to the user of the synthetic video, it is preferable to generate usage information in a configuration including a plurality of changed usage conditions.

[0028] The disclosure condition information generation unit 20 is for generating disclosure condition information regarding one or more pieces of information included in the synthetic video information, which is determined according to the attribute information regarding the output destination of the synthetic data (if not integrated, the synthetic video and the synthetic video information respectively) obtained by integrating all or part of the synthetic video and the synthetic video information. The disclosure condition information preferably includes information specifying whether to disclose each item of the synthetic video information generated by the synthetic video information generation unit 19 according to the attribute information regarding the output destination of the synthetic video, but may include other information. For example, when "high-paying user", "low-paying user", and "non-paying user" are assumed as the attributes of the output destination (usually the user of the synthetic video), the disclosure condition of the "synthetic video information stating that only viewing of the synthetic video is permitted as the usage condition" is set to the case where "the attribute of the output destination is a non-paying user", and the disclosure condition of the "synthetic video information stating that only viewing and video distribution of the synthetic video are permitted as the usage condition" is set to the case where "the attribute of the output destination is a low-paying user", and the disclosure condition information is generated.

[0029] The synthetic video update information generation unit 9 is for generating update information of the synthetic video information. Specifically, the synthetic video update information generation unit 9 includes an update information generation unit 22 that generates update information, which is information for updating the content of the synthetic video information, and a disclosure condition information generation unit 23 that generates disclosure condition information regarding one or more pieces of information included in the update information, which is determined according to the attribute information of the output destination.

[0030] The update information generation unit 22 is for generating update information which is information for updating the content of the composite video information. Specifically, the update information generation unit 22 has a function of generating, in a state where information for specifying an item to be changed in the composite video information as the update information and information after the change regarding the item are associated with each other. For example, the update information generation unit 22 generates, as the update information, information indicating that the usage condition of the composite video, which was only for video distribution as the usage, is changed to a content enabling both video distribution and output to a paper medium.

[0031] The disclosure condition information generation unit 23 is for generating disclosure condition information which is information regarding the disclosure condition of one or more pieces of information included in the update information determined according to the attribute information which is information regarding the attribute of the output destination. The disclosure condition information is, for example, information including content that defines whether or not to disclose the information of each item of the update information generated by the update information generation unit 22 according to the attribute information which is information regarding the attribute of the output destination, but may also include other information. For example, the disclosure condition information generation unit 23 generates, as the disclosure condition information, disclosure condition information that enables the disclosure of the disclosure condition of "update information indicating that the fee for newly joining a paid service is lower than before" only when the attribute of the output destination is "non-payer" when "non-payer" and "payer" are assumed as the attributes of the output destination.

[0032] The token generation unit 10 is for generating a synthetic video token, which is a non-fungible token having a one-to-one correspondence with the generated synthetic video. The so-called "non-fungible token" means a so-called NFT (Non-Fungible Token), that is, a token having the property of being non-substitutable with other tokens by having unique data, and is issued based on, for example, ERC721 which is a standard of Ethereum (registered trademark). The non-fungible token in the first embodiment of the present invention is issued based on ERC721 or other predetermined standards, and the transaction history of the synthetic video token which is the object, information about the holder, synthetic video information which is information about the corresponding synthetic video, disclosure condition information about the synthetic video information, update information about the synthetic video information, disclosure condition information about the update information, etc. are recorded in a distributed ledger on a blockchain in a corresponding relationship with the synthetic video token. The information recorded regarding the synthetic video token is generated by the transaction generation unit 11 described later, and is output to the distributed ledger by the output unit 13 together with the electronic signature generated by the electronic signature generation unit 12.

[0033] A "blockchain" is a technology for data synchronization while utilizing cryptographic techniques among a plurality of computers constituting a distributed network. Specifically, each block is composed of a set of information regarding tokens such as agreed transaction records and information for connecting to other blocks (information of the previous block), and the blockchain is constituted by connecting a plurality of such blocks. Even if data tampering occurs in a part of the plurality of computers, correct data is selected by a majority vote with other computers, so it has the characteristic that it is extremely difficult to destroy or tamper with data. As classifications of blockchains, there are a public blockchain in which there is no restriction on the participants who participate in the majority vote, that is, an unspecified person can participate in the recording process of the distributed ledger, and a consortium blockchain and a private blockchain in which only a plurality or a single specified person can participate in the majority vote, that is, only a predetermined specified person can participate in the recording process of the distributed ledger.

[0034] As a specific aspect of associating the synthetic video token with the synthetic video, it is in the form of associating the identifier of the synthetic video token with the identification information of the synthetic video. More specifically, it may be in the form of making the identifier of the synthetic video token match the identification information of the synthetic video. Also, regarding the generation of non-fungible tokens, the token generation unit 10 may perform it on its own, or it may be in the form that an external system generates by giving a predetermined command to an external system directly or indirectly connected to the token generation unit 10. Also, regarding the specific form of non-fungible tokens, it is not limited to those compliant with ERC721 which is the standard of Ethereum (registered trademark), and any form may be used as long as it has non-fungible properties and information such as transaction history can be stored in a distributed ledger.

[0035] The transaction generation unit 11 is for generating a transaction which is information stored in a distributed ledger having a one-to-one correspondence with the synthetic video. The transaction generation unit 11 has a function of generating a transaction consisting of information including information about the owner of the synthetic video, synthetic video information which is information about the synthetic video, disclosure condition information about the synthetic video information, update information which is information for updating the synthetic video information, and disclosure condition information about the update information, and outputting it to the electronic signature generation unit 12 and the output unit 13. The transaction generation unit 11 may generate a transaction in a form including the synthetic video information and the update information in a single transaction, but more preferably, it generates a transaction including the synthetic video information when the synthetic video information is generated, and then generates another transaction including the update information when the update information is generated.

[0036] The electronic signature generation unit 12 is for generating an electronic signature which is data for proving that the information included in the transaction is regular. Specifically, the electronic signature generation unit 12 has a function of generating a hash value of the transaction and encrypting the hash value with a private key to generate an electronic signature. Note that the "hash value" is a fixed-length value obtained by applying a certain calculation procedure to the original data. Since the calculation procedure is irreversible, it is considered impossible to restore the original data from the hash value. Also, the "private key" is a sequence of numbers used for the encryption process, and it has a configuration that can be decrypted by the corresponding "public key".

[0037] The output unit 13 is for outputting the transaction generated by the transaction generation unit 11, together with the corresponding electronic signature, to the distributed ledger associated with the composite video token. In this embodiment, the output unit 13 is configured to directly output the transaction and the corresponding electronic signature. However, it may also be configured to only instruct other components (including those provided outside the system) to output. In the first embodiment, the output unit 13 is directly or indirectly connected to the network where the distributed ledger is installed and is configured to output predetermined data to the distributed ledger. Regarding the data output from the output unit 13, a hash value is generated for the transaction, and the hash value is compared with the data obtained by decrypting the electronic signature corresponding to the transaction with the public key. If the two values are different, it is determined that the transaction is not a legitimate transaction and the registration to the distributed ledger is rejected. If the two values match, it is determined that the transaction is a legitimate transaction, and the information constituted by the transaction is stored in the block constituting the distributed ledger. Regarding the correspondence between the transaction and the distributed ledger, for example, it may be configured by associating the identification information of the distributed ledger with the identification information of the transaction. More simply, the same information as the identification information of the transaction may be used as the identification information of the distributed ledger. The identification information of the distributed ledger is used not only when the output unit 13 outputs information but also when the composite video information acquisition unit 14 and the update information acquisition unit 16 acquire information.

[0038] The composite video information acquisition unit 14 is for acquiring composite video information, which is information regarding the composite video, and disclosure condition information regarding the composite video information, and outputting the same to the composite video output unit 15. Specifically, the composite video information acquisition unit 14 accesses a distributed ledger that corresponds one-to-one with the target composite video, and has a function of acquiring the composite video information and the disclosure condition information regarding the composite video information from among the transactions, which are the information stored in the blocks of the distributed ledger. As a simpler configuration, it is also possible to directly acquire the composite video information and the disclosure condition information regarding the composite video information generated by the video information generation unit 8. However, from the perspective of preventing forgery and the like, in the first embodiment, the composite video information is acquired by accessing the distributed ledger and acquiring the transactions.

[0039] The composite video output unit 15 is for generating and outputting composite data in which all or part of the composite video information and the composite video are integrated. Specifically, the composite video output unit 15 includes an attribute information acquisition unit 24 that acquires attribute information, which is information regarding the attribute of the output destination of the composite video; a determination unit 25 that determines whether or not the disclosure condition in which the attribute information of the output destination is included in the disclosure condition information is satisfied; an information extraction unit 26 that extracts the information determined by the determination unit 25 to satisfy the disclosure condition from among the composite video information; a composite data generation unit 27 that generates composite data in which all or part of the information of the composite video information extracted by the information extraction unit 26 and the composite video are integrated; and a composite data output unit 28 that outputs the composite data to a predetermined output destination.

[0040] The attribute information acquisition unit 24 is for acquiring attribute information, which is information regarding the attribute of the output destination of the composite video. The attribute information acquired by the attribute information acquisition unit 24 is for comparison with the disclosure condition information regarding the composite video information, and is constituted by information including items corresponding to the items of the disclosure condition information, that is, information such as the attribute of the output destination, in addition to the information for specifying the output destination. The attribute information acquisition unit 24 acquires, for example, information such as that the user of the output destination is a high-paying user, a low-paying user, or a non-paying user as the attribute information.

[0041] When generating composite data for output to a predetermined output destination, the determination unit 25 is for determining whether the attribute information regarding the output destination satisfies the disclosure conditions indicated in the disclosure condition information regarding the composite video information. Specifically, the determination unit 25 compares and contrasts the respective disclosure conditions and the attribute information attached to each of one or more pieces of information constituting the composite video information, and determines whether the attribute information satisfies the disclosure conditions for each of the respective disclosure conditions. For example, assuming that disclosure conditions A, disclosure conditions B, disclosure conditions C,... are respectively attached to information A, information B, information C,... included in the composite video information by the disclosure condition information, the determination unit 25 determines whether the attribute information of the output destination satisfies disclosure condition A, whether the attribute information of the output destination satisfies disclosure condition B, and whether the attribute information of the output destination satisfies disclosure condition C, and as a determination result, outputs that the attribute information of the predetermined output destination satisfies the disclosure condition A regarding information A, does not satisfy the disclosure condition B regarding information B, does not satisfy the disclosure condition C regarding information C, etc.

[0042] The information extraction unit 26 is for extracting, from among one or more pieces of information included in the composite video information acquired by the composite video information acquisition unit 14, the information determined by the determination unit 25 to satisfy the disclosure conditions. Specifically, the information extraction unit 26 has a function of extracting, from among each piece of information included in the composite video information, the information for which the attribute information of the output destination satisfies the disclosure conditions regarding the respective pieces of information, that is, the information that can be disclosed to the output destination, and outputting it to the composite data generation unit 27. It is for extracting all or part of the information that can be disclosed to the output destination. Specifically, the information extraction unit 26 has a function of comparing and contrasting the attribute information acquired by the attribute information acquisition unit 24 with the disclosure condition information regarding the composite video information, and extracting the composite video information determined that the attribute information satisfies the disclosure conditions. For example, when it is shown by the attribute information that the user of the output destination is a high spender, the information extraction unit 26 has a function of extracting the composite video information associated with the disclosure condition of "high spender" and outputting it to the composite data generation unit 27.

[0043] The synthetic data generation unit 27 is for generating synthetic data in which all or part of the synthetic video information and the synthetic video are integrated. Specifically, the synthetic data generation unit 27 has a function of integrating all or part of the synthetic video information, for example, the information extracted by the information extraction unit 26, and the synthetic video. As a mode of information integration, it is desirable to adopt a format in which data constituting all or part of the synthetic video information is embedded in the data constituting the synthetic video, but integration may also be performed by other modes.

[0044] The synthetic data output unit 28 is for outputting synthetic data in which part or all of the synthetic video information and the synthetic video are integrated and generated by the synthetic data generation unit 27. Specifically, the synthetic data output unit 28 has a function of specifying an output destination based on the attribute information acquired by the attribute information acquisition unit 24 and outputting a synthetic video in which the synthetic video information is embedded toward the specified output destination. Note that as a simpler configuration, the synthetic data generation unit 27 may be omitted and the synthetic video and the synthetic video information may be output as separate data (in this case, the synthetic data output unit 28 functions as means for realizing the output step shown in claim 4 of the claims). However, in the first embodiment, a configuration is adopted in which the synthetic data generation unit 27 adds the synthetic video information to the synthetic video and outputs it so that the user can more reliably confirm the content of the synthetic video information.

[0045] The update information acquisition unit 16 is for acquiring update information of synthetic video information related to the synthetic video and disclosure condition information related to the update information and outputting it toward the update information output unit 17. Specifically, the update information acquisition unit 16 accesses a distributed ledger that corresponds one-to-one with the synthetic video that is the target of the update information, and has a function of acquiring the update information and the disclosure condition information related to the update information from among the transactions that are the information stored in the blocks of the distributed ledger. As a simple configuration, the update information acquisition unit 16 may directly acquire the update information and the disclosure condition information from the synthetic video update information generation unit 9, but in the first embodiment, a configuration is adopted in which information stored in the distributed ledger is acquired from the viewpoint of preventing falsification and the like.

[0046] The update information output unit 17 is for extracting a part of the information that can be disclosed to the output destination from the update information acquired by the update information acquisition unit 16 based on the disclosure condition information and then outputting it to the output destination. Specifically, the update information output unit 17 includes an attribute information acquisition unit 29 that acquires the attribute information of the output destination, a determination unit 30 that determines whether the attribute information of the output destination satisfies the disclosure conditions included in the disclosure condition information, and an information extraction unit 31 that extracts the information determined by the determination unit 30 to satisfy the disclosure conditions from the update information.

[0047] The attribute information acquisition unit 29 is for acquiring the attribute information that is information regarding the attribute of the output destination of the update information. The attribute information is for comparison with the disclosure condition information in the update information, and in addition to the information for specifying the output destination, it is composed of information indicating the conditions of the output destination, that is, the attributes of the output destination, etc., which are in a correspondence relationship with the disclosure condition information. The attribute information acquisition unit 29 acquires, for example, information such as whether the user of the output destination is a high-charging user, a low-charging user, or a non-charging user as the attribute information.

[0048] The determination unit 30 is for determining whether the attribute information regarding a predetermined output destination satisfies the disclosure conditions indicated by the disclosure condition information regarding the update information when outputting the update information to the output destination. Specifically, the determination unit 30 compares and contrasts each of the disclosure conditions attached to each of the one or more pieces of information constituting the update information with the attribute information, and determines whether the attribute information satisfies each of the disclosure conditions. For example, assuming that disclosure conditions 1, 2, 3,... are attached to information 1, information 2, information 3,... included in the update information, the determination unit 30 determines whether the attribute information satisfies disclosure condition 1, whether the attribute information satisfies disclosure condition 2, whether the attribute information satisfies disclosure condition 3, etc., and as a determination result, outputs that the attribute information of the predetermined output destination satisfies the disclosure condition 1 regarding information 1, does not satisfy the disclosure condition 2 regarding information 2, satisfies the disclosure condition 3 regarding information 3, etc.

[0049] The information extraction unit 31 is for extracting information determined by the determination unit 30 to satisfy the disclosure conditions from among one or more pieces of information included in the update information acquired by the update information acquisition unit 16. Specifically, the information extraction unit 31 has a function of extracting, from among each piece of information included in the update information, information for which the attribute information of the output destination satisfies the disclosure conditions regarding each piece of information, that is, information that can be disclosed to the output destination, and outputting it to the composite data output unit 28. For example, when the user of the output destination is shown to be a minor by the attribute information of the output destination, the information extraction unit 31 extracts update information associated with the condition of "minor" as disclosure condition information, for example, update information such as "Output to a postcard has newly become possible as a usage condition", and has a function of outputting it to the composite data output unit 28. The composite data output unit 28 outputs all or part of the update information extracted by the information extraction unit 31 to a predetermined output destination, separately from the output of the composite data.

[0050] Next, the advantages of the composite video management system according to Embodiment 1 will be described. First, the composite video management system according to Embodiment 1 of the present invention adopts a configuration in which a composite video is generated and composite video information, which is information regarding the composite video, is generated and provided to an output destination such as a user of the composite video. By adopting such a configuration, users of the composite video and the like have the advantage that they can always check the contents of the generated information, right information, usage information, etc., even when not connected to the Internet, for example. Further, in Embodiment 1 of the present invention, since a configuration is adopted in which the composite video information is output after being integrated with the composite video, even when a malicious user or the like attempts unauthorized use by ignoring the usage conditions, unauthorized use can be effectively suppressed by automatically reading and responding to the usage conditions with an electronic computer that browses the composite video or the like.

[0051] In addition, the composite video management system according to Embodiment 1 adopts a configuration in which update information of composite video information is generated and output to the output destination that has output the composite video. By adopting such a configuration, when the usage conditions of the output composite video change, for example, the composite video management system according to Embodiment 1 generates and outputs update information regarding the usage information included in the composite video information, so that the user of the composite video can be made to use it with accurate usage content according to the changed usage conditions.

[0052] Furthermore, the composite video management system according to Embodiment 1 adopts a configuration in which disclosure condition information is generated for both the composite video information and the update information, and information that satisfies the disclosure conditions according to the attributes of the output destination is provided to the output destination from among the composite video information and the update information. By adopting such a configuration, the composite video management system according to Embodiment 1 has the advantages of suppressing unnecessary information provision, reducing communication volume, and optimizing and differentiating services according to the attributes of the user.

[0053] (Embodiment 2) Next, the composite video management system according to Embodiment 2 will be described. In Embodiment 2, with respect to components having the same name and the same reference numerals as those in Embodiment 1, unless otherwise specified, they shall exhibit the same functions as the components in Embodiment 1.

[0054] As shown in FIG. 2, the composite video management system according to Embodiment 2 includes a disclosure condition information generation unit 33 that generates disclosure condition information, which is information that defines the disclosure conditions of the update information and the disclosure timing when the disclosure conditions are satisfied, and a composite data output unit 34 that outputs, at the disclosure timing defined by the disclosure condition information, the update information that satisfies the disclosure conditions included in the disclosure condition information generated by the disclosure condition information generation unit 33.

[0055] The disclosure condition information generation unit 33 is for generating disclosure condition information including information on the disclosure timing when the conditions for disclosure are met, in addition to disclosing one or more pieces of information included in the update information determined according to the attribute information, which is information on the attributes of the output destination of the synthetic data, similar to the disclosure condition generation unit 23 in the first embodiment. Specifically, for example, with respect to the information in a predetermined item of the update information, the disclosure condition information generation unit 33 determines that the user of the output destination is an adult as the disclosure condition, and sets that the disclosure timing is three months after the generation of the update information, and generates disclosure condition information as information including these contents. Note that the disclosure condition information generation unit 33 can also generate disclosure condition information in which a plurality of disclosure conditions are determined for the same information. For example, with respect to the information in a predetermined item in the update information, it is made possible to disclose when the output destination user is a high-charging user and a low-charging user, and disclosure condition information may be generated such that the high-charging user is disclosed one month after the generation of the update information, and the low-charging user is disclosed one year after the generation of the update information.

[0056] The synthetic data output unit 34 has a function of outputting update information that satisfies the disclosure conditions determined by the disclosure condition information, similar to the synthetic data output unit 28 in the first embodiment, and also has a function of outputting at the disclosure timing determined by the disclosure condition information with respect to the timing of outputting the synthetic data. Specifically, the synthetic data output unit 34 does not directly output the update information that is made outputtable as satisfying the disclosure conditions separately from the output of the synthetic data. Instead, after further referring to the disclosure condition information to obtain information regarding the output timing, it has a function of outputting information to the output destination at the output timing. As a result, the synthetic video management system according to the second embodiment of the present invention enables output at different times according to the attributes of the output destination and the like, not only for different update information but also for the same update information.

[0057] Next, the advantages of the composite video management system according to the second embodiment will be described. The composite video management system according to the second embodiment includes information regarding the output timing of update information when the disclosure condition is satisfied in addition to the disclosure condition in the disclosure condition information regarding the update information. By adopting such a configuration, the composite video management system according to the second embodiment can not only determine whether to transmit the update information but also shift the timing of transmitting the update information, and has the advantage of being able to optimize and differentiate services according to the attributes of the user.

Industrial Applicability

[0058] The present invention can be used as a technology for appropriately managing a composite video generated by inserting an avatar generated based on a person video into an insertion target video composed of one or more video materials and an output mode of information regarding the composite video.

Explanation of Signs

[0059] 1 Video material input unit 2 Material information generation unit 3 Insertion target video generation unit 4 Person video input unit 5 Avatar generation unit 6 Avatar information generation unit 7 Avatar insertion unit 8 Video information generation unit 9, 35 Composite video update information generation unit 10 Token generation unit 11 Transaction generation unit 12 Electronic signature generation unit 13 Output unit 14 Composite video information acquisition unit 15, 36 Composite video output unit 16 Update information acquisition unit 17 Update information output unit 19 Composite video information generation unit 20, 23, 33 Disclosure condition information generation unit 22 Update information generation unit 24, 29 Attribute information acquisition unit 26, 31 Information Extraction Unit 27 Synthetic Data Generation Unit 28, 34 Synthetic Data Output Unit 25, 30 Judgment Unit

Claims

1. A composite video management system that manages an output state of a composite video generated by inserting an avatar generated based on a predetermined person video into a partial area of ​​an insertion target video, comprising: an insertion target image generating means for generating the insertion target image based on one or more image materials; an avatar generating means for generating the avatar based on the person image; an avatar inserting means for inserting the avatar generated by the avatar generating means into all or a part of the video material in the insertion target video to generate a composite video; a composite image information generating means for generating composite image information including one or more of generation information which is information regarding a generation mode of the composite image, rights information which is information regarding a rights relationship of the composite image, and usage information which is information regarding a usage condition of the composite image; a synthetic data generating means for generating synthetic data by integrating the synthetic image with all or a part of the synthetic image information generated by the synthetic image information generating means; a synthetic data output means for outputting the synthetic data generated by the synthetic data generation means; a disclosure condition information generating means for generating disclosure condition information, which is information regarding a disclosure condition of one or more pieces of information included in the composite image information, determined according to attribute information, which is information regarding an attribute of an output destination of the composite data; a determination means for determining whether or not the attribute information related to a predetermined output destination satisfies the disclosure condition in the disclosure condition information when generating the composite data related to the predetermined output destination; an information extraction means for extracting information determined by the determination means to satisfy the disclosure condition from one or more pieces of information included in the composite image information; Equipped with The synthetic image management system is characterized in that the synthetic data generation means generates synthetic data by integrating the information extracted by the information extraction means with the synthetic image as a part of the synthetic image information.

2. an update information generating means for generating update information which is information for updating the content of the composite video information; a disclosure condition information generating means for generating disclosure condition information, which is information on a disclosure condition of one or more pieces of information included in the update information, determined according to attribute information, which is information on an attribute of an output destination; a determination means for determining whether or not the attribute information relating to a predetermined output destination satisfies the disclosure condition in the disclosure condition information when the update information is output to the predetermined output destination; an information extraction means for extracting information determined by the determination means to satisfy the disclosure condition from one or more pieces of information included in the update information; Equipped with 2. The composite image management system according to claim 1, wherein said composite data output means outputs information extracted by said information extraction means separately from said composite data.

3. A composite video management method for managing an output mode of a composite video generated by inserting an avatar generated based on a predetermined person video into a partial area of ​​an insertion target video, comprising: an insertion target image generating step of generating the insertion target image based on one or more image materials; and an avatar generating step of generating the avatar based on a person image. an avatar inserting step of inserting the avatar generated in the avatar generating step into all or a part of the video material in the insertion target video to generate a composite video; a composite image information generating step of generating composite image information including one or more of generation information which is information regarding a generation mode of the composite image, rights information which is information regarding a rights relationship of the composite image, and usage information which is information regarding a usage condition of the composite image; a synthetic data generating step of generating synthetic data by integrating the synthetic image with all or a part of the synthetic image information generated in the synthetic image information generating step; an output step of outputting the composite data obtained by integrating the whole or part of the composite video information generated in the composite video information generating step with the composite video; an update information generating step of generating update information which is information for updating the content of the composite video information; a disclosure condition information generating step of generating disclosure condition information that is information that defines a condition for disclosing one or more pieces of information included in the update information determined according to attribute information that is information regarding an attribute of an output destination of the composite data, and a disclosure time when the disclosure condition is satisfied; a determination step of determining whether or not the attribute information related to the predetermined output destination satisfies a condition for disclosing the one or more pieces of information in the disclosure condition information when the update information is output to the predetermined output destination; an information extraction step of extracting information determined in the determination step to satisfy a condition to be disclosed from one or more pieces of information included in the update information; Including, A composite video management method characterized in that in the output step, information extracted in the information extraction step from one or more pieces of information contained in the update information, separate from all or a part of the composite video information and the composite video, is output at a time specified by the disclosure condition information.

4. A composite video management program that causes a computer to manage an output state of a composite video generated by inserting an avatar generated based on a predetermined person video into a partial area of ​​an insertion target video, the program comprising: The computer is an insertion target image generating function for generating the insertion target image based on one or more image materials; and an avatar generating function for generating the avatar based on the person image. an avatar insertion function for inserting the avatar generated by the avatar generation function into all or a part of the video material in the insertion target video to generate a composite video; a composite image information generating function for generating composite image information including one or more of generation information, which is information regarding a generation mode of the composite image, rights information, which is information regarding a rights relationship of the composite image, and usage information, which is information regarding a usage condition of the composite image; a synthetic data generating function for generating synthetic data by integrating the synthetic image with all or a part of the synthetic image information generated by the synthetic image information generating function; a synthetic data output function for outputting the synthetic data generated by the synthetic data generation function; an update information generating function for generating update information that updates the content of the composite video information; a disclosure condition information generating function for generating disclosure condition information, which is information regarding a disclosure condition of one or more pieces of information included in the update information, determined according to attribute information, which is information regarding an attribute of an output destination; a determination function for determining whether or not the attribute information related to a predetermined output destination satisfies the disclosure condition in the disclosure condition information when the update information is output to the predetermined output destination; an information extraction function that extracts information that is determined by the determination function to satisfy the disclosure condition from one or more pieces of information included in the update information; Run the command, A composite video management program, comprising: a composite data output function that outputs information extracted by said information extraction function separately from said composite data.

Images