Composite video management system, composite video management method, and composite video management program

Abstract

This invention provides a synthetic video management system, method, and program for appropriately managing the synthetic video generated by inserting an avatar and the manner in which information is output to the synthetic video. [Solution] The composite video management system includes: an insertion target video generation unit 3 that generates an insertion target video based on video material input from a video material input unit 1; a person video input unit 4 that inputs a person video that will be the basis of the avatar; an avatar generation unit 5 that generates an avatar from the person video; an avatar insertion unit 7 that inserts the generated avatar into the insertion target video; a video information generation unit 8 that generates composite video information, which is information related to the composite video; a composite video update information generation unit 9 that generates update information for the composite video information; a composite video information acquisition unit 14 that acquires composite video information from a distributed ledger where it is stored; a composite video output unit 15 that outputs a composite video with the composite video information added; an update information acquisition unit 16 that acquires update information from a distributed ledger; and an update information output unit 17 that outputs the acquired update information.

Description

Technical Field

[0001] The present invention relates to a technique for managing a composite video generated by inserting an avatar generated based on a person video into an insertion target video composed of one or more video materials.

Background Art

[0002] In recent years, with the improvement of processing capabilities in electronic computers such as computers, a large number of computer graphics of human figures, so-called avatars, that reflect the characteristics of real people have been widely used. For example, an avatar is used as one's own icon when using SNS (Social Networking Service), or one's own avatar is used as the character of the protagonist in an online game or the like. In addition, in contents such as still images and moving images, a service has been proposed in which some of the characters are replaced with one's own avatar for viewing and the like.

[0003] By using an avatar that reflects one's own characteristics in this way, for example, when a protagonist composed of an avatar expressing the characteristics of a user in a game battles with an enemy character, an effect of improving the user's immersion in the game world occurs. By using an avatar that abstractly represents the user himself / herself as an icon indicating the user in SNS, it is expected that an effect such as promoting communication between users in a virtual space in the same sense as the real world will occur.

[0004] Both Patent Documents 1 and 2 disclose a technique of using an avatar that imitates the actual appearance of the player himself / herself or a co-player in a computer game that expresses a virtual space using a head-mounted display.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] However, when generating images or videos in which some characters are replaced with the user's own avatar, the handling of such images becomes problematic. As the quality of such composite images improves with recent advancements in image processing technology, there is a growing possibility that the generated composite images will not be recognized as being the result of a composite process that differs from reality. Since such misunderstandings can lead to confusion such as so-called fake news, it is necessary to take measures to ensure that composite images are easily understood as fictional works that differ from reality. However, if measures are taken to perform image processing that detracts from the sense of reality, there is a problem in that the quality of the composite image will decrease.

[0007] Furthermore, in recent years, it has become common not only to enjoy the generated composite images personally, but also to widely share them on social media, etc., and it is also conceivable that these composite images may be used by third parties, whether for a fee or free of charge. However, if, for example, the existence and content of the usage rights for the still images, videos, etc. that are to be inserted into the avatar used as the basis for the composite image are unclear, there is a problem that services that publish composite images on social media, etc., or third parties that use the composite images may unintentionally infringe on the rights of legitimate rights holders. No technology to solve these problems is disclosed in either Patent Document 1 or 2.

[0008] The present invention has been made in view of the above problems, and aims to provide a technology for appropriately managing the output of a composite video and information related to the composite video, which is generated by inserting an avatar generated based on a person's image into a video to be inserted, which is composed of one or more video materials. [Means for solving the problem]

[0009] To achieve the above objective, the composite video management system according to claim 1 is a composite video management system that manages the output manner of a composite video generated by inserting an avatar generated based on a predetermined person video into a part of a target video, and is characterized by comprising: an insertion target video generation means for generating the target video based on one or more video materials; an avatar generation means for generating the avatar based on the person video; an avatar insertion means for generating a composite video by inserting the avatar generated by the avatar generation means into all or part of the video materials in the target video; a composite video information generation means for generating composite video information including one or more of the following: generation information which is information relating to the generation manner of the composite video; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video; a composite data generation means for generating composite data which integrates all or part of the composite video information generated by the composite video information generation means with the composite video; and a composite data output means for outputting the composite data generated by the composite data generation means.

[0010] Furthermore, in order to achieve the above objective, the composite video management system according to claim 2 further comprises: a disclosure condition information generation means that generates disclosure condition information, which is information relating to the disclosure conditions of one or more pieces of information included in the composite video information, as determined according to attribute information, which is information relating to the attributes of the output destination of the composite data; a determination means that determines whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when generating the composite data relating to the predetermined output destination; and an information extraction means that extracts information from one or more pieces of information included in the composite video information that the determination means has determined to satisfy the disclosure conditions, wherein the composite data generation means generates composite data that integrates the information extracted by the information extraction means with the composite video as part of the composite video information.

[0011] Furthermore, in order to achieve the above objective, the composite video management system according to claim 3 is characterized in that, in the above invention, it comprises: update information generation means for generating update information which is information for updating the content of the composite video information; disclosure condition information generation means for generating disclosure condition information which is information regarding the disclosure conditions of one or more pieces of information included in the update information, determined according to attribute information which is information regarding the attributes of the output destination; determination means for determining whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when outputting the update information to a predetermined output destination; and information extraction means for extracting information which the determination means has determined to satisfy the disclosure conditions from one or more pieces of information included in the update information, wherein the composite data output means outputs the information extracted by the information extraction means separately from the composite data.

[0012] Furthermore, in order to achieve the above objective, the composite video management method according to claim 4 is a composite video management method that manages the output manner of a composite video generated by inserting an avatar generated based on a predetermined person video into a part of the video to be inserted, and is characterized by including: an insertion target video generation step of generating the video to be inserted based on one or more video materials; an avatar generation step of generating the avatar based on a person video; an avatar insertion step of generating a composite video by inserting the avatar generated in the avatar generation step into all or part of the video materials in the video to be inserted; a composite video information generation step of generating composite video information that includes one or more of the following: generation information which is information relating to the generation manner of the composite video; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video; and an output step of outputting all or part of the composite video information generated in the composite video information generation step and the composite video.

[0013] Furthermore, in order to achieve the above objective, the composite video management method according to claim 5, in the above invention, comprises: an update information generation step of generating update information which is information that updates the content of the composite video information; a disclosure condition information generation step of generating disclosure condition information which is information that defines the conditions for disclosing one or more pieces of information included in the update information, determined according to attribute information which is information relating to the attributes of the output destination of the composite data, and the timing of disclosure when the conditions for disclosure are met; a determination step of determining whether the attribute information relating to the predetermined output destination satisfies the conditions for disclosing the one or more pieces of information in the disclosure condition information when the update information is output to a predetermined output destination; and an information extraction step of extracting information from the one or more pieces of information included in the update information that has been determined to satisfy the conditions for disclosure in the determination step, wherein in the output step, all or part of the composite video information and the information extracted in the information extraction step from the one or more pieces of information included in the update information are output at the time specified by the disclosure condition information.

[0014] Furthermore, in order to achieve the above objective, the composite video management program according to claim 6 is a composite video management program that causes a computer to manage the output manner of a composite video generated by inserting an avatar generated based on a predetermined person video into a part of the video to be inserted, and is characterized in that it causes the computer to execute: an insertion target video generation function that generates the insertion target video based on one or more video materials; an avatar generation function that generates the avatar based on the person video; an avatar insertion function that generates a composite video by inserting the avatar generated by the avatar generation function into all or part of the video materials in the insertion target video; a composite video information generation function that generates composite video information including one or more of the following: generation information which is information relating to the generation manner of the composite video; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video; a composite data generation function that generates composite data which integrates all or part of the composite video information generated by the composite video information generation function with the composite video; and a composite data output function which outputs the composite data generated by the composite data generation function. [Effects of the Invention]

[0015] According to the present invention, it is possible to appropriately manage the output modes of the composite video and information related to the composite video, which are generated by inserting an avatar generated based on a person's image into a video to be inserted, which is composed of one or more video materials. [Brief explanation of the drawing]

[0016] [Figure 1] This is a schematic diagram showing the configuration of the composite video management system according to Embodiment 1. [Figure 2] This is a schematic diagram showing the configuration of the composite video management system according to Embodiment 2. [Modes for carrying out the invention]

[0017] The embodiments of the present invention will be described in detail below with reference to the drawings. The following embodiments describe the most appropriate examples of the present invention, and naturally, the content of the present invention should not be limited to the specific examples shown in these embodiments. It goes without saying that any configuration other than the specific configurations shown in the embodiments that produces similar functions and effects is also included in the technical scope of the present invention.

[0018] (Embodiment 1) First, the composite video management system according to Embodiment 1 will be described. As shown in Figure 1, the composite video management system according to Embodiment 1 includes: a video material input unit 1 for inputting video materials to be used as material for composite video; a material information generation unit 2 for generating video material information, which is information about the video materials to be input; an insertion target video generation unit 3 for generating an insertion target video to which an avatar will be inserted, based on one or more video materials input via the video material input unit 1; a person video input unit 4 for inputting a person video that will be the basis for an avatar to be inserted into the insertion target video; an avatar generation unit 5 for generating an avatar from a person video input via the person video input unit 4; an avatar information generation unit 6 for generating information about the avatar generated by the avatar generation unit 5; an avatar insertion unit 7 for inserting the avatar generated by the avatar generation unit 5 into the insertion target video; a video information generation unit 8 for generating composite video information, which is information about the composite video, based on the video material information and avatar information; and a composite video update information generation unit for generating update information for the composite video information. The system comprises: a report generation unit 9; a token generation unit 10 for generating composite video tokens, which are non-fungible tokens that have a one-to-one correspondence with the composite video; a transaction generation unit 11 for generating transactions, which are information stored in a block in a distributed ledger (described later) that has a one-to-one correspondence with the composite video token and include composite video information and update information; an electronic signature generation unit 12 for generating electronic signatures, which are data that proves that the generation and output of the information included in the transaction is in accordance with the intentions of the holder of the composite video token; an output unit 13 for outputting the transaction and the electronic signature corresponding to the transaction to the distributed ledger that has a correspondence with the composite video token; a composite video information acquisition unit 14 for acquiring composite video information from the distributed ledger; a composite video output unit 15 for outputting a composite video with the composite video information added; an update information acquisition unit 16 for acquiring update information from the distributed ledger; and an update information output unit 17 for outputting the acquired update information.

[0019] The video material input unit 1 is for inputting video material, which is image data used to generate a composite video. The composite video consists of the video to be inserted and an avatar inserted into all or part of the video to be inserted. The video material input via the video material input unit 1 is directly used to generate the video to be inserted. The video to be inserted consists of one or more video materials. In a simple configuration, a single video material may be used as is, or a single video material may be used as the video to be inserted after image processing such as enlarging, shrinking, rotating, or partially modifying it. Furthermore, when the video to be inserted is composed of multiple video materials, the video to be inserted is formed by performing image processing on each video material as necessary, arranging each video material in a predetermined position, or by overlapping multiple video materials. The form of the video material may be either 2D video or 3D video, and may be either video or still image. The content of the video material may be realistic (for example, a real landscape photographed using an imaging device such as a camera) or pictorial (for example, an illustration created by an illustrator). In this embodiment 1, the following explanation assumes that the video material is generated using an imaging device.

[0020] The material information generation unit 2 is for generating video material information, which is information about video material input via the video material input unit 1. Specifically, the material information generation unit 2 includes at least one of the following: generation information, which is information about the method of generating the video material; rights information, which is information about the rights related to the video material; and usage information, which is information about the conditions for using the video material in the finished video. However, it may also include other information. For example, if the video material is realistically generated using imaging equipment, the generation information includes information about the photographer, shooting location, shooting equipment, shooting date, shooting environment (temperature, weather, etc.), and the content of processing from the original video. If the video material is realistically generated, such as an illustration, or realistically generated without using imaging equipment, the generation information includes at least one of the following: the creator, creation equipment (art materials, computer used for creation, software, etc.), creation date, and specific details of the original video if one was referenced. However, it may also include other information. Rights information includes information that includes one or more of the following: information concerning the creator, the rights holder of intellectual property rights such as copyright, the duration of the intellectual property rights, the licensee, and the content of the usage rights held by the licensee, but it may also include other information. Usage information includes information that includes one or more of the following: information concerning the usage conditions of composite videos using video materials (to what extent the scope of publication of composite videos is permitted, to what extent the uses of composite videos (video distribution, output to paper media such as glossy paper or postcards as still images, etc.) are permitted, to what extent commercial use is permitted, etc.), information concerning the usage conditions of video materials in the generation of composite videos (to what extent processing is permitted when processing is performed, which video materials are permitted to be used in combination when other video materials are used in the same composite video, etc.), information concerning the range of attributes that may be permitted for each person who generates composite videos using video materials and the person who uses the generated composite videos (gender, age, nationality, beliefs, identification information, eligibility to use this system (e.g., paid user or unpaid user), etc.), and information concerning the usage fees of video materials, but it may also include other information.As a specific mode of generating material information by the material information generation unit 2, it may be configured to artificially input data constituting the content of each piece of information when inputting the material video. For example, regarding information such as the shooting date and shooting equipment, it may be generated based on the information included in the data of the material video. Furthermore, when the video material is publicly available video data, it may be generated based on the public information referring to the video material.

[0021] The insertion target video generation unit 3 is for generating an insertion target video based on one or more video materials. The insertion target video is the video that is the basis of the composite video. More specifically, it has the relationship that the composite video is the one obtained by performing avatar insertion processing on the insertion target video. The insertion target video generation unit 3 performs necessary processing on one or more video materials, and when using a single video material, it directly uses it as the insertion target video, and when using multiple video materials, it generates the insertion target video by performing processing such as arranging each at a predetermined position. As a specific configuration of the insertion target video generation unit 3, for example, it can be configured to have an image processing function using the prior art, and the configuration of the generated insertion target video can also be a moving image / still image, color video / black-and-white video, two-dimensional video / three-dimensional video, etc. according to the form of the completed video.

[0022] The human image input unit 4 is for inputting a human image which is a material used for generating an avatar that constitutes a composite image. Specifically, the human image input unit 4 has a function of inputting a human image which is an image related to the whole or a part of a person who is the origin of the avatar to be inserted into the target video for insertion. As a specific configuration, it may be configured to input a video from the outside, or may be configured to include an imaging mechanism for acquiring a video. As a specific form of the human image, it may be a full-body image of the target person, or may be a partial image such as a face image. Also, it may be either a still image or a moving image, and may be either a 2D image or a 3D image. In the first embodiment, although an explanation is made using a 2D and still image related to the face of the target person as the human image, it is of course possible to use a human image other than such a configuration in the composite image generation system according to the first embodiment.

[0023] The avatar generation unit 5 generates an avatar for inserting and replacing all or part of the video material constituting the video to be inserted, based on a person video input via the person video input unit 4. Specifically, the avatar generation unit 5 may consist of skeletal information (bones), surface information (skin), and weight information defining the relationship between the two, but it is also preferable to consist only of surface information including information on the three-dimensional shape and color tone on the surface, and in an even simpler configuration, it may consist of the image data itself. Furthermore, the avatar generated by the avatar generation unit 5 may be an avatar corresponding to the full body image of a person, but it may also be an avatar consisting only of a part of a person, for example, only a part of the face. In this embodiment 1, an avatar consisting only of the head from the neck up is generated. The avatar generation unit 5 extracts feature points based on the human video input via the human video input unit 4, according to the position and shape of the surface features (eyes, eyebrows, nose, mouth, ears, hairstyle, etc.) and internal features (joints, etc.) of the human video. By reflecting the positional relationships between the extracted feature points onto the avatar, it generates a realistic avatar that reflects the physical characteristics of the model. However, if a simpler configuration is adopted, for example, the avatar may be generated using the human video as is, or with only the minimum necessary modifications such as adjusting the size and orientation of the human video.

[0024] The avatar information generation unit 6 is for generating avatar information, which is information about the avatar generated by the avatar generation unit 5. Specifically, the avatar information generation unit 6 includes at least one of the following: generation information, which is information about the manner in which the avatar was generated; rights information, which is information about the rights related to the avatar; and usage information, which is information about the conditions for use when using the avatar in the finished video. However, it may also include other information. The generation information includes at least one of the following: information about the photographer, shooting location, shooting equipment, shooting date, shooting environment, etc., related to the person video that forms the basis of the avatar; the content of the processing applied to the person video during avatar generation; the computer and software used for the processing; the date the avatar was created; the creator, etc. However, it may also include other information. The rights information includes at least one of the following: information about the author of the avatar and the person video that forms the basis of the avatar; the rights holder of intellectual property rights such as copyrights; the duration of the intellectual property rights; the licensee; the content of the usage rights held by the licensee. However, it may also include other information. Usage information shall include one or more of the following: conditions for using composite videos using avatars (how far the scope of public access to the composite video is permitted, how far the uses of the composite video are permitted (video distribution, output to paper media such as glossy paper or postcards as still images, etc.), how far commercial use is permitted, etc.); conditions for using avatars in the generation of composite videos (how far processing is permitted when processing is performed, conditions for video materials that can be used in the same composite video, etc.); and the range of permissible attributes for each person who generates a composite video using an avatar and the person who uses the generated composite video (gender, age, nationality, beliefs, identification information, eligibility to use this system (e.g., paid user or free user), etc.). However, it may also include other information.

[0025] The avatar insertion unit 7 is for generating a composite image by inserting the generated avatar into all or part of the video material in the video to be inserted. Specifically, the avatar insertion unit 7 has the function of inserting the avatar into the video to be inserted and completing the composite image by replacing all or part of the video material area in the video to be inserted with the avatar generated by the avatar generation unit 5. If the video to be inserted is a still image, the avatar insertion unit 7 acquires information regarding the size and direction of the replacement target in the video to be inserted, and appropriately changes the size and direction of the avatar based on that information, and then inserts the avatar into the video to be inserted in a manner that replaces the replacement target. If the video to be inserted is a video, the avatar insertion unit 7 acquires information regarding the operation content of the replacement target in the video to be inserted, for example by detecting changes in the position of feature points, and appropriately changes the size, direction and operation of the avatar based on that information, and then inserts the avatar into the video to be inserted in a manner that replaces the replacement target. While it is generally preferable to replace the avatar with footage of a person in the video being inserted, it is not limited to this. For example, if the content video is of a locomotive, an avatar consisting of a part of a face image could be inserted on the front of the locomotive.

[0026] The video information generation unit 8 is for generating composite video information, which is information relating to the composite video. Specifically, the video information generation unit 8 includes a composite video information generation unit 19 that generates composite video information including information relating to the generation method of the composite video, usage conditions, etc., and a disclosure condition information generation unit 20 that generates disclosure condition information, which is information relating to the disclosure conditions of the composite video information generated by the composite video information generation unit 19.

[0027] The composite video information generation unit 19 generates composite video information, which is information relating to a composite video, and includes one or more of the following: generation information, which is information relating to the generation method of the composite video; rights information, which is information relating to the rights relationship of the composite video; and usage information, which is information relating to the conditions for using the composite video. Here, the generation information includes, for example, generation information relating to the video materials and avatars that form the basis of the composite video, as well as one or more of the following: the processor who performed the avatar insertion process when generating the composite video, the equipment used for the process (computer, software, etc.), the date on which the avatar insertion process was performed, and information relating to the authenticity of the composite video (information relating to whether it is a composite video and a creative work (= the video content itself does not actually exist), whether the individual video materials and avatars that make up the composite video are videos of things that actually exist, etc.). However, it may also include other information. The rights information includes one or more of the following information relating to the composite video and the video materials and avatars used in the composite video: the creator, the rights holder of intellectual property rights such as copyright, the duration of the intellectual property rights, the licensee, the content of the usage rights held by the licensee, etc., however, it may also include other information. The usage information shall include one or more of the following: conditions for using the composite video (how far the scope of public disclosure of the composite video is permitted, how far the uses of the composite video are permitted (video distribution, output to paper media such as glossy paper or postcards as still images, etc.), how far commercial use is permitted, etc.), the range of attributes that the user of the generated composite video may accept (including cases where use under all conditions is permitted, as well as cases where only certain conditions of use are permitted), the qualifications for using this system (e.g., paid user or free user), etc.), and information regarding the usage fees for the composite video. However, it may also include other information. Furthermore, regarding the usage information, if the conditions for using the composite video are limited to satisfying the conditions for using the video material / avatar, a condition determination means may be provided when generating the usage information to determine whether the conditions for using the composite video are within the scope of the conditions for using the video material / avatar, and if they are outside the scope, the conditions may not be included in the usage information.Furthermore, if the usage conditions are to be changed according to the user of the synthesized video, it is preferable to generate usage information in a configuration that includes multiple variations of the usage conditions.

[0028] The disclosure condition information generation unit 20 generates disclosure condition information, which is information regarding the disclosure conditions of one or more pieces of information included in the composite video information, determined according to attribute information relating to the attributes of the output destination of the composite data (or the composite video and composite video information separately if they are not integrated) which is a composite video and a composite video information, in whole or in part. Preferably, the disclosure condition information includes information that specifies whether or not to disclose each item of the composite video information generated by the composite video information generation unit 19, according to attribute information relating to the attributes of the output destination of the composite video, but it may also include other information. For example, if the attributes of the output destination (usually the user of the composite video) are assumed to be "high payer," "low payer," and "non-payer," the disclosure condition information generation unit 20 generates disclosure condition information such that the disclosure condition for "composite video information that only allows viewing of the composite video as a condition of use" is "the attribute of the output destination is a non-payer," and the disclosure condition for "composite video information that only allows viewing and video distribution of the composite video as a condition of use" is "the attribute of the output destination is a low payer."

[0029] The composite video update information generation unit 9 is for generating update information for composite video information. Specifically, the composite video update information generation unit 9 includes an update information generation unit 22 that generates update information, which is information that updates the content of the composite video information, and a disclosure condition information generation unit 23 that generates disclosure condition information, which is information regarding the disclosure conditions of one or more pieces of information included in the update information, as determined according to the attribute information of the output destination.

[0030] The update information generation unit 22 is for generating update information, which is information that updates the content of the composite video information. Specifically, the update information generation unit 22 has the function of generating update information that identifies the items in the composite video information that have been changed, and the changed information regarding those items, in a manner that is correlated with each other. For example, the update information generation unit 22 generates information that changes the usage conditions of the composite video, which previously allowed only video distribution, to allow both video distribution and output to paper media.

[0031] The disclosure condition information generation unit 23 is for generating disclosure condition information, which is information about the disclosure conditions of one or more pieces of information included in the update information, determined according to the attribute information, which is information about the attributes of the output destination. For example, the disclosure condition information includes information that specifies whether or not to disclose the information of each item of the update information generated by the update information generation unit 22, determined according to the attribute information, which is information about the attributes of the output destination, but it may also include other information. For example, if the attributes of the output destination are assumed to be "non-paying users" and "paying users", the disclosure condition information generated by the disclosure condition unit 23 will specify that the disclosure condition for "update information stating that the fee for new subscriptions to paid services will be lower than before" can only be disclosed if "the attribute of the output destination is a non-paying user".

[0032] The token generation unit 10 is for generating synthetic video tokens, which are non-fungible tokens that have a one-to-one correspondence with the generated synthetic video. A "non-fungible token" is a so-called NFT (Non-Fungible Token), that is, a token that has the property of being infungible with other tokens by possessing unique data, and is issued based on the Ethereum® standard ERC721, for example. The non-fungible token in this embodiment 1 is issued based on ERC721 or other predetermined standards, and the transaction history of the synthetic video token, information about the holder, and information about the corresponding synthetic video, such as synthetic video information, disclosure conditions information regarding the synthetic video information, and update information regarding the synthetic video information and disclosure conditions information regarding the update information, are recorded on a distributed ledger on the blockchain that corresponds to the synthetic video token. The information recorded regarding the synthetic video token is generated by the transaction generation unit 11, which will be described later, and is output to the distributed ledger by the output unit 13 along with the electronic signature generated by the electronic signature generation unit 12.

[0033] Blockchain is a technology that uses cryptographic techniques to synchronize data among multiple computers that make up a decentralized network. Specifically, each block is composed of a collection of information about tokens, such as agreed-upon transaction records, and information to connect to other blocks (information from the previous block). A blockchain is formed by linking multiple such blocks together. Even if data is tampered with on some of the computers, the correct data is selected by majority vote among the other computers, making it extremely difficult to destroy or tamper with the data. Blockchains can be classified into public blockchains, which do not restrict who can participate in the majority vote, meaning that an unspecified number of people can participate in the recording process on the distributed ledger; and consortium blockchains and private blockchains, where only a select number of specific individuals can participate in the majority vote, meaning that only predetermined individuals can participate in the recording process on the distributed ledger.

[0034] Specific methods for linking a synthesized video token with a synthesized video include associating the identifier of the synthesized video token with the identification information of the synthesized video. More simply, the identifier of the synthesized video token may be matched with the identification information of the synthesized video. Furthermore, the generation of non-fungible tokens may be performed by the token generation unit 10 itself, or by an external system generating the tokens by issuing a predetermined command to an external system directly or indirectly connected to the token generation unit 10. In addition, the specific format of the non-fungible token is not limited to that conforming to the Ethereum® standard ERC721, but may be any format as long as it has the property of being non-fungible and information such as transaction history can be stored in a distributed ledger.

[0035] The transaction generation unit 11 is for generating transactions, which are information stored in a distributed ledger that has a one-to-one correspondence with the synthesized video. The transaction generation unit 11 has the function of generating a transaction that includes information about the owner of the synthesized video, as well as information about the synthesized video, such as synthesized video information and disclosure conditions information related to the synthesized video information, and information about updating the synthesized video information, such as update information and disclosure conditions information related to the update information, and outputting it to the electronic signature generation unit 12 and the output unit 13. The transaction generation unit 11 may generate a transaction in a format that includes the synthesized video information and the update information within a single transaction, but more preferably, it generates a transaction containing the synthesized video information when the synthesized video information is generated, and then generates another transaction containing the update information when the update information is generated thereafter.

[0036] The electronic signature generation unit 12 is for generating an electronic signature, which is data that proves that the information contained in a transaction is legitimate. Specifically, the electronic signature generation unit 12 has the function of generating a hash value of the transaction and encrypting the hash value with a private key to generate an electronic signature. The "hash value" is a fixed-length value obtained by applying a certain calculation procedure to the original data. Because this calculation procedure is irreversible, it is considered impossible to recover the original data from the hash value. The "private key" is a sequence of numbers used in the encryption process and is configured to be decryptable using the corresponding "public key".

[0037] The output unit 13 is for outputting the transaction generated by the transaction generation unit 11, along with the corresponding digital signature, to the distributed ledger associated with the synthesized video token. In this embodiment, the output unit 13 is configured to directly output the transaction and the corresponding digital signature, but it may also be configured to merely instruct other components (including those provided outside the system) to output. In this embodiment 1, the output unit 13 is directly or indirectly connected to the network where the distributed ledger is installed, and is configured to output predetermined data to the distributed ledger. Regarding the data output from the output unit 13, a hash value is generated for the transaction, and this hash value is compared with the data obtained by decrypting the transaction and the corresponding digital signature using the public key. If the two values ​​are different, it is determined that it is not a legitimate transaction and registration to the distributed ledger is rejected. If the two values ​​match, it is determined to be a legitimate transaction, and the information composed of the transaction is stored in a block that constitutes the distributed ledger. The correspondence between transactions and distributed ledgers can be established, for example, by linking the identification information of the distributed ledger with the identification information of the transaction. More simply, the same information as the identification information of the transaction may be used as the identification information of the distributed ledger. The identification information of the distributed ledger is used when outputting information by the output unit 13, as well as when acquiring information in the composite image information acquisition unit 14 and the update information acquisition unit 16.

[0038] The composite video information acquisition unit 14 acquires composite video information and disclosure condition information related to the composite video information, which are information related to the composite video, and outputs them to the composite video output unit 15. Specifically, the composite video information acquisition unit 14 has the function of accessing a distributed ledger that corresponds one-to-one with the target composite video and acquiring composite video information and disclosure condition information related to the composite video information from the transactions, which are information stored in blocks of the distributed ledger. As a simpler configuration, it is also possible to directly acquire the composite video information and disclosure condition information related to the composite video information generated by the video information generation unit 8, but from the viewpoint of preventing tampering, etc., in this embodiment 1, the composite video information is acquired by accessing the distributed ledger and acquiring transactions.

[0039] The composite video output unit 15 is for generating and outputting composite data that integrates all or part of the composite video information with the composite video. Specifically, the composite video output unit 15 includes an attribute information acquisition unit 24 that acquires attribute information which is information relating to the attributes of the output destination of the composite video; a determination unit 25 that determines whether or not the attribute information of the output destination satisfies the disclosure conditions included in the disclosure condition information; an information extraction unit 26 that extracts information that the determination unit 25 has determined to satisfy the disclosure conditions from the composite video information; a composite data generation unit 27 that generates composite data that integrates all or part of the composite video information extracted by the information extraction unit 26 with the composite video; and a composite data output unit 28 that outputs the composite data to a predetermined output destination.

[0040] The attribute information acquisition unit 24 is for acquiring attribute information, which is information relating to the attributes of the output destination of the synthesized video. The attribute information acquired by the attribute information acquisition unit 24 is for comparison with disclosure condition information relating to the synthesized video information, and consists of information including information for identifying the output destination, as well as information for items corresponding to items in the disclosure condition information, i.e., information including the attributes of the output destination. As attribute information, the attribute information acquisition unit 24 acquires information such as whether the user of the output destination is a high-paying user, a low-paying user, or a non-paying user.

[0041] The determination unit 25 is used to determine whether the attribute information relating to a predetermined output destination satisfies the disclosure conditions indicated in the disclosure condition information relating to the composite video information when generating composite data to be output to a predetermined output destination. Specifically, the determination unit 25 compares the attribute information with the respective disclosure conditions attached to one or more pieces of information constituting the composite video information, and determines whether the attribute information satisfies the disclosure conditions for each disclosure condition. For example, if information A, information B, information C, ... is included in the composite video information and each is assigned disclosure conditions A, B, C, ... according to the disclosure condition information, the determination unit 25 determines whether the attribute information of the output destination satisfies disclosure condition A, whether the attribute information of the output destination satisfies disclosure condition B, and whether the attribute information of the output destination satisfies disclosure condition C, and has the function of outputting as a determination result whether the attribute information of the predetermined output destination satisfies disclosure condition A for information A, does not satisfy disclosure condition B for information B, does not satisfy disclosure condition C for information C, etc.

[0042] The information extraction unit 26 is for extracting information that the determination unit 25 has determined to satisfy the disclosure conditions from among one or more pieces of information contained in the composite video information acquired by the composite video information acquisition unit 14. Specifically, the information extraction unit 26 has the function of extracting information from among the pieces of information contained in the composite video information in which the attribute information of the output destination satisfies the disclosure conditions for each piece of information, that is, information that can be disclosed to the output destination, and outputting it to the composite data generation unit 27. This is for extracting all or part of the information that can be disclosed to the output destination. Specifically, the information extraction unit 26 has the function of comparing the attribute information acquired by the attribute information acquisition unit 24 with the disclosure condition information related to the synthesized video information, and extracting the synthesized video information for which the attribute information is determined to satisfy the disclosure conditions. For example, if the attribute information indicates that the user of the output destination is a high-spending user, the information extraction unit 26 has the function of extracting the synthesized video information linked to the disclosure condition "high-spending user" and outputting it to the synthesized data generation unit 27.

[0043] The composite data generation unit 27 is for generating composite data that integrates all or part of the composite video information with the composite video. Specifically, the composite data generation unit 27 has the function of integrating all or part of the composite video information with information extracted by, for example, the information extraction unit 26 and the composite video. As for the form of information integration, it is desirable to embed the data constituting all or part of the composite video information into the data constituting the composite video, but integration may also be carried out by other means.

[0044] The composite data output unit 28 is for outputting composite data in which part or all of the composite video information generated by the composite data generation unit 27 is integrated with the composite video. Specifically, the composite data output unit 28 has the function of identifying an output destination based on attribute information acquired by the attribute information acquisition unit 24 and outputting a composite video with the composite video information embedded to the identified output destination. As a simpler configuration, the composite data generation unit 27 may be omitted and the composite video and composite video information may be output as separate data (in this case, the composite data output unit 28 functions as a means to realize the output step shown in claim 4 of the claims), but in this embodiment 1, a configuration is adopted in which the composite data generation unit 27 adds the composite video information to the composite video and outputs it so that the user can more reliably confirm the content of the composite video information.

[0045] The update information acquisition unit 16 acquires update information for the composite video information related to the composite video and disclosure condition information related to said update information, and outputs it to the update information output unit 17. Specifically, the update information acquisition unit 16 has the function of accessing a distributed ledger that corresponds one-to-one with the composite video to which the update information is to be updated, and acquiring the update information and disclosure condition information related to said update information from the transactions, which are information stored in blocks of the distributed ledger. In a simple configuration, the update information acquisition unit 16 may directly acquire the update information and disclosure condition information from the composite video update information generation unit 9, but in this embodiment 1, from the viewpoint of preventing tampering, the information stored in the distributed ledger is acquired.

[0046] The update information output unit 17 extracts some information that can be disclosed to the output destination based on the disclosure conditions information from the update information acquired by the update information acquisition unit 16, and then outputs it to the output destination. Specifically, the update information output unit 17 includes an attribute information acquisition unit 29 that acquires attribute information of the output destination, a determination unit 30 that determines whether the attribute information of the output destination satisfies the disclosure conditions included in the disclosure conditions information, and an information extraction unit 31 that extracts information from the update information that the determination unit 30 has determined to satisfy the disclosure conditions.

[0047] The attribute information acquisition unit 29 is for acquiring attribute information, which is information about the attributes of the destination to which the update information is output. The attribute information is for comparison with the disclosure condition information in the update information and consists of information that identifies the destination, as well as information that indicates the conditions of the destination, such as the attributes of the destination, which are in a corresponding relationship with the disclosure condition information. As attribute information, the attribute information acquisition unit 29 acquires information such as whether the user of the destination is a high-paying user, a low-paying user, or a non-paying user.

[0048] The determination unit 30 is used to determine whether the attribute information relating to a predetermined output destination satisfies the disclosure conditions indicated in the disclosure conditions information relating to the update information when outputting update information to that destination. Specifically, the determination unit 30 compares the attribute information with each disclosure condition attached to one or more pieces of information constituting the update information, and determines whether the attribute information satisfies each disclosure condition. For example, assuming that information 1, information 2, information 3, ... are each attached to disclosure conditions 1, 2, 3, ... in the update information, the determination unit 30 determines whether the attribute information satisfies disclosure condition 1, whether the attribute information satisfies disclosure condition 2, and whether the attribute information satisfies disclosure condition 3, and has the function of outputting as a determination result that the attribute information of the predetermined output destination satisfies disclosure condition 1 for information 1, does not satisfy disclosure condition 2 for information 2, satisfies disclosure condition 3 for information 3, etc.

[0049] The information extraction unit 31 is for extracting information that the determination unit 30 has determined to satisfy the disclosure conditions from among one or more pieces of information included in the update information acquired by the update information acquisition unit 16. Specifically, the information extraction unit 31 has the function of extracting information from among the pieces of information included in the update information in which the attribute information of the output destination satisfies the disclosure conditions for each piece of information, that is, information that can be disclosed to the output destination, and outputting it to the composite data output unit 28. For example, if the attribute information of the output destination indicates that the user of the output destination is a minor, the information extraction unit 31 has the function of extracting update information linked to the condition "minor" as disclosure condition information, for example, update information such as "It has become possible to output to postcards as a new usage condition," and outputting it to the composite data output unit 28. The composite data output unit 28 outputs all or part of the update information extracted by the information extraction unit 31 to a predetermined output destination, separately from the output of the composite data.

[0050] Next, the advantages of the composite video management system according to Embodiment 1 will be described. First, the composite video management system according to Embodiment 1 employs a configuration that generates composite video and composite video information, which is information related to the composite video, and provides it to the output destination of the composite video user, etc. By adopting this configuration, the composite video user, etc. has the advantage that they can always check the contents of the generated information, rights information, usage information, etc., even when they are not connected to the internet, for example. Furthermore, since Embodiment 1 employs a configuration that outputs the composite video information integrated with the composite video, even if a malicious user, for example, attempts to misuse the system while ignoring the usage conditions, the computer used to view the composite video can automatically read the usage conditions and respond accordingly, thereby effectively suppressing misuse.

[0051] Furthermore, the composite video management system according to this embodiment 1 employs a configuration that generates update information for the composite video and outputs it to the output destination from which the composite video was output. By adopting this configuration, the composite video management system according to this embodiment 1 has the advantage that, for example, if the usage conditions of an already outputted composite video change, it generates and outputs update information regarding the usage information contained in the composite video information, thereby enabling users of the composite video to use it appropriately according to the changed usage conditions.

[0052] Furthermore, the composite video management system according to this embodiment 1 generates disclosure condition information for both the composite video information and the update information, and employs a configuration that provides the output destination with information that satisfies the disclosure conditions from among the composite video information and the update information, according to the attributes of the output destination. By adopting this configuration, the composite video management system according to this embodiment 1 has the advantage of reducing communication volume by suppressing the provision of unnecessary information, and optimizing and differentiating services according to the attributes of the user.

[0053] (Embodiment 2) Next, a composite video management system according to Embodiment 2 will be described. In Embodiment 2, components that have the same name and reference numerals as those in Embodiment 1 will perform the same functions as those in Embodiment 1 unless otherwise specified.

[0054] As shown in Figure 2, the composite video management system according to Embodiment 2 includes a disclosure condition information generation unit 33 that generates disclosure condition information, which is information that defines the disclosure conditions for update information and the disclosure timing when the disclosure conditions are met, and a composite data output unit 34 that outputs the update information that satisfies the disclosure conditions included in the disclosure condition information generated by the disclosure condition information generation unit 33 at the disclosure timing defined in the disclosure condition information.

[0055] The disclosure condition information generation unit 33, similar to the disclosure condition generation unit 23 in Embodiment 1, generates disclosure condition information that includes conditions for disclosing one or more pieces of information included in the update information, which are determined according to attribute information that is information about the attributes of the output destination of the synthesized data, as well as information that specifies the timing of disclosure when the disclosure conditions are met. Specifically, for example, the disclosure condition information generation unit 33 sets the disclosure condition for a predetermined item of information in the update information to be that the user of the output destination is an adult, and sets the disclosure timing to be three months after the generation of the update information, and generates disclosure condition information that includes these contents. It is also possible for the disclosure condition information generation unit 33 to generate disclosure condition information that sets multiple disclosure conditions for the same information. For example, it may generate disclosure condition information that sets the disclosure of a predetermined item of information in the update information to be possible when the output destination user is a high-spending user or a low-spending user, and then sets the disclosure to be one month after the generation of the update information for high-spending users and one year after the generation of the update information for low-spending users.

[0056] The composite data output unit 34, like the composite data output unit 28 in Embodiment 1, has the function of outputting update information that satisfies the disclosure conditions defined in the disclosure conditions information, and also has the function of outputting the composite data according to the disclosure period defined in the disclosure conditions information. Specifically, the composite data output unit 34 does not simply output the update information that has been deemed outputtable as satisfying the disclosure conditions, separate from the output of the composite data, but rather has the function of further referring to the disclosure conditions information to obtain information regarding the output period, and then outputting the information to the output destination at that output period. As a result, the composite video management system according to Embodiment 2 makes it possible to output not only different update information, but also the same update information, at different times depending on the attributes of the output destination, etc.

[0057] Next, the advantages of the composite video management system according to this second embodiment will be described. The composite video management system according to this second embodiment includes, in addition to the disclosure conditions, information regarding the timing of outputting the update information when the disclosure conditions are met, within the disclosure conditions information regarding the update information. By adopting this configuration, the composite video management system according to this second embodiment has the advantage of being able to not only decide whether or not to send update information, but also to stagger the timing of sending the update information, thereby enabling the optimization and differentiation of services according to the attributes of the user. [Industrial applicability]

[0058] This invention can be used as a technology for appropriately managing the output of a composite video and information related to the composite video, which is generated by inserting an avatar, generated based on a person's image, into a video to be inserted, which is composed of one or more video materials. [Explanation of Symbols]

[0059] 1. Video material input section 2. Material Information Generation Unit 3. Insertion target video generation unit 4. Human Video Input Section 5. Avatar Generation Section 6. Avatar Information Generation Unit 7 Avatar insertion section 8. Video Information Generation Unit 9.35 Composite Video Update Information Generation Unit 10 Token generation unit 11 Transaction Generation Unit 12 Electronic signature generation section 13 Output section 14. Composite Image Information Acquisition Unit 15, 36 Composite video output section 16 Update information acquisition section 17 Update Information Output Unit 19. Synthetic Video Information Generation Unit 20, 23, 33 Disclosure condition information generation section 22 Update information generation section 24, 29 Attribute information acquisition part 26, 31 Information extraction part 27. Synthetic Data Generation Unit 28, 34 Synthetic Data Output Section 25, 30 Judgment section

Claims

1. A composite video management system that manages the output mode of a composite video generated by inserting an avatar, generated based on a predetermined person's video, into a portion of the target video, An insertion target video generation means that generates the insertion target video based on one or more video materials, An avatar generation means for generating the avatar based on the aforementioned person video, An avatar insertion means that generates a composite image by inserting the avatar generated by the avatar generation means into all or part of the video material in the video to be inserted, A composite video information generation means for generating composite video information which includes one or more of the following: generation information which is information relating to the manner in which the composite video is generated; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video. A composite data generation means that generates composite data by integrating all or part of the composite video information generated by the composite video information generation means with the composite video, A composite data output means for outputting the composite data generated by the composite data generation means, A composite video management system characterized by having the following features.

2. Disclosure condition information generation means generates disclosure condition information, which is information regarding the disclosure conditions of one or more pieces of information included in the composite video information, determined according to attribute information, which is information regarding the attributes of the output destination of the composite data. When generating the composite data relating to a predetermined output destination, a determination means for determining whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure conditions information, Information extraction means for extracting information that the determination means determines satisfies the disclosure conditions from among one or more pieces of information contained in the synthesized video information, Furthermore, The composite video management system according to claim 1, characterized in that the composite data generation means generates composite data in which the information extracted by the information extraction means is integrated with the composite video as part of the composite video information.

3. Update information generation means that generates update information which is information that updates the content of the aforementioned composite video information, Disclosure condition information generation means generates disclosure condition information, which is information regarding the disclosure conditions of one or more pieces of information included in the update information, as determined according to attribute information, which is information regarding the attributes of the output destination. When outputting the update information to a predetermined output destination, a determination means for determining whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure conditions information, An information extraction means for extracting information from among one or more pieces of information included in the update information that the determination means determines to satisfy the disclosure conditions, Equipped with, The composite video management system according to claim 1, characterized in that the composite data output means outputs information extracted by the information extraction means separately from the composite data.

4. A composite video management method for managing the output mode of a composite video generated by inserting an avatar, generated based on a predetermined person's video, into a portion of the target video, A step of generating an insertion target video, which generates the insertion target video based on one or more video materials, An avatar generation step of generating the aforementioned avatar based on a person's video, An avatar insertion step is performed to insert the avatar generated in the avatar generation step into all or part of the video material in the video to be inserted to generate a composite video, A composite video information generation step that generates composite video information including one or more of the following: generation information which is information relating to the manner in which the composite video is generated; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video. An output step which outputs all or part of the composite video information generated in the composite video information generation step and the composite video, A composite video management method characterized by including [this].

5. The update information generation step generates update information which is information that updates the content of the aforementioned composite video information, A disclosure condition information generation step generates disclosure condition information which is information that specifies the conditions for disclosing one or more pieces of information included in the update information, which are determined according to the attribute information which is information relating to the attributes of the output destination of the synthesized data, and the timing of disclosure when the conditions for disclosure are met. A determination step in which, when outputting the update information to a predetermined output destination, the attribute information relating to the predetermined output destination satisfies the conditions for disclosing one or more pieces of information in the disclosure condition information, An information extraction step in which information is extracted from one or more pieces of information included in the aforementioned update information that is determined to satisfy the conditions disclosed in the aforementioned determination step, Equipped with, The composite video management method according to claim 4, characterized in that in the output step, all or part of the composite video information and, separately from the composite video, information extracted in the information extraction step from one or more pieces of information included in the update information, are output at a time specified by the disclosure condition information.

6. A composite video management program that causes a computer to manage the output format of a composite video generated by inserting an avatar, which is generated based on a predetermined person's image, into a portion of the target video, To the aforementioned computer, An insertion target video generation function that generates the insertion target video based on one or more video materials, An avatar generation function that generates the avatar based on the aforementioned person video, An avatar insertion function that generates a composite image by inserting the avatar generated by the avatar generation function into all or part of the video material in the video to be inserted, A composite video information generation function that generates composite video information including one or more of the following: generation information which is information relating to the manner in which the composite video is generated; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video. A composite data generation function that generates composite data by integrating all or part of the composite video information generated by the composite video information generation function with the composite video, A composite data output function that outputs the composite data generated by the composite data generation function, A composite video management program characterized by its ability to execute certain actions.

Images