Combined video management system, combined video management method, and combined video management program

Abstract

[Problem] The present invention comprises: an insertion target video generation unit 3 for generating, on the basis of one or more video materials that are input via a video material input unit 1, an insertion target video into which an avatar is to be inserted; a person video input unit 4 for inputting a person video upon which the avatar that is to be inserted into the insertion target video is based; an avatar generation unit 5 for generating the avatar from the person video that is input via the person video input unit 4; an avatar insertion unit 7 for inserting the avatar that is generated by the avatar generation unit 5 into the insertion target video; a video information generation unit 8 for generating combined video information that relates to a combined video; a combined video update information generation unit 9 for generating update information for the combined video information; a combined video information acquisition unit 14 for acquiring the combined video information from a distributed ledger in which the information is stored; a combined video output unit 15 for outputting the combined video that has the combined video information added thereto; an update information acquisition unit 16 for acquiring the update information from the distributed ledger; and an update information output unit 17 for outputting the acquired update information.

Description

Composite video management system, composite video management method, and composite video management program

[0001] The present invention relates to a technology for managing a composite video generated by inserting an avatar generated based on a person video into an insertion target video composed of one or more video materials.

[0002] In recent years, with the improvement of processing capabilities in electronic computers such as computers, a large number of computer graphics of human figures, so-called avatars, that reflect the characteristics of real people have been widely used. For example, an avatar is used as one's own icon when using SNS (Social Networking Service), or one's own avatar is used as the character of the protagonist in an online game or the like. In addition, a service has been proposed in which some of the characters appearing in contents such as still images and moving images are replaced with one's own avatar for viewing and the like.

[0003] By using an avatar that reflects one's own characteristics in this way, for example, when a protagonist composed of an avatar expressing the characteristics of a user in a game battles against an enemy character, an effect of improving the user's immersion in the game world occurs. By using an avatar that abstractly represents the user himself / herself as an icon indicating the user in SNS, it is expected that an effect such as promoting communication between users in a virtual space in the same sense as in the real world will occur.

[0004] Patent Documents 1 and 2 both disclose technologies for using avatars that mimic the actual appearance of the player himself / herself or co-players in a computer game that uses a head-mounted display to represent a virtual space.

[0005] Japanese Unexamined Patent Application Publication No. 2019-012509, Japanese Unexamined Patent Application Publication No. 2019-139673

[0006] However, when generating images or videos in which some characters are replaced with the user's own avatar, the handling of such images becomes problematic. As the quality of such composite images improves with recent advancements in image processing technology, there is a growing possibility that the generated composite images will not be recognized as being the result of a composite process that differs from reality. Since such misunderstandings can lead to confusion such as so-called fake news, it is necessary to take measures to ensure that composite images are easily understood as fictional works that differ from reality. However, if measures are taken to perform image processing that detracts from the sense of realism, there is a problem in that the quality of the composite image will deteriorate.

[0007] Furthermore, in recent years, it has become common not only to enjoy the generated composite images personally, but also to widely share them on social media, etc., and it is also conceivable that these composite images may be used by third parties, whether for a fee or free of charge. However, if, for example, the existence and content of the usage permission for the still images, videos, etc. that are to be inserted into the avatar used as the basis for the composite image is unclear, there is a problem that services that publish composite images on social media, etc., or third parties that use the composite images may unintentionally infringe on the rights of legitimate rights holders. No technology to solve these problems is disclosed in either Patent Document 1 or 2.

[0008] The present invention has been made in view of the above problems, and aims to provide a technology for appropriately managing the output of a composite video and information related to the composite video, which is generated by inserting an avatar generated based on a person's image into an insertion target video composed of one or more video materials.

[0009] To achieve the above objective, the composite video management system according to claim 1 is a composite video management system that manages the output manner of a composite video generated by inserting an avatar generated based on a predetermined person video into a part of a target video, comprising: an insertion target video generation means for generating the target video based on one or more video materials; an avatar generation means for generating the avatar based on the person video; an avatar insertion means for generating a composite video by inserting the avatar generated by the avatar generation means into all or part of the video materials in the target video; a composite video information generation means for generating composite video information including one or more of the following: generation information which is information relating to the generation manner of the composite video; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video; and all or part of the composite video information generated by the composite video information generation means and The composite data generation means generates composite data that integrates composite images; the composite data output means outputs the composite data generated by the composite data generation means; the disclosure condition information generation means generates disclosure condition information, which is information regarding the disclosure conditions of one or more pieces of information included in the composite image information, as determined according to attribute information, which is information regarding the attributes of the output destination of the composite data; the determination means determines whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when generating composite data relating to a predetermined output destination; and the information extraction means extracts information from one or more pieces of information included in the composite image information that the determination means determines satisfies the disclosure conditions, wherein the composite data generation means generates composite data that integrates the information extracted by the information extraction means with the composite image information.

[0010] Furthermore, in order to achieve the above objective, the composite video management system according to claim 2 is characterized in that, in the above invention, it comprises: update information generation means for generating update information which is information for updating the content of the composite video information; disclosure condition information generation means for generating disclosure condition information which is information for disclosing one or more pieces of information included in the update information, determined according to attribute information which is information about the attributes of the output destination; determination means for determining whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when outputting the update information to a predetermined output destination; and information extraction means for extracting information which the determination means has determined to satisfy the disclosure conditions from one or more pieces of information included in the update information, wherein the composite data output means outputs the information extracted by the information extraction means separately from the composite data.

[0011] Furthermore, in order to achieve the above objective, the composite video management method according to claim 3 is a composite video management method that manages the output manner of a composite video generated by inserting an avatar generated based on a predetermined person image into a part area of ​​an insertion target video, comprising: an insertion target video generation step of generating the insertion target video based on one or more video materials; an avatar generation step of generating the avatar based on a person image; an avatar insertion step of generating a composite video by inserting the avatar generated in the avatar generation step into all or part of the video materials in the insertion target video; a composite video information generation step of generating composite video information including one or more of generation information which is information relating to the generation manner of the composite video, rights information which is information relating to the rights relationship of the composite video, and usage information which is information relating to the usage conditions of the composite video; a composite data generation step of generating composite data which integrates all or part of the composite video information generated in the composite video information generation step with the composite video; an output step of outputting the composite data which integrates all or part of the composite video information generated in the composite video information generation step with the composite video; and an update information which generates update information which is information that updates the content of the composite video information. The method includes an information generation step; a disclosure condition information generation step which generates disclosure condition information which is information that defines the conditions for disclosing one or more pieces of information included in the update information, determined according to attribute information which is information relating to the attributes of the output destination of the composite data, and the timing of disclosure when the conditions for disclosure are met; a determination step which, when outputting the update information to a predetermined output destination, determines whether the attribute information relating to the predetermined output destination satisfies the conditions for disclosing one or more pieces of information in the disclosure condition information; and an information extraction step which extracts information from the one or more pieces of information included in the update information that has been determined to satisfy the conditions for disclosure in the determination step, wherein in the output step, all or part of the composite video information and the information extracted in the information extraction step from the one or more pieces of information included in the update information, separately from the composite video, are output at the time specified by the disclosure condition information.

[0012] Furthermore, in order to achieve the above objective, the composite video management program according to claim 4 is a composite video management program that causes a computer to manage the output manner of a composite video generated by inserting an avatar generated based on a predetermined person image into a part of the video to be inserted, wherein the computer is provided with: an insertion target video generation function that generates the insertion target video based on one or more video materials; an avatar generation function that generates the avatar based on the person image; an avatar insertion function that generates a composite video by inserting the avatar generated by the avatar generation function into all or part of the video materials in the insertion target video; a composite video information generation function that generates composite video information including one or more of the following: generation information which is information relating to the generation manner of the composite video; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video; a composite data generation function that generates composite data which integrates all or part of the composite video information generated by the composite video information generation function with the composite video; and a composite data output function which outputs the composite data generated by the composite data generation function. The system is characterized by the following functions being executed: an update information generation function that generates update information which is information that updates the content of the composite video information; a disclosure condition information generation function that generates disclosure condition information which is information about the disclosure conditions of one or more pieces of information included in the update information, as determined according to attribute information which is information about the attributes of the output destination; a determination function that determines whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when outputting the update information to a predetermined output destination; and an information extraction function that extracts information which the determination function has determined to satisfy the disclosure conditions from one or more pieces of information included in the update information, and the composite data output function that outputs the information extracted by the information extraction function separately from the composite data.

[0013] According to the present invention, it is possible to appropriately manage the output manner of the composite video and information related to the composite video, which are generated by inserting an avatar generated based on a person's image into a video to be inserted, which is composed of one or more video materials.

[0014] This is a schematic diagram showing the configuration of the composite video management system according to Embodiment 1. This is a schematic diagram showing the configuration of the composite video management system according to Embodiment 2.

[0015] The embodiments of the present invention will now be described in detail with reference to the drawings. The following embodiments describe the most appropriate examples of the present invention, and naturally, the content of the present invention should not be limited to the specific examples shown in these embodiments. It goes without saying that any configuration other than the specific configurations shown in the embodiments that produces similar functions and effects is also included in the technical scope of the present invention. Furthermore, while embodiments 1 and 2 below describe an avatar insertion system as an example, the present invention is not limited to physical devices such as systems, and may be configured by a method that includes the content described below, or by a program that causes a computer to execute the content described below, or by a storage medium in which such a program is stored and readable by the computer.

[0016] (Embodiment 1) First, a composite video management system according to Embodiment 1 will be described. As shown in Figure 1, the composite video management system according to Embodiment 1 includes: a video material input unit 1 for inputting video materials to be used as material for a composite video; a material information generation unit 2 for generating video material information, which is information about the video materials to be input; an insertion target video generation unit 3 that generates an insertion target video to which an avatar will be inserted, based on one or more video materials input via the video material input unit 1; a person video input unit 4 for inputting a person video that will be the basis for an avatar to be inserted into the insertion target video; an avatar generation unit 5 that generates an avatar from the person video input via the person video input unit 4; an avatar information generation unit 6 that generates information about the avatar generated by the avatar generation unit 5; an avatar insertion unit 7 that inserts the avatar generated by the avatar generation unit 5 into the insertion target video; a video information generation unit 8 that generates composite video information, which is information about a composite video, based on the video material information and avatar information; and a composite video update information generation unit that generates update information for the composite video information. The system comprises: a report generation unit 9; a token generation unit 10 for generating composite video tokens, which are non-fungible tokens that have a one-to-one correspondence with the composite video; a transaction generation unit 11 for generating transactions, which are information stored in a block in a distributed ledger (described later) that has a one-to-one correspondence with the composite video token and include composite video information and update information; an electronic signature generation unit 12 for generating electronic signatures, which are data that proves that the generation and output of the information included in the transaction is in accordance with the intentions of the holder of the composite video token; an output unit 13 for outputting the transaction and the electronic signature corresponding to the transaction to the distributed ledger that has a correspondence with the composite video token; a composite video information acquisition unit 14 for acquiring composite video information from the distributed ledger; a composite video output unit 15 for outputting a composite video with the composite video information added; an update information acquisition unit 16 for acquiring update information from the distributed ledger; and an update information output unit 17 for outputting the acquired update information.

[0017] The video material input unit 1 is for inputting video material, which is image data used to generate a composite video. The composite video consists of the video to be inserted and an avatar inserted into all or part of the video to be inserted. The video material input via the video material input unit 1 is directly used to generate the video to be inserted. The video to be inserted consists of one or more video materials. In a simple configuration, a single video material may be used as is, or a single video material may be used as the video to be inserted after image processing such as enlarging, shrinking, rotating, or partially modifying it. When the video to be inserted is composed of multiple video materials, the video to be inserted is formed by performing image processing on each video material as necessary, arranging each video material in a predetermined position, or by overlapping multiple video materials. The video material may be in the form of a two-dimensional video or a three-dimensional video, and may be a video or a still image. The content of the video material may be realistic (for example, a real landscape photographed using an imaging device such as a camera) or pictorial (for example, an illustration created by an illustrator). In this embodiment 1, the following explanation will be given assuming that the video material is generated using an imaging device.

[0018] The material information generation unit 2 is for generating video material information, which is information about video material input via the video material input unit 1. Specifically, the material information generation unit 2 includes at least one of the following: generation information, which is information about the method of generating the video material; rights information, which is information about the rights related to the video material; and usage information, which is information about the conditions for using the video material in the finished video. However, it may also include other information. For example, if the video material is realistically generated using an imaging device, the generation information includes information about the photographer, shooting location, shooting equipment, shooting date, shooting environment (temperature, weather, etc.), and the content of processing from the original video. If the video material is realistically generated, such as an illustration, or realistically generated without using an imaging device, the generation information includes at least one of the following: the creator, creation equipment (art materials, computer used for creation, software, etc.), creation date, and, if there is an original video used as a reference, specific details of the original video. However, it may also include other information. Rights information includes information that includes one or more of the following: information concerning the creator, the rights holder of intellectual property rights such as copyright, the duration of the intellectual property rights, the licensee, and the content of the usage rights held by the licensee, but it may also include other information. Usage information includes information that includes one or more of the following: information concerning the usage conditions of composite videos using video materials (to what extent the scope of publication of composite videos is permitted, to what extent the uses of composite videos (video distribution, output to paper media such as glossy paper or postcards as still images, etc.) are permitted, to what extent commercial use is permitted, etc.), information concerning the usage conditions of video materials in the generation of composite videos (to what extent processing is permitted when processing is performed, which video materials are permitted to be used in combination when other video materials are used in the same composite video, etc.), the range of attributes that may be permitted for each person who generates composite videos using video materials and the person who uses the generated composite videos (gender, age, nationality, beliefs, identification information, eligibility to use this system (e.g., paid user or unpaid user), etc.), and information concerning the usage fees of video materials, but it may also include other information.Regarding the specific methods of generating material information by the material information generation unit 2, the data constituting the content of each piece of information may be manually input when the material video is input, or information such as the shooting date and shooting equipment may be generated based on the information contained in the material video data. Furthermore, if the video material is publicly available video data, the information may be generated based on publicly available information that mentions the video material.

[0019] The insertion target video generation unit 3 is for generating an insertion target video based on one or more video materials. The insertion target video is the video that will be used as the basis for the composite video, and more specifically, the composite video is the insertion target video to which avatar insertion processing has been applied. The insertion target video generation unit 3 performs the necessary processing on one or more video materials, and if a single video material is used, it is used as the insertion target video as is, and if multiple video materials are used, it generates the insertion target video by processing such as arranging each of them in predetermined positions. The specific configuration of the insertion target video generation unit 3 can be, for example, a configuration that has an image processing function using conventional technology, and the configuration of the generated insertion target video can also be video / still image, color video / black and white video, 2D video / 3D video, etc., according to the form of the finished video.

[0020] The person video input unit 4 is for inputting person video, which is material used to generate avatars that make up the composite video. Specifically, the person video input unit 4 has the function of inputting person video, which is video of the whole or a part of the person that will be used to create the avatar to be inserted into the video to be inserted. Specifically, it may be configured to input video from an external source, or it may be configured to have an imaging mechanism for acquiring video. Specifically, the person video may be a full-body video of the person in question, or a video of a part of the person, such as a facial image. It may also be a still image or a video, and may be a two-dimensional video or a three-dimensional video. In this embodiment 1, the explanation uses a two-dimensional still image of the face of the person in question as the person video, but it goes without saying that person video with a configuration other than this can also be used in the composite video generation system according to this embodiment 1.

[0021] The avatar generation unit 5 generates an avatar for inserting and replacing all or part of the video material constituting the video to be inserted, based on a person video input via the person video input unit 4. Specifically, the avatar generation unit 5 may consist of skeletal information (bones), surface information (skin), and weight information defining the relationship between the two, but it is also preferable to have a configuration consisting only of surface information including three-dimensional shape and color tone information on the surface, and in an even simpler configuration, it may consist of the image data itself. Furthermore, the avatar generated by the avatar generation unit 5 may be an avatar corresponding to the full body image of a person, but it may also be an avatar consisting only of a part of a person, for example, only a part of the face. In this embodiment 1, an avatar consisting only of the head from the neck up is generated. The avatar generation unit 5 extracts feature points based on the human video input via the human video input unit 4, according to the position and shape of the surface features (eyes, eyebrows, nose, mouth, ears, hairstyle, etc.) and internal features (joints, etc.) of the human video. By reflecting the positional relationships between the extracted feature points onto the avatar, the unit generates a realistic avatar that reflects the physical characteristics of the model. However, if a simpler configuration is adopted, for example, the avatar may be generated using the human video as is, or with only the minimum necessary modifications such as adjusting the size and orientation of the human video.

[0022] The avatar information generation unit 6 is for generating avatar information, which is information about the avatar generated by the avatar generation unit 5. Specifically, the avatar information generation unit 6 includes at least one of the following: generation information, which is information about the manner in which the avatar is generated; rights information, which is information about the rights related to the avatar; and usage information, which is information about the conditions for use when using the avatar in the completed video. However, it may also include other information. The generation information includes at least one of the following: information about the photographer, shooting location, shooting equipment, shooting date, shooting environment, etc., related to the person video that forms the basis of the avatar; the content of the processing applied to the person video during avatar generation; the computer and software used for the processing; the date the avatar was created; the creator, etc. However, it may also include other information. The rights information includes at least one of the following: information about the author of the avatar and the person video that forms the basis of the avatar; the rights holder of intellectual property rights such as copyrights; the duration of the intellectual property rights; the licensee; the content of the usage rights held by the licensee. However, it may also include other information. Usage information shall include one or more of the following: conditions for using composite videos using avatars (to what extent the scope of public disclosure of composite videos is permitted, to what extent the uses of composite videos (video distribution, output to paper media such as glossy paper or postcards as still images, etc.) are permitted, to what extent commercial use is permitted, etc.), conditions for using avatars in the generation of composite videos (to what extent processing is permitted when processing is performed, conditions for video materials that can be used in the same composite video, etc.), and the range of attributes that may be permitted for each person who generates composite videos using avatars and the person who uses the generated composite videos (gender, age, nationality, beliefs, identification information, eligibility to use this system (e.g., paid user or free user), etc.), but other information may also be included.

[0023] The avatar insertion unit 7 is for generating a composite image by inserting the generated avatar into all or part of the video material in the video to be inserted. Specifically, the avatar insertion unit 7 has the function of inserting the avatar into the video to be inserted and completing the composite image by replacing all or part of the video material area in the video to be inserted with the avatar generated by the avatar generation unit 5. If the video to be inserted is a still image, the avatar insertion unit 7 acquires information regarding the size and direction of the replacement target in the video to be inserted, and after appropriately changing the size and direction of the avatar based on that information, inserts the avatar into the video to be inserted in a manner that replaces the replacement target. If the video to be inserted is a video, the avatar insertion unit 7 acquires information regarding the operation content of the replacement target in the video to be inserted, for example by detecting changes in the position of feature points, and after appropriately changing the size, direction and operation of the avatar based on that information, inserts the avatar into the video to be inserted in a manner that replaces the replacement target. While it is generally preferable to replace the avatar with footage of a person in the video being inserted, it is not limited to this. For example, if the content video is of a locomotive, an avatar consisting of a part of a face image could be inserted on the front of the locomotive.

[0024] The video information generation unit 8 is for generating composite video information, which is information relating to the composite video. Specifically, the video information generation unit 8 includes a composite video information generation unit 19 that generates composite video information including information relating to the generation method of the composite video, usage conditions, etc., and a disclosure condition information generation unit 20 that generates disclosure condition information, which is information relating to the disclosure conditions of the composite video information generated by the composite video information generation unit 19.

[0025] The composite video information generation unit 19 generates composite video information, which is information relating to a composite video, and includes one or more of the following: generation information, which is information relating to the generation method of the composite video; rights information, which is information relating to the rights relationship of the composite video; and usage information, which is information relating to the conditions for using the composite video. Here, the generation information includes, for example, generation information relating to the video materials and avatars that form the basis of the composite video, as well as one or more of the following: the person who performed the avatar insertion process when generating the composite video, the equipment used for the process (computer, software, etc.), the date on which the avatar insertion process was performed, and information relating to the authenticity of the composite video (information relating to whether it is a composite video and a creative work (i.e., the video content itself does not actually exist), whether the individual video materials and avatars that make up the composite video are videos of things that actually exist, etc.). However, it may also include other information. The rights information includes one or more of the following information relating to the composite video and the video materials and avatars used in the composite video: the creator, the rights holder of intellectual property rights such as copyright, the duration of the intellectual property rights, the licensee, the content of the usage rights held by the licensee, etc.. However, it may also include other information. The usage information shall include one or more of the following: usage conditions for the composite video (to what extent the public release of the composite video is permitted, to what extent the uses of the composite video are permitted (video distribution, output to paper media such as glossy paper or postcards as still images, etc.), to what extent commercial use is permitted, etc.), the range of attributes that the user of the generated composite video may accept (including cases where use under all usage conditions is permitted, as well as cases where only certain usage conditions are permitted), the scope of attributes (gender, age, nationality, beliefs, identification information, eligibility to use this system (e.g., paid user or free user)), and information regarding the usage fees for the composite video. However, it may also include other information. Furthermore, regarding the usage information, if the usage conditions for the composite video are limited to satisfying the usage conditions for the video material and avatar, a condition determination means may be provided when generating the usage information to determine whether the usage conditions for the composite video are within the scope of the usage conditions for the video material and avatar, and if they are outside the scope, the usage conditions may not be included in the usage information.Furthermore, if the usage conditions are to be changed according to the user of the synthesized video, it is preferable to generate usage information in a configuration that includes multiple variations of the usage conditions.

[0026] The disclosure condition information generation unit 20 generates disclosure condition information, which is information regarding the disclosure conditions of one or more pieces of information included in the composite video information, determined according to attribute information relating to the attributes of the output destination of the composite data (or the composite video and composite video information separately if they are not integrated) which is a composite video and a composite video information, in whole or in part. Preferably, the disclosure condition information includes information that specifies whether or not to disclose each item of the composite video information generated by the composite video information generation unit 19, according to attribute information relating to the attributes of the output destination of the composite video, but it may also include other information. For example, if the attributes of the output destination (usually the user of the composite video) are assumed to be "high payer," "low payer," and "non-payer," the disclosure condition information generation unit 20 generates disclosure condition information such that the disclosure condition for "composite video information that only allows viewing of the composite video as a condition of use" is "the attribute of the output destination is a non-payer," and the disclosure condition for "composite video information that only allows viewing and video distribution of the composite video as a condition of use" is "the attribute of the output destination is a low payer."

[0027] The composite video update information generation unit 9 is for generating update information for composite video information. Specifically, the composite video update information generation unit 9 includes an update information generation unit 22 that generates update information, which is information that updates the content of the composite video information, and a disclosure condition information generation unit 23 that generates disclosure condition information, which is information regarding the disclosure conditions of one or more pieces of information included in the update information, as determined according to the attribute information of the output destination.

[0028] The update information generation unit 22 is for generating update information, which is information that updates the content of the composite video information. Specifically, the update information generation unit 22 has the function of generating update information that identifies the items in the composite video information that have been changed, and the changed information regarding those items, in a manner that is correlated with each other. For example, the update information generation unit 22 generates information that changes the usage conditions of the composite video, which previously allowed only video distribution, to allow both video distribution and output to paper media.

[0029] The disclosure condition information generation unit 23 is for generating disclosure condition information, which is information about the disclosure conditions of one or more pieces of information included in the update information, determined according to the attribute information, which is information about the attributes of the output destination. For example, the disclosure condition information includes information that specifies whether or not to disclose the information of each item of the update information generated by the update information generation unit 22, determined according to the attribute information, which is information about the attributes of the output destination, but it may also include other information. For example, if the attributes of the output destination are assumed to be "non-paying users" and "paying users", the disclosure condition information generated by the disclosure condition unit 23 will specify that the disclosure condition for "update information stating that the fee for new subscriptions to paid services will be lower than before" can only be disclosed if "the attribute of the output destination is a non-paying user".

[0030] The token generation unit 10 is for generating synthetic video tokens, which are non-fungible tokens that have a one-to-one correspondence with the generated synthetic video. A "non-fungible token" is a so-called NFT (Non-Fungible Token), that is, a token that has the property of being infungible with other tokens by possessing unique data, and is issued based on the Ethereum® standard ERC721, for example. The non-fungible token in this embodiment 1 is issued based on ERC721 or other predetermined standards, and the transaction history of the synthetic video token, information about the holder, and information about the corresponding synthetic video, such as synthetic video information, disclosure conditions information regarding the synthetic video information, and update information regarding the synthetic video information and disclosure conditions information regarding the update information, are recorded in a distributed ledger on the blockchain that corresponds to the synthetic video token. The information recorded regarding the synthetic video token is generated by the transaction generation unit 11, which will be described later, and is output to the distributed ledger by the output unit 13 along with the electronic signature generated by the electronic signature generation unit 12.

[0031] Blockchain is a technology that uses cryptographic techniques to synchronize data among multiple computers that make up a decentralized network. Specifically, each block is composed of a collection of token information, such as agreed-upon transaction records, and information to connect to other blocks (information from the previous block). A blockchain is formed by linking multiple such blocks together. Even if data is tampered with on some of the computers, the correct data is selected by majority vote among the other computers, making it extremely difficult to destroy or tamper with the data. Blockchains can be classified into public blockchains, which do not restrict who can participate in the majority vote, meaning that an unspecified number of people can participate in the recording process on the distributed ledger; and consortium blockchains and private blockchains, where only a select number of specific individuals can participate in the majority vote, meaning that only predetermined individuals can participate in the recording process on the distributed ledger.

[0032] Specific methods for linking a synthesized video token with a synthesized video include associating the identifier of the synthesized video token with the identification information of the synthesized video. More simply, the identifier of the synthesized video token may be matched with the identification information of the synthesized video. Furthermore, the generation of non-fungible tokens may be performed by the token generation unit 10 itself, or by an external system generating the tokens by issuing a predetermined command to an external system directly or indirectly connected to the token generation unit 10. In addition, the specific format of the non-fungible token is not limited to that conforming to the Ethereum® standard ERC721, but may be any format as long as it has the property of being non-fungible and information such as transaction history can be stored in a distributed ledger.

[0033] The transaction generation unit 11 is for generating transactions, which are information stored in a distributed ledger that has a one-to-one correspondence with the synthesized video. The transaction generation unit 11 has the function of generating a transaction that includes information about the owner of the synthesized video, as well as information about the synthesized video, such as synthesized video information and disclosure condition information related to the synthesized video information, and information about updating the synthesized video information, such as update information and disclosure condition information related to the update information, and outputting it to the electronic signature generation unit 12 and the output unit 13. The transaction generation unit 11 may generate a transaction in a format that includes the synthesized video information and the update information within a single transaction, but more preferably, it generates a transaction containing the synthesized video information when the synthesized video information is generated, and then generates another transaction containing the update information when the update information is generated thereafter.

[0034] The electronic signature generation unit 12 is for generating an electronic signature, which is data that proves that the information contained in a transaction is genuine. Specifically, the electronic signature generation unit 12 has the function of generating a hash value of the transaction and encrypting the hash value with a private key to generate an electronic signature. The "hash value" is a fixed-length value obtained by applying a certain calculation procedure to the original data. Because this calculation procedure is irreversible, it is considered impossible to recover the original data from the hash value. The "private key" is a sequence of numbers used in the encryption process and is configured to be decryptable using the corresponding "public key".

[0035] The output unit 13 is for outputting transactions generated by the transaction generation unit 11, along with their corresponding digital signatures, to a distributed ledger associated with the synthesized video token. In this embodiment, the output unit 13 is configured to directly output transactions and their corresponding digital signatures, but it may also be configured to merely instruct other components (including those located outside the system) to output. In this embodiment 1, the output unit 13 is directly or indirectly connected to the network where the distributed ledger is installed, and is configured to output predetermined data to the distributed ledger. Regarding the data output from the output unit 13, a hash value is generated for the transaction, and this hash value is compared with the data obtained by decrypting the transaction and its corresponding digital signature using a public key. If the two values ​​are different, it is determined that it is not a legitimate transaction and registration to the distributed ledger is rejected. If the two values ​​match, it is determined to be a legitimate transaction, and the information composed of the transaction is stored in a block that constitutes the distributed ledger. The correspondence between transactions and distributed ledgers can be established, for example, by linking the identification information of the distributed ledger with the identification information of the transaction. More simply, the same information as the identification information of the transaction may be used as the identification information of the distributed ledger. The identification information of the distributed ledger is used when outputting information by the output unit 13, as well as when acquiring information in the composite image information acquisition unit 14 and the update information acquisition unit 16.

[0036] The composite video information acquisition unit 14 acquires composite video information and disclosure condition information related to the composite video information, which are information related to the composite video, and outputs them to the composite video output unit 15. Specifically, the composite video information acquisition unit 14 has the function of accessing a distributed ledger that corresponds one-to-one with the target composite video and acquiring composite video information and disclosure condition information related to the composite video information from the transactions, which are information stored in blocks of the distributed ledger. As a simpler configuration, it is also possible to directly acquire the composite video information and disclosure condition information related to the composite video information generated by the video information generation unit 8, but from the viewpoint of preventing tampering, etc., in this embodiment 1, the composite video information is acquired by accessing the distributed ledger and acquiring transactions.

[0037] The composite video output unit 15 is for generating and outputting composite data that integrates all or part of the composite video information with the composite video. Specifically, the composite video output unit 15 includes an attribute information acquisition unit 24 that acquires attribute information which is information relating to the attributes of the output destination of the composite video; a determination unit 25 that determines whether or not the attribute information of the output destination satisfies the disclosure conditions included in the disclosure condition information; an information extraction unit 26 that extracts information that the determination unit 25 has determined to satisfy the disclosure conditions from the composite video information; a composite data generation unit 27 that generates composite data that integrates all or part of the composite video information extracted by the information extraction unit 26 with the composite video; and a composite data output unit 28 that outputs the composite data to a predetermined output destination.

[0038] The attribute information acquisition unit 24 is for acquiring attribute information, which is information relating to the attributes of the output destination of the synthesized video. The attribute information acquired by the attribute information acquisition unit 24 is for comparison with disclosure condition information relating to the synthesized video information, and consists of information including information for identifying the output destination, as well as information for items corresponding to items in the disclosure condition information, i.e., information including the attributes of the output destination. As attribute information, the attribute information acquisition unit 24 acquires information such as whether the user of the output destination is a high-paying user, a low-paying user, or a non-paying user.

[0039] The determination unit 25 is used to determine whether the attribute information relating to a predetermined output destination satisfies the disclosure conditions indicated in the disclosure condition information relating to the composite video information when generating composite data to be output to a predetermined output destination. Specifically, the determination unit 25 compares the attribute information with the respective disclosure conditions attached to one or more pieces of information constituting the composite video information, and determines whether the attribute information satisfies the disclosure conditions for each disclosure condition. For example, assuming that information A, information B, information C, ... included in the composite video information are each assigned disclosure conditions A, B, C, ... according to the disclosure condition information, the determination unit 25 determines whether the attribute information of the output destination satisfies disclosure condition A, whether the attribute information of the output destination satisfies disclosure condition B, whether the attribute information of the output destination satisfies disclosure condition C, and has the function of outputting as a determination result whether the attribute information of the predetermined output destination satisfies disclosure condition A relating to information A, does not satisfy disclosure condition B relating to information B, does not satisfy disclosure condition C relating to information C, etc.

[0040] The information extraction unit 26 is for extracting information that the determination unit 25 has determined to satisfy the disclosure conditions from among one or more pieces of information contained in the composite video information acquired by the composite video information acquisition unit 14. Specifically, the information extraction unit 26 has the function of extracting information from among the pieces of information contained in the composite video information in which the attribute information of the output destination satisfies the disclosure conditions for each piece of information, that is, information that can be disclosed to the output destination, and outputting it to the composite data generation unit 27. It is for extracting all or part of the information that can be disclosed to the output destination. Specifically, the information extraction unit 26 has the function of comparing the attribute information acquired by the attribute information acquisition unit 24 with the disclosure condition information related to the composite video information, and extracting composite video information in which the attribute information has been determined to satisfy the disclosure conditions. For example, if the attribute information indicates that the user of the output destination is a high-spending user, the information extraction unit 26 has the function of extracting composite video information linked to the disclosure condition "high-spending user" and outputting it to the composite data generation unit 27.

[0041] The composite data generation unit 27 is for generating composite data that integrates all or part of the composite video information with the composite video. Specifically, the composite data generation unit 27 has the function of integrating all or part of the composite video information with information extracted by, for example, the information extraction unit 26 and the composite video. As for the form of information integration, it is desirable to embed the data constituting all or part of the composite video information into the data constituting the composite video, but integration may also be performed by other means.

[0042] The composite data output unit 28 is for outputting composite data in which part or all of the composite video information generated by the composite data generation unit 27 is integrated with the composite video. Specifically, the composite data output unit 28 has the function of identifying an output destination based on attribute information acquired by the attribute information acquisition unit 24 and outputting a composite video with the composite video information embedded to the identified output destination. As a simpler configuration, the composite data generation unit 27 may be omitted and the composite video and composite video information may be output as separate data (in this case, the composite data output unit 28 functions as a means to realize the output step shown in claim 4 of the claims), but in this embodiment 1, a configuration is adopted in which the composite data generation unit 27 adds the composite video information to the composite video and outputs it so that the user can more reliably confirm the content of the composite video information.

[0043] The update information acquisition unit 16 acquires update information for the composite video information related to the composite video and disclosure condition information related to said update information, and outputs it to the update information output unit 17. Specifically, the update information acquisition unit 16 has the function of accessing a distributed ledger that corresponds one-to-one with the composite video to which the update information is to be updated, and acquiring the update information and disclosure condition information related to said update information from the transactions, which are information stored in blocks of the distributed ledger. In a simple configuration, the update information acquisition unit 16 may directly acquire the update information and disclosure condition information from the composite video update information generation unit 9, but in this embodiment 1, from the viewpoint of preventing tampering, the information stored in the distributed ledger is acquired.

[0044] The update information output unit 17 is for extracting a part of the information that can be disclosed to the output destination from the update information acquired by the update information acquisition unit 16 based on the disclosure condition information, and then outputting it to the output destination. Specifically, the update information output unit 17 includes an attribute information acquisition unit 29 that acquires the attribute information of the output destination, a determination unit 30 that determines whether the attribute information of the output destination satisfies the disclosure conditions included in the disclosure condition information, and an information extraction unit 31 that extracts the information determined by the determination unit 30 to satisfy the disclosure conditions from the update information.

[0045] The attribute information acquisition unit 29 is for acquiring the attribute information that is information regarding the attribute of the output destination of the update information. The attribute information is for comparison with the disclosure condition information in the update information, and in addition to the information for specifying the output destination, it is constituted by information including the information of the output destination in a correspondence relationship with the disclosure condition information, that is, the content indicating the conditions such as the attributes of the output destination. The attribute information acquisition unit 29 acquires, for example, information such as whether the user of the output destination is a high-paying user, a low-paying user, or a non-paying user as the attribute information.

[0046] The determination unit 30 is for determining whether the attribute information regarding a predetermined output destination satisfies the disclosure conditions indicated by the disclosure condition information regarding the update information when outputting the update information to the output destination. Specifically, the determination unit 30 compares and contrasts each disclosure condition attached to each of one or more pieces of information constituting the update information with the attribute information, and determines whether the attribute information satisfies each disclosure condition. For example, assuming that disclosure conditions 1, 2, 3,... are attached to information 1, information 2, information 3,... included in the update information, the determination unit 30 determines whether the attribute information satisfies disclosure condition 1, whether the attribute information satisfies disclosure condition 2, whether the attribute information satisfies disclosure condition 3, and as a determination result, outputs that the attribute information of the predetermined output destination satisfies disclosure condition 1 regarding information 1, does not satisfy disclosure condition 2 regarding information 2, satisfies disclosure condition 3 regarding information 3, etc.

[0047] The information extraction unit 31 is for extracting information determined by the determination unit 30 to satisfy the disclosure conditions from among one or more pieces of information included in the update information acquired by the update information acquisition unit 16. Specifically, the information extraction unit 31 has a function of extracting, from each piece of information included in the update information, information for which the attribute information of the output destination satisfies the disclosure conditions regarding the respective pieces of information, that is, information that can be disclosed to the output destination, and outputting it to the composite data output unit 28. For example, when the attribute information of the output destination indicates that the user of the output destination is a minor, the information extraction unit 31 extracts update information associated with the condition of "minor" as the disclosure condition information, for example, update information such as "Output to a postcard has newly become possible as a usage condition", and outputs it to the composite data output unit 28. The composite data output unit 28 outputs all or part of the update information extracted by the information extraction unit 31 to a predetermined output destination, separately from the output of the composite data.

[0048] Next, the advantages of the composite video management system according to Embodiment 1 will be described. First, the composite video management system according to Embodiment 1 of the present invention adopts a configuration in which a composite video is generated and composite video information, which is information regarding the composite video, is generated and provided to an output destination such as a user of the composite video. By adopting such a configuration, users of the composite video and the like have the advantage that they can always check the contents such as generation information, right information, usage information, etc., even when not connected to the Internet, for example. Further, in Embodiment 1 of the present invention, since a configuration is adopted in which the composite video information is output after being integrated with the composite video, even when a malicious user or the like attempts unauthorized use by ignoring the usage conditions, unauthorized use can be effectively suppressed by automatically reading and responding to the usage conditions with an electronic computer that browses the composite video and the like.

[0049] Furthermore, the composite video management system according to this embodiment 1 employs a configuration that generates update information for the composite video and outputs it to the output destination from which the composite video was output. By adopting this configuration, the composite video management system according to this embodiment 1 has the advantage that, for example, if the usage conditions of an already outputted composite video change, it generates and outputs update information regarding the usage information contained in the composite video information, thereby enabling users of the composite video to use it appropriately according to the changed usage conditions.

[0050] Furthermore, the composite video management system according to this embodiment 1 generates disclosure condition information for both the composite video information and the update information, and employs a configuration that provides the output destination with information that satisfies the disclosure conditions from among the composite video information and the update information according to the attributes of the output destination. By adopting this configuration, the composite video management system according to this embodiment 1 has the advantage of reducing communication volume by suppressing the provision of unnecessary information, and optimizing and differentiating services according to the attributes of the user.

[0051] (Embodiment 2) Next, a composite video management system according to Embodiment 2 will be described. In Embodiment 2, components that have the same name and the same reference numerals as those in Embodiment 1 will perform the same functions as those in Embodiment 1 unless otherwise specified.

[0052] The composite video management system according to Embodiment 2, as shown in Figure 2, includes a disclosure condition information generation unit 33 that generates disclosure condition information, which is information that defines the disclosure conditions for update information and the disclosure timing when the disclosure conditions are met, and a composite data output unit 34 that outputs the update information that satisfies the disclosure conditions included in the disclosure condition information generated by the disclosure condition information generation unit 33 at the disclosure timing defined in the disclosure condition information.

[0053] The disclosure condition information generation unit 33, similar to the disclosure condition generation unit 23 in Embodiment 1, generates disclosure condition information that includes conditions for disclosing one or more pieces of information included in the update information, which are determined according to attribute information that is information about the attributes of the output destination of the synthesized data, as well as information that specifies the timing of disclosure when the disclosure conditions are met. Specifically, the disclosure condition information generation unit 33 generates disclosure condition information that includes, for example, information in a predetermined item of the update information, specifying as a disclosure condition that the user of the output destination is an adult, and as a disclosure timing of three months after the generation of the update information. The disclosure condition information generation unit 33 can also generate disclosure condition information that specifies multiple disclosure conditions for the same information. For example, it may generate disclosure condition information that specifies that information in a predetermined item of the update information can be disclosed if the output destination user is a high-spending user or a low-spending user, and that it will be disclosed to high-spending users one month after the generation of the update information, and to low-spending users one year after the generation of the update information.

[0054] The composite data output unit 34, like the composite data output unit 28 in Embodiment 1, has the function of outputting update information that satisfies the disclosure conditions defined in the disclosure conditions information, and also has the function of outputting the composite data according to the disclosure period defined in the disclosure conditions information. Specifically, the composite data output unit 34 does not simply output the update information that has been deemed outputtable as satisfying the disclosure conditions, separate from the output of the composite data, but rather has the function of further referring to the disclosure conditions information to obtain information regarding the output period, and then outputting the information to the output destination at that output period. As a result, the composite video management system according to Embodiment 2 makes it possible to output not only different update information, but also the same update information, at different times depending on the attributes of the output destination, etc.

[0055] Next, the advantages of the composite video management system according to this second embodiment will be described. The composite video management system according to this second embodiment includes, in addition to the disclosure conditions, information regarding the timing of outputting the update information when the disclosure conditions are met, within the disclosure conditions information for update information. By adopting this configuration, the composite video management system according to this second embodiment has the advantage of not only being able to decide whether or not to send update information, but also being able to stagger the timing of sending the update information, thereby enabling the optimization and differentiation of services according to the attributes of the user.

[0056] This invention can be used as a technology for appropriately managing the output of a composite video and information related to the composite video, which is generated by inserting an avatar, generated based on a person's image, into a video to be inserted, which is composed of one or more video materials.

[0057] 1. Video material input unit 2. Material information generation unit 3. Insertion target video generation unit 4. Person video input unit 5. Avatar generation unit 6. Avatar information generation unit 7. Avatar insertion unit 8. Video information generation unit 9, 35. Composite video update information generation unit 10. Token generation unit 11. Transaction generation unit 12. Electronic signature generation unit 13. Output unit 14. Composite video information acquisition unit 15, 36. Composite video output unit 16. Update information acquisition unit 17. Update information output unit 19. Composite video information generation unit 20, 23, 33. Disclosure condition information generation unit 22. Update information generation unit 24, 29. Attribute information acquisition unit 26, 31. Information extraction unit 27. Composite data generation unit 28, 34. Composite data output unit 25, 30. Judgment unit

Claims

1. A composite video management system for managing the output manner of a composite video generated by inserting an avatar generated based on a predetermined person video into a portion of a target video, comprising: an insertion target video generation means for generating the insertion target video based on one or more video materials; an avatar generation means for generating the avatar based on the person video; an avatar insertion means for generating a composite video by inserting the avatar generated by the avatar generation means into all or part of the video materials in the insertion target video; a composite video information generation means for generating composite video information including one or more of the following: generation information which is information relating to the generation manner of the composite video, rights information which is information relating to the rights relationship of the composite video, and usage information which is information relating to the usage conditions of the composite video; a composite data generation means for generating composite data which integrates all or part of the composite video information generated by the composite video information generation means with the composite video; a composite data output means for outputting the composite data generated by the composite data generation means; and a disclosure condition information generation means for generating disclosure condition information which is information relating to the disclosure conditions of one or more pieces of information included in the composite video information, as determined according to attribute information which is information relating to the attributes of the output destination of the composite data. A composite video management system comprising: a determination means for determining whether the attribute information relating to a predetermined output destination satisfies the disclosure conditions in the disclosure conditions information when generating composite data relating to a predetermined output destination; and an information extraction means for extracting information from one or more pieces of information included in the composite video information that the determination means has determined to satisfy the disclosure conditions, wherein the composite data generation means generates composite data that integrates the information extracted by the information extraction means with the composite video as part of the composite video information.

2. The composite video management system according to claim 1, comprising: update information generation means for generating update information which is information for updating the content of the composite video information; disclosure condition information generation means for generating disclosure condition information which is information about the disclosure conditions of one or more pieces of information included in the update information, determined according to attribute information which is information about the attributes of the output destination; determination means for determining whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when outputting the update information to a predetermined output destination; and information extraction means for extracting information which the determination means has determined to satisfy the disclosure conditions from among one or more pieces of information included in the update information, wherein the composite data output means outputs the information extracted by the information extraction means separately from the composite data.

3. A composite video management method for managing the output manner of a composite video generated by inserting an avatar generated based on a predetermined person video into a portion of a target video, comprising: an insertion target video generation step of generating the target video based on one or more video materials; an avatar generation step of generating the avatar based on a person video; an avatar insertion step of generating a composite video by inserting the avatar generated in the avatar generation step into all or part of the video materials in the insertion target video; a composite video information generation step of generating composite video information including one or more of the following: generation information which is information relating to the generation manner of the composite video; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video; a composite data generation step of generating composite data which integrates all or part of the composite video information generated in the composite video information generation step with the composite video; an output step of outputting the composite data which integrates all or part of the composite video information generated in the composite video information generation step with the composite video; and an update information generation step of generating update information which is information that updates the content of the composite video information. A composite video management method comprising: a disclosure condition information generation step that generates disclosure condition information which is information that defines the conditions for disclosing one or more pieces of information included in the update information, which are determined according to attribute information which is information relating to the attributes of the output destination of the composite data, and the timing of disclosure when the conditions for disclosure are met; a determination step that determines whether the attribute information relating to the predetermined output destination satisfies the conditions for disclosing one or more pieces of information in the disclosure condition information when the update information is output to a predetermined output destination; and an information extraction step that extracts information from the one or more pieces of information included in the update information that has been determined to satisfy the conditions for disclosure in the determination step, wherein in the output step, all or part of the composite video information and the information extracted in the information extraction step from the one or more pieces of information included in the update information are output at the time specified by the disclosure condition information.

4. A composite video management program that causes a computer to manage the output manner of a composite video generated by inserting an avatar, generated based on a predetermined person video, into a portion of a target video, wherein the computer is provided with: an insertion target video generation function that generates the target video based on one or more video materials; an avatar generation function that generates the avatar based on the person video; an avatar insertion function that generates a composite video by inserting the avatar generated by the avatar generation function into all or part of the video materials in the target video; a composite video information generation function that generates composite video information including one or more of the following: generation information which is information relating to the generation manner of the composite video; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video; a composite data generation function that generates composite data which integrates all or part of the composite video information generated by the composite video information generation function with the composite video; a composite data output function which outputs the composite data generated by the composite data generation function; and an update information generation function which generates update information which is information that updates the content of the composite video information. A composite video management program characterized by: executing a disclosure condition information generation function that generates disclosure condition information, which is information regarding the disclosure conditions of one or more pieces of information included in the update information, according to attribute information, which is information regarding the attributes of the output destination; a determination function that determines whether the attribute information relating to the predetermined output destination satisfies the disclosure conditions in the disclosure condition information when outputting the update information to a predetermined output destination; and an information extraction function that extracts information from one or more pieces of information included in the update information that has been determined by the determination function to satisfy the disclosure conditions, and then executing the output of information extracted by the information extraction function separately from the composite data in the composite data output function.

5. The composite video management method according to claim 4, comprising: an update information generation step of generating update information which is information that updates the content of the composite video information; a disclosure condition information generation step of generating disclosure condition information which is information that defines conditions for disclosing one or more pieces of information included in the update information, determined according to attribute information which is information relating to the attributes of the output destination of the composite data, and the timing of disclosure when the conditions for disclosure are met; a determination step of determining whether the attribute information relating to the predetermined output destination satisfies the conditions for disclosing the one or more pieces of information in the disclosure condition information when the update information is output to a predetermined output destination; and an information extraction step of extracting information from the one or more pieces of information included in the update information that has been determined to satisfy the conditions for disclosure in the determination step, wherein in the output step, all or part of the composite video information and the information extracted in the information extraction step from the one or more pieces of information included in the update information are output at the time specified by the disclosure condition information.

6. A composite video management program that causes a computer to manage the output manner of a composite video generated by inserting an avatar generated based on a predetermined person video into a portion of a target video, the program characterized by causing the computer to execute: an insertion target video generation function that generates the insertion target video based on one or more video materials; an avatar generation function that generates the avatar based on the person video; an avatar insertion function that generates a composite video by inserting the avatar generated by the avatar generation function into all or part of the video materials in the insertion target video; a composite video information generation function that generates composite video information including one or more of the following: generation information which is information relating to the generation manner of the composite video; rights information which is information relating to the rights relationship of the composite video; and usage information which is information relating to the usage conditions of the composite video; a composite data generation function that generates composite data which integrates all or part of the composite video information generated by the composite video information generation function with the composite video; and a composite data output function which outputs the composite data generated by the composite data generation function.

Images