Similar content management system, similar content management method, and similar content management program

Abstract

To provide a technique for efficiently extracting a content, such as an avatar or an icon, having a feature of an existing person, and simply and accurately conducting information management regarding the presence / absence of a relationship between the content and the existing person.SOLUTION: A similar content management system includes: a content input unit 1 for inputting a target content for which a relationship with a person is to be determined; a personal image input unit 2 for inputting a personal image which is a face image of the person; a reference image input unit 5 for inputting a reference image which is an average face image; a feature point extraction unit 6 for extracting feature points of the personal image and the target content; a feature point comparison unit 7 for comparing the personal image with the target content based on information on positions of the extracted feature points or the like; and a relationship determination unit 8 for determining a relationship between the person and the target content having similarity with the personal image in terms of appearance.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technology that can efficiently extract content such as avatars and icons that have characteristics of real people, and can easily and accurately manage information such as whether or not the content has a relationship with the real person. [Background technology]

[0002] In recent years, with the improvement of the processing power of electronic computers such as computers, many 2D and 3D computer graphics are used in IT services such as games and SNS (Social Networking Services). In these services, not only icons and avatars that imitate fictional characters are used, but also avatars that realistically represent real people such as users, and avatars that abstractly represent people's characteristics are increasingly used. As a result, for example, in a game, the effect of improving the user's immersion in the game world is generated by having the main character, who is an avatar that represents the user's characteristics, fight against an enemy character, and by using an avatar that abstractly represents the user as an icon representing the user in an SNS, it is expected that the effect of promoting interaction between users in a virtual space in a similar way to the real world is generated.

[0003] Patent documents 1 and 2 each disclose a technology that uses an avatar that resembles the actual appearance of the player or a co-player in a computer game that uses a head-mounted display to present a virtual space. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-012509 A [Patent Document 2] JP 2019-139673 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, since there are many different types of services such as games and SNS, and each service has a different world view and character design, users need to prepare avatars with different designs corresponding to each service. Users need to manage these many avatars, such as the correspondence between each service and the latest version when modifications are made, but neither Patent Document 1 nor 2 discloses any technology for managing multiple avatars.

[0006] It is also important for third parties to be able to distinguish whether an avatar, etc. having the characteristics of a particular user is an avatar, etc. actually used by that user. For example, since an avatar, etc. having the characteristics of a celebrity may be a person impersonating that celebrity and there is concern that that person may commit fraud, it is desirable to build a system that allows third parties other than that person to recognize whether an avatar, etc. having the characteristics of a certain person is associated with that person. However, such technology is not disclosed in Patent Documents 1 and 2.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology that can efficiently extract content such as avatars and icons that have the characteristics of real people, and can easily and accurately manage information such as whether or not the content has a relationship with the real person. [Means for solving the problem]

[0008] In order to achieve the above object, the similar content management system according to claim 1 is a similar content management system that manages information regarding a relationship between a principal image, which is a facial image of the principal, and target content having a similarity in appearance, and is characterized in that it comprises: a feature point extraction means that extracts feature points, which are points corresponding to each element that constitutes a face, from the facial image of the target content; a feature point comparison means that determines whether there is a similarity in appearance between the target content and the principal image based on a comparison result between the feature points of the target content extracted by the feature point extraction means and the feature points of the principal image that correspond to the feature points; a relationship determination means that determines the relationship between the principal and the target content that is determined by the feature point comparison means to have an identical appearance to the principal image; an information generation means that generates identification information of the target content and information regarding the relationship with the principal, which are recorded in a distributed ledger on a distributed network corresponding to a principal token, which is a non-fungible token that corresponds to the principal; and an output instruction means that instructs the distributed network to output the information generated by the information generation means.

[0009] In addition, in order to achieve the above object, the similar content management system of claim 2, in the above invention, is characterized in that the feature point comparison means determines that the target content has an appearance similarity with the person's image when, for one or more feature points to be compared, the distance between the position of a feature point in an image of a face part of the target content and the position of a feature point in the person's image is smaller than a first threshold value.

[0010] Furthermore, in order to achieve the above object, the similar content management system of claim 3, in the above invention, is characterized in that the feature point comparison means determines that the target content has an appearance similarity with the person's image when the value of the dot product of a first vector having a start point at the position of a feature point in a reference image, which is a facial image used as a reference, and an end point at the position of a feature point in the person's image, and a second vector having a start point at the position of the feature point in the reference image and an end point at the position of the feature point in the target content, for one or more feature points to be compared, is equal to or greater than a second threshold value.

[0011] In order to achieve the above object, the similar content management method according to claim 4 is a similar content management method for managing information regarding a relationship between a personal image, which is a facial image of the person, and target content having an external similarity with the person, and is characterized in that it includes a feature point extraction step of extracting feature points, which are points corresponding to each element of a face, from the facial image of the target content; a feature point comparison step of determining whether or not there is an external similarity between the target content and the personal image based on a comparison result between the feature points of the target content extracted in the feature point extraction step and the feature points of the personal image corresponding to the feature points; a relationship determination step of determining a relationship between the person and the target content determined to have an external identity with the personal image in the feature point comparison step; an information generation step of generating information regarding an identification information of the target content and a relationship with the person, which is recorded in a distributed ledger on a distributed network corresponding to a personal token, which is a non-fungible token corresponding to the person; and an output instruction step of instructing the distributed network to output the information generated in the information generation step.

[0012] In order to achieve the above object, the similar content management method of claim 5, in the above invention, further includes a feature point selection step of selecting, from one or more feature points, a feature point for which the distance between the position of the feature point in a reference image, which is a facial image used as a reference, and the position of the feature point corresponding to the feature point in the person's image is equal to or greater than a third threshold value, and is characterized in that, in the feature point comparison step, a comparison of feature points is performed for the feature points selected in the feature point selection step.

[0013] In order to achieve the above object, the similar content management program according to claim 6 is a similar content management program that causes a computer to manage information regarding a relationship between a personal image, which is a facial image of the person, and target content having an external similarity between the person and the person, and is characterized in that the program causes the computer to execute a feature point extraction function that extracts feature points, which are points corresponding to each element constituting a face, from the facial image of the target content; a feature point comparison function that determines whether or not there is an external similarity between the target content and the personal image based on a comparison result between the feature points of the target content extracted by the feature point extraction function and the feature points of the personal image that correspond to the feature points; a relationship determination function that determines the relationship between the person and the target content that is determined to have an external identity with the personal image by the feature point comparison function; an information generation function that generates identification information of the target content and information regarding the relationship with the person, which are recorded in a distributed ledger on a distributed network corresponding to a personal token, which is a non-fungible token that corresponds to the person; and an output instruction function that instructs the computer to output information generated by the information generation function to the distributed network. Effect of the Invention

[0014] According to the present invention, it is possible to efficiently extract content such as avatars and icons that have characteristics of real people, and to easily and accurately manage information such as whether or not the content has a relationship with the real person. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of a similar content management system according to a first embodiment. [Diagram 2] 11 is a flowchart showing a process of determining whether or not there is a similarity in appearance between a target content and a person's image in the similar content management system according to the first embodiment. [Diagram 3] FIG. 11 is a schematic diagram showing a configuration of a similar content management system according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings. In the following embodiments, examples considered to be the most appropriate embodiments of the present invention will be described, and it goes without saying that the contents of the present invention should not be interpreted as being limited to the specific examples shown in the present embodiments. Of course, any configuration that achieves the same action and effect, even if it is a specific configuration other than that shown in the embodiments, is included in the technical scope of the present invention.

[0017] (Embodiment 1) First, a similar content management system according to a first embodiment will be described. As shown in FIG. 1, the similar content management system according to the first embodiment includes a content input unit 1 for inputting target content to be determined for its relevance to the principal, a principal image input unit 2 for inputting a principal image which is a facial image of the principal, a principal information input unit 3 for inputting principal information such as a name for identifying the principal, a principal token generation unit 4 for generating a principal token which is a non-fungible token having a one-to-one correspondence with the input principal image, a reference image input unit 5 for inputting a reference image which is an average facial image, a feature point extraction unit 6 for extracting feature points of the principal image and the target content, a feature point comparison unit 7 for comparing the principal image with the target content based on information such as the positions of the extracted feature points, a relationship determination unit 8 for determining the relationship between the principal and the target content which has similarity to the principal image in terms of appearance, a transaction generation unit 9 for generating a transaction including information regarding the relationship between the target content and the principal, an electronic signature generation unit 10 for generating an electronic signature which is data proving that the content of the transaction is in line with the principal's intention, and an output instruction unit 11 for instructing to output the transaction and the electronic signature to a distributed network which is in correspondence with the principal token.

[0018] The content input unit 1 is for inputting target content that is the subject of determining the relevance to the person in the similar content management system according to the embodiment 1. Specifically, the content input unit 1 has a function of inputting, as the target content, any video content such as a photo or a video (including those accompanied by audio) that includes a facial image of a person, in addition to icons and avatars used in services such as games and SNS.

[0019] The personal image input unit 2 is for inputting a personal image, which is an image including a facial image of the person to be judged for relevance to the target content. Specifically, the personal image input unit 2 has a configuration capable of inputting any image, whether it is a moving image or a still image, or whether it is two-dimensional or three-dimensional, as long as the image is clear enough to allow extraction of facial feature points (described later). In a more preferred embodiment, it is preferable that the image is compatible with the expression mode of the target content, and multiple personal images with different expression modes may be input in accordance with various expression modes.

[0020] The personal information input unit 3 is for inputting information for identifying the person, such as the person's name, date of birth, etc. The contents of the personal information inputted via the personal information input unit 3 are stored in a form linked to the personal token.

[0021] The principal token generating unit 4 is for generating a principal token, which is a non-fungible token having a one-to-one correspondence with the principal. A "non-fungible token" refers to a so-called NFT (Non-Fangible Token), that is, a token that has unique data and has properties that make it non-fungible with other tokens, and is issued based on, for example, ERC721, which is the standard of Ethereum (registered trademark). The principal token in the present embodiment 1 is issued based on ERC721 or other predetermined standards, and is configured such that the transaction history of the token and other information are stored on the blockchain.

[0022] "Blockchain" is a technology that uses encryption technology to synchronize data between multiple computers that make up a distributed network. Specifically, each block is made up of a collection of information about the individual's token, such as agreed-upon transaction records, and information for connecting to other blocks (information from the previous block), and a blockchain is made up of multiple blocks that are linked together. Even if data is tampered with on some of the multiple computers, the correct data is selected by majority vote among the other computers, making it extremely difficult to destroy or tamper with data.

[0023] The correspondence between the principal token and the principal information is a one-to-one association between the principal token identifier and the principal's identification information (which may be information that identifies the principal, such as name and date of birth, or commonly used identification information, such as My Number). More directly, the principal token identifier and the principal's identification information may be the same, or a correspondence may be established between the identifier and the principal's identification information that are composed of different character strings, etc. Alternatively, the identifier and the principal information may be associated with URL information in the Internet space where the principal information is stored.

[0024] In the first embodiment, the blocks constituting the distributed ledger corresponding to the principal token are assumed to record transaction records related to the transfer of ownership of the principal token as well as information related to the relationship between the target content and the principal. The relationship between the target content and the principal is determined by a relationship determination unit 8, which will be described later, and specific information is generated by a transaction generation unit 9.

[0025] The principal token may be generated by the principal token generating unit 4 itself, or may be generated by an external system that is directly or indirectly connected to the principal token generating unit 4 by issuing a predetermined command to the external system. The specific format of the principal token is not limited to one that complies with ERC721, the standard of Ethereum (registered trademark), and may be any format as long as it has non-fungible properties and the transaction history is stored on the blockchain.

[0026] The reference image input unit 5 is for inputting a reference image which is an average face image. Specifically, the reference image input unit 5 may simply have a function of inputting an image to be a given reference image, or may have a function of inputting an unspecified number of face images and then generating the reference image. The function of the reference image input unit 5 may also be to input and generate reference images corresponding to different attributes (e.g., reference images for men in a certain age group) in advance, and select a reference image that matches or is similar to the attributes of the person from the multiple reference images.

[0027] The feature point extraction unit 6 is for extracting feature points in a face image. A "feature point" refers to a point corresponding to each element constituting a face, for example, each organ such as the eyes, nose, mouth, and ears, and more specifically, a part of each organ, for example, the inner corner and outer corner of the eye, the pupil, the tip of the nose, the corners of the mouth, etc. Also, feature points are not necessarily defined only for parts having external characteristics, but may include parts that do not have external characteristics when stationary, such as parts that move in a characteristic manner when facial expressions change due to facial muscles or skeletal structure.

[0028] As a specific configuration of the feature point extraction unit 6, for example, an image recognition technology realized by deep learning, machine learning, etc. may be used, or a combination of multiple image recognition technologies may be used. For example, when the target content is composed of three-dimensional computer graphics, the feature point extraction unit 6 may extract vertices corresponding to the feature parts from among the vertices on the surface that has been modeled. The feature point extraction unit 6 extracts the position and meaning of each feature point (for example, the feature point corresponds to the inner corner of the eye, etc.), but may assign a predetermined identification code to each feature point instead of the meaning (for example, a for the feature point corresponding to the center point of the pupil of the left eye, and b for the feature point corresponding to the tip of the nose). The meaning or identification code is intended to clarify the correspondence between each feature point among the target content, the reference image, and the person's image, and other information may be added as long as it can clarify the correspondence.

[0029] The feature point comparison unit 7 is for comparing the feature points of the target content and the personal image extracted by the feature point extraction unit 6. Specifically, the feature point comparison unit 7 includes a position derivation unit 13 which derives the positions of the feature points of the target content and the personal image, a distance derivation unit 14 which derives the distance between each feature point in the target content and the personal image and a feature point of a reference image corresponding to each feature point, an orientation derivation unit 15 which derives the displacement direction of each feature point from the corresponding feature point in the reference image, and a similarity determination unit 16 which compares the position, distance and orientation of each derived feature point to determine the similarity in appearance between the target content and the person shown in the personal image.

[0030] The position derivation unit 13 is for deriving the positions of feature points of the target content and the person's image. The position information of each feature point derived by the position derivation unit 13 is output to the similarity determination unit 16, and the similarity determination unit 16 determines that the positions of the corresponding feature points are the same when the positional deviation between the corresponding feature points remains below a predetermined threshold value (corresponding to a first threshold value in the claims).

[0031] In this embodiment 1, the positions of feature points are identified based on a predetermined origin and coordinate system after normalizing the facial images of the target content and the person's image (for example, enlarging or reducing the volume of the facial part in a 3D image, or the area of ​​the facial part in a 2D image, so that they are equal to a certain reference value). Also, in this embodiment 1, the same normalization process, origin and coordinate system are used for all feature point extractions. The positions of feature points identified based on such standards correspond to examples of "the positions of feature points in the image of the facial part of the target content", "the positions of feature points in the person's image" and "the positions of feature points in the reference image" in the claims.

[0032] The distance derivation unit 14 is for deriving the distance between each feature point in the personal image and the target content and a feature point in the reference image that corresponds to that feature point. Specifically, the distance derivation unit 14 has a function of deriving the distance between a corresponding feature point, for example, a feature point corresponding to the inner corner of the right eye, in each of the personal image and the reference image, and deriving the distance between that feature point in the target content and the reference image. For example, if the position coordinates of a feature point in the personal image are (x 1 , y 1 , z 1 ), and the position coordinates of the corresponding feature point in the reference image are (x 0 , y 0 , z 0 ), the distance between the two feature points is {(x 1 -x 0 ) 2 +(y 1 -y 0 ) 2 +(z 1 -z 0 ) 2} 1 / 2 In the first embodiment, the reference image is also normalized in the same manner as the target content and the person's image, and the positions of the feature points are expressed using a common origin and coordinate system. Information about the distance between the feature points derived by the distance derivation unit 14 is output to the similarity determination unit 16, and is used in combination with information about the orientation of the feature points, which will be described later, to determine the similarity.

[0033] The orientation derivation unit 15 is for deriving the direction of a line segment connecting each feature point in the personal image and the target content with a feature point in the reference image that corresponds to that feature point. Specifically, the orientation derivation unit 15 has a function of deriving the orientation of a line segment formed between each feature point in the personal image and the target content and a feature point in the reference image that corresponds to that feature point. For example, when the position coordinates of a feature point in the personal image are (x 1 , y 1 , z 1 ), and the position coordinates of the corresponding feature point in the reference image are (x 0 , y 0 , z0 ), the direction of the line segment formed between them is (x 1 -x 0 , y 1 -y 0 , z 1 -z 0 ) / {(x 1 -x 0 ) 2 +(y 1 -y 0 ) 2 +(z 1 -z 0 ) 2} 1 / 2 The information on the inter-feature point orientation derived by the orientation derivation unit 15 is output to the similarity determination unit 16, and is used in the similarity determination together with the information on the inter-feature point distance derived by the distance derivation unit 14. More specifically, the similarity determination unit 16 performs the determination process using a vector that combines the distance and orientation. That is, a first vector (x 1 -x 0 , y 1 -y 0 , z 1 -z 0 ) and the feature point of the target content (x 2 , y 2 , z 2 ) as the end point of the second vector (x 2 -x 0 , y 2 -y 0 , z 2 -z 0 The determination process is performed based on the

[0034] The similarity determination unit 16 is for determining whether there is a similarity between the person shown in the target content and the personal image, based on the position of each feature point in the target content and the personal image, and the distance and direction between each feature point and the feature point in the reference image. Specifically, the similarity determination unit 16 determines that the positions of the corresponding feature points are the same when the positional deviation between each feature point in the target content and the corresponding feature point in the personal image is less than a predetermined threshold (corresponding to the "first threshold" in the claims). In addition, based on the derivation results of the distance derivation unit 14 and the direction derivation unit 15, the similarity determination unit 16 generates a first vector whose start point is a feature point on the reference image and whose end point is a feature point on the personal image, and a second vector whose start point is a feature point on the reference image and whose end point is a feature point on the target content, and determines that the target content and the personal image have the same specificity with respect to the reference image when the inner product of the first vector and the second vector is equal to or greater than a certain threshold (corresponding to the "second threshold" in the claims). When it is determined that the feature points of the target content that are the subject of similarity judgment are located in the same position as the feature points of the person's image or that the specificity of the feature points relative to the reference image is the same, the similarity judgment unit 16 judges that the person depicted in the target content is similar in appearance to the person shown in the person's image.

[0035] In addition, in the similarity determination unit 16, regarding the similarity determination of the specificity of corresponding feature points with respect to the reference image, if the difference value between the distance between feature points between the target content and the reference image and the distance between feature points between the person's image and the reference image is less than a predetermined threshold value, and the angle formed by the line segment connecting the feature point of the target content and the feature point of the reference image and the line segment connecting the feature point of the person's image and the feature point of the reference image is less than a predetermined threshold value, the specificity of the feature points may be determined to be identical.

[0036] The relationship determination unit 8 is configured to determine, for target content determined by the similarity determination unit 16 to have an appearance similar to the person's image, the relationship between the target content and the person shown in the person's image, i.e., whether the content is related to the person, for example, if the target content is a person's icon, avatar, etc., whether it is an icon, avatar, etc. of the person.

[0037] Specifically, the relationship determination unit 8 compares the personal information input via the personal information input unit 3 with information on the target content determined to have a similar appearance (such as the generation process of the target content, the contents used, and the characteristics of the service in which the target content is used), and if the target content matches the personal information (for example, if the attribute information of the user or creator of the target content, such as the name, sex, age, and place of origin, matches or is similar to the personal information in a certain number or more), it presumes that there is a relationship, and if the target content does not match the personal information (if there is a contradiction in important information, or if the attribute information differs in a certain number or more), it judges that there is no relationship. The relationship determination unit 8 uses the personal information to output information on the target content that is presumed to have a relationship to an information device such as a portable terminal managed by the person, and makes a final judgment on whether or not there is a relationship with the person based on the response from the information device. As a response from the information device, for example, a configuration may be used in which an answer on whether or not there is a relationship is automatically output based on the person's sensitive information recorded in the information device, or a configuration may be used in which an answer is generated and output by the operation of the person who has confirmed the information on the target content.

[0038] The transaction generating unit 9 has a function of generating transactions to be stored in blocks in the distributed ledger, and corresponds to an example of an information generating means in the claims. Specifically, the transaction generating unit 9 generates, as the contents of the transaction, information on the change of the name of the holder of the principal's token, as well as information on the determination result by the relationship determining unit 8 regarding the target content determined by the similarity determining unit 16 to have similarity to the principal in appearance. The transaction generating unit 9 has a function of generating a transaction including information for identifying the target content determined to have similarity to the principal in appearance (information for identifying the token if a non-fungible token that corresponds one-to-one with the target content is generated. In addition, the URL information of the website where the target content is recorded, the identification code of the target content, etc. may be used.) and information on the presence or absence of a relationship with the principal.

[0039] The electronic signature generating unit 10 has a function of generating an electronic signature, which is data that proves that the creation and output of information included in a transaction is in accordance with the intention of the holder of the principal token. Specifically, the electronic signature generating unit 10 has a function of generating a hash value of the transaction generated by the transaction generating unit 9, and generating an electronic signature by encrypting the hash value with a private key held by the holder of the principal token. Note that the "hash value" is a fixed-length value obtained by performing a certain calculation procedure on the original data. Since the calculation procedure is irreversible, it is considered impossible to restore the original data from the hash value. Also, the "private key" is a sequence of numbers used in the encryption process, and is configured to be decryptable by a corresponding "public key." Note that, in the first embodiment, the transaction generating unit 9 and the electronic signature generating unit 10 are configured to generate the transaction and electronic signature, which are one aspect of the held name change information, by themselves, but this is not necessary, and the transaction generating unit 9 and the electronic signature generating unit 10 may have a function of only instructing an external predetermined device to generate the transaction and electronic signature.

[0040] The output instruction unit 11 is for instructing to output the transaction generated by the transaction generation unit 9 and the electronic signature generated by the electronic signature generation unit 10 to the distributed network associated with the principal token. In the first embodiment, the output instruction unit 11 is also provided with an output function itself, and specifically, the output instruction unit 11 is directly or indirectly connected to the distributed network and is configured in a manner capable of outputting predetermined data to the distributed network. The distributed network generates a hash value of the output transaction and compares it with data obtained by decrypting the electronic signature with a public key (= hash value of the transaction). If the two values ​​are different, it is determined that the transaction is not made by the owner of the target tangible object and rejects the registration, and if the two values ​​match, it is determined that the transaction is made by the owner and stores the information constituted by the transaction in a block constituting the distributed ledger. The transaction output by the output instruction unit 11 includes information regarding the change of the ownership name, and by outputting the transaction to the distributed network, this information is stored in a block constituting the distributed ledger provided on the distributed network.

[0041] Next, the operation of the similarity determination unit 16 in the similar content management system according to the first embodiment will be described. As shown in Fig. 2, the similarity determination unit 16 derives the distance between the feature points of the target content and the feature points of the person's image that correspond to each other, based on the positions of the feature points derived by the position derivation unit 13 (step S101), and when the distance exceeds a predetermined threshold value L 0 It is determined whether the threshold value L is equal to or greater than the threshold value L (step S102). 0 If it is less than the threshold value L (step S102, No), proceed to step S106. 0 If it is equal to or greater than that (Yes in step S102), the process proceeds to step S103.

[0042] Thereafter, the similarity determination unit 16 generates a first vector starting from the feature point on the reference image and ending at the feature point on the person's image based on the derivation results by the distance derivation unit 14 and the orientation derivation unit 15 regarding the corresponding feature points on the person's image and the reference image (step S103). Also, the similarity determination unit 16 generates a second vector starting from the feature point on the reference image and ending at the feature point on the target content based on the derivation results by the distance derivation unit 14 and the orientation derivation unit 15 regarding the corresponding feature points on the target content and the reference image (step S104).

[0043] Thereafter, the similarity determination unit 16 calculates the inner product of the first vector and the second vector, and if the calculation result exceeds a predetermined threshold M 0 It is determined whether the threshold value M 0 If it is less than the threshold value M (step S105, No), the similarity determination unit 16 determines that the target content does not have similarity in appearance to the person image (step S106), and ends the process. 0 If so (step S105, Yes), it is determined whether the processing has been completed for all feature points (step S107), and if not (step S107, No), the process returns to step S101 and the above processing is repeated for other feature points. If the processing has been completed (step S107, Yes), the similarity determination unit 16 determines that the target content has similarity in appearance to the person's image (step S108), and all processing ends.

[0044] Next, advantages of the similar content management system according to the first embodiment will be described. First, the similar content management system according to the first embodiment has a configuration in which, for target content having similar appearance to the image of the person, the relationship with the person is determined by the relationship determination unit 8, and information including the determination result regarding the target content and the relationship between the target content and the person is generated as a transaction. By adopting such a configuration, it has an advantage that the person and other service users can accurately recognize whether the target content having an appearance similar to the person has a relationship with the person, or whether it is unrelated and is merely a coincidence or a third party has intentionally aimed to confuse the person.

[0045] Moreover, the similar content management system according to the first embodiment has a configuration for determining whether the positions of corresponding feature points in both the target content and the personal image are within a similarity range when determining the similarity in appearance between the target content and the personal image. Since feature points are arranged in correspondence with the shapes of each part on the face, when it is determined that the deviation of the positions of the corresponding feature points is within a predetermined threshold and within the similarity range, similarity is also recognized in the shapes of each part on the face corresponding to the feature points. By adopting such a configuration, the similar content management system according to the first embodiment has the advantage of being able to easily and quickly determine the similarity in appearance between the personal image and the target content.

[0046] Furthermore, the similar content management system according to the first embodiment is configured to set, between corresponding feature points, a first vector having a start point at a feature point of the reference image and an end point at a feature point of the person's image, and a second vector having a start point at a feature point of the reference image and an end point at a feature point of the target content, and when the dot product of the first vector and the second vector is a value equal to or greater than a predetermined threshold, determine that the shapes of the parts on the face corresponding to the feature points are similar.

[0047] When the inner product is equal to or greater than a predetermined threshold, the length of both vectors is equal to or greater than a certain value, and the angle between the two vectors is small. When the length of the vectors is equal to or greater than a certain value, the shape of the part on the face corresponding to the feature point is significantly different from that of the reference image. In addition, when the angle between the two vectors is small, the feature points of the person's image and the feature points of the target content move in a similar direction compared to the feature points of the reference image, so that the shapes of the face parts of the person's image and the face parts of the target content differ to the same extent in comparison with the face parts in the reference image, that is, they have similarly characteristic shapes. In the first embodiment, a configuration is adopted in which the shapes of the parts on the face corresponding to the feature points are judged to have similar appearances even in such a case, thereby providing an advantage that accurate similarity judgment can be made for the characteristic parts of the person that differ from the reference image. Such an advantage is not only obtained when the inner product of the vectors is calculated, but also obtained when the angle between the two vectors is less than a certain threshold and the difference in the lengths of the two vectors is less than a certain threshold and similarity is recognized.

[0048] Furthermore, the similar content management system according to the first embodiment has a configuration in which information regarding the relationship between the target content and the person is generated as a transaction and recorded in a block constituting a distributed ledger that corresponds to the principal's token. By adopting such a configuration, it is possible to prevent the records from being altered or forged by a third party, and the person and third parties such as other service users can obtain accurate information regarding the relationship between the target content and the person.

[0049] (Embodiment 2) Next, a similar content management system according to the embodiment 2 will be described. In the embodiment 2, components having the same names and the same reference numerals as those in the embodiment 1 have the same functions as those in the embodiment 1, unless otherwise specified.

[0050] Fig. 3 is a schematic diagram showing a configuration of a similar content management system according to embodiment 2. As shown in Fig. 3, the similar content management system according to embodiment 2 includes a feature point selection unit 18 that compares a personal image with a reference image and selects a feature point at a unique position in the personal image based on a distance between corresponding feature points, a position derivation unit 20 that derives positions in the target content, the personal image, and the reference image for feature points that correspond to the feature points selected by the feature point selection unit 18, a distance derivation unit 21 that derives a distance, an orientation derivation unit 22 that derives an orientation, and a similarity determination unit 23 that determines whether or not there is a similarity in appearance between the target content and the personal image based on the derived positions, distances, and orientations.

[0051] The feature point selection unit 18 is for extracting feature points located at unique positions in the person's image from among a plurality of feature points set for the face image. Specifically, the feature point selection unit 18 has a function of extracting feature points located at unique positions, specifically positions spaced apart by a predetermined threshold value (corresponding to the "third threshold value" in the claims), as compared with the positions of corresponding feature points in the reference image.

[0052] Since the facial features such as the eyes, nose, and mouth in the reference image have average shapes, the feature points arranged corresponding to each of these features are located at average positions. Therefore, if the position of a feature point in the image of the person in question differs significantly from that in the reference image, the facial feature related to the feature point will have a shape significantly different from that in the reference image, and such different shape will be the facial feature of the person in question. In the second embodiment, the feature point selection unit 18 extracts feature points corresponding to such facial features, and the feature point comparison unit 19 compares the feature points with the target content, etc.

[0053] Next, advantages of the similar content management system according to the second embodiment will be described. The similar content management system according to the second embodiment enables a quick and realistic external similarity judgment by using only the feature points selected by the feature point selection unit 18 among many existing feature points as the comparison target in the feature point comparison unit 19. The feature points selected by the feature point selection unit 18 are feature points on the person's image that are located at positions deviated from the feature points in the reference image, and such feature points correspond to feature parts in the person's image that are different from the reference image. Therefore, by performing a similarity judgment on such feature points, it becomes possible to perform a similarity judgment on the feature parts in the face image of the person. In content that uses the person as a model, such as a portrait, creative content is created while maintaining the facial features of the person (for example, a face image that matches the style of a specific creator while maintaining the features of the eyes and nose, or a whole-body image that has the appearance of another animal while maintaining the features, etc.). In such a case, if all feature points are compared, there is a possibility that the similarity will be denied based on differences between the creative parts of the target content and the image of the person himself / herself. However, in this embodiment 2, by comparing feature points only for those that correspond to the characteristic parts of the image of the person himself / herself, it has the advantage of being able to accurately determine whether or not there is a similarity in appearance.

[0054] Moreover, the similar contents management system according to the second embodiment has an advantage that the amount of information processing is significantly reduced by adopting a configuration in which comparison processing is performed not for all feature points but only for feature points selected by the feature point selection unit 18. As a result, the similar contents management system according to the second embodiment has an advantage that it can easily and quickly determine the presence or absence of appearance similarity.

[0055] Although the contents of the present invention have been described above in the first and second embodiments, the technical scope of the present invention should not be interpreted as being limited to the specific configurations described in the embodiments. As long as the functions of the present invention can be realized, various modifications and applications of the above-mentioned embodiments also belong to the technical scope of the present invention.

[0056] For example, the similar content management system according to the first and second embodiments is configured to determine the similarity in appearance by using both the distance between feature points of the corresponding target content and the person's image and the dot product of the first and second vectors described above, but it is also possible to determine the similarity in appearance by only the distance between feature points or only the dot product of the first and second vectors (or the angle between the two vectors and the difference in the lengths of the vectors).

[0057] Furthermore, the similar content management systems according to the first and second embodiments are configured to input a person's image and a reference image, respectively, and extract feature points of both images by the feature point extraction unit 6, but may also be configured to input and store only previously extracted feature points without inputting images. Even if such a configuration is adopted, the function of determining similarity in appearance is not deteriorated, and there is an advantage in that the amount of data to be stored can be reduced.

[0058] Furthermore, in the similar content management systems according to the first and second embodiments, in the feature point comparison process by the feature point comparison units 7 and 19, the threshold value for the distance between feature points and the threshold value for the inner product of the first and second vectors may be set to different threshold values ​​for each feature point, rather than being set to a uniform value. By adopting such a configuration, it becomes possible to perform a more accurate similarity determination by setting a small threshold value for a part such as the eyes, whose shape can give a completely different impression even with a slight difference.

[0059] In the similar content management system according to the first and second embodiments, instead of using a distributed ledger corresponding to the principal token as a target for recording information regarding the relationship between the target content having similar appearance to the principal image and the principal, a non-fungible token corresponding to the principal information may be generated regardless of the principal image and recorded in a distributed ledger corresponding to the token. Furthermore, a non-fungible token may be generated for each target content, and information regarding the relationship with the principal may be recorded in a distributed ledger corresponding to each non-fungible token. [Industrial Applicability]

[0060] The present invention can be used as a technology that can efficiently extract content such as avatars and icons that have the characteristics of real people, and can easily and accurately manage information such as whether or not the content has a relationship with the real person. [Explanation of symbols]

[0061] 1 Content input section 2. Personal image input section 3. Personal information input section 4. Token Generation Unit 5. Reference image input section 6 Feature point extraction section 7, 19 Feature point comparison section 8 Relationship determination section 9 Transaction Generation 10 Electronic signature generation section 11 Output instruction section 13, 20 Position derivation part 14, 21 Distance derivation part 15, 22 Orientation derivation part 16, 23 Similarity judgment part 18 Feature point selection section

Claims

1. A similar content management system for managing information regarding a relationship between a person's face image, which is a face image of the person, and target content having an appearance similarity between the person and the person, comprising: A feature point extraction means for extracting feature points corresponding to each element constituting a face from a face image of the target content; a feature point comparison means for determining whether there is a similarity in appearance between the target content and the personal image based on a comparison result between the feature points of the target content extracted by the feature point extraction means and the feature points of the personal image corresponding to the extracted feature points; a relationship determination means for determining a relationship between the target content determined by the feature point comparison means to have an appearance identical to the person's image and the person; An information generating means for generating identification information of the target content and information regarding a relationship with the principal, the identification information being recorded in a distributed ledger on a distributed network corresponding to a principal token, which is a non-fungible token that corresponds to the principal; an output instruction means for instructing the distributed network to output the information generated by the information generation means; A similar content management system comprising:

2. The similar content management system according to claim 1, characterized in that the feature point comparison means determines that the target content has an appearance similarity to the person's image when, for one or more feature points to be compared, the distance between the position of a feature point in the image of the face part of the target content and the position of a feature point in the person's image is smaller than a first threshold value.

3. The similar content management system described in claim 1 or 2, characterized in that the feature point comparison means determines that the target content has an appearance similarity to the person's image when the value of the dot product of a first vector having a starting point at the position of the feature point in a reference image, which is a facial image used as a reference, and an ending point at the position of the feature point in the person's image, and a second vector having a starting point at the position of the feature point in the reference image and an ending point at the position of the feature point in the target content, is equal to or greater than a second threshold value, for one or more feature points to be compared.

4. A similar content management method for managing information on a relationship between a person's face image, which is a face image of the person, and target content having an appearance similarity between the person and the person, comprising: a feature point extraction step of extracting feature points corresponding to each element constituting a face from a face image of the target content; a feature point comparison step of determining whether there is a similarity in appearance between the target content and the personal image based on a comparison result between the feature points of the target content extracted in the feature point extraction step and the feature points of the personal image corresponding to the extracted feature points; a relationship determination step of determining a relationship between the target content determined to have an appearance identical to the person's image in the feature point comparison step and the person; An information generating step of generating identification information of the target content and information regarding a relationship with the principal, which are recorded in a distributed ledger on a distributed network corresponding to a principal token that is a non-fungible token corresponding to the principal; an output instruction step of instructing the distributed network to output the information generated in the information generation step; A similar content management method comprising:

5. a feature point selection step of selecting, from one or more feature points, a feature point for which a distance between a position of the feature point in a reference image, which is a face image used as a reference, and a position of a feature point corresponding to the feature point in the person image is equal to or greater than a third threshold value; 5. The similar contents management method according to claim 4, wherein in said feature point comparison step, a comparison of feature points is performed for the feature points selected in said feature point selection step.

6. A similar content management program that causes a computer to manage information regarding a relationship between a person's face image, which is a face image of the person, and target content having an appearance similarity between the person and the person, comprising: The computer is A feature point extraction function that extracts feature points corresponding to each element constituting a face from a facial image of the target content; a feature point comparison function that determines whether there is a similarity in appearance between the target content and the user's image based on a comparison result between the feature points of the target content extracted by the feature point extraction function and the feature points of the user's image that correspond to the extracted feature points; a relationship determination function that determines a relationship between the target content determined to have an appearance identical to the person's image by the feature point comparison function and the person; An information generation function that generates identification information of the target content and information regarding a relationship with the principal, which are recorded in a distributed ledger on a distributed network corresponding to a principal token that is a non-fungible token corresponding to the principal; an output instruction function for instructing the distributed network to output the information generated by the information generation function; A similar content management system characterized by executing the above.

Images