Avatar authenticity registration method, avatar authenticity registration system, representation data management system, and representation data management method

Abstract

To provide an avatar authenticity registration method capable of registering an avatar which looks exactly like a real person and determining whether the authenticity of an AI avatar can be verified even when the AI avatar is damaged by a deep fake.SOLUTION: An avatar authenticity registration method includes: an AI avatar generation step of generating an AI avatar which looks like a user by acquiring at least appearance data and voiceprint data of the user; a biological information acquisition step of acquiring biological information including at least acquisition of a specimen of blood or oral mucosa of the user of the AI avatar generated in the AI avatar generation step and subjecting the obtained specimen to a sequencer to analyze whole genome information and convert the whole genome information into data; an NFT making step of making the whole NFT by associating the biological information acquired by the biological information acquisition step, the AI avatar information, and information of identity confirmation results stored in advance with each other; and an assignment step of assigning an identity confirmed badge to the AI avatar prior to the NFT making by the NFT making step.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to an expression data management system and an expression data management method for managing expression data, and further to an avatar authenticity registration method and an avatar authenticity registration system. [Background technology]

[0002] In recent years, there has been a frequent occurrence of fake news in which expression data such as videos of characters including real people are processed to alter the character's facial expressions and statements so that they differ from the contents of the original expression data. For this reason, a technology has been proposed that makes expression data resistant to tampering by managing it with a blockchain, as shown in Patent Document 1, for example.

[0003] Patent Document 1 discloses a system in which a user's posting of content to an SNS is regarded as a transaction, and posting is permitted when the blocks that make up the posted data are approved by multiple approvers with authority independent of the SNS management authority, and by maintaining the block chain, the system guarantees the authenticity of the content posted by users and is resistant to data tampering.

[0004] Recently, mainly due to the rapid development and spread of AI (Artificial Intelligence) technology, deep fakes, a new cyber attack method that creates political risks, economic risks (stock prices, etc.), credit risks, and rumors by impersonating a person in cyberspace and making statements contrary to the person's intentions, has become a major social issue. If this problem is not solved, it will hinder AI technology and its social implementation, making it difficult to enjoy the benefits of AI, and it is thought that it will disrupt economic activity not only in the real world but also in Web2, Web3, and the metaverse.

[0005] Patent Document 2 discloses an identity verification and management system that performs identity verification by conducting a live interview during registration processing. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6762057 [Patent Document 2] Special Publication No. 2022-532677 Summary of the Invention [Problem to be solved by the invention]

[0007] However, a problem with systems such as that described in Patent Document 1 is that it is easy to tamper with expression data by accessing the expression data stored in the blockchain, storing the accessed expression data outside the blockchain, and tampering with the expression data stored outside the blockchain.

[0008] In addition, in Patent Document 2, identity verification is performed through an online interview, but this is still not a sufficient security measure and there is a possibility of fraud.

[0009] Therefore, the present invention has been devised in consideration of the above-mentioned problems, and its purpose is to provide an expression data management system and an expression data management method that can share expression data while restricting access to the expression data using a blockchain.

[0010] Another object of the present invention is to provide an avatar authenticity registration method and system that enables an authentic avatar that looks exactly like the person in cyberspace to be registered, and that enables the authenticity of an AI avatar to be determined without the need for court proceedings or other procedures in the unlikely event that a person falls victim to deepfake. [Means for solving the problem]

[0011] The avatar authenticity registration method of the first invention is an avatar authenticity registration method in an avatar authenticity registration system, characterized in that it comprises an AI avatar generation step of acquiring at least the user's appearance data and voiceprint data to generate an AI avatar that looks exactly like the user, a biometric information acquisition step of acquiring biometric information of the AI ​​avatar generated by the AI ​​avatar generation step, including acquiring a sample of at least the user's blood or oral mucosa, and running the acquired sample through a sequencer to analyze and digitize the entire genome information, an NFT conversion step of linking the biometric information acquired by the biometric information acquisition step, information about the AI ​​avatar, and information about the identity verification result stored in advance to convert the entirety into an NFT, and an assignment step of assigning an identity verification badge to the AI ​​avatar prior to NFT conversion by the NFT conversion step.

[0012] The authentic avatar registration method of the second invention is characterized in that in the first invention, the pre-verification step includes a face-to-face verification of the user's appearance and / or voiceprint, a verification of legal attribute information including at least the user's address and family register, and a verification of one or more of the user's driver's license, driving history certificate, passport, Basic Resident Registration Card (only those with a photograph), Individual Number Card (only those with a photograph), residence card or special permanent resident certificate, various health insurance cards (including the side with the address), public pension book (certificate), various disability certificates (certificate), family register full (individual) details certificate, and resident registration card.

[0013] The method for authentically registering an avatar according to a third aspect of the present invention is characterized in that, in the first or second aspect of the present invention, it further comprises an encryption step of encrypting the digitized whole genome information.

[0014] The avatar authenticity registration method according to a fourth aspect of the present invention is characterized in that, in the first aspect of the present invention, the originals of the personal identification documents or copies of the originals of the personal identification documents used in the preliminary comparison confirmation step are physically stored in a distributed manner.

[0015] The avatar authenticity registration system of the fifth invention is characterized in having an AI avatar generation means that acquires at least a user's appearance data and voiceprint data and generates an AI avatar that looks exactly like the user; a biometric information acquisition means that acquires biometric information of the user, including acquiring at least a blood or oral mucosa sample of the user, for the AI ​​avatar generated by the AI ​​avatar generation means, and runs the acquired sample through a sequencer to analyze and digitize the entire genome information; an NFT conversion means that links the biometric information acquired by the biometric information acquisition means, information about the AI ​​avatar, and information about identity verification results stored in advance to convert the entire system into an NFT; and an assignment means that assigns an identity verification badge to the AI ​​avatar prior to NFT conversion by the NFT conversion means. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a data management system in the first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing an example of the configuration of a data management unit in the first embodiment. [Diagram 3] FIG. 3 is a schematic diagram illustrating an example of the functions of the data management system in the first embodiment. [Figure 4] FIG. 4 is a schematic diagram showing an example of the operation of the data management system in the first embodiment. [Diagram 5] FIG. 5 is a flowchart showing an example of an operation for outputting data to the management server in the first embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of an operation for restricting access authority in the first embodiment. [Figure 7] FIG. 7 is a schematic diagram showing an example of the operation of the data management system in the second embodiment. [Figure 8] FIG. 8 is a schematic diagram showing an example of a character model. [Figure 9] FIG. 9 is a conceptual diagram of authentic avatar registration in the third embodiment. [Figure 10]FIG. 10 is a diagram showing an example of an identity verification document in the third embodiment. [Figure 11] FIG. 11 is a schematic flowchart of an avatar authenticity registration method according to the third embodiment. [Figure 12] FIG. 12 is a configuration diagram of an avatar authenticity registration system according to the third embodiment. [Figure 13] FIG. 13 is a detailed flowchart of the avatar authenticity registration system according to the third embodiment. [Figure 14] FIG. 14 is a diagram showing an example of a genome sequence in a fasta file format. [Figure 15] Fig. 15(a) is a diagram showing an example of a genome sequence in the fasq file format. Fig. 15(b) is a diagram showing an example of a file structure in the fasq file format. Fig. 15(c) is a diagram showing the correspondence between fasq quality and an ASCII code table. [Figure 16] FIG. 16 is a diagram showing an example of a file of a genome sequence used in the third embodiment. [Figure 17] FIG. 17 is a detailed flowchart of step 511 in FIG. [Figure 18] FIG. 18 is a diagram showing the acquisition, digitization, and encryption of iris data in step 513 in FIG. [Figure 19] FIG. 19 is a diagram showing the acquisition, digitization, and encryption of palm and fingerprint data in step 513 in FIG. [Figure 20] FIG. 20 is a diagram showing the acquisition, digitization, and encryption of voiceprint data. [Figure 21] FIG. 21 is a diagram showing an overall flow of the avatar authenticity registration method according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an example of a data management system in a first embodiment to which the present invention is applied will be described with reference to the drawings.

[0018] (First embodiment: configuration of data management system 100) An example of the configuration of a data management system 100, a management server 1, and a data management device 2 in the first embodiment will be described with reference to Figures 1 to 3. Figure 1 is a block diagram showing the overall configuration of the data management system 100 in the first embodiment.

[0019] The data management system 100 in the first embodiment has a management server 1, as shown in Fig. 1, for example. The management server 1 is connected to a plurality of data management devices 2, such as devices 2a and 2b, and other public communication networks 4a, for example, via a communication network 4. In addition to the public communication network 4a, the data management system 100 also has, for example, a distributed network 4b (for example, a blockchain network), and the management server 1 is connected to, for example, other data management devices 2 and data management systems (not shown) connected to the distributed network 4b, for example, via the communication networks 4.

[0020] The data management system 100 also has a management database 3 connected to, for example, the management server 1. In the management database 3, in addition to recording management information related to, for example, various data related to the various data management devices 2 managed by the data management system 100, various data or information such as information related to various types of transmitted and received data handled by the management server 1, communication setting information between the management server 1 and the data management device 2 connected via the communication network 4, various setting information for transmitting and receiving various types of data to and from the data management device 2, and rules may also be stored.

[0021] The distributed network 4b in the first embodiment uses a known block chain technology, and for example, hash values ​​output from a plurality of data management devices 2 connected by the distributed network 4b are linked to expression data related to character expression and recorded as a chain-like block. Each block of transmitted and received data is recorded as a legitimate block linked in a chain by storing, for example, the hash value of the content of the block transmitted and received before, in addition to the content of the transmitted and received data, inside the block. When new expression data is transmitted, the distributed network 4b compares the hash value linked to the newly acquired expression data with the hash value linked to the recorded expression data to determine whether the data has been tampered with. In addition, each piece of data to be transmitted and received is performed using, for example, P2P (Peer to Peer) communication, but communication other than P2P communication may be used. In addition, the distributed network 4b may manage the recording of the expression data using a public key cryptosystem.

[0022] <Administration Server 1> FIG. 2 is a schematic diagram showing an example of the configuration of the data management system 100 in the first embodiment. As the data management system 100, the management server 1 may be configured as, for example, a client server system or a cloud system (not shown), and includes a housing 10, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a storage unit 104, and a plurality of I / Fs 105 to 107, and each configuration is connected by an internal bus 110. The CPU 101 restricts the entire management server 1. The ROM 102 stores the operation code of the CPU 101. The RAM 103 is a working area used when the CPU 101 operates. The storage unit 104 stores information about the data management device 2, types and conditions of various communication procedures, and the like. As the storage unit 104, for example, a data storage device (not shown) such as an SSD (Solid State Drive), a hard disk, or a semiconductor memory, in addition to an HDD (Hard Disk Drive), may be used.

[0023] The CPU 101 is realized by executing a program stored in the storage unit 104 or the like, using the RAM 103 as a working area. For example, the management server 1 may have a GPU (Graphics Processing Unit) not shown. By having a GPU, faster calculation processing than usual becomes possible.

[0024] The I / F 105 is an interface for transmitting and receiving various information connected via the communication network 4. The I / F 105 transmits and receives various information between devices at multiple locations via the communication network 4. The I / F 106 is an interface for transmitting and receiving information from the input unit 108. For example, a keyboard is used as the input unit 108, and the administrator of the management server 1 inputs, via the input unit 108, transmission and reception of various information, or a restriction command for setting various information. The I / F 107 is an interface for transmitting and receiving various information with the output unit 109. The output unit 109 outputs various information stored in the storage unit 104, the processing status of the management server 1 or the data management device 2, etc. A display is used as the output unit 109, and may be, for example, a touch panel type.

[0025] <Data management device 2> As the data management device 2, for example, a known electronic device such as a personal computer (PC), a smartphone, or a tablet terminal managed for each user is used. An example of the configuration of the data management device 2 may be the same as the schematic diagram shown in Fig. 2, and includes, for example, a housing, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a storage unit, each I / F, an input unit, and an output unit. Each configuration is connected by an internal bus.

[0026] Like the management server 1, the CPU restricts the entire data management device 2. The ROM of the data management device 2 stores the operation code of the CPU. The RAM is a working area used when the CPU is operating. The storage unit of the data management device 2 is a storage area in which various information is stored as well as expression data. For example, a hard disk drive (HDD) or a solid state drive (SSD) is used as the storage unit.

[0027] The various I / Fs of the data management device 2 are interfaces for transmitting and receiving various information to and from the management server 1, devices (not shown) connected to the data management device 2, the communication network 4, and the like. The data management device 2 may have, for example, the same configuration as the management server 1. In that case, the data management device 2 has, for example, an interface for transmitting and receiving various information to and from the input unit 108, and an interface for transmitting and receiving various information to and from the output unit 109. In addition, the data management device 2 may use a keyboard as the input unit 108, or may use a sound collection device such as a microphone. A user who uses the data management device 2 can input various information such as text data and voice via the input unit 108, for example.

[0028] The output unit 109 of the data management device 2 displays various information such as for data acquisition stored in the storage unit 104, the processing status of the data management device 2, expression data, etc. As the output unit 109, a display may be used, or for example, a speaker may be used. As the various I / Fs of the data management device 2, for example, the same one may be used for each, or for example, a plurality of I / Fs may be used for each I / F. Furthermore, when a touch panel type display is used as the output unit 109 of the data management device 2, the output unit 109 may be configured to include an input unit.

[0029] The data management device 2 may be various devices such as a mobile phone, a smartphone, a TV, and a PC, as well as home appliances such as a refrigerator, a microwave oven, and an air conditioner (not shown). Various expression data acquired by the data management device 2 may be acquired from the data management device 2 or other external devices (not shown).

[0030] Furthermore, the data management device 2 may store access authority information for determining whether or not to permit access to the storage unit 104 in the storage unit 104. The data management device 2 may store access information for determining access authority such as whether or not to permit access to the storage area, for example, linked to a storage area of ​​the storage unit 104.

[0031] <Communication Network 4> In the data management system 100, the communication network 4 is an internet network or the like to which the management server 1 and the data management device 2 are connected via a communication circuit. The communication network 4 may be configured as a so-called optical fiber communication network. The communication network 4 may be realized by a public communication network 4a such as a wired communication network or a wireless communication network, or may be realized by a blockchain constructed in a distributed network 4b.

[0032] <<Public communication network 4a>> The public communication network 4a is one type of known communication network 4, and is a known network such as the Internet to which the management server 1 and the data management device 2 are connected via communication circuits in the data management system 100.

[0033] <<Distributed Network 4b>> The distributed network 4b connects the data management system 100, the management server 1, and the data management device 2 by a known block chain technology. The distributed network 4b records various data output from the data management device 2 as blocks in a block chain constructed in the distributed network 4b, for example. The distributed network 4b may be connected to another network (public communication network 4a). The distributed network 4b may also be connected to a plurality of block chains constructed in the distributed network 4b, in which data is exchanged with other data management systems 100 or services (not shown), via the public communication network 4a. It is arbitrary which distributed network 4b is constructed and which block chain data is exchanged with.

[0034] 3 is a schematic diagram showing an example of the functions of the data management system 100 in the first embodiment. As the data management device 2 constituting the data management system 100, at least an acquisition unit 11 that acquires expression data sequentially acquired in the data management device 2 and user data indicating information of a user who accesses the expression data, a generation unit 12 that generates a hash value based on the expression data when the expression data is newly acquired, a selection unit 13 that refers to a permission list in which permission data indicating whether or not access to the previously acquired user data is permitted is recorded, and selects permission data for the user data acquired by the acquisition unit 11, an output unit 14 that outputs the generated hash value to the management server 1 via the distributed network 4b, a restriction unit 15 that restricts the user's access authority to the expression data according to the permission data selected by the selection unit 13, a determination unit 17 that determines whether or not the expression data stored by the management server 1 has been tampered with according to the permission data selected by the selection unit 13, a recording unit 16 that stores various other information, and an expression generation unit 18 that generates new expression data based on the expression data accessed by the user. The acquisition unit 11, generation unit 12, selection unit 13, output unit 14, restriction unit 15, recording unit 16, determination unit 17, and expression generation unit 18 are connected to each other. It is preferable to use a highly collision-resistant hash function for the generated hash value. For example, the hash function may be SHA (Secure Hash Algorithm)-224, SHA-256, SHA-384, SHA-512, SHA-512 / 224, SHA-512 / 256, etc.

[0035] <Management Database 3> The data management system 100 includes a management database 3 used, for example, when implementing the above-mentioned process. The management database 3 may include at least, for example, identification information of the data management device 2 connected to the management server 1, various communication data or setting information related to the operation status or operation results acquired by the data management device 2, initial data related to data acquisition by the data management device 2, index information including hash and nonce, expression data and hash values ​​output from the data management device 2, security codes related to interactions with each data management device 2, and the like.

[0036] (First embodiment: an example of the operation of the data management system 100) Next, an example of the operation of the data management system 100 in this embodiment will be described. For example, the data management system 100 acquires expression data transmitted from the data management device 2 and transmits it to the management server 1. The management server 1 that has received the expression data generates a hash value based on the expression data, associates the hash value with the expression data, and stores them in a block linked by the blockchain. The data management system 100 manages the expression data stored in the blocks linked by the blockchain.

[0037] The data management system 100 acquires user data as shown in Fig. 4. Next, the data management system 100 refers to a permission list in which permission data indicating whether or not access to the acquired user data is permitted is recorded, and selects permission data for the acquired user data. The data management system 100 restricts the user's access authority to the expression data according to the selected permission data. The operation of the data management system 100 will be described in detail below.

[0038] Fig. 5 is a flowchart showing an example of an operation performed by the data management system 100 in this embodiment to store expression data in the management server 1. Fig. 6 is a flowchart showing an example of an operation performed by the management server 1 in this embodiment to restrict access authority.

[0039] The data management device 2 of the data management system 100 includes an acquisition means S110, a generation means S120, an addition means S130, and a storage means S140, as shown in Fig. 5. The data management device 2 of the data management system 100 further includes a reception means S150, a determination means S160, and a recording means S170, as shown in Fig. 6.

[0040] <Acquisition means S110> The acquiring means S110 acquires expression data. For example, the acquiring unit 11 of the data management device 2 acquires the expression data via the input unit 108 of the data management device 2 (device 2a, device 2b). The acquiring unit 11 may also acquire expression data stored in a server such as an SNS via the public communication network 4a. The acquiring unit 11 can use various known technologies as various protocols and communication formats for acquiring the expression data. The acquiring unit 11 of the data management device 2 sequentially stores, for example, the acquired various expression data and various related data in the storage unit 104.

[0041] Expression data is data indicating a character's expression, which is composed of an image or video including the character and the character's voice. Examples of expressions include visual expressions, audio expressions, and physical expressions. Visual expressions are expressions that appeal to the sense of sight, such as gestures and facial expressions. Audio expressions are expressions that appeal to the sense of hearing, such as words, speech, and song. Physical expressions are expressions that appeal to the sense of touch, such as body touch. Expression data may include pseudo data that is generated in a simulated manner.

[0042] The expression data also includes internal representation data, which includes at least one of the following data: self-awareness data indicating the character's self-awareness, priority order data indicating the character's priority with respect to events, emotion expression data indicating the character's emotion expression with respect to events, and causal relationship data indicating an estimate of the causal relationship with respect to the character's events.

[0043] Self-awareness data is data that indicates a character's self-awareness. Self-awareness refers to emotions that arise from one's mission, role, and position in society. For example, if one is in a leader position in a group, self-awareness is the "expression that a leader should adopt." Self-awareness also includes emotions such as those that are used to avoid disrupting the atmosphere in a group. Examples of self-awareness data include joy, anger, melancholy, and happiness.

[0044] Priority data is data that indicates the character's priority for events. A character's priority for events is a permutation of what the character prioritizes with respect to events. For example, if a character prioritizes self-awareness over emotional expression, this indicates that the character tends to express things based on self-awareness. Examples of priority data include self-awareness, modality, emotional expression, causality, etc.

[0045] Emotion expression data is data that indicates the emotional expression of a character in response to an event. The emotional expression of a character in response to an event indicates the emotion the character felt in response to the event. Examples of emotional expression data include joy, anger, sadness, and happiness.

[0046] Causal relationship data is data that indicates an inference of a causal relationship to an event. An inference of a causal relationship to an event indicates what events a character associates with an event. For example, it indicates an association of a traffic jam with an accident.

[0047] The above-mentioned "character" may be a person or animal that is simulated to resemble a real person or animal, or may be a person or animal that is simulated, such as an animation.

[0048] The acquisition unit 11 of the data management device 2 may acquire a plurality of pieces of expression data accumulated over a certain period of time at once, for example.

[0049] <Generation means S120> When new expression data is acquired, the generating means S120 generates a hash value based on the expression data. The generating unit 12 may generate a hash value based on the expression data, for example, by referring to a hash function. The hash value is hashed expression data.

[0050] The generating means S120 may also generate a public key that identifies the expression data acquired in the acquiring means S110.

[0051] <Output Means S130> The output means S130 outputs the generated hash value to the management server 1. The output unit 14 outputs the hash value generated by, for example, the generation unit 12 to the management server 1 via the distributed network 4b. In addition, the hash value may be output to the management server 1 as appropriate, for example, according to each setting value such as the timing set by the management server 1, the date and time, the importance of the expression data acquired by the acquisition unit 11, or the state such as capacity.

[0052] The management server 1 records the hash value in a block connected by a blockchain constructed in the distributed network 4b. The recording unit 16 sequentially records the hash value in a block connected by a blockchain constructed in the distributed network 4b by the management server 1. When the recording unit 16 determines that the received hash value is normal, for example, by the determination unit 17, the recording unit 16 records the hash value in the blockchain constructed in the distributed network 4b based on the data of the hash value and the nonce. The recording unit 16 may manage, for example, by an index based on the hash and nonce of the hash value 23. The recording unit 16 may store, for example, the generated index in the management database 3 and manage the hash value based on this index (not shown). In addition, the management server 1 may record the hash value and the expression data in a block connected by a blockchain constructed in the distributed network 4b by linking them to each other.

[0053] The management server 1 of the data management system 100 includes, for example, a user data acquisition means S150, a selection means S160, and a restriction means S170, as shown in FIG. 6, in order to manage the expression data outputted and stored in S130.

[0054] <User data acquisition means S150> The user data acquiring means S150 acquires user data indicating information of a user who accesses the stored expression data. The acquiring unit 11 acquires the user data, for example, via the input unit 108 of the data management device 2 (device 2a, device 2b). The acquiring unit 11 may also acquire the user data from a user's account or the like when the user accesses the stored hashed expression data.

[0055] User data is information that can identify a user accessing expression data, and includes one or more of the following: text data containing information about the user, image data including an image of the user, and audio data related to the user's voice.

[0056] <Selection Means S160> The selection means S160 selects permission data for the user data acquired by the acquisition unit 11, for example, by referring to a permission list in which permission data indicating the access authority of the user data acquired in advance to the expression data is recorded.

[0057] Access authority is the authority to operate data set for users or groups of users who use the system. Examples of data operations include viewing data, deleting data, updating data, and outputting data.

[0058] The permission list is a table in which permission data indicating the access authority of user data to expression data, such as that shown in Table 1, is recorded.

[0059] [Table 1] In addition, the permission list may be a trained model for generating output data from input data by machine learning using the training data, with user data as input data, permission data for the user data as output data, and the input data and output data as a set of training data.

[0060] <Restriction Means S170> The restriction means S170 restricts the user's access to the expression data according to the selected permission data, so that only users permitted by the permission list can access the expression data, and the expression data can be shared while restricting access to the expression data.

[0061] Furthermore, the restriction unit 15 may restore the hashed expression data using a private key or the like in accordance with the permission data, or may present the expression data associated with the hash value to the user.

[0062] In addition, in the restriction means S170, the determination unit 17 may determine whether the stored expression data has been tampered with in accordance with the selected permission data when the user accesses the expression data. In this case, for example, when a user who is permitted to view data accesses the expression data, the determination unit 17 may determine whether the stored expression data has been tampered with.

[0063] For example, when the hash and nonce of the blocks linked by the blockchain constructed in the distributed network 4b do not match, the determination unit 15 determines that the block is abnormal (possible tampering or forgery) and does not link the block in the blockchain, and the data is returned as a block following the normal block. Therefore, the user can access highly reliable expression data that has been determined to have not been tampered with.

[0064] By carrying out each of the above steps, the operation of the data management system 100 in the first embodiment is completed.

[0065] <Second embodiment> Next, the operation of the data management system 100 in the second embodiment to which the present invention is applied will be described. The second embodiment differs from the first embodiment in that, as shown in Fig. 7, character data indicating character data is obtained from stored expression data, and the permission list corresponding to the obtained character data is referenced. Descriptions of the same parts as those in the first embodiment will be omitted.

[0066] The acquiring unit 11 acquires character data from the expression data recorded in the recording unit 16. The character data is information capable of identifying a character included in the expression data, and includes, for example, the name and ID of the character.

[0067] The acquiring unit 11 may acquire character data from image data included in the expression data by using, for example, a known face recognition technique. The acquiring unit 11 may also acquire character data from features of voice data included in the expression data.

[0068] The acquisition unit 11 may acquire a character model generated using machine learning based on a neural network model, for example. The character model may be generated using machine learning based on a neural network model such as a CNN (Convolution Neural Network), or any other model may be used.

[0069] In the character model, for example, an association having the degree of association between reference expression data (input data) and reference character data (output data) is stored. The degree of association indicates the degree of association between the input data and the output data, and it can be determined that the higher the degree of association, the stronger the connection between the data. The degree of association may be indicated by three or more values ​​(three or more levels) such as a percentage, or may be indicated by two values ​​(two levels). The reference expression data and the reference character data are expression data and character data, respectively, acquired in advance as learning data for generating a character model.

[0070] For example, the association is constructed based on the degree of connection between many-to-many information (multiple input data, pair, multiple output data). The association is appropriately updated during the machine learning process, and indicates a function (classifier) ​​optimized based on, for example, multiple input data and multiple output data. Note that the association may have multiple association degrees indicating the degree of connection between each data. For example, when the database is constructed using a neural network, the association degree can correspond to a weight variable.

[0071] For this reason, in the data management system 100, output data suitable for input data is selected using, for example, associations based on all the results determined by the classifiers. This makes it possible to quantitatively select output data suitable for input data not only when the input data is the same as or similar to the output data, but also when the input data is dissimilar.

[0072] The association may indicate the degree of connection between a plurality of output data and a plurality of input data, for example, as shown in Fig. 8. In this case, by using the association, the degree of relationship between a plurality of input data ("Expression Data A" to "Expression Data C" in Fig. 8) can be linked and stored for each of the plurality of output data ("Character Data A" to "Character Data C" in Fig. 8). Therefore, for example, a plurality of input data can be linked to one output data via the association. This makes it possible to realize a multifaceted selection of output data for the input data.

[0073] The association has, for example, a plurality of associations linking each output data with each input data. The association is shown in three or more stages, such as a percentage, a 10-stage scale, or a 5-stage scale, and is shown, for example, by the characteristics of the line (e.g., thickness, etc.). For example, "output data A" included in the output data shows an association AA of "73%" with "output data A" included in the input data, and an association AB of "12%" with "output data B" included in the input data. In other words, the "association" indicates the degree of connection between each data, and for example, the higher the association, the stronger the connection between each data.

[0074] Furthermore, the character model may be machine-learned by providing at least one or more hidden layers between the input data and the output data. The above-mentioned correlation is set in either the input data or the hidden layer data, or in both, and this becomes the weighting of each data, and the output is selected based on this. Then, when this correlation exceeds a certain threshold, the output may be selected.

[0075] The acquisition unit 11 may acquire character data from expression data by referring to the above-mentioned character model.

[0076] Next, the selection unit 13 refers to the permission list corresponding to the character data acquired by the acquisition unit 11. In this case, the recording unit 16 may record a plurality of permission lists, and each permission list may be recorded in association with character data. Therefore, when a user accesses expression data, it becomes possible to automatically refer to the permission list corresponding to each character in the expression data, and appropriate access management can be automatically performed for each expression data.

[0077] <Third embodiment> The avatar authenticity registration method of the third embodiment adds encrypted biometric information (fingerprint, iris, genome information, etc.) as attributes to the appearance and voiceprint acquired during a face-to-face interview only at the time of initial generation of the look-alike AI avatar, and registers it on the blockchain as an NFT (Non-fungible Token) using a hash value and a nonce as an authentic initial product of the look-alike AI avatar that has been verified. The blockchain is not used to maintain the validity of the transaction chain, but rather as a notary public office for the authentic initial product of the look-alike AI avatar, and digitally judges the authenticity. Here, the blockchain described in the first and second embodiments is used. In addition, it is desirable to use a high collision-resistant hash function for the hash value to be generated. For example, SHA-224, SHA256, SHA-384, SHA-512, SHA-512 / 224, SHA-512 / 256, etc. can be used as the hash function.

[0078] That is, the avatar authentic registration method of the third embodiment includes a verification step in which, when a user registers an AI avatar that looks exactly like the user, the entire identity verification process in which identity verification is performed using the original identity verification material is verified in front of the user, an AI avatar generation step in which an AI avatar that looks exactly like the user is generated if the user is verified and confirmed to be the user in the verification step, a biometric information acquisition step in which biometric information of the user of the AI ​​avatar generated in the AI ​​avatar generation step is acquired, and an NFT conversion step in which the biometric information acquired in the biometric information acquisition step, the information of the AI ​​avatar, and the information of the identity verification result in the verification step are linked to convert the whole into an NFT. In addition, the original identity verification material or a copy of the original identity verification material used in the verification step is physically distributed and stored. The physical distributed storage will be described later.

[0079] 9 is a conceptual diagram of avatar authenticity registration in the third embodiment. The principles of the present invention will be described below with reference to FIG.

[0080] With current AI avatars, the user's appearance data is not linked to voiceprints or biometric data (unique and constant (unchanging) data such as DNA information), making them vulnerable to deep fakes. Since identity verification cannot be performed immediately, fake avatars have spread and become a social problem. Once linked, biometric data is the key to identity verification, so it is desirable to use the following method as a general rule for identity verification before linking. As shown in Figure 9, in principle, the individual (A) visits the identity verification agency (C) to conduct face-to-face verification. At that time, A submits an AI avatar creation request form and identity verification documents (A) to the AI ​​avatar creator (B) to begin the process of authenticating the avatar. In this case, Party A submits to Party B several official photo IDs (driver's license, passport, My Number card, etc.), a certified copy of family register (certificate of all family register details), a residence certificate with My Number, and a seal certificate, and has Party B scan their biometric data (fingerprints, iris, voiceprint, palm, DNA information, etc.) on the spot. In accordance with the important principle of "cash on the spot" handling, which is an important principle of financial institution transactions, these are obtained in person at the same time, and the AI ​​avatar creator (Party B) runs the sequencer on the spot without going through a third party or inspection agency, and stores the data offline in several safe deposit boxes, etc., as identity verification data (this is called physically distributed storage). Party B also collects the image data and voiceprint data required for generating an AI avatar that looks just like the person (Party C) on the spot. Party B directly inquires with the identity verification agency (Party C), which is the issuing agency, that the official ID submitted by Party A to Party B is genuine and not forged.

[0081] Party B submits the identity verification documents (A), the response from the identity verification agency (B), the generated AI avatar (C), and the individual's (Party A's) biometric information (D) to the AI ​​avatar registration agency (Party D), and registers the NFT as a whole. If the biometric information collected is genomic information, the highest level of identity verification can be achieved by using the results of a whole-genome sequence analysis of the human genome.

[0082] As shown in Fig. 10, identity verification materials (A) include face-to-face verification of the user's appearance and / or voiceprint, verification of legal attribute information including at least the user's address and family register, and information on the user's driver's license, driving history certificate, passport, basic resident registration card (only with a photograph), personal number card (only with a photograph), residence card or special permanent resident certificate, various health insurance cards (including the side with the address), public pension book (certificate), various disability certificates (certificate), family register full (individual) details certificate, resident's card, etc. Of identity verification materials, certificates certified by public institutions and resident's cards are kept as originals, while driver's licenses and personal number cards are kept as copies, and these are stored in a physically distributed manner.

[0083] The flow of the above-mentioned avatar authenticity registration method is shown in Fig. 11. The avatar authenticity registration method will be described below according to the flow of Fig. 11.

[0084] First, the individual (Person A) submits an AI avatar creation request form and identity verification documents (A) to the AI ​​avatar creator (Person B) (Step 301).

[0085] Next, the person is referred to an identity verification agency (C) by the person B (step 302).

[0086] Next, a response (B) to either Party A or Party B is obtained from the identity verification agency (Party C) (Step 303).

[0087] Next, an AI avatar (C) is generated by Z (step 304). If necessary, Party A provides biometric information (D) to Party B (step 305).

[0088] Based on A's request, Z submits the above A, B, C, and D to the AI ​​avatar registration agency (D) (step 306).

[0089] D adds the encrypted information of A, B, and D to the AI ​​avatar (C), then adds a hash and nonce and registers it as an NFT on the blockchain. D then notifies A and B of C's verified AI avatar (E) with an indication that the identity has been verified (F), and stores A, B, C, D, E, and F in a physically distributed manner for A and B (step 307).

[0090] Party A and Party B generate a video or other object (G) based on F received from Party D, and if necessary, request Party D to register G, a by-product of F generated based on F, as an NFT (step 308).

[0091] D will register the NFT of G for A and B, and will cooperate with any confirmation request from A or B to verify the deepfake (step 309).

[0092] According to this third embodiment, in addition to the first or second embodiment, it becomes possible to register an authentic avatar that looks exactly like the person in cyberspace, and even if a person falls victim to deepfake, it becomes possible to determine the authenticity of the AI ​​avatar without going through a court case or other procedure.

[0093] <Avatar Authentic Registration System> FIG. 12 shows the configuration of an avatar authenticity registration system according to the third embodiment.

[0094] As shown in FIG. 12, the genuine avatar registration system 200 has a camera 211, a scanner 212, a control unit 213, a memory unit 214, an avatar generation unit 215, an NFT unit 216, a personal identification information registration unit 217, a biometric information registration unit 218, a communication unit 219, and a sequencer 220.

[0095] <Configuration Description> The camera 211 is a camera for acquiring appearance data of the person who is to register an avatar, such as a digital camera or a digital video camera.

[0096] The scanner 212 is a scanner for acquiring biometric information.

[0097] The control unit 213 controls the entire system.

[0098] The storage unit 214 is a memory that stores various data or registered programs.

[0099] The avatar generation unit 215 generates an AI avatar of the person using various acquired data.

[0100] The NFT unit 216 registers information linked to the AI ​​avatar as an NFT on the blockchain.

[0101] The personal identification information registration unit 217 registers the above-mentioned personal identification documents.

[0102] The biometric information registration unit 218 registers the collected biometric information.

[0103] The communication unit 219 registers NFTs or performs various communications in the distributed communication network.

[0104] The sequencer 220 performs whole genome sequencing of the human genome.

[0105] The above-mentioned hardware may be realized as a standalone server, a cloud server, a distributed server, or the like.

[0106] FIG. 13 is a diagram showing an avatar authenticity registration process in the third embodiment.

[0107] The authentic avatar registration process can be broadly divided into the identity verification process 501 and the AI ​​avatar (β) generation and NFT process 507 that looks just like the individual.

[0108] The identity verification process 501 includes a face-to-face verification process 502 of the individual's appearance and voiceprint, a verification process 503 of the address and family register (legal attribute information), identity verification material (α) 504, and a process 505 for verifying these. The results of the verification and verification are stored in a physically distributed manner 506.

[0109] The AI ​​avatar (β) generation and NFT process 507 generates an AI avatar (β) that looks just like the person from the appearance data acquisition 508 and the voiceprint data acquisition 509 of the person (510). Samples such as the person's blood and oral mucosa are acquired (γ) (511) and digitized (γ) by a sequencer (512). Iris, palm, fingerprints, etc. may also be acquired and digitized (γ). These digitized data are encrypted and linked to the generated AI avatar (β) that looks just like the person, and the whole is NFTized (514). An example of a genome sequence digitized by a sequencer is shown in Figure 14 and Figures 15(a), (b), and (c). In the fasta example in Figure 14, the first line shows the header starting with ">", and the second line shows the genome sequence. In the example of Figure 15(a), a fastq file example shows the read information output from the sequencer, with four lines showing one read. As shown in Figure 15(b), each line consists of required information and options. The first line shows the sequence ID starting with @ and the addition / removal options. The second line shows the base sequence of the read, the third line shows +, and the fourth line shows the quality of the read. As shown in Figure 15(c), the quality of fastaq corresponds to "ASCII code of symbol -33". Multiple file formats are shown in Figure 16. Files such as fasta, fastq, str, and ab1 can be used.

[0110] Figure 17 shows the acquisition of biometric data, the acquisition of digitized samples, and the process of digitizing genomes (γ) using a sequencer or similar device. When a request is made to create an avatar that looks just like the individual, blood or oral mucosa is taken as a sample in the individual's presence under the supervision of a doctor (step 601). The sample is pre-sequenced in the individual's presence using a specified solution according to the sequencer's operating instructions (step 602). The pre-processed sample is run through a sequencer (step 603), and the individual's genome sequence (approximately 3 billion bases arranged in ATGC quadruplets) is detected and output as data (step 604). Genome base sequence analysis data is in the form of FASTA, FASQ, str, ab1, etc.

[0111] These may be encrypted directly, or may be temporarily converted (obfuscated) into binary data before encryption (steps 605, 606, 607). Encryption using ECDH, RAS, DAS, elliptic curve cryptography, PQC, etc. can make the data even more unreadable.

[0112] Fig. 18 shows the process of acquiring biometric data, converting it to digital data, and acquiring and converting the iris to digital data (γ) using an optical scanner or the like. Iris data (step 1001) is acquired using a medical optical scanner for eyeball examinations or the like (step 1002), and when a request is made to create an avatar that looks exactly like the person, the data is acquired in the person's presence and output (step 1003).

[0113] This data is image data and the data format is BMP, JPEG, PNG, GIF, TIFF, PSD, etc.

[0114] When the image data is binarized (obfuscated), it may be made obfuscated (step 1004). After binarization, the image data can be further encrypted using ECDH, RAS, DAS, elliptic curve cryptography, PQC, or the like to make it even more unreadable (steps 1005 and 1006).

[0115] Fig. 19 shows the acquisition and digitization of biometric data, and the acquisition and digitization (γ) of palm, fingerprint, etc., using an optical scanner, etc. Palm, fingerprint, etc. are acquired in the presence of the person using a medical or general optical scanner, and the data is output (steps 1101, 1102, 1103). This data is image data, etc., and the data format is BMP, JPEG, PNG, GIF, TIFF, PSD, etc.

[0116] When the image data is binarized (obfuscated), it may be made obfuscated (step 1104). After binarization, the image data can be further encrypted using ECDH, RAS, DAS, elliptic curve cryptography, PQC, or the like to make it even more unreadable (steps 1105 and 1106).

[0117] Fig. 20 shows the acquisition and digitization of biometric data, and the acquisition and digitization (γ) of a voiceprint using a microphone or the like. A voiceprint is acquired using a microphone or the like (steps 1201, 1202), and when a request is made to create an avatar that looks exactly like the person, the voiceprint is acquired in the presence of the person and the data is output (step 1203).

[0118] These data are audio data and the like, and the data formats are WAV, MP3, OGG, etc. These data may be encrypted directly, or may be temporarily converted (obfuscated) into binary data before being encrypted (steps 1204, 1205, 1206).

[0119] FIG. 21 shows a method for linking biometric information to an AI avatar that resembles the person and the registration method (1301).

[0120] In order to link biometric information to an AI avatar that looks just like the person, the person (A) must request the creation of an AI avatar from an AI avatar creation company (B), verify the identity of the person in person with personal identification documents (driver's license, passport, My Number card, etc.), and then obtain (1302, 1303, 1304) the biometric information (appearance, voiceprint, iris, palm, fingerprints, genome, etc.) on the spot, digitize, obfuscate, and encrypt (1305, 1306, 1307), add the encrypted information to the AI ​​avatar that looks just like the person (1308), and establish ownership of the entire thing in cyberspace using an NFT or the like. The identity verification agency (C) checks the details of the identity verification procedures of Party A and Party B, and if it determines that there are no problems, it attaches an identity verification badge to an AI avatar that Party B has created for Party A that looks exactly like the person prior to the NFT conversion (1309), and converts the data together with the encrypted biometric data in accordance with the following procedure. The NFT conversion is carried out by Party B or a company commissioned by Party B in the following procedure (1310).

[0121] Open an account at a virtual currency exchange (cryptocurrency exchange), create a wallet, create and register NFT art, and transfer funds to the wallet. For example, on the Ethereum blockchain, you can create an ERC721 (Ethereum Request For Resale (ERC) token), which is the token standard for issuing NFTs. Comments) can be used.

[0122] We will explain the image of a badge verifying the legitimacy of the identity verification procedure by an identity verification agency. The identity verification agency (C) checks the details of the identity verification procedures of A and B, and if it determines there are no problems, it will grant an identity verified badge to the look-alike AI avatar created by B for A prior to NFTization. The same confirmation number will always be displayed as a digital watermark or similar. If C is asked by A or B to authenticate the authenticity of a deepfake because they have encountered one, they will compare the data with the data when the look-alike AI avatar was registered and the NFTized data and respond. Based on this response, A or B will make an appropriate request to the SNS service provider to remove the fake (1311).

[0123] Although an embodiment of the present invention has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. The embodiment can be implemented in various forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. Such embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0124] 1 Management Server 10. Chassis 11 Acquisition Department 12 Generation part 13 Selection section 14 Output section 15 Restrictions 16 Recording Section 17 Judgment section 18 Expression generator 100 Data Management System 101 CPU 102 ROM 103 RAM 104 Preservation Department 105 Interface 106 Interfaces 107 Interface 108 Input section 109 Output section 110 Internal Bus 2 Data management device 2a equipment 2b equipment 3 Management Database 4. Communication Network 4a Public communication network 4b Decentralized Networks S110 Acquisition method S120 generation means S130 Output means S150 User data acquisition means S160 Selection means S170 Restrictive measures 200 Avatar Authentic Registration System 211 Camera 212 Scanner 213 Control Unit 214 Storage section 215 Avatar Generation Department 216 NFT conversion department 217 Personal Identification Information Registration Department 218 Biometric Information Registration Department 219 Communications Department 220 Sequencer 501 Identity Verification Process 502 Appearance and voiceprint of the person (in person) 503 Address / family register (verification of legal attributes) 504 Identification Documents (α) 505 Verification / Confirmation 506 Physically distributed storage 507 AI Avatar (β) Generation and NFT Process 508 Acquire the person's appearance data 509 Acquisition of the individual's voiceprint data 510 Create an AI avatar (β) that looks just like you 511 Obtaining blood, oral mucosa, and other samples from the individual (γ) 512 Data conversion by sequencer (γ) 513 Acquisition and digitization of iris, palm, fingerprints, etc. (γ) 514 Encryption 515 NFT the whole thing 1301 Authentic registration process for AI avatars 1302 Personal appearance data 1303 Your voiceprint data 1304 Personal movement data 1305 Encrypted genome sequence data 1306 Encrypted Iris Data 1307 Encrypted palm and fingerprint data 1308 Create an AI avatar that looks just like you 1309 Application for authenticity of AI avatar to identity verification agency (C) 1310 Authenticity verification of AI avatar by identity verification agency (C)

Claims

1. In the avatar authenticity registration method in the avatar authenticity registration system, an AI avatar generating step of acquiring at least the user's appearance data and voiceprint data to generate an AI avatar that closely resembles the user; a biometric information acquisition step of acquiring biometric information including acquiring a blood or oral mucosa sample of at least the user of the AI ​​avatar generated by the AI ​​avatar generation step, subjecting the acquired sample to a sequencer to analyze and digitize the whole genome information; An NFT conversion step of linking the biometric information acquired by the biometric information acquisition step, the AI ​​avatar information, and the previously stored identity verification result information to convert the entire data into an NFT; a step of granting an identity verification badge to the AI ​​avatar prior to the NFT conversion step; 13. An avatar authenticity registration method comprising the steps of:

2. The method for authenticating an avatar as described in claim 1, characterized in that the pre-verification step includes a face-to-face verification of the user's appearance and / or voiceprint, a verification of legal attribute information including at least the user's address and family register, and a verification of one or more of the user's driver's license, driving history certificate, passport, resident registration card (only those with a photograph), personal identification number card (only those with a photograph), residence card or special permanent resident certificate, various health insurance cards (including the side with the address), public pension book (certificate), various disability certificates (certificates), family register (individual) details certificate, and resident registration card.

3. 3. The method for authenticating an avatar according to claim 1, further comprising an encryption step of encrypting the digitized whole genome information.

4. 2. The method for registering authenticity of an avatar according to claim 1, further comprising physically distributing and storing the originals of the personal identification documents or copies of the originals of the personal identification documents used in said preliminary verification step.

5. In the avatar authentic registration system, an AI avatar generating means for acquiring at least the user's appearance data and voiceprint data and generating an AI avatar that closely resembles the user; a biometric information acquisition means for acquiring biometric information of the user, including acquiring a sample of at least the user's blood or oral mucosa, of the AI ​​avatar generated by the AI ​​avatar generation means, and subjecting the acquired sample to a sequencer to analyze and digitize the whole genome information; An NFT conversion means for linking the biometric information acquired by the biometric information acquisition means, the AI ​​avatar information, and the previously stored identity verification result information to convert the entirety into an NFT; An assignment means for assigning an identity verification badge to the AI ​​avatar prior to the NFT conversion by the NFT conversion means; An avatar authenticity registration system comprising:

Images