Information processing device, brain information communication device, information processing method, and information processing program

Abstract

To express information to be communicated between recognition systems in common form not dependent on recognition systems.SOLUTION: An information processing device 1 comprises, where n, m, N are defined as natural numbers, respectively: a first feature representation vector generation unit 10 for generating n first feature representation vectors Sa from n concepts recognized by a first recognition system; a second feature representation vector generation unit 20 for generating m second feature representation vectors Sb from m concepts recognized by a second recognition system; and a common variable representation vector generation unit 30 for projecting the first feature representation vectors Sa and second feature representation vectors Sb to an N-dimension hypersphere by conversion F and conversion G, respectively, so as to generate a common variable representation vector. When F-1 and G-1 are defined as the inverse conversions of the conversion F and conversion G, respectively, the expressions Sa=F-1×F×Sa, Sb=G-1×G×Sb are satisfied, and the projection onto the hypersphere is trained so that the F×Sa and G×Sb distributions overlap.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a brain information communication device, an information processing method, and an information processing program. [Background technology]

[0002] A technology has been proposed in which a common feature vector common to image data and text data is generated, this common feature vector is compared with a brain activity feature vector, and the brain activity feature vector is selectively supplied to a channel that generates a vector with the highest correlation (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2018-205885 Summary of the Invention [Problem to be solved by the invention]

[0004] Brain information communication technology, which communicates recognized information between systems with recognition functions (hereinafter referred to as "recognition systems"), such as human or animal brains or AI (artificial intelligence), has attracted attention. However, recognition through neural activity in humans and animals varies from person to person. Therefore, the representation format of concepts recognized and encoded in the brain varies from person to person. Furthermore, even between AIs that have learned from different initial values, each represents information in a different format. Therefore, in order to communicate information between multiple recognition systems, it is necessary to represent the information to be communicated (i.e., various concepts) in a common format that is independent of the recognition systems.

[0005] The present invention has been made in view of the above circumstances, and its object is to express information to be communicated between recognition systems in a common format that is independent of the recognition system. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to one aspect (first embodiment) of the present invention comprises: When n, m, and N are natural numbers, n first feature representation vectors from the n concepts recognized by the first recognition system Sa=[Sa_1,Sa_2,…,Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m concepts recognized by the second recognition system Sb=[Sb_1,Sb_2,…,Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa and the second feature representation vector Sb are projected onto an N-dimensional hypersphere by transformation F and transformation G, respectively, according to the rules of contrastive learning or instance learning, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1',Sa_2',…,Sa_n'] and Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector, The inverse transformations of the transformations F and G are F -1 and G -1 When Sa=F -1 ·F·Sa Sb=G -1 G Sb Fulfilling We learn to project onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0007] Another aspect (second embodiment) of the present invention is also an information processing device. This device further comprises an identification unit in addition to the configuration of the information processing device of the first embodiment. This identification unit identifies whether the common variable representation vectors Sa' and Sb' are derived from the first feature representation vector Sa or the second feature representation vector Sb. The common variable representation vector generation unit learns transformations F and G so that the distributions of the two vectors match, using a generative adversarial network (GAN) or the like.

[0008] Another aspect (third embodiment) of the present invention is also an information processing device. Let n, m, and N be natural numbers, and when n>m, n first feature representation vectors from the n concepts recognized by the first recognition system Sa=[Sa_1,Sa_2,…,Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m concepts recognized by the second recognition system Sb=[Sb_1,Sb_2,…,Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa is projected onto an N-dimensional hypersphere by a transformation F according to the rules of contrastive learning or instance learning, resulting in a common variable representation vector Sa'=F·Sa=[Sa_1',Sa_2',…,Sa_n'] a common variable representation vector generation unit that generates a common variable representation vector, By projecting the second feature representation vector Sb onto the N-dimensional hypersphere by the transformation G, the common variable representation vector Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector, The inverse transformations of the transformations F and G are F -1 and G -1 When Sa=F -1 ·F·Sa Sb=G -1 G Sb Learning to fulfill the A projection onto the hypersphere is learned so that the distribution of Sb' matches the distribution of Sa'.

[0009] Yet another aspect (fourth embodiment) of the present invention is also an information processing device. When n, m, Na, Nb, and N are natural numbers, n first feature representation vectors from the n concepts recognized by the first recognition system Sa=[Sa_1,Sa_2,…,Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m concepts recognized by the second recognition system Sb=[Sb_1,Sb_2,…,Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa is projected onto the Na-dimensional hypersphere by the transformation F to obtain the first variable representation vector Sa^=F·Sa=[Sa_1^,Sa_2^,…,Sa_n^] a first variable representation vector generation unit that generates a first variable representation vector; The second feature representation vector Sb is projected onto the Nb-dimensional hypersphere by the transformation G, and the second variable representation vector Sb^=G·Sb=[Sb_1^,Sb_2^,…,Sb_m^] a second variable representation vector generation unit that generates a The first variable representation vector Sa^ and the second variable representation vector Sb^ are projected onto the N-dimensional hypersphere by the transformation F2 and the transformation G2, respectively, to obtain the common variable representation vector Sa' = F2 · Sa^ = [Sa_1',Sa_2',…,Sa_n'] and Sb'=G2·Sb^=[Sb_1',Sb_2',…,Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector, The inverse transformations of the transformations F and G are F -1 and G -1 Let F2 be the inverse transformation of the transformation F2 and the transformation G2, respectively. -1 and G2 -1 When Sa=F -1 ·F·Sa Sb=G -1 G Sb Sa^=F2 -1 ·F2·Sa^ Sb^=G2 -1 ·G2·Sb^ Fulfilling We learn to project onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0010] Yet another aspect (fifth embodiment) of the present invention is also an information processing device for communicating information between recognition systems, comprising: When n, m, k, and N are natural numbers, n first feature representation vectors from the n concepts recognized by the first recognition system Sa=[Sa_1,Sa_2,…,Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m concepts recognized by the second recognition system Sb=[Sb_1,Sb_2,…,Sb_m] a second feature representation vector generation unit that generates a feature representation vector k third feature representation vectors from the k concepts recognized by the third recognition system Sc = [Sc_1,Sb_2,…,Sb_k] a third feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa, the second feature representation vector Sb, and the third feature representation vector Sc are projected onto the N-dimensional hypersphere by transformations F, G, and H, respectively, to obtain a common variable representation vector Sa'=F·Sa=[Sa_1',Sa_2',…,Sa_n'], Sb' = G·Sb = [Sb_1',Sb_2',…,Sb_m'] and Sc'=H·Sc=[Sc_1',Sc_2',…,Sc_k'] a common variable representation vector generation unit that generates a common variable representation vector, The inverse transformations of transformations F, G, and H are F, respectively. -1 , G -1 and H -1 When Sa=F -1 ·F·Sa Sb=G -1 G Sb Sc=H -1 ·H·Sc Fulfilling We learn to project onto the hypersphere so that the distributions of Sa', Sb', and Sc' overlap.

[0011] Yet another aspect (sixth embodiment) of the present invention is also an information processing device. In this device, in the configuration of the information processing device of the fifth embodiment, the common variable representation vector generation unit projects the third feature representation vector Sc onto an N-dimensional hypersphere by a transformation H, thereby generating a common variable representation vector Sc'=H·Sc=[Sc_1',Sc_2',…,Sc_k'] The common variable representation vector generator learns the projection onto the hypersphere so that the distribution of Sc' overlaps with the distribution of Sa'.

[0012] Yet another aspect (seventh embodiment) of the present invention is a brain information communication device for communicating information between a brain and an AI, comprising: When n, m, and N are natural numbers, n first feature representation vectors from n pieces of information recognized by the brain Sa=[Sa_1,Sa_2,…,Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from m pieces of information recognized by AI Sb=[Sb_1,Sb_2,…,Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa and the second feature representation vector Sb are projected onto the N-dimensional hypersphere by the transformation F and the transformation G, respectively, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1',Sa_2',…,Sa_n'] and Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector; a communication unit that transmits the common variable representation vectors Sa′ and Sb′ from the common variable representation vector generation unit to AI; The inverse transformations of the transformations F and G are F -1 and G -1 When Sa=F -1 ·F·Sa Sb=G -1 G Sb Fulfilling We learn to project onto the hypersphere so that the distributions of Sa' and Sb' overlap, The AI ​​reconstructs the n pieces of information recognized by the brain by decoding them based on the common variable representation vectors Sa' and Sb'.

[0013] Yet another aspect (eighth embodiment) of the present invention is also a brain information communication device. This device is a brain information communication device for communicating information between brains, and includes: When n, m, and N are natural numbers, n first feature representation vectors from n pieces of information recognized by the first brain Sa=[Sa_1,Sa_2,…,Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from m pieces of information recognized by the second brain Sb=[Sb_1,Sb_2,…,Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa and the second feature representation vector Sb are projected onto the N-dimensional hypersphere by the transformation F and the transformation G, respectively, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1',Sa_2',…,Sa_n'] and Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector; a communication unit, the communication unit transmits information of the first brain to the second brain and transmits information of the second brain to the first brain based on the common variable representation vectors Sa′ and Sb′; The inverse transformations of the transformations F and G are F -1 and G -1 When Sa=F -1 ·F·Sa Sb=G -1 G Sb Fulfilling We learn to project onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0014] Yet another aspect (ninth embodiment) of the present invention is also a brain information communication device. This device further comprises a decoding unit and a feature representation vector estimation unit in addition to the components of the brain information communication device of the eighth embodiment. The communication unit transmits the common variable representation vectors Sa' and Sb' generated by the common variable representation vector generation unit to the feature representation vector estimation unit. The feature representation vector estimation unit estimates first and second feature representation vectors for decoding. The decoding unit decodes the n pieces of information recognized by the first brain based on the first and second feature representation vectors estimated by the feature representation vector estimation unit and the feature representation vector generated by the second feature representation vector generation unit.

[0015] Yet another aspect (tenth embodiment) of the present invention is also a brain information communication device. This device, compared to the brain information communication device of the eighth embodiment, further comprises: a first brain stimulation unit that generates information in the first brain by stimulating the first brain with electricity, magnetism, ultrasound, chemicals, light, or heat; a second brain stimulation unit that generates information in the second brain by stimulating the second brain with electricity, magnetism, ultrasound, chemicals, light, or heat; a first feature representation vector estimation unit; and a second feature representation vector estimation unit. The communication unit transmits the common variable representation vector Sb' generated by the common variable representation vector generation unit to the first feature representation vector estimation unit, and transmits the common variable representation vector Sa' generated by the common variable representation vector generation unit to the second feature representation vector estimation unit. The first feature representation vector estimation unit estimates a feature representation vector related to the second brain information based on the common variable representation vector Sb'. The second feature representation vector estimation unit estimates a feature representation vector related to the first brain information based on the common variable representation vector Sa'. The first brain stimulation unit stimulates the first brain to generate second brain information in the first brain based on the feature representation vector estimated by the first feature representation vector estimation unit. The second brain stimulation unit stimulates the second brain to generate first brain information in the second brain based on the feature representation vector estimated by the second feature representation vector estimation unit.

[0016] Yet another aspect (eleventh embodiment) of the present invention is also a brain-based information communication device. This device further comprises an AI and a display unit in addition to the components of the brain-based information communication device of the eighth embodiment. The AI ​​decodes the information recognized by the first brain and the information recognized by the second brain based on the common variable representation vector, and reconstructs the information to be transmitted to the display unit. The display unit displays the information reconstructed by the AI.

[0017] Yet another aspect (twelfth embodiment) of the present invention is also a brain information communication device. This device further comprises a first feature representation vector estimation unit, a second feature representation vector estimation unit, a first decoding unit, a second decoding unit, a first display unit, and a second display unit, in addition to the components of the brain information communication device of the ninth embodiment. The communication unit transmits the common variable representation vector Sb' generated by the common variable representation vector generation unit to the first feature representation vector estimation unit. The communication unit transmits the common variable representation vector Sa' generated by the common variable representation vector generation unit to the second feature representation vector estimation unit. The first feature representation vector estimation unit estimates a feature representation vector related to second brain information based on the common variable representation vector Sb' and transmits it to the first decoding unit. The second feature representation vector estimation unit estimates a feature representation vector related to first brain information based on the common variable representation vector Sa' and transmits it to the second decoding unit. The first decoding unit decodes the second brain information based on the feature representation vector estimated by the first feature representation vector estimation unit and reconstructs it as information to be sent to the second display unit. The second decoding unit decodes the first brain information based on the feature representation vector estimated by the second feature representation vector estimation unit and reconstructs it as information to be sent to the first display unit. The first display unit displays the second brain information reconstructed by the first decoding unit. The second display unit displays the first brain information reconstructed by the second decoding unit.

[0018] Yet another aspect (thirteenth embodiment) of the present invention is an information processing method for communicating information between recognition systems, which includes the steps of: When n, m, and N are natural numbers, n first feature representation vectors from the n pieces of information recognized by the first recognition system Sa=[Sa_1,Sa_2,…,Sa_n] generating a m second feature representation vectors from the m pieces of information recognized by the second recognition system Sb=[Sb_1,Sb_2,…,Sb_m] generating a The first feature representation vector Sa and the second feature representation vector Sb are projected onto the N-dimensional hypersphere by the transformation F and the transformation G, respectively, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1',Sa_2',…,Sa_n'] and Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] and generating The inverse transformations of the transformations F and G are F -1 and G -1 When Sa=F -1 ·F·Sa Sb=G -1 G Sb Fulfilling We learn to project onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0019] Yet another aspect (fourteenth embodiment) of the present invention is an information processing program. When n, m, and N are natural numbers, n first feature representation vectors from the n pieces of information recognized by the first recognition system Sa=[Sa_1,Sa_2,…,Sa_n] generating a m second feature representation vectors from the m pieces of information recognized by the second recognition system Sb=[Sb_1,Sb_2,…,Sb_m] generating a The first feature representation vector Sa and the second feature representation vector Sb are projected onto the N-dimensional hypersphere by the transformation F and the transformation G, respectively, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1',Sa_2',…,Sa_n'] and Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] generating a signal; The inverse transformations of the transformations F and G are F -1and G -1 When Sa=F -1 ·F·Sa Sb=G -1 G Sb Fulfilling We learn to project onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0020] Yet another aspect (fifteenth embodiment) of the present invention is an information processing device. This information processing device includes a mapping unit and an anchor point extraction unit in addition to the configuration of the information processing device of the first embodiment. The mapping unit projects the first feature representation vector Sa and the second feature representation vector Sb onto an N-dimensional hypersphere, respectively, and extracts the first latent variable vector Sa. ~ and the second latent variable vector Sb ~ The anchor point extraction unit generates the first latent variable vector Sa ~ and the second latent variable vector Sb ~ The first latent variable vector Sa ~ and the second latent variable vector Sb ~ First, a first cluster region group and a second cluster region group, each densely distributed, are extracted, and feature representation vectors corresponding to latent variable vectors contained in clusters that are close to each other are extracted as anchor points. A projection is learned in which the feature representation vectors extracted as anchor points match each other as common variable representation vectors.

[0021] Yet another aspect (sixteenth embodiment) of the present invention is a brain information communication device. This brain information communication device includes a mapping unit and an anchor point extraction unit in addition to the configuration of the brain information communication device of the seventh embodiment. The device learns a projection such that the feature representation vectors extracted as anchor points match each other as common variable representation vectors.

[0022] Yet another aspect (17th embodiment) of the present invention is an information processing method for communicating information between recognition systems. This method differs from the information processing method of the 13th embodiment in that it projects the first feature representation vector Sa and the second feature representation vector Sb onto an N-dimensional hypersphere, and calculates the first latent variable vector Sa. ~ and the second latent variable vector Sb ~ a mapping step to generate a first latent variable vector Sa ~ and the second latent variable vector Sb ~ The first latent variable vector Sa ~ and the second latent variable vector Sb ~ The method further comprises the steps of extracting a first cluster region group and a second cluster region group in which the latent variable vectors are densely distributed, and extracting, as anchor points, feature representation vectors corresponding to latent variable vectors contained in clusters that are close to each other from the first cluster region group and the second cluster region group. A projection is learned in which the feature representation vectors extracted as anchor points match each other as common variable representation vectors.

[0023] Yet another aspect (18th embodiment) of the present invention is an information processing program. This program differs from the program of the 14th embodiment in that it projects the first feature representation vector Sa and the second feature representation vector Sb onto an N-dimensional hypersphere, and calculates the first latent variable vector Sa. ~ and the second latent variable vector Sb ~ a mapping step to generate a first latent variable vector Sa ~ and the second latent variable vector Sb ~ The first latent variable vector Sa ~ and the second latent variable vector Sb ~The computer further executes the steps of extracting a first cluster region group and a second cluster region group in which the latent variable vectors are densely distributed, and extracting, as anchor points, feature representation vectors corresponding to latent variable vectors contained in clusters that are close to each other from the first cluster region group and the second cluster region group. The feature representation vectors extracted as anchor points are trained to project in such a way that they coincide with each other as common variable representation vectors.

[0024] Any combination of the above components, and any transformation of the present invention into an apparatus, method, system, recording medium, computer program, etc., are also valid aspects of the present invention. [Effects of the Invention]

[0025] According to the present invention, information to be communicated between recognition systems can be expressed in a common format that is independent of the recognition system. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a functional block diagram of an information processing device according to a first embodiment. [Figure 2] FIG. 2 is a schematic diagram illustrating the operation of the information processing device of FIG. [Figure 3] FIG. 10 is a functional block diagram of an information processing device according to a second embodiment. [Figure 4] FIG. 10 is a functional block diagram of an information processing device according to a fourth embodiment. [Figure 5] FIG. 5 is a schematic diagram showing the operation of the information processing device of FIG. [Figure 6] FIG. 10 is a functional block diagram of an information processing device according to a fifth embodiment. [Figure 7] FIG. 7 is a schematic diagram showing the operation of the information processing device of FIG. 6. [Figure 8] FIG. 13 is a functional block diagram of a brain information communication device according to a seventh embodiment. [Figure 9] FIG. 13 is a functional block diagram of a brain information communication device according to an eighth embodiment. [Figure 10] FIG. 13 is a functional block diagram of a brain information communication device according to a ninth embodiment. [Figure 11] FIG. 4 is a functional block diagram of a brain information communication device according to a tenth embodiment. [Figure 12] FIG. 23 is a functional block diagram of a brain information communication device according to an eleventh embodiment. [Figure 13] FIG. 23 is a functional block diagram of a brain information communication device according to a twelfth embodiment. [Figure 14] 13 is a flowchart showing the processing steps of an information processing method and an information processing program according to the thirteenth and fourteenth embodiments. [Figure 15] FIG. 23 is a functional block diagram of an information processing device according to a fifteenth embodiment. [Figure 16] FIG. 22 is a functional block diagram of a brain information communication device according to a sixteenth embodiment. [Figure 17] 16 is a flowchart showing the processing steps of an information processing method and an information processing program according to the seventeenth and eighteenth embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0027] The present invention will be described below based on preferred embodiments with reference to the drawings. In the embodiments and modifications, identical or equivalent components and members are designated by the same reference numerals, and redundant descriptions will be omitted where appropriate. The dimensions of the components in the drawings are enlarged or reduced as appropriate for ease of understanding. Some components that are not important for explaining the embodiments are omitted from the drawings. Terms including ordinal numbers such as "first" and "second" are used to describe various components, but these terms are used only to distinguish one component from another and do not limit the components.

[0028] [First embodiment] 1 is a functional block diagram of an information processing device 1 according to a first embodiment. The information processing device 1 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, and a common variable representation vector generation unit 30.

[0029] The information processing device 1 is connected to an external first recognition system and a second recognition system. The first recognition system and the second recognition system are systems with recognition functions, such as human or animal brains, or AI (e.g., CNN, Transformer, etc. implemented as neural networks). These recognition systems can recognize information such as images, text, and voice, and store them as concepts. Note that the concepts stored in the first recognition system and the second recognition system are expressed in different ways. For example, in the case of the human brain, the representation format of recognized concepts varies from person to person. Also, in the case of AI, when systems learn correspondences between each other, the types and number of concepts that can be learned do not necessarily match. For this reason, these concepts cannot be directly communicated between the first recognition system and the second recognition system.

[0030] The first feature representation vector generation unit 10 receives information recognized by the first recognition system from the first recognition system. The first feature representation vector generation unit 10 generates, from the received information, first feature representation vectors corresponding to concepts included in the information. For example, when the first feature representation vector generation unit 10 receives n pieces of information from the first recognition system, it generates n first feature representation vectors from the n pieces of information. Sa=[Sa_1,Sa_2,…,Sa_n] where n is a natural number.

[0031] Similarly, the second feature representation vector generation unit 20 receives information recognized by the second recognition system from the second recognition system. The second feature representation vector generation unit 20 generates, from the received information, second feature representation vectors corresponding to concepts included in the information. For example, when the second feature representation vector generation unit 20 receives m pieces of information from the second recognition system, it generates m first feature representation vectors from the m pieces of information. Sb=[Sb_1,Sb_2,…,Sb_m] where m is a natural number.

[0032] To generate the first feature representation vector and the second feature representation vector, a technique such as instance learning may be used.

[0033] The common variable representation vector generation unit 30 projects the first feature representation vector Sa onto the N-dimensional hypersphere by the transformation F, thereby generating a common variable representation vector Sa'=F·Sa=[Sa_1',Sa_2',…,Sa_n'] Generate. Furthermore, the common variable representation vector generation unit 30 projects the second feature representation vector Sb onto the N-dimensional hypersphere by the transformation G, thereby generating a common variable representation vector Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] where N is a natural number.

[0034] Below, the inverse transformations of transformations F and G are respectively -1 and G -1 The feature representation vectors Sa and Sb satisfy the following rules 1, 1-2, and 1-3. (Rule 1) Sa=F -1 ·F·Sa Sb=G -1 G Sb (Rule 1-2) Rules to follow for contrastive or instance learning (Rules 1-3) The rule to match the distribution of Sa' and Sb', i.e., overlap

[0035] Rule 1 means that when the first and second feature representation vectors are transformed into a common variable representation vector, the original feature representation vector is restored by inverse transformation. Learning that satisfies Rule 1 is also called AutoEncoder learning.

[0036] Rule 1-2 is a learning rule that maps similar concepts closer together and dissimilar concepts further apart. One type of this rule is known as contrastive learning. Furthermore, when it is not clear what concepts each data sample represents, instance learning is known as an extension rule, which learns to correctly distinguish each sample from other samples based on the feature vectors of each data sample. By following rule 1-2, it is possible to learn the appropriate projection onto the hypersphere.

[0037] Furthermore, by following rules 1-3, the common variable representation vector generation unit 30 learns the projection onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0038] Furthermore, the feature representation vectors Sa and Sb satisfy the following rule 2. (Rule 2) Sa=F -1 G G -1 ·F·Sa Sb=G -1 F F -1 G Sb

[0039] Rule 2 means that when a first feature representation vector is transformed into a second common variable representation vector via a common variable representation vector, the original first feature representation vector is restored by inverse transformation (similarly, when a second feature representation vector is transformed into the first common variable representation vector via a common variable representation vector, the original second feature representation vector is restored by inverse transformation). Learning that satisfies Rule 2 is also called learning with CycleConsistencyLoss added.

[0040] As mentioned above, the representation formats of the concepts held by the first and second recognition systems are different. However, there is a commonality in the conceptual structure of these systems. For example, the concepts of "cat," "lion," and "car" are represented in different formats in the first and second recognition systems. However, in both recognition systems, similar concepts such as "cat" and "lion" are represented in close positions as vector representations. On the other hand, dissimilar concepts such as "cat" and "car" are represented in distant positions as vector representations. In this way, the relationship structure between concepts is thought to be common between different recognition systems.

[0041] Because there is a commonality in the relational concept structure between different recognition systems, the common variable representation vector generation unit 30 projects the first feature representation vector and the second feature representation vector onto an N-dimensional hypersphere while satisfying Rule 1, Rule 1-2, Rule 1-3, and Rule 2, thereby arranging these feature representation vectors on the N-dimensional hypersphere. As a result, the generated common variable representation vector is expressed in a common format that is independent of the recognition system.

[0042] FIG. 2 shows a schematic diagram of the operation of the information processing device 1.

[0043] The common variable representation vector generation unit 30 may learn the transformations F and G using a neural network.

[0044] According to this embodiment, in order to realize brain information communication, information to be communicated between recognition systems can be expressed in a common format that is independent of the recognition system.

[0045] [Second embodiment] 3 is a functional block diagram of an information processing device 2 according to the second embodiment. The information processing device 2 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, and a classification unit 40. That is, the information processing device 2 further includes the classification unit 40 in addition to the information processing device 1 in FIG. The other configuration of the information processing device 2 is the same as the configuration of the information processing device 1.

[0046] The identification unit 40 identifies whether the common variable representation vectors Sa′ and Sb′ are derived from the first feature representation vector Sa or the second feature representation vector Sb. The common variable representation vector generation unit 30 learns the transformations F and G using a generative adversarial network (GAN).

[0047] For example, the calculation performed by the discrimination unit 40 is denoted as D, and the discrimination evaluation value is denoted as D(·). In this case, the common variable representation vector generation unit 30 may learn D so that the expected value of Log(D([F·Sa, G·Sb]) is maximized, and may learn the transformations F and G so that the expected value of Log(1-D([F·Sa, G·Sb]) is maximized. Alternatively, the transformations F and G may be learned using the MMD (Maximum Mean Discrepancy) method or the like so that the distributions after the transformations become closer to each other.

[0048] By learning the transformations F and G using a generative adversarial network in this way, the first feature representation vector Sa and the second feature representation vector Sb are not projected onto separate subspaces, but are projected so that they overlap more closely onto the N-dimensional hypersphere. This allows the common variable representation vector to be generated without distinguishing whether it is derived from the first feature representation vector Sa or the second feature representation vector Sb.

[0049] According to this embodiment, the common variable representation vector is generated without distinguishing whether it is derived from the first feature representation vector Sa or the second feature representation vector Sb, so that the common variable representation vector can be generated with greater accuracy.

[0050] [Third embodiment] For example, if the number of first feature representation vectors is sufficiently greater than the number of second feature representation vectors, the transformation F can be calculated by applying only rule 1 and rule 1-2, the first feature representation vectors can be projected onto an N-dimensional hypersphere, and then the second feature representation vectors can be learned using rule 1, rule 1-2, rule 1-3, and rule 2 in accordance with the projection of the first feature representation vectors.

[0051] In the third embodiment, in the information processing device 1 of Figure 1, when the number of concepts recognized by the first recognition system is n and the number of concepts recognized by the second recognition system is m, it is assumed that n>m. The first feature representation vector generation unit 10 generates n first feature representation vectors from n concepts recognized by the first recognition system. Sa=[Sa_1,Sa_2,…,Sa_n] Generate. Similarly, the second feature representation vector generation unit 20 generates m second feature representation vectors from the m concepts recognized by the second recognition system. Sb=[Sb_1,Sb_2,…,Sb_m] Generate.

[0052] The common variable representation vector generation unit 30 projects the first feature representation vector Sa and the second feature representation vector Sb onto the N-dimensional hypersphere by the transformation F and the transformation G, respectively, to generate a common variable representation vector Sa' = F·Sa = [Sa_1',Sa_2',…,Sa_n'] and Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] The common variable representation vector generation unit 30 learns the projection onto the hypersphere so that the distribution of Sb' coincides with the distribution of Sa'.

[0053] The inverse transformations of the transformations F and G are F -1 and G -1 When Sa=F -1 ·F·Sa=F -1 G G -1 ·F·Sa Sb=G -1 G Sb=G -1 F F -1 G Sb Meet the following.

[0054] According to this embodiment, when there is a sufficiently large difference between the number of first feature representation vectors and the number of second feature representation vectors, it is possible to generate common variable representation vectors more efficiently.

[0055] [Fourth embodiment] When the representation formats of the first and second recognition systems are significantly different, it may be efficient to first project the first and second feature representation vectors onto different hyperspheres with different dimensions, and then generate a common variable representation vector.

[0056] 4 is a functional block diagram of an information processing device 3 according to a fourth embodiment. The information processing device 3 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a first variable representation vector generation unit 50, a second variable representation vector generation unit 60, and a common variable representation vector generation unit 30. That is, the information processing device 3 further includes the first variable representation vector generation unit 50 and the second variable representation vector generation unit 60 in addition to the information processing device 1 in FIG. 1. The other configuration of the information processing device 3 is the same as the configuration of the information processing device 1.

[0057] The first feature representation vector generation unit 10 generates n first feature representation vectors from n concepts recognized by the first recognition system. Sa=[Sa_1,Sa_2,…,Sa_n] Generate. The second feature representation vector generation unit 20 generates m second feature representation vectors from the m concepts recognized by the second recognition system. Sb=[Sb_1,Sb_2,…,Sb_m] Generate.

[0058] The first variable representation vector generation unit 50 projects the first feature representation vector Sa onto the Na-dimensional hypersphere by the transformation F, thereby generating a first variable representation vector Sa^=F·Sa=[Sa_1^,Sa_2^,…,Sa_n^] The second variable representation vector generation unit 60 generates the second variable representation vector Sb by projecting the second feature representation vector Sb onto the Nb-dimensional hypersphere by the transformation G. Sb^=G·Sb=[Sb_1^,Sb_2^,…,Sb_m^] Generate.

[0059] The common variable representation vector generation unit 30 generates the common variable representation vectors Sa^ and Sb^ by projecting the first variable representation vector Sa^ and the second variable representation vector Sb^ onto the N-dimensional hypersphere by the transformation F2 and the transformation G2, respectively. Sa' = F2 · Sa^ = [Sa_1',Sa_2',…,Sa_n'] and Sb'=G2·Sb^=[Sb_1',Sb_2',…,Sb_m'] The common variable representation vector generation unit 30 learns the projection onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0060] The inverse transformations of the transformations F and G are F -1 and G -1 Let F2 be the inverse transformation of the transformation F2 and the transformation G2, respectively. -1 and G2 -1 When Sa=F -1 ·F·Sa=F -1 F2 -1 G2 G G-1 G2 -1 ·F2·F·Sa Sb=G -1 G Sb=G -1 G2 -1 ·F2·F·F -1 F2 -1 F2 G Sb Sa'=F2 -1 ·F2·Sa' Sb'=G2 -1 ·G2·Sb' Meet the following.

[0061] FIG. 5 shows a schematic diagram of the operation of the information processing device 3.

[0062] According to this embodiment, when the representation formats of the first recognition system and the second recognition system are significantly different, a common variable representation vector can be generated efficiently.

[0063] [Fifth embodiment] The following description relates to an embodiment that realizes brain information communication between three or more recognition systems. FIG. 6 is a functional block diagram of an information processing device 4 according to a fifth embodiment. The information processing device 4 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a third feature representation vector generation unit 70, and a common variable representation vector generation unit 30. That is, the information processing device 4 further includes the third feature representation vector generation unit 70 in addition to the information processing device 1 of FIG. 1. The other configuration of the information processing device 4 is the same as the configuration of the information processing device 1.

[0064] The first feature representation vector generation unit 10 generates n first feature representation vectors from n concepts recognized by the first recognition system. Sa=[Sa_1,Sa_2,…,Sa_n] Generate. The second feature representation vector generation unit 20 generates m second feature representation vectors from the m concepts recognized by the second recognition system. Sb=[Sb_1,Sb_2,…,Sb_m] Generate. The third feature representation vector generation unit 70 generates k three feature representation vectors from the k concepts recognized by the third recognition system. Sc = [Sc_1,Sc_2,…,Sc_k] Generate.

[0065] The common variable representation vector generation unit 30 generates a common variable representation vector by projecting the first feature representation vector Sa, the second feature representation vector Sb, and the third feature representation vector Sc onto an N-dimensional hypersphere using transformations F, G, and H, respectively. Sa'=F·Sa=[Sa_1',Sa_2',…,Sa_n'], Sb' = G·Sb = [Sb_1',Sb_2',…,Sb_m'] and Sc'=H·Sc=[Sc_1', Sc_2', ..., Sc_k'] is generated. The common variable representation vector generation unit 30 learns the projection onto the hypersphere so that the distributions of Sa', Sb', and Sc' overlap.

[0066] The inverse transformations of transformations F, G, and H are F, respectively. -1 , G -1 and H -1 When Sa=F -1 ·F·Sa=F -1 G G -1 ·F·Sa=F -1 H H -1 ·F·Sa Sb=G -1 G Sb=G -1 F F -1 G Sb=G -1 H H -1 G Sb Sc=H -1 H Sc=H -1 F F -1 H Sc=H -1 G G -1 ·H·Sc Meet the following.

[0067] FIG. 7 shows a schematic diagram of the operation of the information processing device 4.

[0068] In the above example, we have explained how to generate a common variable representation vector for concepts recognized by three recognition systems. It is clear that a common variable representation vector can be generated in the case of four or more recognition systems by adding a similar configuration.

[0069] According to this embodiment, in order to realize brain information communication among three or more recognition systems, information to be communicated among the recognition systems can be expressed in a common format that is independent of the recognition systems.

[0070] [Sixth embodiment] In the configuration of the information processing device 4 in Figure 6, if a highly reliable common variable representation vector has already been generated between the first feature representation vector and the second feature representation vector, it is considered that a common variable representation vector may be generated for the third feature representation vector only between the first feature representation vector and the third feature representation vector.

[0071] In the sixth embodiment, in the configuration of the information processing device 4 in FIG. 6, the common variable representation vector generation unit 30 projects the third feature representation vector Sc onto the N-dimensional hypersphere by the transformation H, thereby generating the common variable representation vector Sc'=H·Sc=[Sc_1',Sc_2',…,Sc_k'] The common variable representation vector generation unit 30 learns the projection onto the hypersphere so that the distribution of Sc' overlaps the distribution of Sa'.

[0072] The inverse transformation of transformation H is H -1 When Sc=H -1 H Sc=H -1 F F -1 ·H·Sc Meet the following.

[0073] According to this embodiment, in order to realize brain information communication among three or more recognition systems, information to be communicated among the recognition systems can be expressed more efficiently in a common format that is independent of the recognition systems.

[0074] [Seventh embodiment] 8 is a functional block diagram of a brain information communication device 100 according to the seventh embodiment. The brain information communication device 100 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, and a communication unit 200.

[0075] The brain information communication device 100 is a communication device for communicating information between a brain and an AI. The brain information communication device 100 is connected to an external brain and an AI.

[0076] The first feature representation vector generation unit 10 receives information recognized by the brain from the brain. The first feature representation vector generation unit 10 generates, from the received information, first feature representation vectors corresponding to concepts included in the information. For example, when the first feature representation vector generation unit 10 receives n pieces of information from the brain, it generates n first feature representation vectors from the n pieces of information. Sa=[Sa_1,Sa_2,…,Sa_n] where n is a natural number.

[0077] Similarly, the second feature representation vector generation unit 20 receives information recognized by the AI ​​from the AI. The second feature representation vector generation unit 20 generates, from the received information, second feature representation vectors corresponding to the concepts contained in the information. For example, when the second feature representation vector generation unit 20 receives m pieces of information from the AI, it generates m first feature representation vectors from these m pieces of information. Sb=[Sb_1,Sb_2,…,Sb_m] where m is a natural number.

[0078] The common variable representation vector generation unit 30 projects the first feature representation vector Sa onto the N-dimensional hypersphere by the transformation F, thereby generating a common variable representation vector Sa'=F·Sa=[Sa_1',Sa_2',…,Sa_n'] Generate. Furthermore, the common variable representation vector generation unit 30 projects the second feature representation vector Sb onto the N-dimensional hypersphere by the transformation G, thereby generating a common variable representation vector Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] where N is a natural number.

[0079] Below, the inverse transformations of transformations F and G are respectively -1 and G -1 The feature representation vectors Sa and Sb satisfy the following rules 1, 1-2, 1-3, and 2. (Rule 1) Sa=F -1 ·F·Sa Sb=G -1 G Sb (Rule 1-2) Rules to follow for contrastive or instance learning (Rules 1-3) The rule to match the distribution of Sa' and Sb', i.e., overlap (Rule 2) Sa=F -1 G G -1 ·F·Sa Sb=G -1 F F -1 G Sb This allows the brain and the AI ​​to share a common representation format for the concepts they each possess. The common variable representation vector generation unit 30 learns how to project onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0080] The communication unit 200 transmits the common variable expression vectors Sa' and Sb' from the common variable expression vector generation unit 30 to the AI.

[0081] The AI ​​reconstructs the n pieces of information recognized by the brain by decoding them based on the common variable representation vectors Sa' and Sb'. In this case, the AI ​​may use a neural network such as GAN, Variational Autoencoder, or Transformer to decode the information.

[0082] According to this embodiment, by sharing a common representation format for concepts between the brain and AI, it is possible to realize information communication between the brain and AI.

[0083] [Eighth embodiment] 9 is a functional block diagram of a brain information communication device 110 according to the eighth embodiment. The brain information communication device 110 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, and a communication unit 202.

[0084] The brain information communication device 110 is a communication device for communicating information between brains. The brain information communication device 110 is connected to an external first brain and a second brain. In the following description, the flow of information recognized by the first brain is shown by a solid line, and the flow of information recognized by the second brain is shown by a dashed line.

[0085] The first feature representation vector generation unit 10 receives information recognized by the first brain from the first brain. The first feature representation vector generation unit 10 generates, from the received information, first feature representation vectors corresponding to concepts included in the information. For example, when the first feature representation vector generation unit 10 receives n pieces of information from the first brain, it generates n first feature representation vectors from the n pieces of information. Sa=[Sa_1,Sa_2,…,Sa_n] where n is a natural number.

[0086] Similarly, the second feature representation vector generation unit 20 receives information recognized by the second brain from the second brain. The second feature representation vector generation unit 20 generates, from the received information, second feature representation vectors corresponding to concepts included in the information. For example, when the second feature representation vector generation unit 20 receives m pieces of information from the second brain, it generates m first feature representation vectors from the m pieces of information. Sb=[Sb_1,Sb_2,…,Sb_m] where m is a natural number.

[0087] The common variable representation vector generation unit 30 projects the first feature representation vector Sa onto the N-dimensional hypersphere by the transformation F, thereby generating a common variable representation vector Sa'=F·Sa=[Sa_1',Sa_2',…,Sa_n'] Generate. Furthermore, the common variable representation vector generation unit 30 projects the second feature representation vector Sb onto the N-dimensional hypersphere by the transformation G, thereby generating a common variable representation vector Sb'=G·Sb=[Sb_1',Sb_2',…,Sb_m'] where N is a natural number.

[0088] Below, the inverse transformations of transformations F and G are respectively -1 and G -1 The feature representation vectors Sa and Sb satisfy the following rules 1, 1-2, and 2. (Rule 1) Sa=F -1 ·F·Sa Sb=G -1 G Sb (Rule 1-2) Rules to follow for contrastive or instance learning (Rules 1-3) The rule to match the distribution of Sa' and Sb', i.e., overlap (Rule 2) Sa=F -1 G G -1 ·F·Sa Sb=G -1 F F -1 G Sb This allows the first and second brains to share a common representation format for the concepts they hold. The common variable representation vector generation unit 30 learns the projection onto the hypersphere so that the distributions of Sa' and Sb' overlap.

[0089] The communication unit 202 transmits the common variable representation vector Sa' to the second brain. That is, the information of the first brain is transmitted to the second brain. The communication unit 202 transmits the common variable representation vector Sb' to the first brain. That is, the information of the second brain is transmitted to the first brain.

[0090] According to this embodiment, by standardizing the representation format of concepts between the first brain and the second brain, it is possible to realize information communication between the first brain and the second brain.

[0091] [Ninth embodiment] 10 is a functional block diagram of a brain information communication device 111 according to a ninth embodiment. The brain information communication device 111 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, a communication unit 202, a decoding unit 300, and a feature representation vector estimation unit 700. That is, the brain information communication device 111 further includes the decoding unit 300 and the feature representation vector estimation unit 700 in addition to the components of the brain information communication device 110 in FIG. 9. The other components of the brain information communication device 111 are the same as those of the brain information communication device 110.

[0092] The communication unit 202 transmits the common variable representation vectors Sa' and Sb' generated by the common variable representation vector generation unit 30 to the feature representation vector estimation unit 700. The feature representation vector estimation unit 700 estimates first and second feature representation vectors for decoding. The decoding unit 300 decodes and reconstructs the n pieces of information recognized by the first brain based on the first and second feature representation vectors estimated by the feature representation vector estimation unit 700 and the feature representation vector generated by the second feature representation vector generation unit 20. In this case, the decoding unit 300 may decode the information using a neural network such as a GAN, a Variational Autoencoder, or a Transformer.

[0093] According to this embodiment, an individual having a second brain can present information by decoding and reconstructing concepts such as images, text, and sounds recognized by the first brain. Alternatively, information can be presented to an individual having a first brain by decoding and reconstructing concepts such as images, text, and sounds recognized by the second brain.

[0094] [Tenth embodiment] 11 is a functional block diagram of a brain information communication device 112 according to a tenth embodiment. The brain information communication device 112 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, a communication unit 202, a first brain stimulation unit 401, a second brain stimulation unit 402, a first feature representation vector estimation unit 701, and a second feature representation vector estimation unit 702. That is, the brain information communication device 112 further includes the first brain stimulation unit 401, the second brain stimulation unit 402, the first feature representation vector estimation unit 701, and the second feature representation vector estimation unit 702 in addition to the components of the brain information communication device 110 in FIG. 9. The other components of the brain information communication device 112 are the same as those of the brain information communication device 110.

[0095] The first brain stimulation unit 401 stimulates the first brain with electricity, magnetism, ultrasound, chemicals, light, or heat to generate information in the first brain. Similarly, the second brain stimulation unit 402 stimulates the second brain with electricity, magnetism, ultrasound, chemicals, light, or heat to generate information in the second brain. For example, when stimulating the brain with light or heat, nerve cells may be modified in advance using genetic techniques.

[0096] The communication unit 202 transmits the common variable representation vector Sb' generated by the common variable representation vector generation unit 30 to the first feature representation vector estimation unit 701. The communication unit 202 transmits the common variable representation vector Sa' generated by the common variable representation vector generation unit 30 to the second feature representation vector estimation unit 702. The first feature representation vector estimation unit 701 estimates a feature representation vector related to the second brain information based on the common variable representation vector Sb' and transmits it to the first brain stimulation unit 401. The second feature representation vector estimation unit 702 estimates a feature representation vector related to the first brain information based on the common variable representation vector Sa' and transmits it to the second brain stimulation unit 402. The first brain stimulation unit 401 stimulates the first brain based on the feature representation vector estimated by the first feature representation vector estimation unit 701 to generate second brain information in the first brain. The second brain stimulation unit 402 stimulates the second brain based on the feature representation vector estimated by the second feature representation vector estimation unit 702 so as to generate information from the first brain in the second brain.

[0097] According to this embodiment, information from the second brain can be transmitted to the first brain, and the first brain can generate the information by stimulation. Alternatively, information from the first brain can be transmitted to the second brain, and the second brain can generate the information by stimulation.

[0098] [Eleventh embodiment] 12 is a functional block diagram of a brain information communication device 113 according to the eleventh embodiment. The brain information communication device 113 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, an AI 500, a communication unit 202, and a display unit 600. That is, the brain information communication device 113 further includes the AI ​​500 and the display unit 600 in addition to the brain information communication device 110 in FIG. 9. The other configuration of the brain information communication device 113 is the same as that of the brain information communication device 110.

[0099] The AI ​​500 reconstructs the information recognized by the first brain and the information recognized by the second brain by decoding them based on the common variable representation vectors Sa' and Sb'. The display unit 600 displays the information reconstructed by the AI ​​500. By displaying the information on the display unit 600, the information of the first brain can be conveyed to the second brain, and the information of the second brain can be conveyed to the first brain.

[0100] According to this embodiment, information communication between brains can be realized via the information displayed on the display unit.

[0101] [Twelfth embodiment] 13 is a functional block diagram of a brain information communication device 114 according to a twelfth embodiment. The brain information communication device 114 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, a communication unit 202, a first feature representation vector estimation unit 701, a second feature representation vector estimation unit 702, a first decoding unit 301, a second decoding unit 302, a first display unit 601, and a second display unit 602. That is, the brain information communication device 114 further includes the first feature representation vector estimation unit 701, the second feature representation vector estimation unit 702, the first decoding unit 301, the second decoding unit 302, the first display unit 601, and the second display unit 602 in addition to the components of the brain information communication device 111 in FIG. The other configurations of the brain information communication device 114 are common to the configurations of the brain information communication device 111.

[0102] The communication unit 202 transmits the common variable representation vector Sb' generated by the common variable representation vector generation unit 30 to the first feature representation vector estimation unit 701. The communication unit 202 transmits the common variable representation vector Sa' generated by the common variable representation vector generation unit 30 to the second feature representation vector estimation unit 702. The first feature representation vector estimation unit 701 estimates a feature representation vector related to the second brain information based on the common variable representation vector Sb' and transmits it to the first decoding unit 301. The second feature representation vector estimation unit 702 estimates a feature representation vector related to the first brain information based on the common variable representation vector Sa' and transmits it to the second decoding unit 302. The first decoding unit 301 decodes and reconstructs the second brain information based on the feature representation vector estimated by the first feature representation vector estimation unit 701. The second decoding unit 302 decodes and reconstructs the first brain information based on the feature representation vector estimated by the second feature representation vector estimation unit 702. The first display unit 601 displays the second brain information reconstructed by the first decoding unit 301. The second display unit 602 displays the first brain information reconstructed by the second decoding unit 302.

[0103] According to this embodiment, information communication between brains can be realized via the information displayed on the first and second display units.

[0104] [Thirteenth embodiment] 14 is a flowchart showing the processing procedure of the information processing method according to the thirteenth embodiment. Hereinafter, n, m, and N are all natural numbers. In step S10, the method includes generating n first feature representation vectors from the n pieces of information recognized by the first recognition system. Sa=[Sa_1,Sa_2,…,Sa_n] Generate. In step S20, the method further comprises: generating m second feature representation vectors from the m pieces of information recognized by the second recognition system; Sb=[Sb_1,Sb_2,…,Sb_m] Generate. In step S30, the method projects the first feature representation vector Sa and the second feature representation vector Sb onto an N-dimensional hypersphere by a transformation F and a transformation G, respectively, to generate a common variable representation vector Sa' = F·Sa = [Sa_1',Sa_2',…,Sa_n'] and Generate Sb' = G·Sb = [Sb_1', Sb_2', …, Sb_m'].

[0105] When the inverse transformations of transformation F and transformation G are F-1 and G-1, respectively, Sa=F-1·F·Sa=F-1·G·G-1·F·Sa Sb=G-1·G·Sb=G-1·F·F-1·G·Sb The projection onto the hypersphere is learned so that the distributions of Sa' and Sb' overlap.

[0106] According to this embodiment, in order to realize brain information communication, information to be communicated between recognition systems can be generated using a computer in a common expression format that is independent of the recognition system.

[0107] [Fourteenth embodiment] The fourteenth embodiment is an information processing program. Hereinafter, n, m, and N are all natural numbers. This program generates n first feature representation vectors from n pieces of information recognized by a first recognition system. Sa=[Sa_1,Sa_2,…,Sa_n] generating a m second feature representation vectors from the m pieces of information recognized by the second recognition system Sb=[Sb_1,Sb_2,…,Sb_m] generating a The first feature representation vector Sa and the second feature representation vector Sb are projected onto the N-dimensional hypersphere by the transformation F and the transformation G, respectively, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1',Sa_2',…,Sa_n'] and and generating Sb'=G·Sb=[Sb_1', Sb_2', ..., Sb_m'].

[0108] The inverse transformations of the transformations F and G are F -1 and G -1 When Sa=F -1 ·F·Sa=F -1 G G -1 ·F·Sa Sb=G -1 G Sb=G -1 F F -1 G Sb The projection onto the hypersphere is learned so that the distributions of Sa' and Sb' overlap.

[0109] According to this embodiment, in order to realize brain information communication, a program can be implemented as computer software that generates information to be communicated between recognition systems in a common expression format that is independent of the recognition system.

[0110] In general, in learning based on instance learning, it is important to adjust how widely data is scattered. For this reason, a learning parameter called a temperature parameter is sometimes taken into consideration in instance learning. This temperature parameter adjusts the repulsive force between sample data that acts when projection is performed. In this embodiment, setting a learning parameter such as a temperature parameter is important to appropriately scatter feature representation vectors on an N-dimensional hypersphere.

[0111] In one embodiment, the cosine distances between sample data scattered on an N-dimensional hypersphere are histogrammed, and the temperature parameter is adjusted to maximize the skewness of the histogram. A high skewness in the data distribution means that some highly similar sample data are located close to each other, while many other sample data are located far apart. In this embodiment, the distribution of feature representation vectors projected onto the N-dimensional hypersphere must have a characteristic bias. By plotting a histogram of the similarity between each sample pair of feature representation vectors and calculating the skewness, it is possible to evaluate how the feature representation vectors are distributed on the N-dimensional hypersphere. By adjusting learning parameters such as the temperature parameter to maximize the skewness, only specific sample data with high similarity can be located close to each other. That is, in this embodiment, the common variable representation vector generation unit performs projection by setting learning parameters such as the temperature parameter to maximize the skewness. This allows for more accurate projection.

[0112] [Fifteenth embodiment] 15 is a functional block diagram of an information processing device 5 according to a fifteenth embodiment. The information processing device 5 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, a mapping unit 80, and an anchor point extraction unit 90. That is, the information processing device 5 includes the mapping unit 80 and the anchor point extraction unit 90 in addition to the configuration of the information processing device 1 in FIG. 1. Since the other configuration of the information processing device 5 is the same as the configuration of the information processing device 1, the following description will focus on the added configuration.

[0113] The mapping unit 80 projects the first feature representation vector Sa and the second feature representation vector Sb onto an N-dimensional hypersphere, and generates a first latent variable vector Sa ~ and the second latent variable vector Sb ~ Generate.

[0114] The anchor point extraction unit 90 extracts the first latent variable vector Sa ~ and the second latent variable vector Sb ~ The system reduces the dimension of the latent variable vectors to extract a first cluster region group and a second cluster region group where these latent variable vectors are densely distributed, and extracts feature representation vectors corresponding to latent variable vectors contained in clusters that are close to each other from the first cluster region group and the second cluster region group as anchor points.The system learns projections such that the feature representation vectors extracted as anchor points match each other as common variable representation vectors.

[0115] First, the anchor point extraction unit 90 performs dimensional compression on the latent variable vector mapped on the N-dimensional hypersphere, for example, by isomap. At this time, the anchor point extraction unit 90 compresses the dimension so as to preserve the distance relationship between the sample data of the latent variable vector on the N-dimensional hypersphere (isometry). The dimensional compression is performed by ~ and the second latent variable vector Sb ~ In isomap, dimensionality reduction is performed based on the proximity of the data, so if the concepts recognized by the first recognition system and the concepts recognized by the second recognition system are similar, it is expected that the distributions of the latent variable vectors of both systems will be similar after dimensionality reduction.

[0116] Next, the anchor point extraction unit 90 extracts areas where data are densely distributed within the dimensionally compressed space as cluster area groups. For example, by calculating the distance between specific sample data and other sample data, if there are sample data distributed within a distance within a predetermined threshold, the set of sample data can be determined to be a cluster. Cluster extraction is performed by using the first latent variable vector Sa ~ and the second latent variable vector Sb ~ That is, the anchor point extraction unit 90 extracts the first latent variable vector Sa ~ and the second latent variable vector Sb~ The regions where the cluster regions are densely distributed are extracted as a first cluster region group and a second cluster region group, respectively.

[0117] Next, the anchor point extraction unit 90 calculates the distance between the first cluster region group and the second cluster region group in the dimensionally reduced space, and extracts the data sample in the cluster region pair with the closest distance as the anchor point. The distance between clusters may be defined as the Euclidean distance between their centers of gravity, or may be defined by the Kullback-Leibler distance, etc. The first data sample and the second data sample extracted as the anchor point are considered to have the same content of the corresponding concepts, since the distance between the corresponding clusters is the closest. In other words, the concepts expressed by the anchor points can be considered to be a set of the same or similar concepts.

[0118] According to this embodiment, anchor points, which are sets of concepts that are the same or similar to each other, are extracted, and the system learns to generate common variable representation vectors so that the anchor points match, thereby enabling more accurate information processing.

[0119] In one embodiment, the anchor point extraction unit extracts the first latent variable vector Sa generated by the mapping unit. ~ and the second latent variable vector Sb ~ Alternatively, anchor points may be extracted using the first variable expression vector Sa^ and the second variable expression vector Sb^.

[0120] In this case, the anchor point extraction unit dimensionally compresses the first variable representation vector Sa^ and the second variable representation vector Sb^ to extract a first cluster area group and a second cluster area group in which the first variable representation vector Sa^ and the second variable representation vector Sb^ are densely distributed, respectively.Then, of the first cluster area group and the second cluster area group, variable representation vectors included in clusters that are close to each other are extracted as anchor points.A projection is learned in which the variable representation vectors extracted as anchor points match each other as common variable representation vectors.

[0121] In one embodiment, the anchor point extractor extracts a first latent variable vector Sa ~ and the second latent variable vector Sb ~ In addition to this, a third latent variable vector Sc ~ The anchor points may be extracted using the following formula:

[0122] In this case, the mapping unit projects the first feature representation vector Sa, the second feature representation vector Sb, and the third feature representation vector Sc onto the N-dimensional hypersphere, and generates the first latent variable vector Sa ~ , the second latent variable vector Sb ~ and the third latent variable vector Sc ~ The anchor point extraction unit generates the first latent variable vector Sa ~ , the second latent variable vector Sb ~ and the third latent variable vector Sc ~ The first latent variable vector Sa ~ , the second latent variable vector Sb ~ and the third latent variable vector Sc ~First, a first cluster region group, a second cluster region group, and a third cluster region group are extracted, each of which has a dense distribution of latent variable vectors. Then, feature representation vectors corresponding to latent variable vectors contained in clusters that are close to each other among the first cluster region group, the second cluster region group, and the third cluster region group are extracted as anchor points. A projection is learned in which the feature representation vectors extracted as anchor points match each other as common variable representation vectors.

[0123] [Sixteenth embodiment] 16 is a functional block diagram of a brain information communication device 120 according to the sixteenth embodiment. The brain information communication device 120 includes a first feature representation vector generation unit 10, a second feature representation vector generation unit 20, a common variable representation vector generation unit 30, a communication unit 200, a mapping unit 80, and an anchor point extraction unit 90. That is, the brain information communication device 120 includes the mapping unit 80 and the anchor point extraction unit 90 in addition to the configuration of the brain information communication device 100 in FIG. 8. The other configuration of the brain information communication device 120 is the same as the configuration of the brain information communication device 100. The operations of the mapping unit 80 and the anchor point extraction unit 90 are as described in the fifteenth embodiment.

[0124] According to this embodiment, by extracting anchor points, which are sets of concepts that are the same or similar to each other, and learning to generate common variable representation vectors so that the anchor points match each other, it is possible to achieve more accurate brain information communication.

[0125] [Seventeenth embodiment] 17 is a flowchart showing the processing procedure of an information processing method according to the seventeenth embodiment. This method differs from the information processing method shown in FIG. 14 in that the first feature representation vector Sa and the second feature representation vector Sb are each projected onto an N-dimensional hypersphere, and the first latent variable vector Sa ~ and the second latent variable vector Sb ~ a mapping step S40 to generate a first latent variable vector Sa ~and the second latent variable vector Sb ~ The first latent variable vector Sa ~ and the second latent variable vector Sb ~ The method further includes step S50 of extracting a first cluster region group and a second cluster region group in which latent variable vectors are densely distributed, and extracting, as anchor points, feature representation vectors corresponding to latent variable vectors contained in clusters that are close to each other from the first cluster region group and the second cluster region group. A projection is learned in which the feature representation vectors extracted as anchor points match each other as common variable representation vectors.

[0126] According to this embodiment, anchor points, which are sets of concepts that are the same or similar to each other, are extracted, and learning is performed to generate common variable representation vectors so that the anchor points match, thereby achieving more accurate brain information communication.

[0127] [Eighteenth embodiment] The eighteenth embodiment is an information processing program. This program differs from the program of the fourteenth embodiment in that it projects the first feature representation vector Sa and the second feature representation vector Sb onto an N-dimensional hypersphere, and calculates the first latent variable vector Sa. ~ and the second latent variable vector Sb ~ a mapping step S40 to generate a first latent variable vector Sa ~ and the second latent variable vector Sb ~ The first latent variable vector Sa ~ and the second latent variable vector Sb ~ The computer further executes step S50 of extracting a first cluster region group and a second cluster region group in which the latent variable vectors are densely distributed, and extracting, as anchor points, feature representation vectors corresponding to latent variable vectors contained in clusters that are close to each other from the first cluster region group and the second cluster region group. The feature representation vectors extracted as anchor points are trained to project in such a way that they coincide with each other as common variable representation vectors.

[0128] According to this embodiment, a program that realizes more accurate brain information communication can be provided by extracting anchor points, which are sets of concepts that are the same or similar to each other, and performing learning to generate common variable representation vectors so that the anchor points match each other.

[0129] The present invention has been described above based on the embodiments. These embodiments are merely examples, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each treatment process, and that such modifications are also within the scope of the present invention. [Explanation of symbols]

[0130] 1. Information processing device, 2. Information processing device, 3. Information processing device, 4. Information processing device, 5. Information processing device, 10. First feature representation vector generation unit; 20. Second feature representation vector generation unit, 30··Common variable representation vector generation unit, 40··Identification unit, 50 ··First variable representation vector generation unit; 60 ··Second variable representation vector generation unit; 70··Third feature representation vector generation unit, 80··Mapping section, 90··Anchor point extraction section, 100··Brain information communication device, 110··Brain information communication device, 111··Brain information communication device, 112··Brain information communication device, 113··Brain information communication device, 114··Brain information communication device, 120··Brain information communication device, 200··Communications Department, 202··Communications Department, 300··Decoding unit, 300··Decoding unit, 301: First decoding unit; 302 a second decoding unit; 401··First brain stimulation unit, 402··Second brain stimulation unit, 500··AI, 600...display section, 601: First display unit; 602: Second display unit; 700··Feature representation vector estimation unit, 701: First feature representation vector estimation unit; 702: A second feature representation vector estimation unit; S10: generating a first feature representation vector; S20: generating a second feature representation vector; S30: generating a common variable representation vector; S40··mapping step, S50: extracting anchor points; Sa··first feature representation vector, Sb: second feature representation vector, Sc··third feature representation vector, Sa'··first common variable representation vector, Sb': second common variable representation vector, Sc': the third common variable representation vector, Sa^··first variable representation vector, Sb^·· second variable representation vector, Sc^··third variable representation vector, Sa ~ the first latent variable vector, Sb ~ the second latent variable vector, Sc ~ ··Third latent variable vector.

Claims

1. An information processing device for communicating information between recognition systems, comprising: When n, m, and N are natural numbers, n first feature representation vectors from the n concepts recognized by the first recognition system Sa=[Sa_1, Sa_2,..., Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m concepts recognized by the second recognition system Sb=[Sb_1, Sb_2,..., Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa and the second feature representation vector Sb are projected onto an N-dimensional hypersphere by transformations F and G, respectively, according to a rule that follows contrastive learning or instance learning, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1', Sa_2', ..., Sa_n'] and Sb'=G・Sb=[Sb_1', Sb_2',..., Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector, The inverse transformations of the transformation F and the transformation G are respectively F －１ and G －１ When S=F －１ ・F・Sa Sb=G －１ ・G・Sb Fulfilling An information processing device that learns a projection onto a hypersphere so that the distributions of Sa' and Sb' overlap.

2. 2. The information processing apparatus according to claim 1, wherein the common variable representation vector generation unit learns the transformation F and the transformation G using a neural network.

3. an identification unit that identifies whether the common variable representation vectors Sa′ and Sb′ are derived from the first feature representation vector Sa or the second feature representation vector Sb; The information processing apparatus according to claim 2 , wherein the common variable representation vector generation unit learns the transformations F and G using a generative adversarial network.

4. When n, m, and N are natural numbers and n>m, n first feature representation vectors from the n concepts recognized by the first recognition system Sa=[Sa_1, Sa_2,..., Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m concepts recognized by the second recognition system Sb=[Sb_1, Sb_2,..., Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa is projected onto an N-dimensional hypersphere by a transformation F according to a rule that follows contrastive learning or instance learning, thereby obtaining a common variable representation vector Sa'=F・Sa=[Sa_1', Sa_2',..., Sa_n'] Generate The second feature representation vector Sb is projected onto the N-dimensional hypersphere by the transformation G to obtain a common variable representation vector Sb'=G・Sb=[Sb_1', Sb_2',..., Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector, The inverse transformations of the transformation F and the transformation G are respectively F －１ and G －１ When S=F －１ ・F・Sa Sb=G －１ ・G・Sb Learning to fulfill the An information processing device that learns a projection onto a hypersphere so that the distribution of Sb' coincides with the distribution of Sa'.

5. When n, m, Na, Nb, and N are natural numbers, n first feature representation vectors from the n concepts recognized by the first recognition system Sa=[Sa_1, Sa_2,..., Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m concepts recognized by the second recognition system Sb=[Sb_1, Sb_2,..., Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa is projected onto the Na-dimensional hypersphere by a transformation F to obtain a first variable representation vector Sa^=F・Sa=[Sa_1^, Sa_2^,..., Sa_n^] a first variable representation vector generation unit that generates a The second feature representation vector Sb is projected onto the Nb-dimensional hypersphere by the transformation G to obtain a second variable representation vector Sb^=G・Sb=[Sb_1^, Sb_2^,..., Sb_m^] a second variable representation vector generation unit that generates a The first variable representation vector Sa^ and the second variable representation vector Sb^ are each transformed F ２ and transformation G ２ By projecting onto the N-dimensional hypersphere by Sa' = F ２ Sa^=[Sa_1′, Sa_2′, ..., Sa_n′] and Sb'=G ２ ・Sb^=[Sb_1', Sb_2',…, Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector, The inverse transformations of the transformation F and the transformation G are respectively F －１ and G －１ Then, the transformation F ２ and transformation G ２ The inverse transformation of F ２ －１ and G ２ －１ When S=F －１ ・F・Sa Sb=G －１ ・G・Sb S^=F ２ －１ ・F ２ ・Sa^ Sb^=G ２ －１ ・G ２ ・Sb^ Fulfilling An information processing device that learns a projection onto a hypersphere so that the distributions of Sa' and Sb' overlap.

6. An information processing device for communicating information between recognition systems, comprising: When n, m, k, and N are natural numbers, n first feature representation vectors from the n concepts recognized by the first recognition system Sa=[Sa_1, Sa_2,..., Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m concepts recognized by the second recognition system Sb=[Sb_1, Sb_2,..., Sb_m] a second feature representation vector generation unit that generates a feature representation vector k third feature representation vectors from the k concepts recognized by the third recognition system Sc=[Sc_1, Sc_2,..., Sc_k] a third feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa, the second feature representation vector Sb, and the third feature representation vector Sc are projected onto an N-dimensional hypersphere by transformations F, G, and H, respectively, to obtain a common variable representation vector Sa'=F・Sa=[Sa_1', Sa_2',..., Sa_n'], Sb' = G · Sb = [Sb_1', Sb_2', ..., Sb_m'] and Sc'=H・Sc=[Sc_1', Sc_2',..., Sc_k'] a common variable representation vector generation unit that generates a common variable representation vector, The inverse transformations of the transformations F, G and H are respectively F －１ , G －１ and H －１ When S=F －１ ・F・Sa Sb=G －１ ・G・Sb Sc=H －１ ・H・Sc Fulfilling An information processing device that learns projection onto a hypersphere so that distributions of Sa', Sb', and Sc' overlap.

7. When k is a natural number, k third feature representation vectors from the k concepts recognized by the third recognition system Sc=[Sc_1, Sb_2,..., Sb_k] a third feature representation vector generation unit that generates a feature representation vector The common variable representation vector generation unit The third feature representation vector Sc is projected onto the N-dimensional hypersphere by a transformation H to obtain a further common variable representation vector Sc'=H・Sc=[Sc_1', Sc_2',..., Sc_k'] Generate The inverse transformation of the transformation H is H －１ When Sc=H －１ ・H・Sc Fulfilling 2. The information processing apparatus according to claim 1, wherein the projection onto the hypersphere is learned so that the distribution of Sc' overlaps the distribution of Sa'.

8. A brain information communication device for communicating information between a brain and AI, When n, m, and N are natural numbers, n first feature representation vectors from n pieces of information recognized by the brain Sa=[Sa_1, Sa_2,..., Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from m pieces of information recognized by AI Sb=[Sb_1, Sb_2,..., Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa and the second feature representation vector Sb are projected onto an N-dimensional hypersphere by transformation F and transformation G, respectively, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1', Sa_2', ..., Sa_n'] and Sb'=G・Sb=[Sb_1', Sb_2',..., Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector; a communication unit that transmits the common variable representation vectors Sa′ and Sb′ from the common variable representation vector generation unit to the AI; The inverse transformations of the transformation F and the transformation G are respectively F －１ and G －１ When S=F －１ ・F・Sa Sb=G －１ ・G・Sb Fulfilling Learning the projection onto the hypersphere so that the distributions of Sa' and Sb' overlap, The brain information communication device is characterized in that the AI ​​reconstructs the n pieces of information recognized by the brain by decoding them based on the common variable representation vectors Sa' and Sb'.

9. A brain information communication device for communicating information between brains, When n, m, and N are natural numbers, n first feature representation vectors from the n pieces of information recognized by the first brain Sa=[Sa_1, Sa_2,..., Sa_n] a first feature representation vector generation unit that generates a feature representation vector m second feature representation vectors from the m pieces of information recognized by the second brain Sb=[Sb_1, Sb_2,..., Sb_m] a second feature representation vector generation unit that generates a feature representation vector The first feature representation vector Sa and the second feature representation vector Sb are projected onto an N-dimensional hypersphere by transformation F and transformation G, respectively, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1', Sa_2', ..., Sa_n'] and Sb'=G・Sb=[Sb_1', Sb_2',..., Sb_m'] a common variable representation vector generation unit that generates a common variable representation vector; a communication unit, the communication unit transmits information of the first brain to the second brain and transmits information of the second brain to the first brain based on the common variable representation vectors Sa′ and Sb′; The inverse transformations of the transformation F and the transformation G are respectively F －１ and G －１ When S=F －１ ・F・Sa Sb=G －１ ・G・Sb Fulfilling A brain information communication device that learns projection onto a hypersphere so that the distributions of Sa' and Sb' overlap.

10. a decoding unit that decodes the second brain information by machine learning; A feature representation vector estimation unit is further provided, the communication unit transmits the common variable representation vectors Sa′ and Sb′ generated by the common variable representation vector generation unit to the feature representation vector estimation unit; the feature representation vector estimation unit estimates first and second feature representation vectors for decoding; The brain information communication device according to claim 9, characterized in that the decoding unit reconstructs the n pieces of information recognized by the first brain by decoding them based on the first and second feature representation vectors estimated by the feature representation vector estimation unit and the feature representation vector generated by the second feature representation vector generation unit.

11. a first feature representation vector estimation unit; a second feature representation vector estimation unit; a first brain stimulation unit that stimulates the first brain with electricity, magnetism, ultrasound, chemical substances, light, or heat to generate information in the first brain; a second brain stimulation unit that stimulates the second brain with electricity, magnetism, ultrasound, chemical substances, light, or heat to generate information in the second brain; Furthermore, the communication unit transmits the common variable representation vector Sb′ generated by the common variable representation vector generation unit to the first feature representation vector estimation unit, and transmits the common variable representation vector Sa′ generated by the common variable representation vector generation unit to the second feature representation vector estimation unit; The first feature representation vector estimation unit estimates a feature representation vector relating to second brain information based on the common variable representation vector Sb'. the second feature representation vector estimation unit estimates a feature representation vector related to the first brain information based on the common variable representation vector Sa′; the first brain stimulation unit stimulates the first brain based on the feature representation vector estimated by the first feature representation vector estimation unit so as to generate information of the second brain in the first brain; The brain information communication device described in claim 9, characterized in that the second brain stimulation unit stimulates the second brain so as to generate information of the first brain in the second brain based on the feature representation vector estimated by the second feature representation vector estimation unit.

12. An AI that reconstructs information recognized by the first brain and information recognized by the second brain by decoding the common variable representation vectors Sa' and Sb'; The brain information communication device according to claim 9, further comprising: a display unit that displays the information reconstructed by the AI.

13. The brain information communication device according to claim 10, further comprising a display unit that displays the reconstructed information.

14. 1. An information processing method for communicating information between recognition systems, comprising: When n, m, and N are natural numbers, a first feature representation vector generation unit generating n first feature representation vectors from the n pieces of information recognized by the first recognition system; Sa=[Sa_1, Sa_2,..., Sa_n] generating a A second feature representation vector generation unit generates m second feature representation vectors from the m pieces of information recognized by the second recognition system. Sb=[Sb_1, Sb_2,..., Sb_m] generating a a common variable representation vector generation unit that projects the first feature representation vector Sa and the second feature representation vector Sb onto an N-dimensional hypersphere by a transformation F and a transformation G, respectively, to generate a common variable representation vector Sa' = F·Sa = [Sa_1', Sa_2', ..., Sa_n'] and Sb'=G・Sb=[Sb_1', Sb_2',..., Sb_m'] and generating The inverse transformations of the transformation F and the transformation G are respectively F －１ and G －１ When S=F －１ ・F・Sa Sb=G －１ ・G・Sb Fulfilling An information processing method characterized by learning a projection onto a hypersphere so that the distributions of Sa' and Sb' overlap.

15. An information processing program for communicating information between recognition systems, When n, m, and N are natural numbers, n first feature representation vectors from the n pieces of information recognized by the first recognition system Sa=[Sa_1, Sa_2,..., Sa_n] generating a m second feature representation vectors from the m pieces of information recognized by the second recognition system Sb=[Sb_1, Sb_2,..., Sb_m] generating a The first feature representation vector Sa and the second feature representation vector Sb are projected onto an N-dimensional hypersphere by transformation F and transformation G, respectively, to obtain a common variable representation vector Sa' = F·Sa = [Sa_1', Sa_2', ..., Sa_n'] and Sb'=G・Sb=[Sb_1', Sb_2',..., Sb_m'] generating a signal; The inverse transformations of the transformation F and the transformation G are respectively F －１ and G －１ When S=F －１ ・F・Sa Sb=G －１ ・G・Sb Fulfilling An information processing program that learns a projection onto a hypersphere so that the distributions of Sa' and Sb' overlap.

16. 8. The information processing device according to claim 1, wherein the common variable representation vector generation unit sets learning parameters and performs projection so that the skewness of the distribution of distances between the common variable representation vector pairs to be generated is maximized.

17. The first feature representation vector Sa and the second feature representation vector Sb are each projected onto an N-dimensional hypersphere to obtain a first latent variable vector Sa ～ and the second latent variable vector Sb ～ a mapping unit that generates The first latent variable vector Sa ～ and the second latent variable vector Sb ～ is dimensionally reduced to the first latent variable vector Sa ～ and the second latent variable vector Sb ～ and an anchor point extraction unit that extracts a first cluster region group and a second cluster region group in which the latent variable vectors are densely distributed, and extracts, as anchor points, feature expression vectors corresponding to latent variable vectors included in clusters that are close to each other from the first cluster region group and the second cluster region group, 5. The information processing apparatus according to claim 1, wherein the information processing apparatus learns a projection in which the feature representation vectors extracted as the anchor points coincide with each other as common variable representation vectors.

18. an anchor point extraction unit that performs dimensional compression on the first variable representation vector Sa^ and the second variable representation vector Sb^ to extract a first cluster area group and a second cluster area group in which the first variable representation vector Sa^ and the second variable representation vector Sb^ are densely distributed, and extracts variable representation vectors included in clusters that are close to each other from the first cluster area group and the second cluster area group as anchor points; 6. The information processing apparatus according to claim 5, wherein the information processing apparatus learns a projection in which the variable expression vectors extracted as the anchor points coincide with each other as common variable expression vectors.

19. The first feature representation vector Sa, the second feature representation vector Sb, and the third feature representation vector Sc are each projected onto an N-dimensional hypersphere to obtain a first latent variable vector Sa ～ , the second latent variable vector Sb ～ and the third latent variable vector Sc ～ a mapping unit that generates The first latent variable vector Sa ～ , the second latent variable vector Sb ～ and the third latent variable vector Sc ～ is dimensionally reduced to the first latent variable vector Sa ～ , the second latent variable vector Sb ～ and the third latent variable vector Sc ～ and an anchor point extraction unit that extracts a first cluster region group, a second cluster region group, and a third cluster region group in which latent variable vectors are densely distributed, and extracts, as anchor points, feature expression vectors corresponding to latent variable vectors included in clusters that are close to each other from among the first cluster region group, the second cluster region group, and the third cluster region group; 8. The information processing apparatus according to claim 6, wherein the information processing apparatus learns a projection in which the feature representation vectors extracted as the anchor points coincide with each other as common variable representation vectors.

20. 20. The information processing device according to claim 17, wherein the anchor point extraction unit calculates the distance between specific sample data and other sample data in the space after dimensional compression, and if there are a number of sample data distributed within a distance within a predetermined threshold that is equal to or greater than a predetermined threshold, determines that the set of sample data is a cluster.

21. 21. The information processing apparatus according to claim 17, wherein the distance between the first cluster region group and the second cluster region group is a Euclidean distance or a Kullback-Leibler distance between centers of gravity.

22. The brain information communication device according to any one of claims 8 to 13, characterized in that the common variable representation vector generation unit sets learning parameters and performs projection so that the skewness of the distribution of distances between the generated common variable representation vector pairs is maximized.

23. The first feature representation vector Sa and the second feature representation vector Sb are each projected onto an N-dimensional hypersphere to obtain a first latent variable vector Sa ～ and the second latent variable vector Sb ～ a mapping unit that generates The first latent variable vector Sa ～ and the second latent variable vector Sb ～ is dimensionally reduced to the first latent variable vector Sa ～ and the second latent variable vector Sb ～ and an anchor point extraction unit that extracts a first cluster region group and a second cluster region group in which the latent variable vectors are densely distributed, and extracts, as anchor points, feature expression vectors corresponding to latent variable vectors included in clusters that are close to each other from the first cluster region group and the second cluster region group, 23. The brain information communication device according to claim 8, wherein the device learns a projection in which the feature representation vectors extracted as the anchor points coincide with each other as common variable representation vectors.

24. The brain information communication device according to claim 23, characterized in that the anchor point extraction unit calculates the distance between specific sample data and other sample data in the space after dimensional compression, and if there are sample data distributed within a distance within a predetermined threshold in number equal to or greater than a predetermined threshold, determines that this set of sample data is a cluster.

25. 25. The brain information communication device according to claim 23, wherein the distance between the first cluster region group and the second cluster region group is a Euclidean distance or a Kullback-Leibler distance between centers of gravity.

26. A mapping unit projects the first feature representation vector Sa and the second feature representation vector Sb onto an N-dimensional hypersphere, respectively, to generate a first latent variable vector Sa ～ and the second latent variable vector Sb ～ a mapping step to generate The anchor point extraction unit extracts the first latent variable vector Sa ～ and the second latent variable vector Sb ～ is dimensionally reduced to the first latent variable vector Sa ～ and the second latent variable vector Sb ～ extracting a first cluster region group and a second cluster region group in which the latent variable vectors are densely distributed, and extracting, as anchor points, feature representation vectors corresponding to latent variable vectors included in clusters that are close to each other from the first cluster region group and the second cluster region group; 15. The information processing method according to claim 14, wherein projections are learned such that the feature representation vectors extracted as the anchor points coincide with each other as common variable representation vectors.

27. The first feature representation vector Sa and the second feature representation vector Sb are each projected onto an N-dimensional hypersphere to obtain a first latent variable vector Sa ～ and the second latent variable vector Sb ～ a mapping step to generate The first latent variable vector Sa ～ and the second latent variable vector Sb ～ is dimensionally reduced to the first latent variable vector Sa ～ and the second latent variable vector Sb ～ and further causing the computer to execute a step of extracting a first cluster region group and a second cluster region group in which the latent variable vectors are densely distributed, and extracting, as anchor points, feature representation vectors corresponding to latent variable vectors included in clusters that are close to each other from the first cluster region group and the second cluster region group; 16. The information processing program according to claim 15, wherein the information processing program learns a projection in which the feature representation vectors extracted as the anchor points coincide with each other as common variable representation vectors.

Images