A Multi-Granularity Brain Semantic Alignment Method Based on a Large Model

Abstract

This invention discloses a multi-granularity brain semantic alignment method based on a large model, comprising: at the sentence granularity, generating a set of sentences with the same semantics but different syntaxes based on word order shuffling with preserved motion; extracting sentence representations from the set using the encoding function of a large language model and then averaging them to obtain a sentence semantic representation vector; and using a linear fitting model to predict the intensity of functional magnetic resonance imaging (fMRI) at various locations in the human brain using the sentence semantic representation to obtain the locations in the human brain related to sentence semantic understanding. At the word granularity, constructing an unsupervised probabilistic statistical model, introducing alignment relationships and output rates to describe the activation relationship between words and brain regions; using the expectation-maximization method for parameter iteration, training the model until convergence, and obtaining the alignment probability from words to brain regions. This invention models at both the sentence and word granularities separately, and conducts experiments on parallel datasets of text-brain activity recordings to obtain the brain region distribution of human semantic understanding at multiple granularities.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of computer natural language processing and neuroscience, and in particular to a multi-granularity brain semantic alignment method based on a large model. Background Technology

[0002] The human brain's processing of natural language is a topic of ongoing research in neuroscience, cognitive linguistics, and many other fields. Although research began relatively early, many questions remain to be explored. In the past, the fields of computer science and artificial intelligence have developed rapidly, with natural language processing (NLP) producing numerous landmark methods and models. In recent years, some works have utilized NLP methods and models (especially the rapidly developing large language models) to aid research on the mechanisms of human language, achieving significant results. These include research on the human brain's semantic understanding process.

[0003] Various language models are used to generate representation vectors for words or sentences. Linear regression is then used to predict the intensity of functional magnetic resonance imaging (fMRI) scans of the human brain from these representation vectors. Finally, the Pearson correlation coefficient is used to evaluate the prediction results. This process is called "brain coding." Word-level language models include word2vec and ELMO, while sentence-level language models include GPT-2 and LLaMA. These models generate representation vectors from words and sentences, which are then mapped onto fMRI images of the human brain using brain coding techniques.

[0004] However, this method is somewhat crude for semantic representation mapping because vectors in language models contain not only semantic information from utterances but also other information such as syntactic structure. Adding an adversarial objective to the model to minimize the grammatical performance of the representation vectors can extract semantic representations. However, the vectors obtained under this adversarial objective do not perform ideally on brain encoding tasks, indicating a decrease in their similarity to the brain. On the other hand, for word meaning, vector representations of words can be extracted from the model and then encoded by the brain, but current language models are increasingly neglecting improvements in word representation methods, hindering research in this area. Furthermore, the lack of parallel brain imaging corpora at the word level makes supervised alignment model training difficult. Summary of the Invention

[0005] This invention provides a multi-granularity brain semantic alignment method based on a large model to systematically explore the semantic understanding process of the human brain at multiple granularities. The semantic understanding process is divided into word semantic understanding of individual words in a sentence and sentence semantic understanding of the overall semantic combination. Sentence semantics requires effective representation extraction, while word semantics requires new alignment modeling approaches. This invention provides alignment modeling methods for sentence semantics and word semantics to human brain fMRI (functional magnetic resonance imaging) images. Combining these two methods allows for the acquisition of the distribution characteristics of semantics at different granularities in the human brain, thus providing a possible path for hierarchical comprehensive analysis. This method is easy to operate and highly scalable.

[0006] This invention provides a multi-granularity brain semantic alignment method based on a large model, comprising the following steps:

[0007] In terms of sentence meaning, the extraction of sentence meaning representation vectors is as follows: each sentence in the dataset is shuffled while preserving the word order. During shuffling, the core action of the sentence, the agent and receiver of the action are preserved. A set of sentences with similar meanings but different grammatical structures is generated. The sentences in this set are input into the large language model, and vectors are extracted from its hidden layer. The vectors are averaged to obtain the sentence meaning representation vector.

[0008] Mapping of semantic representation to the human brain: Semantic representation vectors are mapped onto each voxel of a functional magnetic resonance imaging (fMRI) image of the human brain using brain coding technology. If the nuclear magnetic intensity of a voxel is predicted by the semantic representation vector to be higher than a set threshold, it indicates that this voxel is related to semantic understanding function.

[0009] In terms of word meaning, the modeling and simplification of the alignment model: a probabilistic model is established for the alignment relationship between each word in the dataset text and each brain region in the human brain. At the same time, for the alignment relationship that "a word may cause multiple brain regions to respond", the "output rate" is used for modeling. Assumptions are made on the established model to simplify it.

[0010] Training the alignment model: The obtained model is transformed into an optimization problem and trained on a parallel dataset. The trained model includes the probability of alignment between each word and brain region.

[0011] Optionally, in one embodiment of the present invention, the extraction of the semantic representation vector specifically includes:

[0012] (1) Identify the words corresponding to the "movement" element of the sentence. The movement is the core action of the sentence, the doer and receiver of the action;

[0013] (2) Change the word order of each non-verbal word in the sentence. The position after the change is determined by a random variable and the original position, resulting in a sentence with scrambled word order.

[0014] (3) Repeat step (2) to obtain multiple sentences with scrambled word order;

[0015] (4) Input the set of sentences obtained in step (3) into the large language model and extract the representation vectors from the hidden layer of the model;

[0016] (5) Take the average of the set of language vectors obtained in step (4) to obtain the semantic representation vector of the original sentence.

[0017] Optionally, in one embodiment of the present invention, the mapping of semantic representation to the human brain specifically includes:

[0018] (1) Preprocessing of semantic representation vectors and human brain functional magnetic resonance imaging sequences, including principal component analysis for unified dimension and functional magnetic resonance imaging downsampling;

[0019] (2) Train the ridge regression model and use the semantic representation vector of each sentence in the dataset to predict the corresponding functional magnetic resonance imaging image of the human brain, i.e. the magnetic resonance intensity on each voxel.

[0020] (3) The Pearson correlation coefficient was used to evaluate the predictive effect of the ridge regression model and predict brain regions that are above the set threshold, i.e. brain regions related to sentence meaning processing.

[0021] Optionally, in one embodiment of the present invention, the modeling and simplification of the alignment model specifically includes:

[0022] (1) Use probabilistic models and alignment relationships to describe the process of words triggering brain region activity, that is, given multiple words, the probabilistic model describes the conditional probability of the brain regions they activate.

[0023] (2) Introduce output rate to describe the situation where "one word can trigger responses in multiple brain regions";

[0024] (3) Use the chain rule to decompose the model and propose assumptions to simplify the model.

[0025] Optionally, in one embodiment of the present invention, training the alignment model specifically includes:

[0026] (1) The Lagrange multiplier method is introduced to transform the model into an unconstrained optimization problem;

[0027] (2) Use the expectation-maximization algorithm to iterate the parameters. Calculate the expectation in step E and maximize the probability parameter iteration in step M.

[0028] (3) Repeat steps E and M in step (2) until the parameters converge.

[0029] Optionally, in one embodiment of the present invention, training the alignment model specifically includes:

[0030] Over a period of time, specifically within a functional magnetic resonance imaging (fMRI) measurement slice, suppose m words map to n responding brain regions, denoted as S = {w1, w2, ... wn}. m The responding brain regions are B = {r1, r2, ... r}. n There is an alignment relationship. This process, in which "multiple words trigger responses in multiple brain regions," is described as follows:

[0031]

[0032] For each word w in S i It may activate Each brain region is called the output rate, denoted as _____. Their respective positions in B are denoted as After introducing the output rate, the model is rewritten as:

[0033]

[0034] in, Next, we make the following five assumptions:

[0035]

[0036] The final model is:

[0037]

[0038] p0+p1=1.

[0039] Optionally, in one embodiment of the present invention, during the training of the alignment model, the constrained optimization problem is transformed into a Lagrange unconstrained problem:

[0040]

[0041] Where θ is the objective parameter, the optimal parameter is calculated using the expectation-maximization algorithm. In the E-step, the expected frequency is used for calculation, and the following formula for the expected frequency is defined:

[0042]

[0043] Where δ(x,y) represents whether x and y appear simultaneously; it is 1 if they do, and 0 otherwise. In the M-step, parameter estimation is performed on the training set of K samples. The parameter estimation formula is:

[0044]

[0045] The multi-granularity brain semantic alignment method based on a large model in this invention has the following beneficial effects:

[0046] This paper provides a systematic semantic alignment modeling framework (involving both sentence and word meaning granularities). The sentence meaning representation extraction method is simpler to operate than previous methods, and experiments demonstrate its effectiveness. An innovative unsupervised word-brain region alignment model is proposed, improving the process of mapping language to the human brain. It exhibits good scalability and provides the possibility of statistical analysis for different linguistic classifications, overcoming the limitation of existing models using word embedding techniques that can only perform statistical analysis on the mathematical features of vectors.

[0047] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0048] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0049] Figure 1 A flowchart illustrating a multi-granularity brain semantic alignment method based on a large model, provided according to an embodiment of the present invention;

[0050] Figure 2 This is a flowchart of the semantic representation vector extraction process according to an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram illustrating the correspondence between brain regions and vocabulary in an embodiment of the present invention. Detailed Implementation

[0052] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0053] Figure 1 This is a flowchart illustrating a multi-granularity brain semantic alignment method based on a large model, according to an embodiment of the present invention.

[0054] like Figure 1 As shown, this multi-granularity brain semantic alignment method based on a large model includes the following steps:

[0055] Regarding sentence meaning, (1) extraction of sentence meaning representation vectors. The word order of each sentence in the dataset is shuffled while preserving the "motion path", generating a set of sentences with similar meanings but different grammatical structures. The sentences in this set are input into a large language model, and vectors are extracted from its hidden layer. The vectors are averaged to obtain the sentence meaning representation vector;

[0056] (2) Mapping of semantic representation to the human brain. The semantic representation vector is mapped to each voxel of the human brain fMRI image through brain coding technology. If the nuclear magnetic intensity of a certain voxel is predicted well by the semantic representation vector, it indicates that this voxel is related to the semantic understanding function.

[0057] Specifically, the human brain and semantic alignment framework based on multi-granularity modeling includes two granularities: sentence meaning and word meaning.

[0058] In the extraction of semantic representations, this step involves shuffling the word order of each sentence in the dataset while preserving its "motion path," generating a set of sentences with similar meanings but distinct grammatical structures. These sentences are then input into a large language model such as LLaMA3, and vectors are extracted from each hidden layer. The vectors are averaged to obtain the semantic representation vector of each sentence. The "motion path" refers to the core action in the sentence and its agent and receiver (e.g., in "I ate a cake with a fork last night.", the core action is "ate," the agent is "I," and the receiver is "cake").

[0059] The mapping of the representation vector to the human brain: In this step, the semantic representation vector obtained in the previous step will be mapped to each voxel of the human brain fMRI image through brain coding technology (i.e., linear regression + Pearson correlation coefficient). If the nuclear magnetic intensity on a certain voxel is predicted well by the semantic representation vector, it indicates that this voxel is related to the semantic understanding function.

[0060] Semantic granularity includes the following steps:

[0061] Regarding word meaning, (1) Modeling and simplification of the alignment model. A probabilistic model is established for the alignment relationship between each word in the dataset text and each brain region in the human brain. At the same time, for the alignment relationship that "a word may cause multiple brain regions to respond", the "output rate" is used for modeling. After that, reasonable assumptions are made to simplify the established model;

[0062] (2) Training the alignment model. The obtained model is transformed into an optimization problem and trained on a parallel dataset. The trained model includes the probability of alignment between each word and brain region. Based on this, relevant statistical analyses can be performed, such as the preference relationship between words of different parts of speech and specific brain regions.

[0063] Optionally, in one embodiment of the present invention, the method for extracting semantic representation vectors includes the following steps:

[0064] (1) Identify the words corresponding to the "motion process" (i.e. the core action of the sentence, the doer and receiver of the action) element in the sentence;

[0065] (2) Change the word order of each non-verb word in the sentence. The new position is determined by a random variable and the original position. This step yields a sentence with the word order shuffled.

[0066] (3) Repeat step (2) to obtain multiple sentences with scrambled word order.

[0067] (4) Input the set of sentences obtained in step (3) into the large language model (LLaMA3 is used in this invention) and extract the representation vector from the hidden layer of the model.

[0068] (5) Take the average of the set of language vectors obtained in step (4) to obtain the semantic representation vector of the original sentence.

[0069] Optionally, in one embodiment of the present invention, the mapping of semantic representation to the human brain includes the following steps:

[0070] (1) Preprocess the semantic representation vector and human brain fMRI sequence, including principal component analysis to unify the dimension and fMRI downsampling;

[0071] (2) Train the ridge regression model and use the semantic representation vector of each sentence in the dataset to predict the corresponding human brain fMRI image (i.e., the magnetic resonance intensity on each voxel);

[0072] (3) The Pearson correlation coefficient was used to evaluate the predictive performance of the ridge regression model. The brain regions with good predictive performance were those related to sentence meaning processing.

[0073] Optionally, in one embodiment of the present invention, the modeling and simplification of the alignment model includes the following steps:

[0074] (1) Using probabilistic models and alignment relationships to describe the process by which words trigger brain region activity. That is, given multiple words, the probabilistic model describes the conditional probability of the brain regions they activate;

[0075] (2) Introduce output rate to describe the situation where "one word can trigger responses in multiple brain regions";

[0076] (3) Use the chain rule to decompose the model and propose assumptions to simplify the model.

[0077] Optionally, in one embodiment of the present invention, training the alignment model includes the following steps:

[0078] (1) The Lagrange multiplier method is introduced to transform the model into an unconstrained optimization problem;

[0079] (2) The EM algorithm (Expectation Maximization Algorithm) is used for parameter iteration. The expectation is calculated in the E-step, and the probability parameters are maximized in the M-step.

[0080] (3) Repeat steps E and M in step (2) until the parameters converge.

[0081] Optionally, in one embodiment of the present invention, in the modeling and simplification of the alignment model, it is assumed that m words map to n responding brain regions over a period of time (i.e., within one fMRI measurement time slice), and these words are denoted as S = {w1, w2, ... w...} n The responding brain regions are B = {r1, r2, ... r}. n There is an alignment relationship. This process, in which "multiple words trigger responses in multiple brain regions," is described as follows:

[0082]

[0083] For each word w in S i It may activate Each brain region (called the output rate) is denoted as... Their respective positions in B are denoted as After introducing the output rate, the model is rewritten as:

[0084]

[0085] in, Next, we make the following five assumptions:

[0086]

[0087] The final model is:

[0088]

[0089] p0+p1=1.

[0090] Optionally, in one embodiment of the present invention, during the training of the alignment model, the constrained optimization problem is transformed into a Lagrange unconstrained problem:

[0091]

[0092] Where θ is the objective parameter, the optimal parameter is calculated using the expectation-maximization algorithm. In the E-step, the expected frequency is used for calculation, and the following formula for the expected frequency is defined:

[0093]

[0094] Where δ(x,y) represents whether x and y appear simultaneously; it is 1 if they do, and 0 otherwise. In the M-step, parameter estimation is performed on the training set of K samples. The parameter estimation formula is:

[0095]

[0096] The invention will now be described in further detail with reference to the accompanying drawings.

[0097] In order to conduct a systematic, multi-granular exploration of the semantic understanding process of the human brain, such as Figure 1 As shown, the framework of the invention is divided into two parts: the semantic part, which includes the extraction and mapping of semantic representations; and the word semantic part, which includes the modeling and training of alignment relationships.

[0098] Sentence meaning:

[0099] (1) Extraction of semantic representation

[0100] The overall semantic extraction method is as follows: Figure 2 As shown, in this method, for each sentence, the individual words that make up its "process" are first identified, that is, the core action in the sentence and its agent and receiver. The relative order of these words will remain unchanged to preserve the meaning of the sentence (e.g., "The dog bit the boy." would change its semantics if the relative order were changed to "The boy bit the dog.").

[0101] For non-verbal words in the sentence, each word will be assigned a normally distributed integer p, and this word will be moved p positions before or after it (since p can be positive or negative, the corresponding position can be either before or after). This results in a sentence with a similar meaning but a different grammatical structure than the original sentence. Repeating this process multiple times on each original sentence yields several sentences with similar meanings but different syntaxes. For a sentence S = {w1, w2, w3, ... w...} n}(w i The pseudocode for the algorithm to shuffle the word order of words in the sentence (where i∈[1,n] are words in the sentence) is as follows:

[0102] Predication is the set of dynamic elements in S.

[0103]

[0104] These sentences are input into a language model, and the sentence representation vectors are obtained from the model's hidden layers. The average of the sentence representation vectors of a set of sentences with similar semantics but different syntactic features is taken to obtain the semantic representation vector of this set of sentences (i.e., the original sentences).

[0105] (2) Representation of the mapping from vectors to the human brain

[0106] This section provides a detailed description of the voxel-level brain coding process performed after obtaining the semantic representation.

[0107] Hypothesis representation M represents the number of words corresponding to X, and d represents the vector dimension, such as the GPT-2 model where the vector dimension is 768. These are the fMRI vectors corresponding to these words. N represents the number of time slices (TR). Ridge regression is used to fit the two:

[0108] argmin f ∑(Yf(X)) 2 +λ‖f‖ 2

[0109] Where λ is the regularization coefficient.

[0110] The fitting effect, known as the brain score in brain coding, is the Pearson correlation coefficient between the fMRI predicted value and the true value.

[0111]

[0112] In addition, since X and Y have different dimensions, and X may contain some unimportant information, principal component analysis will be used to unify the dimensions during the fitting process.

[0113] Meaning section:

[0114] (1) Modeling and simplification of alignment relationships

[0115] The model aims to describe a scenario where, during fMRI data measurement, a subject hears several words over a period of time (i.e., within an fMRI measurement time slice), which elicit responses in certain brain regions. Assume m words map to n responding brain regions, and denote these words ("S" representing external stimuli) as S = {w1, w2, ... w...}. m The responding brain regions are B = {r1, r2, ... r}. n They have an alignment relationship. like Figure 3 The goal is to explore this alignment relationship. Note that the introduction of NULL (empty symbol) here describes the possibility that a word does not elicit a response in a brain region, while non-textual elements do elicit a response in some brain regions.

[0116] First, we model this mapping process. We use a probabilistic model approach, where the mapping of an external stimulus S to brain region B within a specific time slice is a probabilistic problem. That is, given an external stimulus S, the probability that some brain regions B will respond is denoted as P(B|S). The alignment relationship is as follows: the mapping first traverses every possible alignment relationship, and each alignment method corresponds to a probability P(B,a|S) of producing a correct mapping under that alignment relationship. The alignment relationships are independent of each other, and according to the addition criterion, the mapping probability is:

[0117]

[0118] A word may correspond to multiple brain regions, and a brain region may be activated by multiple words. This is a "many-to-many" problem, which results in a large number of iterations of "traversing every possible alignment relationship." To simplify the subsequent modeling, we break it down into a bidirectional "one-to-many" problem. That is, a word or non-textual factor can elicit responses from multiple brain regions, while a brain region cannot be activated by multiple words. This alignment relationship is symmetrical, and in the subsequent training phase, we can perform two training iterations (forward and backward), and take the union of the resulting alignments. Here, we rewrite the alignment relationship representation as follows: a = {w i →r j |i∈[1,m],j∈[1,n],w i Activate r j}

[0119] For each word w in S i It may activate Each brain region (called the output rate) is denoted as... Their respective positions in B are denoted as A <τ,π> pair corresponds to one<B,a> Yes, and one<B,a> Can correspond One <τ,π> pair. Transcribing the above alignment model:

[0120]

[0121] in The above equation is derived from the chain rule, and without any assumptions, both sides are strictly equal. However, to simplify the model, some assumptions need to be made:

[0122] 1. Assume that the output rate is only related to the current word, i.e.

[0123] 2. Assume that the probability of a brain region being activated by a certain word is only related to that word, i.e.

[0124] 3. Since the positional order within a set of brain regions has no practical significance, it is assumed that the brain regions are uniformly distributed. For the word w... i Activated brain regions set τ i After determining the locations of k brain regions, there is still... One position, that is

[0125] 4. Assuming everything is determined After each non-empty aligned brain region, a position for placing an empty aligned brain region is generated with probability p1 after each brain region. It follows a binomial distribution and has

[0126] 5. Assume that after determining the brain regions activated by non-empty words and their locations, the remaining brain regions are generated by empty words. After k empty words have been placed, what are the remaining brain regions? There are several locations, evenly distributed, with but

[0127] There is a model:

[0128]

[0129] p0 + p1 = 1

[0130] (2) Model optimization and training

[0131] The constrained optimization problem described above is transformed into an unconstrained Lagrangian problem, where θ is the parameter in the optimization objective, i.e., p0, p1, ... f(r|w):

[0132]

[0133] Since the above equation has no analytical solution, the EM algorithm (Expectation Maximization Algorithm) is used for parameter iteration (calculating the new parameters using the old parameters, and then replacing the old parameters with the new parameters). The EM algorithm consists of an E-step and an M-step. In the E-step, the expected frequency is used for calculation, and the following formula for the expected frequency is defined:

[0134]

[0135] Where δ(x,y) represents whether x and y appear simultaneously; it is 1 if they do, and 0 otherwise. In the M-step, parameter estimation is performed on the training set of K samples. The parameter estimation formula is:

[0136]

[0137] The new values ​​of each parameter can be calculated based on the above formula. Repeat the E-step and M-step until the parameter values ​​converge. Finally, the alignment probability f(r) between each word and each brain region is obtained. j |w i ).

[0138] The multi-granularity brain semantic alignment method based on a large model proposed in this invention includes: at the sentence level, generating a set of sentences with the same semantics but different syntaxes based on word order shuffling with preserved motion; extracting sentence representations from the set using the encoding function of a large language model and then averaging them to obtain a sentence semantic representation vector; and using a linear fitting model to predict the intensity of functional magnetic resonance imaging (fMRI) at various locations in the human brain using the sentence semantic representations to obtain the locations in the human brain related to sentence semantic understanding. At the word level, constructing an unsupervised probabilistic statistical model, introducing alignment relationships and output rates to describe the activation relationship between words and brain regions; using the expectation-maximization method for parameter iteration, training the model until convergence, and obtaining the alignment probability from words to brain regions. This invention, by modeling at both the sentence and word levels separately, conducts experiments on parallel datasets of text-brain activity records to obtain the brain region distribution for human semantic understanding at multiple levels.

[0139] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0140] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0141] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

Claims

1. A multi-granularity brain semantic alignment method based on a large model, characterized in that, Includes the following steps: In terms of sentence meaning, the extraction of sentence meaning representation vectors is as follows: each sentence in the dataset is shuffled while preserving the word order. During shuffling, the core action of the sentence, the agent and receiver of the action are preserved. A set of sentences with similar meanings but different grammatical structures is generated. The sentences in this set are input into the large language model, and vectors are extracted from its hidden layer. The vectors are averaged to obtain the sentence meaning representation vector. Mapping of semantic representation to the human brain: Semantic representation vectors are mapped onto each voxel of a functional magnetic resonance imaging (fMRI) image of the human brain using brain coding technology. If the nuclear magnetic intensity of a voxel is predicted by the semantic representation vector to be higher than a set threshold, it indicates that this voxel is related to semantic understanding function. In terms of word meaning, the modeling and simplification of the alignment model: a probabilistic model is established for the alignment relationship between each word in the dataset text and each brain region in the human brain. At the same time, for the alignment relationship that "a word may cause multiple brain regions to respond", the "output rate" is used for modeling. Assumptions are made on the established model to simplify it. Training the alignment model: The obtained model is transformed into an optimization problem and trained on a parallel dataset. The trained model includes the probability of alignment between each word and brain region. The training of the alignment model specifically includes: Over a period of time, specifically within a functional magnetic resonance imaging (fMRI) measurement slice, suppose m words map to n responding brain regions, denoted as S = {w1, w2, ... wn}. m The responding brain regions are B = {r1, r2, ... r}. n There is an alignment relationship. This process, in which "multiple words trigger responses in multiple brain regions," is described as follows: For each word w in S i It may activate Each brain region is called the output rate, denoted as _____. Their respective positions in B are denoted as After introducing the output rate, the model is rewritten as: in, Next, we make the following five assumptions: The final model is: p0+p1=1.

2. The method according to claim 1, characterized in that, The extraction of semantic representation vectors specifically includes: (1) Identify the words corresponding to the "movement" element of the sentence. The movement is the core action of the sentence, the implementer and the receiver of the action; (2) Change the word order of each non-verbal word in the sentence. The position after the change is determined by a random variable and the original position, resulting in a sentence with scrambled word order. (3) Repeat step (2) to obtain multiple sentences with scrambled word order; (4) Input the set of sentences obtained in step (3) into the large language model and extract the representation vectors from the hidden layer of the model; (5) Take the average of the set of language vectors obtained in step (4) to obtain the semantic representation vector of the original sentence.

3. The method according to claim 1, characterized in that, The mapping of semantic representation to the human brain specifically includes: (1) Preprocessing of semantic representation vectors and human brain functional magnetic resonance imaging sequences, including principal component analysis for unified dimension and functional magnetic resonance imaging downsampling; (2) Train the ridge regression model and use the semantic representation vector of each sentence in the dataset to predict the corresponding functional magnetic resonance imaging image of the human brain, i.e. the magnetic resonance intensity on each voxel. (3) The Pearson correlation coefficient was used to evaluate the predictive effect of the ridge regression model and predict brain regions that are above the set threshold, i.e. brain regions related to sentence meaning processing.

4. The method according to claim 1, characterized in that, The modeling and simplification of the alignment model specifically includes: (1) Use probabilistic models and alignment relationships to describe the process of words triggering brain region activity, that is, given multiple words, the probabilistic model describes the conditional probability of the brain regions they activate. (2) Introduce output rate to describe the situation where "one word can trigger responses in multiple brain regions"; (3) Use the chain rule to break down the model and make assumptions to simplify the model.

5. The method according to claim 1, characterized in that, The training of the alignment model specifically includes: (1) The Lagrange multiplier method is introduced to transform the model into an unconstrained optimization problem; (2) Use the expectation-maximization algorithm to iterate the parameters. Calculate the expectation in step E and maximize the probability parameter iteration in step M. (3) Repeat steps E and M in step (2) until the parameters converge.

6. The method according to claim 5, characterized in that, During the training of the alignment model, the constrained optimization problem is transformed into an unconstrained Lagrangian problem: Where θ is the objective parameter, the optimal parameter is calculated using the expectation-maximization algorithm. In the E-step, the expected frequency is used for calculation, and the following formula for the expected frequency is defined: Where δ(x,y) represents whether x and y appear simultaneously; it is 1 if they do, and 0 otherwise. In the M-step, parameter estimation is performed on the training set of K samples. The parameter estimation formula is:

Images