Scoring dictionary construction method and system based on english lexical linguistic attribute prediction

Abstract

The application provides a scoring dictionary construction method and system based on English vocabulary linguistic attribute prediction, constructs text features and visual features for a target word; a feature fusion method is used to fuse the text features and the visual features, the fused multi-modal features are taken as inputs of a graph attention network model, an adjacency matrix is constructed based on word similarity and word co-occurrence probability, a graph structure is represented by using the adjacency matrix, a relationship graph between words is obtained, an improved graph attention network model is constructed based on the constructed relationship graph between words, a preprocessed feature matrix X and an adjacency matrix A representing a graph structure are input into the graph attention network model, and a prediction value of the specificity or the imageability of a word after the graph attention network model processing; the multi-modal features are taken as inputs of a prediction model, and the overfitting risk of the prediction model in training is reduced; the adjacency matrix is constructed according to the similarity and the co-occurrence probability between words, and the interaction between words with high similarity and high co-occurrence frequency is realized.

Description

Technical Field

[0001] This invention belongs to the field of vocabulary standardization scoring prediction, specifically involving a scoring dictionary construction method and system based on the prediction of English vocabulary linguistic attributes. Background Technology

[0002] In recent years, evaluating the lexical norms of texts such as words and sentences, and providing scores for various linguistic attributes, has been a research focus of considerable interest. Evaluating the lexical norms of texts and constructing large-scale dictionaries can provide crucial data support for experimental and quantitative research in fields such as cognitive science and computational linguistics. Lexical specificity and imageability scores have been applied in numerous natural language processing tasks, such as information retrieval, metaphor detection, and controlling the visual descriptiveness of image captions based on specified imageability values.

[0003] Specificity and imagerability are crucial attributes in linguistics, representing the semantic regularity of words and quantifying human perception of them. In linguistics, specificity refers to the degree to which a word's concept points to a perceptible entity, while imagerability represents the degree to which a word evokes mental intentions through sensory experience. Although most things with low specificity are difficult to imagine, this doesn't mean there's an absolute positive correlation between specificity and imagerability. Currently, these two attributes are applied in numerous natural language processing tasks, and a high-quality specificity and imagerability scoring dictionary is a crucial prerequisite for achieving good results in these applications.

[0004] The evaluation techniques for lexical specificity and imageability scores have evolved from manual scoring to automated calculation using mathematical models. Early methods primarily involved building artificial dictionaries by inviting experts or recruiting volunteers to score the specificity and imageability of words. While this approach yielded relatively reliable evaluation results, it was extremely costly in terms of both money and time, and each dictionary contained a limited number of words. Given the vastness of the natural language vocabulary, obtaining specificity and imageability scores for all words manually is impractical. Therefore, how to automatically estimate the specificity and imageability scores for each word became a pressing issue. With the rapid development of modern information technology, a series of advanced technologies have emerged, such as lexical specificity and imageability score prediction techniques based on weakly supervised learning and machine learning.

[0005] The main idea behind lexical specificity and imagerability rating prediction techniques based on weakly supervised learning is to select a small subset of high-scoring and low-scoring samples from a manually rated dictionary as seed words. Based on the semantic similarity between the unrated word and the seed words, a specific algorithm is constructed to estimate the specificity or imagerability rating of the unrated word. A key characteristic of this technique is that it is weakly supervised, requiring no large datasets and only a small amount of human input to obtain the estimation results. Therefore, it is suitable for languages ​​with scarce word resources in their corpora. However, it has some limitations, such as the quality of the seed word selection directly affecting the final prediction performance, and its performance on large corpora is inferior to that of supervised learning-based prediction techniques.

[0006] Supervised learning-based rating prediction techniques utilize a large portion of the data from a manually rated dictionary as training samples. Features are constructed for each word, and a prediction model is trained. After validation on a test set, the model is used to estimate other unrated words. In this technique, researchers typically construct features based on the semantic or related visual information of words. Semantic information is usually reflected through word vectors, such as pre-trained Word2vec or FastText word vectors. Initially, word-related visual information was represented by the number of word-related images retrieved from image databases; however, this feature contains very limited information. With the development of computer vision technology, some scholars have attempted to encode visual features using word-related images. For example, they crawl images related to each word in a dataset from the internet to form an image set, using patterns, contours, and edges to represent low-level image features, and using the output of the last hidden layer of a neural network model trained for image classification or object detection as high-level image features. Support vector regression, random forests, and neural networks are commonly used when training prediction models. Supervised learning-based vocabulary specificity and imageability rating prediction techniques are currently the mainstream techniques in rating prediction tasks. These techniques are very convenient, but their prediction performance on small datasets still needs improvement. Summary of the Invention

[0007] To address the problems existing in the prior art, this invention provides a method and system for constructing a rating dictionary based on the prediction of the linguistic attributes of English words. It combines the textual features of words with the visual features related to the words to construct multimodal features, establishes an improved graph attention network model to predict the rating of the specificity and imageability of words, and then constructs a rating dictionary of English words based on the predicted values ​​of the specificity or imageability of English words, ensuring a high correlation between the prediction results and human ratings.

[0008] To achieve the above objectives, the technical solution adopted by this invention is: a method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes, comprising the following steps:

[0009] Construct textual and visual features for the target word;

[0010] Feature fusion methods are used to fuse textual and visual features to obtain multimodal features.

[0011] The fused multimodal features are combined to form a feature matrix X. An adjacency matrix A is constructed based on word similarity and word co-occurrence probability. The adjacency matrix A represents the graph structure, and the relationship graph between words is obtained. Based on the relationship graph between words, an improved graph attention network model is constructed. The feature matrix X and the adjacency matrix A representing the graph structure are input into the graph attention network model. After processing by the improved graph attention network model, the predicted value of the specificity or imageability of the words is obtained.

[0012] When constructing text features for a target word, word vectors, word suffix vectors, part-of-speech vectors, sentiment polarity, and semantic depth features are selected, and the word vectors, word suffix vectors, part-of-speech vectors, sentiment polarity, and semantic depth features are concatenated as the text features of the target word.

[0013] A rating dictionary for English words is constructed based on the predicted values ​​of the specificity or imageability of English words.

[0014] Constructing text features for target words includes feature construction methods without context and feature construction methods with context. The text feature construction method without context is as follows:

[0015] For word vectors, FastText pre-trained word vectors are used as word vector features;

[0016] For word suffixes, construct a multi-dimensional vector with an initial value of 0, extract the last, two, three, and four characters of the target word, select several of the most commonly used suffixes to form a set, when there are elements in the set that are the same as the last, two, three, and four characters, record the position of the element, and assign a value of 1 to the multi-dimensional vector at the position corresponding to the element, and finally obtain the word suffix vector;

[0017] For part-of-speech vectors, a 6-dimensional vector with each element initially set to 0 is constructed. If the WordNet pos function can be used to obtain the part-of-speech and frequency of occurrence corresponding to different meanings of a word, then the frequency of occurrence of each part-of-speech of a word is weighted according to the sum of the frequencies of occurrence of all parts of speech of a word in the external corpus, and the part-of-speech vector representing the word is obtained. Then the part-of-speech vector of the next word is processed.

[0018] If WordNet's pos function cannot obtain the part-of-speech and frequency of a word corresponding to different meanings, then use NLTK's pos_tag function to obtain the part-of-speech of the word. Find the element position representing the part-of-speech in the 6-dimensional vector, assign the value 1 to that element, and leave the values ​​of the other elements unchanged; then proceed to process the part-of-speech vector of the next word.

[0019] For sentiment polarity, obtain the sentiment score of a word, and consider the sentiment score of each definition of the word when calculating the sentiment score;

[0020] The positive and negative scores of each word's meaning are obtained. For word meanings that are completely positive and have no negative meaning, the positive score ranges from (0,1] and the negative score is 0. For a neutral word meaning, the positive and negative scores are consistent. The difference between the positive and negative scores is used as the total sentiment score of the corresponding word meaning. The total sentiment score of a word is obtained by weighting the corresponding scores according to the frequency of each meaning in the external corpus.

[0021] For semantic depth, when calculating the total semantic depth of a word, the semantic depth of each definition of the word is obtained, and the semantic depth of each definition is weighted according to the frequency of occurrence of each definition to obtain the total semantic depth of the word.

[0022] The specific method for constructing text features in conjunction with the context is as follows:

[0023] For word vectors, a fine-tuned BERT model is used to obtain word vectors containing contextual semantic information. Specifically, the downstream task of the BERT model is set as a word specificity prediction task or a word imageability prediction task. The input of the task is a complete sentence, and the output is the specificity score of the target word in the sentence. After the BERT model is trained, the embedding representation corresponding to the final hidden state of the target word is extracted, that is, the word vector or imageability score containing contextual semantic information.

[0024] For parts of speech, each word in the sentence containing the target word is tagged with its part of speech, and the part of speech of the target word is returned. Then, a 6-dimensional vector is constructed. In the 6-dimensional vector, the elements of each dimension represent nouns, verbs, adjectives, prepositions, adverbs and conjunctions, respectively. The element position that represents the part of speech of the target word is found in the vector, and the value of that element is set to 1, and the values ​​of the other elements are 0.

[0025] For sentiment polarity, the sentiment score of the sentence containing the target word and the sentiment score of the word are calculated separately. Then, the two are weighted and the weighted result is used as the final sentiment score of the target word.

[0026] For semantic depth, the cosine similarity between the semantic vector of each definition of the target word and the semantic vector of the target word is calculated. The definition with the highest similarity is selected, and the semantic depth corresponding to the definition with the highest similarity is obtained as the semantic depth of the target word.

[0027] When constructing visual features for the target word, ResNet152 is used to extract high-level semantic features of images related to the target word as visual features of the target word.

[0028] The text and visual features are fused using feature fusion methods to obtain multimodal features, including the following steps:

[0029] (1) Based on the constructed text features and visual features, an autoencoder is used to reduce the dimensionality of high-dimensional features, so that the dimensionality of text features and visual features is consistent. Specifically:

[0030] Automatic encoder f θ It contains two subnetworks, one of which is used to learn the input vector. To hidden vector mapping relationship Another subnetwork is used to learn the hidden vectors. to vector mapping relationship g is the data encoding process; g takes the high-dimensional input... Encoded as a low-dimensional hidden vector h represents the data decoding process, which involves encoding the hidden vector. Decoded into high dimension Hidden Vectors The dimensions can be set by the user;

[0031] use Representing image feature vectors, image features are learned using an autoencoder. The process is as follows:

[0032]

[0033]

[0034] Equation (1) represents the data encoding process, and Equation (2) represents the data decoding process. (1) and W (2) This is the weight matrix. and The bias vector is obtained after training via gradient descent. For visual features recovered using an autoencoder, ReLU is the activation function, and its calculation formula is:

[0035]

[0036] The hidden vectors obtained during the training process after training the autoencoder Can be used as image features The low-dimensional representation of .

[0037] Using feature fusion methods to fuse textual and visual features to obtain multimodal features also includes:

[0038] In the task of predicting lexical specificity, the text features and the dimensionality-reduced visual features are weighted according to the imageability score of the target word. The weighted vector and the text feature vector are concatenated to form the final feature vector, which is the input vector of the graph attention network model.

[0039] In the task of predicting the imageability of words, the text features and the dimensionality-reduced visual features are weighted according to the specificity score of the target word. Similarly, the weighted vector and the text feature vector are concatenated to obtain the final feature vector, which is the input vector of the graph attention network model.

[0040] In the graph attention network model, the preprocessed feature matrix and the adjacency matrix representing the graph structure first pass through a single-head graph attention layer, where the output of this layer is processed using the Leaky ReLU activation function. Then, it passes through a multi-head graph attention layer, where the output of the second layer is processed again using the Leaky ReLU activation function. Next, it passes through a fully connected layer, where the output of this layer is processed again using the Leaky ReLU activation function. Finally, it reaches the output layer, which outputs the predicted value of the specificity or imageability of the word.

[0041] The construction of the adjacency matrix includes the following steps:

[0042] The similarity between two words is calculated using the cosine similarity method.

[0043] In the process of constructing the adjacency matrix, words with high co-occurrence probabilities are selected based on the co-occurrence matrix of words. The co-occurrence probability of two words is calculated based on the co-occurrence probability calculation method of the GloVe pre-trained model. The adjacency matrix is ​​constructed based on the similarity and co-occurrence probability between words.

[0044] The present invention also provides a rating dictionary construction system based on the prediction of linguistic attributes of English words, including a feature construction module, a feature fusion module, a prediction module and a dictionary construction module; the feature construction module is used to construct text features and visual features for target words;

[0045] The feature fusion module uses a feature fusion method to fuse textual and visual features to obtain multimodal features.

[0046] The prediction module merges the fused multimodal features to form a feature matrix X, constructs an adjacency matrix A based on word similarity and word co-occurrence probability, uses adjacency matrix A to represent the graph structure, obtains the relationship graph between words, and constructs an improved graph attention network model based on the relationship graph between words. The preprocessed feature matrix X and the adjacency matrix A representing the graph structure are input into the graph attention network model, and after processing by the graph attention network model, the predicted value of the specificity or imageability of the words is obtained. The dictionary construction module is used to construct a scoring dictionary of English words based on the predicted value of the specificity or imageability of English words.

[0047] Compared with the prior art, the present invention has at least the following beneficial effects:

[0048] This invention can construct textual and visual features that fully reflect the semantic information of target words, and effectively fuse these two types of features to obtain multimodal features with moderate dimensionality and rich semantics as input to the prediction model. This reduces the risk of overfitting during the training process and helps improve the prediction accuracy of the model. In the process of constructing the model, an adjacency matrix is ​​built based on the similarity and co-occurrence probability between words, realizing the interaction between similar words with high co-occurrence frequency. Furthermore, attention coefficients are designed based on the similarity between words, making more effective use of the association information between words, which can improve the prediction effect of lexical norms such as specificity and imageability. Even with a small sample size, it can achieve a prediction effect superior to the best existing technology, thus providing high-quality specificity and imageability scores for words outside the corpus based on a small sample corpus. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the overall process of the method described in this invention.

[0050] Figure 2 This describes the implementation process for downstream tasks of the BERT model. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The scheme described in this invention is divided into three parts: feature construction, feature fusion, and obtaining the prediction value of the specificity or imageability of words based on the improved graph attention network model. (1) This invention proposes feature construction methods that meet the corresponding conditions for cases without context and cases with context, so that the constructed text features can fully reflect the semantic information of the target word, and the constructed visual features can fully reflect the visual information in the image related to the target word.

[0053] (2) In order to better combine text features and visual features, this invention proposes a multimodal feature fusion method. This method uses an autoencoder to reduce the dimensionality of visual features, so that the dimensions of text features and visual features are consistent. Furthermore, the two types of features are weighted according to other relevant vocabulary norm scores. The weighted vector and the text feature vector are concatenated to obtain the final multimodal feature.

[0054] (3) This invention proposes an improved graph attention network model suitable for vocabulary specificity and imageability prediction tasks. In the process of constructing the model, an adjacency matrix is ​​built based on the similarity and co-occurrence probability between words to realize the interaction between similar words with high co-occurrence frequency, and attention coefficients are designed based on the similarity between words to effectively utilize the association information between words.

[0055] Figure 1 This paper illustrates the overall framework of the solution described in this invention. The overall framework of the solution remains the same in both tasks without and with context; the only difference lies in the specific methods for feature construction and feature fusion, which need to be discussed in relation to the task. Figure 1 As shown, the overall framework is divided into five modules: module (a) describes the feature construction process, module (b) describes the multimodal feature fusion process, module (c) presents the overall framework of the improved graph attention network, module (d) demonstrates the computation process of the single-head attention layer in the improved graph attention network, and module (e) demonstrates the computation process of the multi-head attention layer in the improved graph attention network. Modules (b), (c), and (d), represented by solid lines, represent the innovative aspects of this solution and will be discussed in detail later. Figure 1In this invention, module (a) is the feature construction part, which constructs text features and visual features respectively. For text features, word vectors, part-of-speech tags, sentiment polarity, semantic depth, and word suffixes are selected as features representing the semantic and attribute information of the target word. For visual features, the ResNet152 model is used to extract high-level semantic features from images related to the target word and encode them. Module (b) is the feature fusion module, one of the innovations of this invention. To ensure that the text feature vector and the visual feature vector have the same dimension, an autoencoder is used to reduce the dimensionality of the visual features. Then, a weighted operation is used to fuse these two types of features to form multimodal features. The multimodal features are then concatenated with the text features as input features to the improved graph attention network model. The improved graph attention network has three layers: a single-head graph attention layer, a multi-head graph attention layer, and a fully connected layer, as shown in module (c). The input of this network is the feature matrix X and the adjacency matrix A representing the graph structure. The calculation process of each core layer in this network will be described below.

[0056] This invention introduces a similarity-based single-head attention mechanism into the single-head graph attention layer, which is also the innovation of the improved graph attention network. Figure 1 Module (d) shows the attention coefficients of the single-head graph attention layer. The calculation process and the feature update process of the nodes, where v i v represents the word vector corresponding to node i. j This represents the word vector corresponding to node j, calculated by vector v. i sum vector v j The cosine similarity, after normalizing the calculation results using the LeakyReLU activation function, can be obtained as follows: Let node 1's neighboring nodes be node 2 and node 3, and the input features of nodes 1, 2, and 3 be respectively... The attention coefficients of node 1 in the single-headed graph attention layer, between node 1 and itself, node 2, and node 3 are respectively represented by... This indicates the features of node 1 after the first layer update. According to The result is obtained by weighted summation of the features of all neighboring nodes of node 1.

[0057] This invention introduces a multi-head attention mechanism into the multi-head graph attention layer, setting the number of attention heads to 8. Module (e) displays the attention coefficient of the r-th head in the multi-head graph attention layer. The calculation process and the feature update process of the nodes, where W (2,r) This represents the weight matrix shared by all nodes in the r-th head of the second layer. Let i and j represent the features of nodes i and j after the first layer update, respectively. By... and The calculation results are concatenated and multiplied by the weight vector of the r-th head in the second layer. After normalizing the calculation results using the LeakyReLU activation function, we can obtain... On the right side of module (e), These represent the features of nodes 1, 2, and 3 after the first layer update, respectively. These represent the attention coefficients between node 1 and itself, node 2, and node 3 in the r-th head of the multi-head graph attention layer. As shown in the diagram, there are 8 connections between node 1 and itself, and between node 2 and node 3. Taking the connection between node 1 and node 2 as an example, each connection represents the attention coefficient between node 1 and node 2 corresponding to a different attention head. The features of node 1 after the second layer update... Attention coefficients of each attention head can be used as a basis. The result is obtained by weighted summation of the features of all neighboring nodes of node 1, followed by averaging.

[0058] In the feature construction module, text features and visual features are constructed separately. For text features, word vectors, part-of-speech tags, sentiment polarity, semantic depth, and word suffixes are selected as features representing the semantic and attribute information of the target words. For visual features, a ResNet152 model is used to extract high-level semantic features from images related to the target words and encode them. In the feature fusion module, to ensure that the text feature vectors and visual feature vectors have the same dimensionality, an autoencoder is used to reduce the dimensionality of the visual features. Then, a weighted operation is used to fuse these two types of features to form multimodal features. The multimodal features are then concatenated with the text features to form the final fused features. After obtaining the fused features of all target words, a feature matrix x is formed. Based on the similarity and co-occurrence probability between target words, an adjacency matrix A is constructed. The feature matrix X and the adjacency matrix A are used as inputs to the improved graph attention network model. The improved graph attention network has three layers: a single-head graph attention layer, a multi-head graph attention layer, and a fully connected layer, as follows... Figure 1 The module (c) is shown.

[0059] The specific implementation plan is as follows: First, construct textual and visual features for the target word:

[0060] In the task of predicting the specificity and imageability of words, it is first necessary to construct text features and visual features for the target word. Then, a feature fusion method is used to fuse these two types of features. The fused multimodal features are used as input to an improved graph attention network. The predicted value of the specificity or imageability of the word is obtained through the improved graph attention network.

[0061] In the feature construction process, two types of features are considered: textual features and visual features. Based on the Dual-Coding Theory, which posits that human memory and thought can be encoded in two different ways: linguistic symbols and non-linguistic symbols. Linguistic symbols correspond to text, while non-linguistic symbols correspond to images. Specifically, when humans memorize information, if they simultaneously use linguistic and non-linguistic symbols for encoding, this information will be easier to retrieve and recall. Therefore, it can be hypothesized that when humans rate the concreteness or imagery of words, related textual and image information will be automatically recalled. Without context, a unique lexical normative score is assigned to the same word by default; with context, different lexical normative scores are assigned to the same word depending on its linguistic context.

[0062] 1) Feature construction method without considering context

[0063] In terms of text features, word vectors, word suffixes, parts of speech, sentiment polarity, and semantic depth are selected as features to represent the semantic and attribute information of words.

[0064] (1) Word vectors

[0065] Word vectors are vectorized representations of word semantics. By encoding the semantics of words, it ensures that two semantically similar words are also positioned very close in the word vector space. This invention uses FastText pre-trained word vectors as word vector features.

[0066] (2) Word suffixes

[0067] In English, affixes are added to the beginning, middle, or end of a word's root, stem, or part of the word to modify its meaning. This invention constructs features based on word suffixes. Adding specific affixes to words that can serve as word bases can form new nouns, verbs, adjectives, adverbs, etc. For example, adding specific affixes to nouns can form corresponding concrete nouns, abstract nouns, etc. Different word formation processes often give words specific semantic meanings, and the newly formed words may be related to concreteness and imagery. To determine whether each word contains an affix, words containing affixes are encoded. This invention constructs an affix extraction algorithm:

[0068]

[0069]

[0070] (3) Part of speech

[0071] Regarding part-of-speech (POS) attributes, words are categorized into six types: nouns, verbs, adjectives, prepositions, adverbs, and conjunctions. As for POS acquisition tools, Python's WordNet library can use the `pos` function to obtain multiple POS of a word and the frequency of each POS in an external corpus, while Python's nltk library's `pos_tag` function can only obtain the most common POS of a word. Considering that a word may have multiple POS, this invention uses WordNet's `pos` function to obtain the POS of a word; for words whose POS cannot be obtained using the `pos` function, nltk's `pos_tag` function is used instead. For the word w to be encoded, let... This represents the frequency of the part of speech k corresponding to the word w in an external corpus, v p This represents the initial part-of-speech vector. The encoded part-of-speech vectors are shown below.

[0072] a) Construct a 6-dimensional vector v p Each dimension contains elements representing nouns, verbs, adjectives, prepositions, adverbs, and conjunctions, with each element's initial value set to 0.

[0073] b) Try using WordNet's pos function to obtain the part of speech and frequency of occurrence of word w for different meanings. If it can be obtained, proceed to step c; otherwise, proceed to step d.

[0074] c) Based on the sum of the frequencies of all parts of speech of word w in the external corpus. Frequency of occurrence of word w for each part of speech By weighting the vectors, we can obtain a vector representing the part-of-speech attribute of each word.

[0075]

[0076] Return to step a and calculate the part-of-speech vector for the next word.

[0077] d) Use the nltk pos_tag function to obtain the part-of-speech tag of a word. Unlike the WordNet pos function, the pos_tag function can only obtain the most common part-of-speech tag of a word in the vector v. p Find the position of the element representing the part of speech, set the value of that element to 1, and leave the values ​​of the other elements unchanged. Return to step a and calculate the part-of-speech vector of the next word.

[0078] (4) Emotional polarity

[0079] The sentiment polarity of a word refers to the emotional tendency it carries, generally categorized into positive, negative, and neutral. A word's sentiment polarity can be quantified into a sentiment score, which is then used as one of the features in training. This invention uses SentiWordNet, a tool provided by Python's NLTK library, to obtain word sentiment scores. Since a word may have multiple parts of speech, and each part of speech may have multiple meanings, the sentiment score for each meaning must be considered when calculating the sentiment score.

[0080] In SentiWordNet, positive and negative scores can be obtained for each word's meaning. For word meanings that are entirely positive and have no negative connotations, the positive score ranges from (0,1], and the negative score is 0. For a neutral word meaning, the positive and negative scores may be consistent. Following this scoring rule, this invention uses the difference between the positive and negative scores as the total sentiment score for the corresponding word meaning. When calculating the total sentiment score for a word, the scores are weighted according to the frequency of each meaning's occurrence in the external corpus.

[0081] Use w i Let w represent the i-th word whose sentiment score is to be calculated. i There are m possible interpretations, represented by w ij (j=1,…,m) represents each definition of the word. To represent the positive score for each interpretation, use This represents the negative score for each interpretation. Let w represent the frequency of each definition j in the external corpus. i Total Emotional Score for:

[0082]

[0083] (5) Semantic depth

[0084] Words with high specificity possess hierarchical semantic structures built upon categorical ontological links, such as synonyms, hypernyms, and hyponyms, while words with low specificity are built upon association. Words with high specificity have richer hierarchical semantic structures, while words with low specificity, being more abstract, lack such structures. Therefore, it is hypothesized that the specificity of a word is related to its semantic depth.

[0085] As an example, in Python's WordNet library, the `min_depth` function can be used to obtain the semantic depth of each definition of a word. When calculating the total semantic depth of a word, the semantic depth is weighted according to the frequency of occurrence of each definition. Using w iLet w represent the i-th word whose semantic depth is to be calculated. i There are m possible interpretations, represented by w ij (j=1,…,m) represents each definition of the word. This indicates the semantic depth of each definition. If we represent the frequency of each definition, then the word w i Total semantic depth for:

[0086]

[0087] After constructing the above 5 types of features, they are concatenated to form the text features of the word.

[0088] In terms of visual features, the main focus is on encoding the high-level semantic features of the image. High-level semantic features refer to abstracted and semantic information related to the content of the image, which can be automatically learned and extracted by deep learning models and is typically represented as vectors or matrices. As one possible method, this invention uses ResNet152 to extract the high-level semantic features of the image.

[0089] When extracting image features using ResNet152, the input image is first converted to a size of 224×224×3. The processed result is then input into the ResNet152 network. After processing through the first network layer (conv1), the second network layer (conv2_x), the third network layer (conv3_x), the fourth network layer (conv4_x), the fifth network layer (conv5_x), and an average pooling layer, a 2048-dimensional feature vector is obtained. This invention uses this 2048-dimensional feature vector as the high-level semantic features of the image.

[0090] 2) Feature construction method combined with context

[0091] Even with context in mind, word vectors, word suffixes, parts of speech, sentiment polarity, and semantic depth are still selected as text features, and ResNet152 is used to extract image features. The construction methods for word suffixes and image features are the same as in the case without context, but the construction methods for word vectors, parts of speech, sentiment polarity, and semantic depth are different.

[0092] (1) Word vectors

[0093] This invention uses a fine-tuned BERT model to obtain word vectors containing contextual semantic information, which are then used as semantic representations of target words.

[0094] For the task of predicting lexical specificity, specifically when predicting the specificity of English words, one can refer to... Figure 2The downstream task of the BERT model is set as the lexical specificity prediction task. The input of the task is a complete sentence, and the output is the specificity score of the target word in the sentence. In the implementation process, the input sentence is first converted into a standard format represented by [CLS] tags and multiple tokens. An embedding representation is generated for each token and input into the BERT pre-trained model. After the model is trained, the final hidden state of each token represents the corresponding word or word sub-word's updated embedding. Due to the task requirements, only the embedding representation corresponding to the final hidden state of the target word is extracted and input into a linear layer to obtain a one-dimensional output, which represents the specificity score of the word. For example, assuming that Token2 corresponds to the target word w in the task, only the final hidden state T2 needs to be extracted and input into the linear layer. The output obtained can be used as the predicted specificity value of the target word w. Let the embedding representation of the target word w output by the BERT pre-trained model be... Where h is the dimension of the embedding representation, and y is the output of the downstream task, the calculation formula for the linear layer is:

[0095]

[0096] In equation (4), Let be the weight vector to be learned. Representing vectors sum vector The inner product is the sum of the element-wise multiplications of the two vectors, where b is the bias term. Furthermore, during the training of downstream tasks, the mean squared error is used as the loss function.

[0097] The lexical imageability prediction task is similar to the lexical concreteness prediction task. The only difference is that the downstream task of the BERT model is changed to the lexical imageability prediction task. The input of the task is a complete sentence, and the output is the imageability score of the target word in the sentence.

[0098] (2) Part of speech

[0099] In terms of part-of-speech (POS) attributes, words are categorized into six types: nouns, verbs, adjectives, prepositions, adverbs, and conjunctions. The Python text preprocessing library spaCy is used to perform POS tagging on each word in the sentence containing the target word and returns the POS of the target word. A 6-dimensional vector v is then constructed. p Each dimension's elements represent nouns, verbs, adjectives, prepositions, adverbs, and conjunctions, respectively. The element position that represents the part of speech of the target word is found in the vector, and that element is set to 1, while the other elements are set to 0.

[0100] (3) Emotional polarity

[0101] To better integrate with the context, the sentiment score of the sentence containing the target word and the sentiment score of the word itself are calculated separately. Then, the two are weighted and the weighted result is used as the final sentiment score of the target word.

[0102] Specifically, this invention uses the Python text processing library TextBlob to obtain the sentiment score of the sentence containing the target word. When calculating the sentiment score, WordNet is first used to obtain each definition of the target word. Next, a BERT pre-trained model is used to obtain the semantic vector of each definition, and the cosine similarity between these semantic vectors and the semantic vector of the target word is calculated. The definition with the highest similarity is then selected. Finally, SentiWordNet, a tool provided by NLTK, is used to obtain the positive and negative sentiment scores of the target word, and the difference between the two is used as the sentiment score of the corresponding definition.

[0103] Let w be the i-th word whose sentiment score is to be calculated. i The sentence containing the target word is The sentiment score for this statement is The positive and negative scores for the target word are as follows: Then the word w i Total Emotional Score for:

[0104]

[0105] In equation (5), α is a parameter that needs to be set by the user.

[0106] (4) Semantic depth

[0107] Similar to the method of obtaining the sentiment score of the corresponding word definition, the cosine similarity between the semantic vector of each definition of the target word and the semantic vector of the target word is calculated. The definition with the highest similarity is selected, and the word meaning depth corresponding to this definition is obtained by using the min_depth function of WordNet as the word meaning depth of the target word.

[0108] After constructing five types of features—word vectors, word suffixes, parts of speech, sentiment polarity, and semantic depth—they are concatenated to form the text features of a word.

[0109] II. Using feature fusion methods to fuse textual and visual features to obtain multimodal features:

[0110] Considering the different dimensions of text features and image features, an autoencoder is first used to reduce the dimensionality of image features. Since the input and output of an autoencoder are the same vector, the input data can be used as a supervisory signal to train the neural network, thereby learning a low-dimensional representation of the input vector.

[0111] Automatic encoder f θ The CCP consists of two subnetworks, one of which is used to learn the input vector. To hidden vector mapping relationship Another subnetwork is used to learn the hidden vectors. to vector mapping relationship g is the data encoding process; g takes the high-dimensional input... Encoded as a low-dimensional hidden vector h represents the data decoding process, which involves encoding the hidden vector. Decoded into high dimension Hidden Vectors The dimensions can be set by the user.

[0112] Each image's features are 2048-dimensional vectors, using... Representing image feature vectors, image features are learned using an autoencoder. The process is as follows:

[0113]

[0114]

[0115] Equation (6) describes the data encoding process, and Equation (7) describes the data decoding process. In Equations (6) and (7), W (1) and W (2) This is the weight matrix. and These are the bias vectors, which are parameters that need to be learned and can be obtained through gradient descent training. ReLU is the activation function, calculated using the following formula:

[0116]

[0117] When training the model, the squared Euclidean distance is used as the loss function, assuming the true image features... Image features recovered using an autoencoder are The formula for calculating the loss function is:

[0118]

[0119] Hidden vectors obtained during training Can be used as image features The low-dimensional representation of the text features allows this invention to determine the hidden vector based on the dimension of the text features. The dimensions are kept consistent between text features and image features. When fusing multimodal features, considering that directly concatenating features would result in excessively high dimensions and reduce the model's generalization ability, a weighted operation is used to fuse various vectors.

[0120] When predicting the specificity score of a target word, in order to incorporate more information, the text features and image features are weighted according to the imageability score of the word to obtain a weighted vector. Then, the weighted vector and the word vector are concatenated as the final input vector of the model.

[0121] Specifically, let the target word be w, and the text features of the target word be... Image features are The image features after dimensionality reduction by the autoencoder are: The imageability of the target word is represented by I. w This indicates that, in order to ensure that the weighting coefficients used in the weighting process remain within a reasonable range, a normalization operation is performed on the imageability of words. Let the maximum and minimum values ​​in the imageability score dataset be I0 and I1, respectively. max I min The final input vector for:

[0122]

[0123] In equation (10), [·||·] refers to performing a concatenation operation on the vectors on the left and right sides. and The products are multiplied by weighting coefficients, and the sum of these two weighting coefficients is 1.

[0124] When predicting the imageability score of a target word, the text features and image features can be weighted according to the specificity score of the target word to obtain a weighted vector. This weighted vector and the word vector are then concatenated as the final input vector of the model. (Using C...) w The specificity of the target word is represented by C, and the maximum and minimum values ​​in the specificity score dataset are C, respectively. max C min The final input vector is obtained. for:

[0125]

[0126] Third, the fused multimodal features are used as input to an improved graph attention network model, which then yields predicted values ​​for the specificity or imageability of words.

[0127] This invention utilizes word similarity and co-occurrence probability to construct a relationship graph between words. Let the relationship graph for a lexical specificity and imageability prediction task be G = (V, E), where each node represents a word w in the corpus. i (i = 1, 2, ..., N), where the number of nodes equals the total number of words N in the corpus. The graph structure is represented by an adjacency matrix A. If there is an edge between node i and node j, A... ij =1, otherwise A ij =0.

[0128] Next, an adjacency matrix is ​​constructed based on word similarity and word co-occurrence probability. The cosine similarity between two words is calculated using the cosine similarity method, where word w is the similarity between two words. i The word vectors are word w j The word vectors are Here, k refers to the dimension of the word vector. and The formula for calculating the similarity between them is:

[0129]

[0130] Constructing an adjacency matrix solely based on word similarity only utilizes local information between words. Considering that two highly specific or image-like words often co-occur, selecting words with high co-occurrence probabilities based on the co-occurrence matrix during adjacency matrix construction incorporates global information. This invention calculates word w based on the co-occurrence matrix X used in the GloVe pre-trained model. i And the word w j The co-occurrence probability. Let X be the co-occurrence matrix X. For the word w j In the word w i The frequency of the word w in the context i And the word w j co-occurrence probability Calculate according to the following formula:

[0131]

[0132] In equation (13), K refers to the size of the vocabulary in the general corpus used in the GloVe pre-trained model. The calculation involves all words in the general corpus at the level of word w. i The sum of frequencies appearing in the context. In the undirected graph G, according to equation (13), the co-occurrence probability between words represented by different nodes can be calculated. Based on the linear combination of the co-occurrence probability and similarity between nodes i and j, a combined score e is calculated. ij , use eij determines whether to set an adjacent edge for node i and node j. Let S be the set of words contained in the general corpus used by the GloVe model, then e ij is calculated by the formula:

[0133]

[0134] Equation (14) indicates that if word w i and word w j belong to the word set S, the same weight can be assigned to the co-occurrence probability and the similarity between node i and node j, and e is calculated after weighting ij . If word w i and word w j any one of the words does not belong to the word set S, or both words do not belong to the word set S, e is calculated only based on the similarity between the word vectors of word w i and word w j ij . Let the threshold be t, then the construction rule of the adjacency matrix is:

[0135]

[0136] In Equation (15), the threshold t is a parameter that needs to be set by oneself. Regarding the value of t, if the adjacency matrix is constructed only according to Equation (15), when the weighted value e between node i and all other nodes ij is less than the threshold t, that is, e ij <t(j=1,2,…,i-1,i+1,n), an isolated node will appear in graph G. In order to prevent this phenomenon, it is necessary to add a construction rule to the adjacency matrix, that is, when constructing adjacent edges for node i, let the maximum value of the similarity between node i and other nodes be sim i,max , and the maximum value of the co-occurrence probability between node i and other nodes be p i,max , the calculation formula is as follows:

[0137]

[0138]

[0139] When constructing the adjacent edge of node i, if e ij <t(j=1,2,…,i-1,i+1,n), then according to sim i,max and p i,max , find the node sn with the maximum similarity and co-occurrence probability with node i respectively i , pn i , at this time, the construction rules of the adjacency matrix are:

[0140] ​

[0141] This invention builds a graph network based on an attention mechanism, using a constructed graph of relationships between words, to achieve specific and image-based prediction tasks. The network structure is as follows: Figure 1 The module (c) is shown.

[0142] from Figure 1 As seen in module (c), the graph attention network model takes a preprocessed feature matrix X and an adjacency matrix A representing the graph structure as input. First, it passes through a single-head graph attention layer, where the Leaky ReLU activation function processes its output. Then, it passes through a multi-head graph attention layer, where the Leaky ReLU activation function is used to process the second layer's output. Next, it passes through a fully connected layer, where the Leaky ReLU activation function is used to process its output. Finally, it reaches the output layer. To achieve the regression task, the output dimension of the output layer is set to 1. Due to the high dimension of the input vector, the number of parameters in the weight matrices of the first three layers is very large, which may lead to overfitting when the number of nodes is small. To prevent overfitting, a dropout strategy is used after the single-head attention layer, multi-head attention layer, and fully connected layer to improve the model's generalization ability. The computational details of each layer are described below.

[0143] In the single-head attention layer, a single-head attention mechanism is introduced and applied to each node. Considering that words with high semantic similarity can learn more valuable information from each other, the cosine similarity between word vectors is used to measure the importance of neighboring nodes for message passing to the central node. Let v be the word vector corresponding to node i. i =[v 1,i ,v 2,i ,…,v M,i The word vector corresponding to node j is v. j =[v 1,j ,v 2,j ,…,v M,j ], M represents the dimension of the word vector, and the neighborhood set of node i (including node i) is . In the first layer, the attention coefficient of neighbor node j to node i is... The calculation formula is:

[0144]

[0145] Then, the attention coefficient is normalized:

[0146]

[0147] get Then, based on this, the neighborhood set is... We perform a weighted summation of the features of all nodes to obtain the features of node i after the first layer update.

[0148]

[0149] In equation (21), W (1) This is the learnable weight matrix in the first layer, shared by all nodes. The attention layer is normalized in the first layer. LeakyReLU is the activation function, which has a hyperparameter α, typically around 0.01. The activation function is calculated as follows:

[0150]

[0151] To integrate more information, a multi-head graph attention layer is constructed, introducing a multi-head attention mechanism and applying it to each node. Since the neighboring nodes of node i have already aggregated their neighboring node information in the first layer, when aggregating the neighboring node information for node i in the second layer, the information of node i's second-order neighbors can also be obtained. This network layer has a total of 8 attention heads, denoted by r (attention head number) and R (number of attention heads), where R = 8. The input to the network layer is the node representation updated by the single-head graph attention layer, and the attention coefficient of the r-th attention head in the second layer. The calculation formula is:

[0152]

[0153] In equation (23), Let r be the weight vector of the r-th head in the second layer. T This indicates the weight vector Transpose, W (2,r) It is the weight matrix shared by each node in the r-th head of the second layer. and These are the features of nodes i and j after the first layer update, respectively. The attention coefficients for each head in the second layer are also shown. Adoption and attention coefficient The same normalization method:

[0154]

[0155] Then, the features are updated based on eight different attention coefficient matrices, and the updated features are averaged to obtain the features of node i after the second layer update.

[0156]

[0157] In equation (25), W represents the normalized value of the attention coefficient between node j and node i in the r-th head of the second layer. (2,r) This represents the weight matrix shared by all nodes in the r-th head of the second layer. This represents the feature of node j after the first layer update.

[0158] Features of node i in a fully connected layer The update formula is:

[0159]

[0160] In equation (26), Let W be the feature vector of node i after being updated by the fully connected layer. T(3) This is the weight matrix. This is the bias vector. Finally, the predicted value of node i is calculated according to the following formula.

[0161]

[0162] In equation (27), W T(4) This is the weight matrix. and W T(4) Multiplying them together yields a scalar, which is then added to the bias term b. i The predicted value of node i can then be calculated.

[0163] When training the model, the mean squared error (MSE) is used as the loss function, and the true value of node i is t. i The predicted value is The formula for calculating the mean square error is:

[0164]

[0165] In equation (28), N is the total number of nodes in the graph.

[0166] Finally, a more accurate scoring dictionary for English words is constructed based on the predicted values ​​of the specificity or imageability of English words.

[0167] Based on the concept of the method described in this invention, a scoring dictionary construction system based on the prediction of English lexical linguistic attributes is also provided, including a feature construction module, a feature fusion module, a prediction module, and a dictionary construction module; the feature construction module is used to construct text features and visual features for the target word;

[0168] The feature fusion module uses a feature fusion method to fuse textual and visual features to obtain multimodal features.

[0169] The prediction module merges the fused multimodal features to form a feature matrix X, constructs an adjacency matrix A based on word similarity and word co-occurrence probability, uses adjacency matrix A to represent the graph structure, obtains the relationship graph between words, and constructs an improved graph attention network model based on the relationship graph between words. The preprocessed feature matrix X and the adjacency matrix A representing the graph structure are input into the graph attention network model, and after processing by the graph attention network model, the predicted value of the specificity or imageability of the words is obtained. The dictionary construction module is used to construct a scoring dictionary of English words based on the predicted value of the specificity or imageability of English words.

[0170] The paper "Imageability estimation using visual and language features" has disclosed a word imageability prediction method based on multimodal feature concatenation. This method can be used as an alternative to feature fusion methods to fuse text and visual features. This method uses text and speech features related to the target word as language features, and low-level and high-level semantic features from images related to the target word as visual features. The concatenated language and visual features are used as input to train a random forest model to estimate the predicted value. However, when constructing multimodal features, this method only concatenates language and visual features, resulting in a lot of redundant information in the constructed features. The feature dimension is too high, which easily leads to overfitting problems during the training of the random forest model. The random forest model used by this method cannot maximize the use of the inter-word association information during training and cannot achieve the same effect as the proposed solution. Therefore, this invention only uses it as an optional feature fusion method.

[0171] In summary, this invention provides a novel method for predicting lexical specificity and imageability. It designs a strategy for constructing both textual and visual features related to the target word and then fuses these two types of features. Based on an existing manually rated dictionary, a series of textual features are constructed according to word vectors, word suffixes, parts of speech, sentiment polarity, and semantic depth. Furthermore, an image set related to the target word is crawled, and high-level semantic features are extracted from the images as visual features related to the target word. To better combine these two types of features, an autoencoder is used to reduce the dimensionality of high-dimensional features, ensuring consistency between the dimensionality of the two types of features. The two types of features are then weighted according to the ratings of other relevant lexical standards. The weighted vector and the text feature vector are concatenated as the final input vector of the prediction model, achieving a prediction effect superior to other feature fusion methods. During the construction of the prediction model, an adjacency matrix is ​​constructed based on word similarity and co-occurrence probability to realize word interaction. An attention coefficient is designed based on word similarity to more effectively utilize the association information between words, improving the prediction effect of lexical specificity and imageability ratings and resulting in a more accurate English lexical rating dictionary.

Claims

1. A method for constructing a rating dictionary based on the prediction of linguistic attributes of English lexical words, characterized in that, Includes the following steps: Construct textual and visual features for the target word; Feature fusion methods are used to fuse textual and visual features to obtain multimodal features. The fused multimodal features are merged to form a feature matrix X. An adjacency matrix A is constructed based on word similarity and word co-occurrence probability. The adjacency matrix A represents the graph structure, and the relationship graph between words is obtained. Based on the relationship graph between words, an improved graph attention network model is constructed. The feature matrix X is preprocessed and the adjacency matrix A representing the graph structure is input into the graph attention network model. After processing by the graph attention network model, the predicted value of the specificity or imageability of English words is obtained. A rating dictionary for English words is constructed based on the predicted values ​​of the specificity or imageability of English words.

2. The method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes according to claim 1, characterized in that, When constructing text features for a target word, word vectors, word suffix vectors, part-of-speech vectors, sentiment polarity, and semantic depth features are selected, and these features are concatenated to form the text features of the target word.

3. The method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes according to claim 2, characterized in that, Constructing text features for target words includes feature construction methods without context and feature construction methods with context. The text feature construction method without context is as follows: For word vectors, FastText pre-trained word vectors are used as word vector features; For word suffixes, construct a multi-dimensional vector with an initial value of 0, extract the last, two, three, and four characters of the target word, select several of the most commonly used suffixes to form a set, when there are elements in the set that are the same as the last, two, three, and four characters, record the position of the element, and assign a value of 1 to the multi-dimensional vector at the position corresponding to the element, and finally obtain the word suffix vector; For part-of-speech vectors, a 6-dimensional vector with each element initially set to 0 is constructed. If the WordNet pos function can be used to obtain the part-of-speech and frequency of occurrence corresponding to different meanings of a word, then the frequency of occurrence of each part-of-speech of a word is weighted according to the sum of the frequencies of occurrence of all parts of speech of a word in the external corpus, and the part-of-speech vector representing the word is obtained. Then the part-of-speech vector of the next word is processed. If WordNet's pos function cannot obtain the part-of-speech and frequency of a word corresponding to different meanings, then use NLTK's pos_tag function to obtain the part-of-speech of the word. Find the element position representing the part-of-speech in the 6-dimensional vector, assign the value 1 to that element, and leave the values ​​of the other elements unchanged. Then proceed to process the part-of-speech vector of the next word; For sentiment polarity, obtain the sentiment score of a word, and consider the sentiment score of each definition of the word when calculating the sentiment score; The positive and negative scores of each word's meaning are obtained. For word meanings that are completely positive and have no negative meaning, the positive score ranges from (0,1] and the negative score is 0. For a neutral word meaning, the positive and negative scores are consistent. The difference between the positive and negative scores is used as the total sentiment score of the corresponding word meaning. The total sentiment score of a word is obtained by weighting the corresponding scores according to the frequency of each meaning in the external corpus. For semantic depth, when calculating the total semantic depth of a word, the semantic depth of each definition of the word is obtained, and the semantic depth of each definition is weighted according to the frequency of occurrence of each definition to obtain the total semantic depth of the word.

4. The method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes according to claim 3, characterized in that, The specific method for constructing text features in conjunction with context is as follows: For word vectors, a fine-tuned BERT model is used to obtain word vectors containing contextual semantic information. Specifically, the downstream task of the BERT model is set as a word specificity prediction task or a word imageability prediction task. The input of the task is a complete sentence, and the output is the specificity score of the target word in the sentence. After the BERT model is trained, the embedding representation corresponding to the final hidden state of the target word is extracted, that is, the word vector or imageability score containing contextual semantic information. For parts of speech, each word in the sentence containing the target word is tagged with its part of speech, and the part of speech of the target word is returned. Then, a 6-dimensional vector is constructed. In the 6-dimensional vector, the elements of each dimension represent nouns, verbs, adjectives, prepositions, adverbs and conjunctions, respectively. The element position that represents the part of speech of the target word is found in the vector, and the value of that element is set to 1, and the values ​​of the other elements are 0. For sentiment polarity, the sentiment score of the sentence containing the target word and the sentiment score of the word are calculated separately. Then, the two are weighted and the weighted result is used as the final sentiment score of the target word. For semantic depth, the cosine similarity between the semantic vector of each definition of the target word and the semantic vector of the target word is calculated. The definition with the highest similarity is selected, and the semantic depth corresponding to the definition with the highest similarity is obtained as the semantic depth of the target word.

5. The method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes according to claim 1, characterized in that, When constructing visual features for the target word, ResNet152 is used to extract high-level semantic features of images related to the target word as visual features of the target word.

6. The method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes according to claim 1, characterized in that, The text and visual features are fused using feature fusion methods to obtain multimodal features, including the following steps: (1) Based on the constructed text features and visual features, an autoencoder is used to reduce the dimensionality of high-dimensional features, so that the dimensionality of text features and visual features is consistent. Specifically: Automatic encoder f θ It contains two subnetworks, one of which is used to learn the input vector. To hidden vector Mapping relationship g: Another subnetwork is used to learn the hidden vectors. to vector Mapping relationship h: g is the data encoding process; g takes the high-dimensional input... Encoded as a low-dimensional hidden vector h represents the data decoding process, which involves encoding the hidden vector. Decoded into high dimension Hidden Vectors The dimensions can be set by the user; use Representing image feature vectors, image features are learned using an autoencoder. The process is as follows: Equation (1) represents the data encoding process, and Equation (2) represents the data decoding process. W (1) and W (2) This is the weight matrix. and The bias vector is obtained after training via gradient descent. For visual features recovered using an autoencoder, ReLU is the activation function, and its calculation formula is: The hidden vectors obtained during the training process after training the autoencoder Can be used as image features The low-dimensional representation of .

7. The method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes according to claim 1, characterized in that, Using feature fusion methods to fuse textual and visual features to obtain multimodal features also includes: In the task of predicting lexical specificity, the text features and the dimensionality-reduced visual features are weighted according to the imageability score of the target word. The weighted vector and the text feature vector are concatenated to form the final feature vector, which is the input vector of the graph attention network model. In the task of predicting the imageability of words, the text features and the dimensionality-reduced visual features are weighted according to the specificity score of the target word. Similarly, the weighted vector and the text feature vector are concatenated to obtain the final feature vector, which is the input vector of the graph attention network model.

8. The method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes according to claim 1, characterized in that, In the graph attention network model, the preprocessed feature matrix and the adjacency matrix representing the graph structure first pass through a single-head graph attention layer, where the output of this layer is processed using the Leaky ReLU activation function. Then, it passes through a multi-head graph attention layer, where the output of the second layer is processed again using the Leaky ReLU activation function. Next, it passes through a fully connected layer, where the output of this layer is processed again using the Leaky ReLU activation function. Finally, it reaches the output layer, which outputs the predicted value of the specificity or imageability of the word.

9. The method for constructing a rating dictionary based on the prediction of English lexical linguistic attributes according to claim 8, characterized in that, The construction of the adjacency matrix includes the following steps: The similarity between two words is calculated using the cosine similarity method. In the process of constructing the adjacency matrix, words with high co-occurrence probabilities are selected based on the word co-occurrence matrix. The co-occurrence probability of two words is calculated based on the co-occurrence probability calculation method of the GloVe pre-trained model. Construct an adjacency matrix based on word similarity and co-occurrence probability.

10. A rating dictionary construction system based on the prediction of linguistic attributes of English lexical words, characterized in that, It includes a feature construction module, a feature fusion module, a prediction module, and a dictionary construction module; the feature construction module is used to construct text features and visual features for the target word; The feature fusion module uses a feature fusion method to fuse textual and visual features to obtain multimodal features. The prediction module is used to merge the fused multimodal features to form a feature matrix X, construct an adjacency matrix A based on word similarity and word co-occurrence probability, use adjacency matrix A to represent the graph structure, obtain the relationship graph between words, and construct an improved graph attention network model based on the relationship graph between words. The feature matrix X is preprocessed and input into the graph attention network model along with the adjacency matrix A representing the graph structure. After processing by the graph attention network model, the predicted value of the specificity or imageability of the words is obtained. The dictionary building module is used to build a scoring dictionary of English words based on the predicted values ​​of the specificity or imageability of the English words.

Images