A Tibetan syntactic component annotation method
Through the Tibetan syntactic component labeling model that incorporates the prior knowledge of grid sequences, the semantic feature representation layer and CRF inference layer are used to solve the problems of incompatibility of Tibetan-dependent syntactic annotation system and the small tree library scale, and improves the Tibetan syntactic component labeling accuracy and sentence representation ability.
Patent Information
- Application Number
- CN202311230632.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-22
- Publication Date
- 2025-01-07
- Estimated Expiration
- 2043-09-22
AI Technical Summary
The existing Tibetan language dependent syntax annotation system is incompatible, and the tree library is small, resulting in low accuracy of the syntax component annotation of Tibetan language.
Using a Tibetan syntactic component labeling model that incorporates grid sequence prior knowledge, semantic embedding of Tibetan language through semantic feature representation layer, obtain word-T, word-level, part-of-speech feature representation and grid sequence constraint features, combining context coding layer and CRF Inferring the layer, perform multi-dimensional semantic representation and syntactic component annotation.
It improves the accuracy of the labeling of syntactic components in Tibetan, improves the representation ability of sentences, and achieves more efficient syntactic analysis.
Smart Images

Figure CN117172241B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of syntactic analysis, and in particular to a Tibetan syntactic component labeling method. Background Art
[0002] At present, Tibetan syntactic analysis mainly relies on phrase structure grammar and dependency grammar, and the research focuses on the deep syntactic analysis of various sentence patterns. Among them, Tibetan phrase structure syntactic analysis is mainly based on manually constructed static rules, using context-free grammar and functional unity grammar to describe the surface structure of Tibetan sentences, and constructing a small-scale Tibetan syntactic tree library on different sentence patterns such as judgment sentences and interrogative sentences. Then, rule methods such as CYK algorithm, Early algorithm, and line graph analysis method are used to realize simple Tibetan phrase structure syntactic analysis. This rule-based method can complete better syntactic analysis tasks on specific sentence patterns. However, the rule construction is highly dependent on manual work, and the quality and quantity of rules seriously affect the syntactic analysis results; the rules have poor portability, and the applicability and syntactic analysis ability of rules are weak when facing new sentence patterns. Especially in noun sentences like Tibetan, which are centered on case markers, the limitations of structural linguistics phrase structure analysis methods for syntactic analysis tasks are clearly highlighted.
[0003] The syntactic analysis method based on dependency theory is the mainstream method in the existing Tibetan syntactic analysis research. In recent years, the academic community has conducted systematic research on Tibetan dependency syntactic analysis and syntactic treebank construction methods. Among them, combining the verb-centered semantic lattice system in Tibetan grammar and the verb-centered "central knot" theory in dependency grammar, using discriminant analysis and dependency classification models as methods, certain results have been achieved in feature engineering construction, and Tibetan dependency syntactic analysis of small-scale simple sentences has been realized. However, this method mainly uses manually defined atomic features and feature combinations in feature representation, which leads to weak syntactic structure performance of complex sentences, and has limitations such as poor robustness to noise and redundant information, and limited ability to handle long-distance dependencies.
[0004] As the requirements for the scale of training corpus for deep learning increase, the academic community has gradually paid attention to the construction of large-scale Tibetan dependency syntax treebanks. A Tibetan dependency syntax annotation system with 23 categories, 25 categories, 33 categories, 5 major categories and 36 minor categories has been developed. With manual and treebank conversion methods as the main channels, dependency treebanks of different sizes have been constructed, laying a certain theoretical foundation for Tibetan syntactic analysis. However, due to different theories and classification standards, the existing dependency syntax annotation systems are incompatible and cannot be shared. The manual construction method has too high labor costs and it is difficult to form a large-scale treebank. At the same time, Tibetan has a complex grammatical structure, rich morphological markers, and insufficient syntactic treebanks. Therefore, deep syntactic analysis is difficult. The research on syntactic component annotation in English and Chinese mainly relies on pre-training models to model the contextual information of words to meet the needs of deep syntactic analysis and language understanding. For low-resource languages such as Tibetan, the method of only extracting word primitive features does not work well in the evaluation of real syntactic component annotation.
[0005] In summary, the existing Tibetan dependency syntax annotation systems are incompatible and the treebank is small, resulting in low accuracy in the annotation of Tibetan syntactic components. Summary of the invention
[0006] The embodiment of the present invention provides a Tibetan syntactic component labeling method, which solves the problem that the existing Tibetan dependency syntactic labeling system is incompatible and the tree bank is small, resulting in low accuracy of Tibetan syntactic component labeling.
[0007] The present invention provides a Tibetan syntactic component labeling method, comprising the following steps:
[0008] Obtain Tibetan sentences to be annotated;
[0009] The Tibetan language to be annotated is input into the Tibetan syntactic component annotation model that incorporates case sequence prior knowledge, and the syntactic component annotation of the Tibetan language to be annotated is performed, including:
[0010] The semantic feature representation layer is used to embed the Tibetan language to be annotated, and various feature representations are obtained, including character-level feature representation, word-level feature representation, part-of-speech feature representation, and Tibetan case sequence constraint feature representation for syntactic components.
[0011] Contextual semantic features are extracted from various feature representations through the context encoding layer to obtain various semantic feature information;
[0012] Through the CRF inference layer, a variety of different semantic feature information is inferred to obtain the syntactic component label of each word in the Tibetan sentence to be annotated.
[0013] Preferably, semantic embedding is performed on the Tibetan language to be annotated through a semantic feature representation layer to obtain a variety of different feature representations, including the following steps:
[0014] The convolutional neural network is used to embed the characters of the Tibetan sentences to be annotated, and the character-level feature representation is obtained;
[0015] The original semantic information of the word level in the Tibetan sentence to be annotated is represented by distributed vectors through Word2Vec to obtain word-level feature representation;
[0016] The original semantic information of the part of speech in the Tibetan sentence to be annotated is represented by a distributed vector through Word2Vec to obtain the part of speech feature representation;
[0017] By embedding the words of the Tibetan sentences to be annotated through prior knowledge labels, we can obtain the prior knowledge feature representation of the Tibetan case sequence constraints on syntactic components.
[0018] Preferably, embedding the characters of the Tibetan sentence to be annotated by a convolutional neural network to obtain character-level feature representation includes the following steps:
[0019] Mapping each character of the Tibetan sentence to be annotated by a character mapping function;
[0020] Convert the mapped Tibetan sentence to be annotated into a fixed-length character embedding matrix;
[0021] Perform convolution operations on the character embedding matrix to extract the local features of each character;
[0022] Perform maximum pooling operation on the local features of each character to obtain the optimal local features;
[0023] The preferred local features of each character are output through the output layer to obtain character-level feature representation.
[0024] Preferably, the convolutional neural network includes m neurons, and each neuron extracts the local features of each character as follows:
[0025] z j =ReLU(V 1:H *K j +b j )
[0026] In the formula, z j is the feature generated by the jth neuron, V 1:H is the word embedding matrix of sentence length H, b j is the bias term, * represents the convolution operation, and ReLU is the nonlinear activation function;
[0027] The preferred local features are as follows:
[0028]
[0029] In the formula, o j To optimize local features, k is a sliding window;
[0030] The character-level feature representation is as follows:
[0031] T1=[o1,o2,…,o m ]
[0032] Where T1 is the character-level feature representation.
[0033] Preferably, the word-level feature representation is as follows:
[0034]
[0035] in,
[0036]
[0037] In the formula, T2 is the word-level feature representation, w i is the i-th word, E1 is the trained Word2Vec word vector, is the first dense vector.
[0038] Preferably, the part-of-speech feature representation is as follows:
[0039]
[0040] in,
[0041]
[0042] In the formula, T3 is the part-of-speech feature representation, p i is the i-th part-of-speech tag, E2 is the trained Word2Vec part-of-speech vector, is the second dense vector.
[0043] Preferably, embedding the words of the Tibetan sentence to be annotated by prior knowledge labels to obtain the prior knowledge feature representation of the Tibetan case sequence constraints on syntactic components includes the following steps:
[0044] The Tibetan sentence to be annotated is transformed by using a priori knowledge label to obtain a syntactic component feature label sequence; the priori knowledge label is m∈{s,i,v,a,d,c,u,o}, where s represents the subject component, i represents the object, v represents the predicate component, a represents the attributive component, d represents the adverbial component, c represents the complement component, u represents the unknown component, and o represents the non-syntactic component;
[0045] Through one-hot encoding, each syntactic component feature label is represented by a distributed vector, and the prior knowledge feature representation of the lattice sequence on the syntactic component constraints is obtained.
[0046] Preferably, the prior knowledge feature representation of the lattice sequence on the syntactic component constraints is as follows:
[0047]
[0048] in,
[0049]
[0050] Where T3 is the prior knowledge feature representation of the constraints of the lattice sequence on the syntactic components, is the feature label of each syntactic component of the i-th word, E3 is the feature vector of noe-hot encoding, is the third dense vector.
[0051] Preferably, extracting contextual semantic features from a plurality of different feature representations through a context coding layer to obtain a plurality of different semantic feature information comprises the following steps:
[0052] Concatenate multiple different feature representations to obtain a fused sentence;
[0053] Input the fused sentence into the Bi-LSTM network to generate forward vector and reverse vector;
[0054] The forward vector and the reverse vector are concatenated to obtain a variety of different semantic feature information.
[0055] Preferably, the CRF inference layer is trained using maximum conditional likelihood estimation.
[0056] Compared with the prior art, the present invention has the following beneficial effects:
[0057] The present invention first needs to semantically embed the Tibetan language to be annotated through the semantic feature representation layer to obtain the Tibetan character level D feature representation, word level feature representation, part of speech feature representation and Tibetan case sequence constraint feature representation of syntactic components. The processed feature representation is then integrated into the context encoding layer to obtain contextual semantic features; finally, a variety of different semantic feature information is input into the CRF layer to infer the syntactic component label of the current word. The Tibetan syntactic component annotation model that incorporates case sequence prior knowledge proposed in the present invention performs semantic representation from a multi-dimensional semantic level, further improves the representation ability of sentences, and thus improves the accuracy of Tibetan syntactic component annotation. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0059] Figure 1 This is a structural diagram of the Tibetan syntactic component annotation model that incorporates prior knowledge of case sequences according to the present invention. DETAILED DESCRIPTION
[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0061] Reference Figure 1 The present invention proposes a Tibetan syntactic component labeling model that incorporates case sequence prior knowledge. First, it is necessary to model the semantic features of sentences that have been segmented and marked with part-of-speech tags, including Tibetan word-level feature representation, word-level feature representation, part-of-speech feature representation, and Tibetan case sequence feature representation for syntactic component constraints, and construct four different input representations. Then the processed feature representation is integrated into Bi-LSTM to obtain contextual semantic features; finally, a variety of different semantic feature information is input into the CRF layer to infer the syntactic component label of the current word. The entire model mainly consists of three parts: semantic feature representation layer, context encoding layer, and CRF inference layer.
[0062] The Tibetan syntactic component labeling model that incorporates case sequence prior knowledge proposed in the present invention performs semantic representation from a multi-dimensional semantic level, further improves the representation ability of sentences, and thus improves the accuracy of Tibetan syntactic component labeling.
[0063] Based on the above model, the present invention proposes a Tibetan syntactic component labeling method, which includes the following steps:
[0064] Step 1: Get the Tibetan sentences to be annotated.
[0065] Step 2: Input the Tibetan language to be annotated into the Tibetan syntactic component annotation model that incorporates case sequence prior knowledge, and perform syntactic component annotation on the Tibetan language sentences to be annotated.
[0066] The Tibetan language to be annotated is semantically embedded through the semantic feature representation layer to obtain a variety of different feature representations.
[0067] Semantic embedding is crucial for the annotation of Tibetan syntactic components, and the syntactic and semantic information in the language can be captured during the pre-training process. Therefore, in order to capture the semantic information of different dimensions in Tibetan sentences, the present invention maps language units of different granularities into the semantic space for embedding through a pre-training method. The semantic feature representation stage mainly maps the Tibetan sentences to be annotated into four different feature representations, namely, Tibetan character-level feature representation, word-level feature representation, part-of-speech-level feature representation, and case sequence feature representation for syntactic component constraints. Finally, the above four features are spliced as the embedding layer of the model.
[0068] (1) Character characteristics
[0069] In the field of Tibetan information processing, syllable embedding is often used due to the existence of syllables. However, in practical applications, syllable adhesion occurs in Tibetan corpus, so this method is limited by the accuracy of syllable segmentation. In addition, although some syntactic and semantic information can be captured in pre-trained word embedding, the morphological and structural information of words is usually ignored, while the state information of words is very useful for syntactic component labeling tasks. Based on this, the present invention uses a convolutional neural network (CNN) to learn the character-level features of words. The so-called character is "any form that can be segmented on the linear arrangement of Tibetan visual symbols, which can contain a single layer of consonants and all vertically superimposed letters." Through the character mapping function f:M→R d Each word in the sentence is mapped to a d-dimensional vector, and then the given sentence is converted into a word embedding matrix with a fixed length of H. Then a convolution operation is performed on the word embedding sequence, which contains m neurons, and each neuron corresponds to a convolution kernel K∈R k×d It is used to perform convolution operation on the character vector to extract the local features of the character. Assume that V 1:H is the word embedding matrix of sentence length H, and the feature generated by the jth neuron is:
[0070] z j =ReLU(V 1:H *K j +b j ) (1)
[0071] where b j is a bias term, * represents a convolution operation, and ReLU is a nonlinear activation function.
[0072] Under the action of sliding window k, the feature generated by the jth neuron is The feature is subjected to the maximum pooling operation, which is mainly used to capture the most important features with the maximum value, which is defined as:
[0073]
[0074] The final output m-dimensional feature vector of the last convolutional layer is the feature representation of the Tibetan character Ding, which is defined as:
[0075] T1=[o1,o2,…,o m ] (3)
[0076] (2) Word-level features
[0077] Words are the basic units that make up sentences, while syntactic components refer to the different parts of a sentence. Words and syntactic components are important elements in language expression. The grammatical category and semantic information of a word can determine its role and function in syntax. Therefore, the semantic features of vocabulary are the key factors affecting the labeling of syntactic components. In order to obtain deeper semantic features, the present invention uses the Word2Vec pre-training method to distribute the original semantic information at the word level and use it as one of the model embedding layer features. Given a sentence S = {w1, w2, …, w n}, for each word w i , Word2Vec is used to represent semantic information into dense vectors That is, the first eigenvector, as shown in formulas (4) to (5).
[0078]
[0079]
[0080] Among them, E1 is the trained Word2Vec word vector, and T2 is the word-level feature representation.
[0081] (2) Part of Speech Features
[0082] The correspondence between parts of speech and syntactic components is not easy to change and is a stable grammatical structure. In Tibetan, this stable structural relationship integrates parts of speech and syntactic components into a specific structural pattern. This structural pattern is complex and reflects the appearance and essential characteristics of the language from a certain perspective. Therefore, as a basic grammatical attribute of words, parts of speech are the key features of syntactic component annotation. In order to obtain deeper semantic features, the present invention learns the correspondence between parts of speech and syntactic components, combines Word2Vec to distribute parts of speech, and uses it as one of the model embedding layer features. Given a part-of-speech sequence p = {p1, p2, ..., p n}, for each part-of-speech tag p i , the sparse vector encoding is converted into a dense vector through Word2Vec That is, the first eigenvector, as shown in formulas (6) to (7).
[0083]
[0084]
[0085] Among them, E2 is the trained Word2Vec part-of-speech vector, and T3 is the part-of-speech feature representation.
[0086] (4) Embedding prior knowledge of lattice sequence
[0087] The present invention incorporates the prior knowledge of the constraints of lattice sequences on syntactic components into the model embedding layer, and learns to reconstruct them into outputs containing different semantic information together with other features, which helps the model retain the important information stored in these prior knowledge, thereby enhancing the effect of the Tibetan syntactic component labeling task.
[0088] Given a sentence S = {w1,w2,…,w n}, contains n words. For each word, there is a corresponding prior knowledge feature label m∈{s,i,v,a,d,c,u,o} that constrains the syntactic components of the lattice sequence. For each word in the sentence S There may be a priori knowledge label m, where s represents the subject component, i represents the object, v represents the predicate component, a represents the attributive component, d represents the adverbial component, c represents the complement component, u represents the unknown component, and o represents the non-syntactic component. These syntactic component feature labels are labels marked based on the priori knowledge of the Tibetan case sequence on the constraints of syntactic components (as shown in Table 1). Based on this priori knowledge, the present invention can obtain the syntactic component feature label sequence l = {N1, N2, ... N n}, for each syntactic component feature label N i , using one-hot encoding for distributed vector representation That is, the first eigenvector is integrated into the model as a priori knowledge feature, as shown in formulas (8) to (9).
[0089]
[0090]
[0091] Where E3 is the feature vector of the noe-hot encoding used, and T4 is the prior knowledge feature representation of the lattice sequence constraints on syntactic components.
[0092] Table 1. Constraints of Tibetan case sequences on syntactic components
[0093]
[0094] The context encoding layer is used to extract contextual semantic features from a variety of feature representations to obtain a variety of different semantic feature information.
[0095] The present invention has obtained the semantic information features of different dimensions such as characters, words and parts of speech and the prior knowledge of the constraints of case sequences on syntactic components. Next, these four feature representations are fused together as the embedding layer of the model and input into the encoding layer. The present invention selects the most intuitive fusion method to connect the four different features in series, as shown in formula (10).
[0096] e i =concat([T1,T2,T3,T4]) (10)
[0097] The present invention represents a sentence that integrates multiple features X = {e1, e2, ..., e n} as the input of the Bi-LSTM network. The Bi-LSTM network consists of forward and backward LSTM units, which have the ability to capture past and future contextual information when modeling sentences. An LSTM unit consists of three gate structures, which control the proportion of information to be forgotten and the proportion of information passed to the next time step. Formally, the formula for updating an LSTM unit at time t is shown as follows (11) to (16):
[0098] i t =σ(W i e t-1 +U i h t +b i ) (11)
[0099] f t =σ(W f e t-1 +U f h t +b f ) (12)
[0100]
[0101]
[0102] o t =σ(W o e t-1 +U o h t +b o ) (15)
[0103] h t =o t ⊙tanh(c t ) (16)
[0104] where i t ,f t ,o tThey represent the three gate structures in the LSTM network structure, e t is the input fusion feature vector at time t, h t is the hidden state (also called output) vector, storing all useful information at time t (and before). σ represents the sigmoid function. ⊙ represents the Hadamard product, and W, U, b are learnable parameters. The forward vector generated by the Bi-LSMT network and the reverse vector And n represents the length of the sentence, and finally and The result of stitching is a sentence-level latent representation vector, which contains the global contextual semantic information of the sentence.
[0105] Through the CRF inference layer, a variety of different semantic feature information is inferred to obtain the syntactic component label of the current word.
[0106] It is beneficial to consider the correlation between syntactic component tokens and jointly decode the best token sequence for a given input sentence. For example, in a syntactic component, an object is more likely to be followed by a case particle. Therefore, a conditional random field (CRF) is used to jointly model the syntactic component token sequence, taking into account the dependencies between consecutive tokens, instead of decoding each label independently.
[0107] In form, the present invention uses z = {h1, h2 ..., h n} to represent a general input sequence, where z i is the input vector of the i-th word. n} represents the general tag sequence of z. y(z) represents the set of possible tag sequences of z. The probability model of sequence CRF defines a conditional probability p(y|z; W,b) on all possible tag sequences y given z, which is in the following form:
[0108]
[0109] in is the potential function, and They correspond to the label pairs The weight vector and bias of .
[0110] For CRF training, the maximum conditional likelihood estimation is used. For a training set {(z i ,y i )}, the logarithm of the likelihood (also called log-likelihood) is calculated by the following formula:
[0111] L(W,b)=∑i logp(y|z;W,b) (18)
[0112] Maximum likelihood training selects parameters to maximize the log-likelihood L(W,b).
[0113] Decoding is to find the syntactic component label sequence y with the highest conditional probability * .
[0114] y * = argmax y∈y(z) p(y|z;W,b) (19)
[0115] For a sequential CRF model (considering only the interaction between two consecutive labels), training and decoding can be effectively solved by adopting the Viterbi algorithm.
[0116] Example
[0117] In order to verify the effectiveness of the present invention, 21,455 sentences were first extracted from Tibetan word segmentation annotated corpus covering a variety of themes such as novels, news, and ancient books and documents. Among them, 8,205 sentences were manually annotated with syntactic components in the BRAT annotation tool, and the other 13,250 sentences were constructed using the "rule annotation + manual proofreading" model. After constructing the data set, we divided the data into a training set, a test set, and a validation set at a ratio of 8:1:1, as shown in Table 2.
[0118] Table 2 Dataset statistics
[0119]
[0120] The syntactic component markers of Tibetan sentences in this experimental data set include 9 categories of markers, namely subject (sub), object (iob), object involved (vob), main predicate (hed), secondary predicate (pre), attributive (atr), adverbial (adv), complement (com) and non-syntactic component (unk) markers (non-syntactic components mainly include various Tibetan function words).
[0121] In order to verify the performance of the Tibetan syntactic component tagging method of the present invention, three primitive features, syllable and word, word + part of speech, were selected in the experiment, and a comparative experiment was carried out in combination with a commonly used sequence tagging model.
[0122] Table 3 Comparative experiment
[0123]
[0124] According to Table 3, the WPCc_BiLSTM+CRF model proposed in the present invention reaches the highest level in all four indicators, which are 90.67%, 87.00%, 87.33% and 87.16% respectively.
[0125] The Tibetan syntactic component annotation method that incorporates prior knowledge of case sequences can produce multiple beneficial effects and economic benefits in the field of language analysis.
[0126] Improve the performance of natural language processing (NLP) systems: By accurately annotating Tibetan syntactic components, it can help NLP systems better understand and process Tibetan natural language. This helps to improve the accuracy and efficiency of NLP systems, thereby improving the results of tasks such as search, translation, and speech recognition.
[0127] Improve the efficiency of Tibetan language education: Syntactic component annotation technology can be used in the development of Tibetan language learning software and tools. By analyzing the structure and grammar of sentences, these tools can provide personalized learning content and feedback to help learners master the Tibetan language faster.
[0128] Promoting cross-cultural communication: In a multilingual environment, Tibetan syntactic component annotation technology can help overcome language and cultural barriers and improve the efficiency and accuracy of communication.
[0129] Generate economic benefits: Tibetan syntactic component annotation technology is also widely used in the business field, including market analysis, automatic text generation, customer service system, etc. By analyzing consumer behavior and needs, companies can develop more effective marketing strategies and provide better customer service.
[0130] In general, Tibetan syntactic component annotation technology has important application value in the fields of language analysis and machine learning, and can produce practical benefits in many fields such as education, business, and cross-cultural communication.
[0131] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0132] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A Tibetan syntactic component labeling method, characterized in that: The following steps are involved: Obtain Tibetan sentences to be annotated; The Tibetan language to be annotated is input into the Tibetan syntactic component annotation model that incorporates case sequence prior knowledge, and the syntactic component annotation of the Tibetan language to be annotated is performed, including: The semantic feature representation layer is used to embed the Tibetan language to be annotated, and different feature representations are obtained, including character-level feature representation, word-level feature representation, part-of-speech feature representation, and Tibetan case sequence constraint feature representation for syntactic components. Contextual semantic features are extracted from various feature representations through the context encoding layer to obtain various semantic feature information; Through the CRF inference layer, a variety of different semantic feature information is inferred to obtain the syntactic component label of each word in the Tibetan sentence to be annotated.
2. A Tibetan syntactic component labeling method as claimed in claim 1, characterized in that: The semantic feature representation layer is used to embed the Tibetan language to be annotated, and various feature representations are obtained, including the following steps: The convolutional neural network is used to embed the characters of the Tibetan sentences to be annotated, and the character-level feature representation is obtained; The original semantic information of the word level in the Tibetan sentence to be annotated is represented by distributed vectors through Word2Vec to obtain word-level feature representation; The original semantic information of the part of speech in the Tibetan sentence to be annotated is represented by a distributed vector through Word2Vec to obtain the part of speech feature representation; By embedding the words of the Tibetan sentences to be annotated through prior knowledge labels, we can obtain the prior knowledge feature representation of the Tibetan case sequence constraints on syntactic components.
3. A Tibetan syntactic component labeling method as claimed in claim 2, characterized in that: The convolutional neural network is used to embed the characters of the Tibetan sentence to be annotated to obtain character-level feature representation, including the following steps: Mapping each character of the Tibetan sentence to be annotated by a character mapping function; Convert the mapped Tibetan sentence to be annotated into a fixed-length character embedding matrix; Perform convolution operations on the character embedding matrix to extract the local features of each character; Perform a maximum pooling operation on the local features of each character to obtain the local features after the operation; The local features of each character after operation are output through the output layer to obtain character-level feature representation.
4. A Tibetan syntactic component labeling method as claimed in claim 3, characterized in that: The convolutional neural network includes m neurons, and each neuron extracts the local features of each character as follows: z j =ReLU(V 1:H *K j +b j ) In the formula, z j is the feature generated by the jth neuron, V 1:H is the word embedding matrix of sentence length H, b j is the bias term, * represents the convolution operation, and ReLU is the nonlinear activation function; The local features after the operation are as follows: In the formula, o j is the local feature after operation, k is the sliding window; The character-level feature representation is as follows: T1=[o1,o2,…,o m ] Where T1 is the character-level feature representation.
5. A Tibetan syntactic component labeling method as claimed in claim 2, characterized in that: The word-level feature representation is as follows: in, In the formula, T2 is the word-level feature representation, w i is the i-th word, E1 is the trained Word2Vec word vector, is the first dense vector.
6. A Tibetan syntactic component labeling method as claimed in claim 2, characterized in that: The part-of-speech feature representation is as follows: in, In the formula, T3 is the part-of-speech feature representation, p i is the i-th part-of-speech tag, E2 is the trained Word2Vec part-of-speech vector, is the second dense vector.
7. A Tibetan syntactic component labeling method as claimed in claim 2, characterized in that: The words of the Tibetan sentences to be annotated are embedded by prior knowledge labels to obtain the prior knowledge feature representation of the Tibetan case sequence constraints on syntactic components, including the following steps: The Tibetan sentence to be annotated is transformed by using a priori knowledge label to obtain a syntactic component feature label sequence; the priori knowledge label is m∈{s,i,v,a,d,c,u,o}, where s represents the subject component, i represents the object, v represents the predicate component, a represents the attributive component, d represents the adverbial component, c represents the complement component, u represents the unknown component, and o represents the non-syntactic component; Through one-hot encoding, each syntactic component feature label is represented by a distributed vector, and the prior knowledge feature representation of the lattice sequence on the syntactic component constraints is obtained.
8. A Tibetan syntactic component labeling method as claimed in claim 7, characterized in that: The prior knowledge feature representation of the lattice sequence constraints on syntactic components is as follows: in, Where T4 is the prior knowledge feature representation of the lattice sequence constraints on syntactic components, N i is the feature label of the ith syntactic component, E3 is the feature vector of noe-hot encoding, is the third dense vector.
9. The Tibetan syntactic component labeling method according to claim 1, characterized in that: Contextual semantic features are extracted from various feature representations through the context coding layer to obtain various semantic feature information, including the following steps: Concatenate different feature representations to obtain a fused sentence; Input the fused sentence into the Bi-LSTM network to generate forward vector and reverse vector; The forward vector and the reverse vector are concatenated to obtain a variety of different semantic feature information.
10. The Tibetan syntactic component labeling method according to claim 1, characterized in that: The CRF inference layer is trained using maximum conditional likelihood estimation.
Citation Information
Patent Citations
United labeling method for syntax of Tibet language and semantic roles
CN103440236A
Syllabic size based method and apparatus for identifying Tibetan syntax chunk
CN105426355A