A method for constructing Vietnamese dependency syntactic treebank based on joint optimization training

Abstract

The present invention relates to a method for constructing a Vietnamese dependency syntax tree bank based on joint optimization training, and belongs to the field of natural language processing. The present invention comprises: using Chinese, English and Vietnamese experimental data to respectively train three bi-affine dependency syntax analysis models, which are a Chinese model, an English model and a Vietnamese model. The three trained bi-affine dependency syntax analysis models are used to parse the dependency syntax tree corresponding to the Vietnamese unlabeled text, and the results of the Vietnamese model analysis are used as a reference standard to screen the parsing results of the Chinese model and the English model. The three bi-affine dependency syntax analysis models are re-iteratively optimized using the screened data and further data selection is performed, thereby constructing a high-quality Vietnamese dependency syntax tree bank. This method can alleviate the problem of scarcity of Vietnamese dependency syntax data, thereby significantly improving the performance of Vietnamese dependency syntax analysis.

Description

Technical Field

[0001] The invention relates to a Vietnamese dependency syntactic tree library construction method based on joint optimization training, and belongs to the technical field of natural language processing. Background Art

[0002] With the deepening development of globalization, exchanges and cooperation between China and Southeast Asian countries, especially Vietnam, are becoming increasingly frequent. In this process, language plays an indispensable role as a bridge for cultural and information exchange. Vietnamese, as a language with deep historical roots in China, can not only promote cultural exchanges and understanding between the peoples of the two countries, but also lay a solid foundation for promoting regional economic cooperation through in-depth learning and mastering of this language.

[0003] However, cross-language natural language processing (NLP) problems, especially dependency parsing, still face many technical challenges. Dependency parsing is one of the core tasks in natural language processing, which aims to parse the grammatical structure of sentences and reveal the dependency relationship between words. This process has an important impact on advanced applications such as machine translation, information retrieval, and sentiment analysis. Especially in Chinese-Vietnamese machine translation, dependency syntactic analysis of Vietnamese is crucial to improving translation results, and the accuracy of the syntactic structure directly affects the readability and accuracy of the translation results.

[0004] However, as a resource-scarce language, Vietnamese faces the dilemma of a lack of high-quality annotated data. Compared with other high-resource languages ​​such as English and Chinese, Vietnamese has its own unique grammatical structure, which poses a great challenge to the generalization ability of dependency syntax models. When dealing with Vietnamese, traditional cross-language transfer methods often cause model performance to degrade due to differences between languages. Therefore, how to build an efficient and accurate Vietnamese dependency syntax tree bank has become a major problem in the field of natural language processing.

[0005] In this context, the present invention proposes a method for constructing a Vietnamese dependency syntactic treebank with joint optimization training. The method aims to construct an efficient and accurate Vietnamese dependency syntactic treebank by making full use of the rich language resources and corpora of Chinese and English, combined with the language characteristics of Vietnamese. Although Vietnamese, Chinese and English belong to different language families, there are some structural similarities between them. For example, Vietnamese and Chinese are both isolating languages ​​and have similarities in syntactic structure; and in the process of modernization, Vietnamese has absorbed many English words and expressions. These similarities provide strong support for cross-language transfer learning.

[0006] The research method of the present invention not only relies on a single language model, but also screens out high-quality Vietnamese dependency syntax trees through joint optimization training of three models of Chinese, English and Vietnamese, and then constructs a syntax tree library. Specifically, the present invention first uses a large amount of annotated Chinese and English data to fine-tune the multilingual pre-trained language model XLM-RoBERTa to make it better adapt to the syntactic structures of different languages. Then, the present invention jointly trains the Vietnamese data with these two models, and through repeated optimization and adjustment, screens out Vietnamese dependency syntax trees that are similar to the Chinese and English syntactic structures and have high performance. This multi-language-based joint optimization strategy not only improves the construction efficiency of the Vietnamese dependency syntax tree library, but also greatly reduces the dependence on manual annotation.

[0007] Experimental results show that the joint optimization training strategy significantly improves the accuracy and stability of Vietnamese dependency parsing, especially when dealing with complex long sentence structures. By effectively utilizing cross-language similarities, the joint optimization training method provides a new solution for dependency parsing in Vietnamese and other low-resource languages, and lays a solid foundation for further natural language processing tasks such as machine translation and semantic analysis.

[0008] In summary, the joint optimization training method not only provides innovative ideas for building Vietnamese dependency syntax treebanks, but also accumulates valuable experience for grammatical analysis research in other low-resource languages. Through continuous optimization and improvement, this method is expected to be applied and promoted in a wider range of cross-language natural language processing tasks, thereby promoting the development of multilingual technology and promoting communication and understanding between different cultures. Summary of the invention

[0009] The present invention provides a Vietnamese dependency syntax treebank construction method based on joint optimization training, which can construct an efficient and accurate Vietnamese dependency syntax treebank, improve the performance of Vietnamese dependency syntax analysis, and is used to solve the problem of scarce Vietnamese dependency syntax treebank resources. It also provides a reference for natural language processing tasks of other low-resource languages.

[0010] The technical solution of the present invention is: a method for constructing a Vietnamese dependency syntax tree bank based on joint optimization training, the method comprising:

[0011] Step 1: Collect Chinese, English and Vietnamese corpora as experimental data and perform preprocessing;

[0012] Step 2: Load the experimental data into the multilingual pre-trained language model, and use the multilingual pre-trained language model (XLM-RoBERTa) to enhance the representation ability of the original bi-affine dependency parsing model (BiAffine Parser); use the Chinese, English and Vietnamese experimental data to train three bi-affine dependency parsing models respectively; after the training is completed, save the three obtained bi-affine dependency parsing models and name them Chinese model (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi );

[0013] Step 3: Collect new unlabeled Vietnamese datasets and build a Vietnamese dependency syntax tree library;

[0014] Step 4. Use the Chinese model (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi ) Parse the constructed Vietnamese dependency syntactic treebank corpus to generate a pseudo-dependency syntactic treebank;

[0015] Step 5: Use the pseudo-dependency syntactic treebank generated by the Vietnamese model as the standard to select high-quality pseudo-dependency syntactic treebanks generated by the Chinese model and the English model;

[0016] Step 6. Mix the selected high-quality pseudo-dependency syntactic treebank with the Chinese, English and Vietnamese corpora in the original experimental data, retrain the double affine dependency syntactic analysis model, and use the trained double affine dependency syntactic analysis model to construct the Vietnamese dependency syntactic treebank.

[0017] Furthermore, the Step 1 includes:

[0018] We downloaded labeled universal dependency syntactic treebanks of Chinese, English and Vietnamese from the Universal Dependencies (UD) dataset as experimental data, segmented the experimental data, and then processed them into dependency syntactic trees that conform to the CoNLL-U format based on the segmentation results.

[0019] Furthermore, the Step 2 includes:

[0020] Step 2.1, load the processed experimental data into the multi-language pre-trained language model for parameter fine-tuning;

[0021] Step 2.2, use the multi-language pre-trained language model (XLM-RoBERTa) for feature extraction;

[0022] Step 2.3: Weighted combination of extracted features to form feature representation rep XLM-R ;

[0023] Step 2.4: Use the character-level bidirectional long short-term memory network Char-BiLSTM to encode the characters of each word and get the character representation ;

[0024] Step 2.5: Represent the feature rep XLM-R With randomly initialized word embeddings Combine to form word representation and combine with character representation Combined to form a word vector representation X i ; The calculation formula is as follows:

[0025] ;

[0026] Step 2.6, represent the word vectors of Chinese, English and Vietnamese as X i Input them into the BiAffine Parser model for training respectively;

[0027] Step 2.7. After the training is completed, save the dual affine dependency parsing models and name them Chinese models (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi ).

[0028] Furthermore, the Step 2.3 includes:

[0029] The extracted features are weighted and combined to form a feature representation, where the features of the last 4 layers of the multilingual pre-trained language model are denoted as H i1 (i1∈[1,4]), the trainable weight parameter is S i1 , γ is the scaling parameter; first, the weight parameters of each layer are normalized by the Softmax function, and then the normalized weight parameters are multiplied by the corresponding feature representation to obtain the weighted feature representation of each layer; finally, the weighted feature representations of each layer are summed to obtain the final feature representation rep XLM-R ; The calculation process is as follows:

[0030] ;

[0031] .

[0032] Furthermore, Step 2.6 and Step 2.7 include:

[0033] A three-layer BiLSTM encoder is used to generate contextualized representations of each word vector in the sentence { }; The output vector of each BiLSTM Input into two multi-layer perceptrons MLP respectively to extract two low-dimensional syntactic vector dependent words and The correct core word ; Then use biaffine to calculate dependent words and The correct core word The dependency relationship between them is ArcScore( j); additional MLP and biaffine operations are also used to calculate the label scores between each word pair , which is used to evaluate the grammatical relationship type of each dependency arc; the bi-affine dependency syntactic analysis model also defines a local cross entropy loss function for each position i; finally, the trained bi-affine dependency syntactic analysis models are saved separately and named as Chinese model, English model and Vietnamese model respectively;

[0034] ArcScore The calculation formula of j) is: Arc Score ( i ← j )= [ r i d 1 ] T U 1 r j h ;

[0035] in, It is a dependent word The vector of yes The correct core word A vector of , U1 is a biaffine matrix used to calculate the dependency arc;

[0036] The definition of the dependency parsing loss function used by the local cross entropy loss function is as follows:

[0037]

[0038] Among them, w j Represents the dependent word w i The correct core word, l represents the dependent word and The correct core word The type of dependency relationship between them.

[0039] Furthermore, the Step 3 includes:

[0040] Step 3.1, download the unlabeled Vietnamese dataset from the general dataset ALT (Asian Language Treebank) as new experimental data;

[0041] Step 3.2. Use the underthesea word segmentation tool to segment and deduplicate the Vietnamese dataset, and manually mark it to build a Vietnamese dependency syntactic tree library.

[0042] Furthermore, the Step 4 includes:

[0043] Step 4.1. Input the corpus in the constructed Vietnamese dependency syntax tree library into the three fine-tuned Chinese models (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi ) for analysis;

[0044] Step 4.2, save the Vietnamese pseudo-dependency syntactic tree library obtained by parsing the Chinese model, English model and Vietnamese model.

[0045] Furthermore, the Step 5 includes:

[0046] Step 5.1. Using the Vietnamese model (Parser Vi ) as the standard to select the Chinese model (Parser Ch ) and the English model (Parser En ) Generate an initial high-quality pseudo-dependency syntactic tree bank;

[0047] Step 5.2, filter out high-quality pseudo-dependency syntactic treebanks according to the rules;

[0048] Among them, the screening is performed by comparing the number of labels and head nodes in the dependency syntactic trees generated by the Chinese model, the English model, and the Vietnamese model; the same number of labels and head nodes are recorded as T, the total number is recorded as A, and then the ratio of the two is calculated to obtain the Correct Rate; the dependency syntactic trees are screened according to the size of the scores, and the high-quality dependency syntactic trees in the Chinese model and the English model are selected, and they are used to form a high-quality pseudo-dependency syntactic tree library; the specific score ratio calculation method is as follows:

[0049] .

[0050] Furthermore, the Step 6 includes:

[0051] Step 6.1, mix the selected high-quality pseudo-dependency syntactic treebank with the Chinese, English and Vietnamese corpora in the original experimental data;

[0052] Step 6.2: Load the mixed processed data into the dual affine dependency syntactic analysis model for model training. The steps are the same as Step 2. Save the trained model and use the trained dual affine dependency syntactic analysis model to construct the Vietnamese dependency syntactic tree bank.

[0053] The present invention also provides a Vietnamese dependency syntax treebank construction system based on joint optimization training, the system comprising: a module for executing the above-mentioned Vietnamese dependency syntax treebank construction method based on joint optimization training.

[0054] The beneficial effects of the present invention are:

[0055] 1. The present invention significantly reduces the need for a large amount of manually annotated data by effectively utilizing the rich resources of Chinese and English and combining the language similarity of Vietnamese, thereby reducing the time and cost of developing a Vietnamese dependency parsing model;

[0056] 2. The present invention uses the similarities between the Chinese and English models and Vietnamese to enhance the generalization ability of the model in a multilingual environment, enabling it to achieve higher accuracy and stability in Vietnamese dependency syntax analysis, especially when dealing with complex syntactic structures;

[0057] 3. The Vietnamese dependency syntactic treebank construction method based on joint optimization training proposed in this invention not only effectively improves the performance of Vietnamese natural language processing tasks, but also provides a useful reference for the processing of other low-resource languages, opening up a new direction for the development of the field of natural language processing;

[0058] 4. The present invention provides technical support for the language and cultural exchanges between China and Vietnam by constructing a Vietnamese dependency syntax tree library, while promoting the application and development of multilingual technology and helping to improve the use and understanding of Vietnamese in the international market. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flow chart of the present invention; DETAILED DESCRIPTION

[0060] Example 1: Figure 1 As shown, a method for constructing a Vietnamese dependency syntactic tree bank based on joint optimization training, the method comprising:

[0061] Step 1: Collect Chinese, English and Vietnamese corpora as experimental data and perform preprocessing;

[0062] Furthermore, the Step 1 includes:

[0063] Download the labeled Chinese (UD) from the Universal Dependencies dataset Ch ), English (UD En ) and Vietnamese (UD Vi ) is used as experimental data, the experimental data is segmented, and then processed into a dependency syntax tree that conforms to the CoNLL-U format based on the segmentation results.

[0064] Step 2: Load the experimental data into the multilingual pre-trained language model, and use the multilingual pre-trained language model (XLM-RoBERTa) to enhance the representation ability of the original bi-affine dependency parsing model (BiAffine Parser); use the Chinese, English and Vietnamese experimental data to train three bi-affine dependency parsing models respectively; after the training is completed, save the three obtained bi-affine dependency parsing models and name them Chinese model (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi );

[0065] Furthermore, the Step 2 includes:

[0066] Step 2.1, load the processed experimental data into the multi-language pre-trained language model for parameter fine-tuning;

[0067] Step 2.2, use the multi-language pre-trained language model (XLM-RoBERTa) for feature extraction;

[0068] Step 2.3: Weighted combination of extracted features to form feature representation rep XLM-R ;

[0069] Step 2.4: Use the character-level bidirectional long short-term memory network Char-BiLSTM to encode the characters of each word and get the character representation ;

[0070] Step 2.5: Represent the feature rep XLM-R With randomly initialized word embeddings Combine to form word representation and combine with character representation Combined to form a word vector representation X i2 ; The calculation formula is as follows:

[0071] ;

[0072] Step 2.6, represent the word vectors of Chinese, English and Vietnamese as X i2Input them into the BiAffine Parser model for training respectively;

[0073] Step 2.7. After the training is completed, save the dual affine dependency parsing models and name them Chinese models (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi ).

[0074] Furthermore, the Step 2.3 includes:

[0075] The extracted features are weighted and combined to form a feature representation, where the features of the last 4 layers of the multilingual pre-trained language model are denoted as H i1 (i1∈[1,4]), the trainable weight parameter is S i1 , γ is the scaling parameter; first, the weight parameters of each layer are normalized by the Softmax function, and then the normalized weight parameters are multiplied by the corresponding feature representation to obtain the weighted feature representation of each layer; finally, the weighted feature representations of each layer are summed to obtain the final feature representation rep XLM-R ; The calculation process is as follows:

[0076] ;

[0077] .

[0078] Furthermore, Step 2.6 and Step 2.7 include:

[0079] A three-layer BiLSTM encoder is used to generate contextualized representations of each word vector in the sentence { }; The output vector of each BiLSTM Input into two multi-layer perceptrons MLP respectively to extract two low-dimensional syntactic vector dependent words and The correct core word ; Then use biaffine to calculate dependent words and The correct core word The dependency relationship between them is ArcScore( j); additional MLP and biaffine operations are also used to calculate the label scores between each word pair , which is used to evaluate the grammatical relationship type of each dependency arc; the bi-affine dependency syntactic analysis model also defines a local cross entropy loss function for each position i; finally, the trained bi-affine dependency syntactic analysis models are saved separately and named as Chinese model, English model and Vietnamese model respectively;

[0080] ArcScore The calculation formula of j) is: Arc Score ( i ← j )= [ r i d 1 ] T U 1 r j h ;

[0081] in, It is a dependent word The vector of yes The correct core word A vector of , U1 is a biaffine matrix used to calculate the dependency arc;

[0082] The definition of the dependency parsing loss function used by the local cross entropy loss function is as follows:

[0083]

[0084] Among them, w j Represents the dependent word w i The correct core word, l represents the dependent word and The correct core word The type of dependency relationship between them.

[0085] Step 3: Collect new unlabeled Vietnamese datasets and build a Vietnamese dependency syntax tree library;

[0086] Furthermore, the Step 3 includes:

[0087] Step 3.1, download the unlabeled Vietnamese dataset from the general dataset ALT (Asian Language Treebank) as new experimental data;

[0088] Step 3.2. Use the underthesea word segmentation tool to segment and deduplicate the Vietnamese dataset, and manually mark it to build a Vietnamese dependency syntactic tree library.

[0089] Step 4. Use the Chinese model (Parser Ch )、English Model(ParserEn ) and Vietnamese model (Parser Vi ) Parse the constructed Vietnamese dependency syntactic treebank corpus to generate a pseudo-dependency syntactic treebank;

[0090] Furthermore, the Step 4 includes:

[0091] Step 4.1. Input the corpus in the constructed Vietnamese dependency syntax tree library into the three fine-tuned Chinese models (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi ) for analysis;

[0092] Step 4.2, save the Vietnamese pseudo-dependency syntactic tree library obtained by parsing the Chinese model, English model and Vietnamese model.

[0093] Step 5: Use the pseudo-dependency syntactic treebank generated by the Vietnamese model as the standard to select high-quality pseudo-dependency syntactic treebanks generated by the Chinese model and the English model;

[0094] Furthermore, the Step 5 includes:

[0095] Step 5.1. Using the Vietnamese model (Parser Vi ) as the standard to select the Chinese model (Parser Ch ) and the English model (Parser En ) Generate an initial high-quality pseudo-dependency syntactic tree bank;

[0096] Step 5.2, filter out high-quality pseudo-dependency syntactic treebanks according to the rules;

[0097] Among them, the screening is performed by comparing the number of labels and head nodes in the dependency syntactic trees generated by the Chinese model, the English model, and the Vietnamese model; the same number of labels and head nodes are recorded as T, the total number is recorded as A, and then the ratio of the two is calculated to obtain the Correct Rate; the dependency syntactic trees are screened according to the size of the scores, and the high-quality dependency syntactic trees in the Chinese model and the English model are selected, and they are used to form a high-quality pseudo-dependency syntactic tree library; the specific score ratio calculation method is as follows:

[0098] .

[0099] Step 6. Mix the selected high-quality pseudo-dependency syntactic treebank with the Chinese, English and Vietnamese corpora in the original experimental data, retrain the double affine dependency syntactic analysis model, and use the trained double affine dependency syntactic analysis model to construct the Vietnamese dependency syntactic treebank.

[0100] Furthermore, the Step 6 includes:

[0101] Step 6.1, mix the selected high-quality pseudo-dependency syntactic treebank with the Chinese, English and Vietnamese corpora in the original experimental data;

[0102] Step 6.2: Load the mixed processed data into the dual affine dependency syntactic analysis model for model training. The steps are the same as Step 2. Save the trained model and use the trained dual affine dependency syntactic analysis model to construct the Vietnamese dependency syntactic tree bank.

[0103] The present invention also provides a Vietnamese dependency syntax treebank construction system based on joint optimization training, the system comprising:

[0104] The collection and preprocessing module is used to collect Chinese, English and Vietnamese corpora as experimental data and perform preprocessing;

[0105] Three bi-affine dependency parsing model building modules are used to load experimental data into the multilingual pre-trained language model and use the multilingual pre-trained language model (XLM-RoBERTa) to enhance the representation ability of the original bi-affine dependency parsing model (BiAffine Parser); use Chinese, English and Vietnamese experimental data to train three bi-affine dependency parsing models respectively; after the training is completed, save the three obtained bi-affine dependency parsing models respectively and name them as Chinese model (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi );

[0106] Vietnamese dependency syntax treebank construction module, which is used to collect new unlabeled Vietnamese datasets and build a Vietnamese dependency syntax treebank;

[0107] Pseudo-dependency syntactic treebank generation module for using the Chinese model (Parser Ch )、English Model(Parser En ) and Vietnamese model (Parser Vi ) Parse the constructed Vietnamese dependency syntactic treebank corpus to generate a pseudo-dependency syntactic treebank;

[0108] A high-quality pseudo-dependency syntactic treebank generation module is used to filter out high-quality pseudo-dependency syntactic treebanks generated by Chinese and English models using the pseudo-dependency syntactic treebank generated by the Vietnamese model as a standard;

[0109] The dependency syntax treebank construction module is used to mix the screened high-quality pseudo-dependency syntax treebank with the Chinese, English and Vietnamese corpora in the original experimental data, retrain the bi-affine dependency syntax analysis model, and use the trained bi-affine dependency syntax analysis model to construct the Vietnamese dependency syntax treebank.

[0110] The new model was evaluated using the test set, and LAS (Labeled Attachment Score) and UAS (Unlabelled Attachment Score) were analyzed. Through the evaluation, it was found that the model trained with mixed corpus was far superior to the model trained with the original experimental data. The specific formula used is as follows:

[0111] (1) LAS predicts the accuracy of the model in correctly predicting dependency relationships and dependency labels;

[0112]

[0113] (2) UAS predicts the accuracy of dependency relationships (regardless of dependency labels);

[0114]

[0115] In order to verify the effectiveness of the method proposed in the present invention, the present invention selected data from the classic database (Universal Dependencies, UD) as experimental data to train the model, and used the trained model as the benchmark model. On this basis, the present invention uses the benchmark model to analyze unlabeled Vietnamese and generate the corresponding Vietnamese syntax tree. By setting certain rules, the present invention screens out high-quality syntax trees, aiming to use limited data through joint optimization training to build a new Vietnamese pseudo-corpus tree library.

[0116] The purpose of this operation is to mix the experimental data with the Vietnamese pseudo-corpus treebank to train the model, thereby significantly improving the efficiency, generalization ability and Vietnamese processing performance of the dependency parsing model. This method can alleviate the problem of scarcity of Vietnamese dependency syntactic data to a certain extent, thereby improving the overall performance of Vietnamese dependency syntactic analysis.

[0117] The experimental results of the present invention are shown in Table 1. The table shows that the method proposed in the present invention exceeds the performance of the baseline model in all evaluation indicators, which indicates that through joint optimization training and cross-language transfer learning, the present invention successfully improves the effect of Vietnamese dependency parsing.

[0118] Finally, the comprehensive application of this series of strategies effectively improved the overall performance of the method of the present invention and significantly enhanced the processing capabilities of Vietnamese natural language processing tasks. This achievement not only provides new ideas and methods for the research on building Vietnamese treebanks, but also provides strong support for the development of multilingual technology and Sino-Vietnamese language and cultural exchanges. Through cross-language collaboration and optimization, the present invention paves the way for the further application of Vietnamese in the field of natural language processing.

[0119] Table 1 is a comparison of the original treebank and the syntactic treebank constructed by this method.

[0120]

[0121] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A method for constructing a Vietnamese dependency syntactic treebank based on joint optimization training, characterized in that: The method comprises: Step 1: Collect Chinese, English and Vietnamese corpora as experimental data and perform preprocessing; Step 2, load the experimental data into the multilingual pre-trained language model, and use the multilingual pre-trained language model to enhance the representation ability of the original dual affine dependency syntactic analysis model; use Chinese, English and Vietnamese experimental data to train three dual affine dependency syntactic analysis models respectively; after the training is completed, save the three dual affine dependency syntactic analysis models respectively, and name them as Chinese model, English model and Vietnamese model respectively; Step 3: Collect new unlabeled Vietnamese datasets and build a Vietnamese dependency syntax tree library; Step 4: Use the Chinese model, English model and Vietnamese model to parse the constructed Vietnamese dependency syntactic treebank corpus to generate a pseudo-dependency syntactic treebank; Step 5: Use the pseudo-dependency syntactic treebank generated by the Vietnamese model as the standard to select high-quality pseudo-dependency syntactic treebanks generated by the Chinese model and the English model; Step 6: Mix the selected high-quality pseudo-dependency syntactic treebank with the Chinese, English and Vietnamese corpora in the original experimental data, retrain the bi-affine dependency syntactic analysis model, and use the trained bi-affine dependency syntactic analysis model to construct the Vietnamese dependency syntactic treebank; The Step 1 includes: Download the universal dependency syntactic treebanks of Chinese, English and Vietnamese with labels from the universal dataset UniversalDependencies as experimental data, segment the experimental data, and then process them into dependency syntactic trees in the CoNLL-U format based on the segmentation results; The Step 2 includes: Step 2.1, load the processed experimental data into the multi-language pre-trained language model for parameter fine-tuning; Step 2.2, use multi-language pre-trained language model for feature extraction; Step 2.3, weighted combination of the extracted features to form feature representation; Step 2.4, use the character-level bidirectional long short-term memory network Char-BiLSTM to encode the characters of each word to obtain character representation; Step 2.5, combine the feature representation with the randomly initialized word embedding to form a word representation, and combine it with the character representation to form a word vector representation; Step 2.6, input the word vector representations of Chinese, English and Vietnamese into the dual affine dependency parsing model for training; Step 2.

7. After the training is completed, save the dual affine dependency parsing models and name them as Chinese model, English model and Vietnamese model respectively. The Step 2.3 includes: Extract the features of the last 4 layers of the multilingual pre-trained language model and record them as H i1 (i1∈[1,4]), the trainable weight parameter is S i1 , γ is the scaling parameter; first, the weight parameters of each layer are normalized by the Softmax function, and then the normalized weight parameters are multiplied by the corresponding feature representation to obtain the weighted feature representation of each layer; finally, the weighted feature representations of each layer are summed to obtain the final feature representation rep XLM-R ; The calculation process is as follows: ； ； Step 2.6 and Step 2.7 include: A three-layer BiLSTM encoder is used to generate contextualized representations of each word vector in the sentence { }; The output vector of each BiLSTM Input into two multi-layer perceptrons MLP respectively to extract two low-dimensional syntactic vector dependent words and The correct core word ; Then use biaffine to calculate dependent words and The correct core word The dependency relationship between them is ArcScore( j); additional MLP and biaffine operations are also used to calculate the label scores between each word pair , which is used to evaluate the grammatical relationship type of each dependency arc; the bi-affine dependency syntactic analysis model also defines a local cross entropy loss function for each position i; finally, the trained bi-affine dependency syntactic analysis models are saved separately and named as Chinese model, English model and Vietnamese model respectively; ArcScore The calculation formula of j) is: ; in, It is a dependent word The vector of yes The correct core word A vector of , U1 is a biaffine matrix used to calculate the dependency arc; The definition of the dependency parsing loss function used by the local cross entropy loss function is as follows: ； Among them, w j Represents the dependent word w i The correct core word, l represents the dependent word and The correct core word The type of dependencies between them; The Step 5 includes: Step 5.

1. Using the Vietnamese pseudo-dependency syntactic treebank generated by the Vietnamese model as the standard, select the Chinese model and the English model to generate the initial high-quality pseudo-dependency syntactic treebank; Step 5.2, filter out high-quality pseudo-dependency syntactic treebanks according to the rules; Among them, the screening is performed by comparing the number of labels and head nodes in the dependency syntactic trees generated by the Chinese model, the English model, and the Vietnamese model; the same number of labels and head nodes are recorded as T, the total number is recorded as A, and then the ratio of the two is calculated to obtain the Correct Rate; the dependency syntactic trees are screened according to the size of the scores, and the high-quality dependency syntactic trees in the Chinese model and the English model are selected, and they are used to form a high-quality pseudo-dependency syntactic tree library; the specific score ratio calculation method is as follows: ； The Step 6 includes: Step 6.1, mix the selected high-quality pseudo-dependency syntactic treebank with the Chinese, English and Vietnamese corpora in the original experimental data; Step 6.2: Load the mixed processed data into the dual affine dependency syntactic analysis model for model training. The steps are the same as Step 2. Save the trained model and use the trained dual affine dependency syntactic analysis model to construct the Vietnamese dependency syntactic tree bank.

2. The method for constructing a Vietnamese dependency syntax treebank based on joint optimization training according to claim 1, characterized in that: The Step 3 includes: Step 3.1, download the unlabeled Vietnamese dataset from the general dataset ALT as new experimental data; Step 3.

2. Use the underthesea word segmentation tool to segment and deduplicate the Vietnamese dataset, and manually mark it to build a Vietnamese dependency syntactic tree library.

3. The method for constructing a Vietnamese dependency syntax treebank based on joint optimization training according to claim 1, characterized in that: The Step 4 includes: Step 4.1, input the corpus in the constructed Vietnamese dependency syntax tree library into three fine-tuned Chinese models, English models and Vietnamese models for parsing; Step 4.2, save the Vietnamese pseudo-dependency syntactic tree library obtained by parsing the Chinese model, English model and Vietnamese model.

4. A Vietnamese dependency syntactic treebank construction system based on joint optimization training, characterized in that: The system comprises: a module for executing the method for constructing a Vietnamese dependency syntax tree bank based on joint optimization training as claimed in any one of claims 1 to 3.

Images