Conditional random field and transformative learning based Vietnamese chunking method

A conditional random field and transformation technology, applied in the creation of semantic tools, natural language data processing, special data processing applications, etc., can solve the problem of low accuracy of Vietnamese chunk recognition, and achieve the effect of improving the F value.

Inactive Publication Date: 2016-07-06
KUNMING UNIV OF SCI & TECH
View PDF2 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The present invention provides a method for learning Vietnamese chunks based on conditional random fields and transformations to solve the problem of low recognition accuracy of Vietnamese chunks. It can construct Vietnamese phrase trees, complete syntax analysis, and machine Upper-level applications such as translation and information acquisition can provide strong support

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Conditional random field and transformative learning based Vietnamese chunking method
  • Conditional random field and transformative learning based Vietnamese chunking method
  • Conditional random field and transformative learning based Vietnamese chunking method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0044] Embodiment 1: as Figure 1-4 As shown, a method for learning Vietnamese chunks based on conditional random fields and conversion, the specific steps of the method are as follows:

[0045] Step1. First, the Vietnamese corpus is preprocessed to obtain sentence-level Vietnamese chunk training corpus as a standard sentence-level Vietnamese chunk training corpus, and the sentence-level Vietnamese chunk corpus is saved in the database;

[0046] Step2. Extract the sentence-level Vietnamese chunk training corpus from the database and perform chunk modeling on it to obtain the Vietnamese chunk conditional random field model;

[0047] Step3. Use the conversion error-driven learning method to learn the training corpus and obtain the conversion method set;

[0048]Step4. Put the sub-level test corpus of Vietnamese sentences to be chunked through the established Vietnamese chunking conditional random field model and the obtained conversion method set to perform chunking marking, an...

Embodiment 2

[0049] Embodiment 2: as Figure 1-4 As shown, a method for learning Vietnamese chunks based on conditional random field and conversion, this embodiment is the same as embodiment 1, wherein:

[0050] The specific steps of preprocessing in the step Step1 are as follows:

[0051] Step1.1, use the crawler program to crawl out the Vietnamese web page information from the Internet;

[0052] Step1.2. Filter and process the crawled webpage information, build a Vietnamese text corpus, and use the word segmentation tool to process the word segmentation of the corpus to form a sentence-level Vietnamese text corpus that has been word-segmented, and manually proofread it. Finally, the Vietnamese The text corpus of Vietnamese and the sentence-level Vietnamese text corpus that has been segmented are stored in the database;

[0053] Step1.3. Take out the sentence-level Vietnamese text corpus that has been segmented from the database, use the Vietnamese part-of-speech tagging tool to tag, ge...

Embodiment 3

[0055] Embodiment 3: as Figure 1-4 As shown, a method for learning Vietnamese chunks based on conditional random field and conversion, this embodiment is the same as embodiment 2, wherein:

[0056] In the step Step2, the specific steps of constructing the conditional random field model of Vietnamese chunks are as follows:

[0057] Step2.1. Obtain the processed sentence-level Vietnamese chunk training corpus from the database;

[0058] Step2.2, according to the language and sentence characteristics of Vietnamese, extract the features of Vietnamese from the sentence-level Vietnamese chunk training corpus in Step 2.1. The extracted Vietnamese features include: word features, part-of-speech features and context information features;

[0059] Step2.3. According to the extracted Vietnamese features, construct the basic feature templates of Vietnamese required in the Vietnamese block conditional random field model;

[0060] Step2.4, with the sentence-level Vietnamese chunk traini...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a conditional random field and transformative learning based Vietnamese chunking method and belongs to the technical field of natural language processing. The method comprises the steps of firstly preprocessing Vietnamese corpora to obtain sentence level Vietnamese chunking training corpora; extracting the sentence level Vietnamese chunking training corpora from a database and performing chunking modeling on the sentence level Vietnamese chunking training corpora to obtain a Vietnamese chunking conditional random field model; obtaining a transformative mode set; and performing chunking marking on to-be-chunked Vietnamese sentence level test corpora through the established Vietnamese chunking conditional random field model and the obtained transformative mode set to obtain a Vietnamese chunking marking result. The method realizes effective chunking analysis for Vietnamese sentences and paves the way for work such as phrase trees, semantic analysis, machine translation and the like; and compared with an existing Vietnamese chunking tool, the Vietnamese chunking method is remarkably improved in accuracy, recall rate and F value.

Description

technical field [0001] The invention relates to a method for learning Vietnamese chunks based on conditional random fields and conversion, and belongs to the technical field of natural language processing. Background technique [0002] The China-ASEAN Free Trade Area is the most populous free trade area in the world. The "Bridgehead Strategy" is a strategic need to promote my country's southwest development and realize good-neighborliness and friendship with ASEAN countries. Yunnan is an important bridgehead for China's opening to the southwest. Language Communication is the prerequisite for political, cultural and economic exchanges between China and ASEAN countries. Vietnam, a member of ASEAN, is connected by mountains and rivers to Yunnan. The people of the two countries have a long history of exchanges. Language communication has played a very important role in the friendly coexistence and mutual learning of the people on the border between the two sides. Therefore, the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/27G06F17/30
CPCG06F16/36G06F40/211
Inventor 余正涛刘艳超郭剑毅
Owner KUNMING UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap