A method for automatically learning Chinese construction based on correlation degree

By employing a relevance-based approach and utilizing word segmentation models such as jieba, LAC, PKUSEG, and HanLP, along with tabu and variable neighborhood search algorithms, Chinese constructions are automatically learned. This addresses the issues of limited quantity and low quality of constructions in existing technologies, enabling efficient construction learning and its application in natural language processing.

CN117094309BActive Publication Date: 2026-06-26ZHEJIANG UNIV
2 Cites 0 Cited by

Patent Information

Application Number
CN202310519881.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2026-06-26
Estimated Expiration
2043-05-10

AI Technical Summary

Technical Problem

Existing technologies lack automated methods for learning Chinese construction patterns. Manual construction pattern libraries are few in number and costly, and word segmentation standards are inconsistent, making them difficult to apply directly to natural language processing technologies.

Method used

A correlation-based approach is adopted, which uses jieba, LAC, PKUSEG and HanLP word segmentation models to obtain vocabulary and part-of-speech information, combines Word2Vec and K-Means clustering to obtain semantic information, uses tabu search and variable neighborhood search algorithms to generate candidate construction sets, and automatically learns Chinese constructions through bidirectional correlation calculation and minimum description length optimization.

Benefits of technology

It automatically learns a large number of high-quality Chinese constructions, solving the problem of time-consuming and labor-intensive manual construction libraries. It achieves a significant increase in the number of constructions and an improvement in quality, with an MDL compression rate of over 70%, making it suitable for natural language processing technology.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application discloses a kind of Chinese construction automatic learning method based on correlation degree, which is divided into background statistical corpus, candidate construction generation corpus, MDL optimization corpus and determine the best correlation degree threshold corpus by corpus;Then the correlation degree of each information pair is calculated in the background statistical corpus, and the locally optimal correlation degree is searched on the small corpus for determining the best correlation degree threshold, so that the construction set obtained with the threshold has the best MDL evaluation index;Then the candidate construction set is generated using the locally optimal correlation degree on the larger corpus generated by candidate construction, and the correlation degree characteristics of candidate construction are calculated;Finally, the best MDL candidate construction set is searched by tabu search algorithm and variable neighborhood search algorithm on the MDL optimization corpus, and the Chinese construction set is obtained.The application can automatically learn to good quality Chinese construction, which can be directly applied to natural language processing technology, and has good application prospect.
Need to check novelty before this filing date? Find Prior Art