Genetic simulated annealing method for solving new words in Chinese segmentation

Abstract

The invention discloses a genetic simulated annealing method for solving new words in Chinese segmentation. The method comprises the steps of firstly acquiring and intelligently searching Internet information by adopting a crawler program to complete data preparation; then performing Chinese segmentation on the acquired data by adopting a dedicated lexicon, namely discovering a public opinion; proposing a genetic simulated annealing algorithm using the characteristics of parallel operation and global convergence of a genetic algorithm in combination with local convergence of a simulated annealing algorithm, and performing relevant design and application on a public opinion monitoring system. By adopting the method, the automatic segmentation problem in the field of Chinese information processing is solved; by combining the solution strategies of the genetic algorithm and the simulated annealing algorithm for new words continuously appearing with the development of society and Internet, the segmentation accuracy is improved, the problems of disperse strings and segmentation errors in the automatic segmentation result are effectively solved, and the method plays an important role in observing, researching and analyzing dynamic changes of language phenomena, normalizing languages and characters and improving the overall effect of automatic Chinese segmentation.

Description

technical field [0001] The invention belongs to the technical field of Chinese search and retrieval, and in particular relates to a genetic simulated annealing method for solving new words in Chinese word segmentation. Background technique [0002] With the improvement of my country's comprehensive national strength, Chinese occupies an increasingly important position on the world stage. At the same time, with the development of Internet technology, there are more and more Chinese information, and Chinese search and retrieval technology has also been greatly developed. How to find the information and materials you need in the vast Chinese information world has become an increasingly important topic. In today's era of information explosion, it has become unrealistic to rely solely on manual labor to deal with the rapidly growing mass of information. Therefore, Chinese automatic word segmentation technology has emerged. Chinese automatic word segmentation is the basis of natur...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a genetic simulated annealing method for solving new words in Chinese word segmentation, aiming at solving the problem that the algorithm flexibility of word segmentation in the current word segmentation system is poor and has strong domain

[0005] Solve the problem that the current new words are constantly appearing, and the automatic word segmentation results are prone to "scattered strings" and low accuracy

Images