Traditional Mongolian webpage recognition method and traditional Mongolian webpage recognition system

Abstract

The invention relates to a traditional Mongolian webpage recognition method and a traditional Mongolian webpage recognition system. The method includes the following steps: the word frequency and document frequency of each word in a traditional Mongolian webpage corpus are obtained and counted, and the harmonic mean of each word is calculated; according to the harmonic means in descending order, a first previous number of words are chosen, and the harmonic means of the first previous number of words are accumulated, so that a first accumulated sum is obtained; the word frequencies of the first previous number of words in a webpage to be recognized are obtained and counted, and are accumulated, so that a second accumulated sum is obtained; when the difference between the first accumulated sum and the second accumulated sum is less than or equal to a first threshold, the webpage to be recognized is determined to be a traditional Mongolian webpage. The traditional Mongolian webpage recognition method provided by the invention can carry out the recognition of traditional Mongolian webpages with high accuracy and high efficiency, and thereby can help to collect traditional Mongolian webpages and implement a traditional Mongolian full-text search engine.

Description

technical field [0001] The invention relates to the field of network technology, in particular to a traditional Mongolian web page recognition method and device. Background technique [0002] Traditional Mongolian is the official way of writing the Mongolian language in the Inner Mongolia Autonomous Region of China (that is, the orthography of Mongolian language written in the Mongolian alphabet). Traditional Mongolian network resources are an important way for the Mongolian people to use their own language to transmit information and share resources. They are also the main platform for the inheritance of Mongolian traditional culture. Full-text search engines are of great significance. The number of traditional Mongolian network resources in my country is relatively small compared with Chinese and English network resources, and the encoding is complex. Therefore, it is very important to collect traditional Mongolian network resources accurately and efficiently. Preliminary...

AI Technical Summary

Problems solved by technology

Among the three existing technologies for identifying webpage languages, the webpage language recognition technology based on high-frequency words is more effective than the other two methods, but this technology only considers the absolute frequency of language units and does not consider the words used in texts in different fields. Therefore, the recognition accuracy of webpage languages ​​varies greatly

Images