Stop word mining method and device, search method and device, evaluation method and device

A mining device and technology of stop words, applied in the Internet field, can solve problems such as low accuracy rate, achieve the effect of improving accuracy rate, saving browsing time, and saving storage space

CN103902552BActive Publication Date: 2019-03-26SHENZHEN SHI JI GUANG SU INFORMATION TECH
4 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Publication Date
2019-03-26

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a stop word mining method. The stop word mining method comprises the following steps that a query log is obtained; the inverse document frequency of search terms in a search string recorded in the query log, the relative term weights of the search terms, search term sets generated due to search string modifying behaviors, and at least one kind of attribute information in the corresponding relation set, generated due to trigger behaviors, between the search string and a web page URL are obtained, and a stop word set is generated according to the attribute information. In addition, the invention provides a stop word mining device, a searching method and device, and an evaluating method and device for a stop word mining algorithm. By the adoption of the stop word mining method and device, the accuracy rate of mined stop words is increased; according to the searching method and device, due to the fact that an original search string is simplified by removing stop words, more relevant web pages can be found out, and the accuracy rate of searching is increased; according to the evaluating method and device for the stop word mining algorithm, evaluation is conducted in a cross validation manner, and an optimal algorithm can be obtained through comparison.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The present invention relates to Internet technology, in particular to a stop word mining method and device, a search method and device, and a stop word mining algorithm evaluation method and device. Background technique

[0002] Stop words are query words that are automatically ignored by search engines when indexing web pages or processing query requests. Stop words usually appear too frequently and have no practical meaning, such as "the", "a", "的", "了". , removing such words is beneficial to reduce the scale of web search and improve the accuracy of search results.

[0003] There are two traditional ways to mine stop words, one is manual selection according to a certain standard; the other is automatic mining from web documents and search engine logs. Manual selection requires a lot of manpower and is inefficient. There are two ways to automatically mine stop words from web documents and search engine logs. One is to use random sampling to genera...

Examples

Embodiment Construction

[0055] The method and device for mining stop words, as well as the technical solution for the evaluation method and device for mining algorithms for stop words will be described in detail below in conjunction with specific embodiments and accompanying drawings, so as to make them more clear.

[0056] Such as figure 1 Shown, in one embodiment, a kind of mining method of stop word, comprises the following steps:

[0057] Step S102, acquiring query logs.

[0058] Specifically, the query log is used to record the information generated by the user inputting the query string to perform the query behavior and triggering the query result behavior. The query log includes the query string, the webpage address obtained from the query, the behavior of modifying the query string, the behavior of triggering the webpage address, and the corresponding relationship between the query string and the webpage address.

[0059] Step S104, obtain the reverse document frequency of the query word in...