Method and system for core process knowledge intelligent pushing based on multi-model fusion

Abstract

The invention discloses a method and a system for core process knowledge intelligent pushing based on multi-model fusion. The method comprises: preprocessing existing corpus data, and then inputting the data into a classification algorithm model for pre-classification, improving effect of classification through model fusion, when user queries or user feeds back, performing similarity calculation on user input and text categories, to determine the categories to which keywords belong, taking first k1 most similar categories, just retrieving in the categories, for each category, using the input keywords to retrieve respectively using different models in the category, combining all previous results, performing relevancy sorting using a BM25 algorithm, taking first k2 results, and using Jaccardsimilarity to remove texts which are too similar in the results, finally, returning the results to users. The method and the system can further adjust user's keyword models according to the feedbackof the users, and better fit needs of the users, so as to optimize user pushing effect and matching degree in the next pushing.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a method and system for intelligently pushing core process knowledge based on multi-model fusion. Background technique [0002] There are many problems in traditional information retrieval systems. In terms of query and access, the main methods include Boolean query, vector space model, probability model, etc., each has its own advantages and disadvantages, but when used alone, the accuracy cannot reach the highest level, and there will be a small number of inconsistencies. In the query results that match keywords, due to semantic problems, the same keyword does not refer to the same thing, that is, although the keyword of the entry matches the keyword retrieved, this part of the information is not needed by the user. From the results of the search, although the comprehensiveness of the information is considered, it cannot meet the needs of the searcher well, resulting in a de...

AI Technical Summary

Problems solved by technology

[0004] fastText Disadvantages: Requires a large amount of labeled data to train

[0005] Disadvantages of Rocchio: Rocchio assumes that the training data is absolutely correct, but the corpus will inevitably have errors; the unbalanced corpus will lead to deviations in the model file, and classes with more samples have more advantages; it is believed that documents of a category are only gathered around a centroid. This is often not the case (such data is called linearly inseparable)

[0006] Disadvantages of multi-classification SVM: Since this classification uses directed acyclic graphs to prevent classifiers, if the first classifier answers wrong (it is obviously an article of category 1, it says 5), then the latter classifier is no matter what It is impossible to correct its mistakes (because the category label "1" does not appear in the subsequent classifiers), in fact, there is such a phenomenon of downward accumulation of errors in the classifiers of each layer below

[0007] Disadvantages of Jaccard coefficient-Knn: the value of the element can only be 0 or 1, and richer information cannot be used

[0009] Disadvantages of the Boolean model: its retrieval strategy is derived from the binary judgment standard, documents are either relevant or irrelevant, and there is no concept of document classification, it is difficult to improve retrieval performance; although Boolean expressions have exact semantics, it is usually difficult to The user's information needs are converted into Boolean expressions. In fact, many users find it difficult to express their query requirements in Boolean expressions

[0010] Disadvantages of the vector space model: the calculation of the similarity is large, and when a new document is added, the weight of the word must be recalculated

[0011] Disadvantages of the probability model: First, the dependence on the text set is too strong, and the documents need to be divided into related and irrelevant sets. In fact, this model does not consider the frequency of index words in the document; second, the model stores and calculates The overhead is very high, parameter estimation is difficult, and the expression of files and queries is also difficult

Images