Ranking method, device, electronic medium and storage medium for document search

Abstract

Embodiments of the present application provide a sorting method and device for document search, electronic equipment and storage medium, comprising: obtaining a user search statement; obtaining a document search result corresponding to the user search statement based on the user search statement, and sorting the document search result according to a similarity result determined by a preset first model; the preset first model determines the similarity result based on the output result of one or more of a preset document type preference weight model, a preset document content prediction model, a preset document theme prediction model, a preset weighted TF-IDF model and a preset clustering model. The sorting method provided by the embodiments of the present application is more suitable for the search needs of enterprise employees, and can help users quickly locate the required documents in work.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a sorting method, apparatus, electronic device, and storage medium for document search. Background Technology

[0002] Faced with the ever-growing information resources within enterprises, users often struggle to quickly retrieve the information they need from vast databases. Therefore, search engines have become an indispensable tool in enterprise IT infrastructure development and application. However, with the massive growth of enterprise data, traditional search capabilities are increasingly overwhelmed by the sheer volume of search results. How to quickly retrieve the truly needed information from these results is one of the key areas of research in current search technology.

[0003] How to prioritize user-relevant information in search results is a continuous direction for search engine technology improvement and optimization. Several major ranking algorithms exist, including term frequency-weighted ranking and link analysis ranking algorithms. Term frequency-weighted ranking, developed from traditional information retrieval technology, posits that the higher the frequency and importance of a user's search term on a webpage, the more relevant the webpage is considered to be, and thus, it appears higher in the search results. Early search engines like InfoSeek, Excite, and Lycos used this method. Link analysis ranking algorithms are based on the principle of academic citations: the more times a paper or document is cited, the higher its academic value. Similarly, the more links pointing to a webpage, the more important that webpage is. Link analysis algorithms can be categorized into those based on random walk models (e.g., PageRank); those based on Hub and Authority mutual reinforcement models (e.g., HITS and its variants); those based on probabilistic models (e.g., SALSA); and those based on Bayesian models (e.g., Bayesian algorithms and their simplified versions).

[0004] Existing search ranking technologies, such as word frequency position weighted ranking and link analysis ranking algorithms, are easy to use and implement, and have been relatively mature. However, they are mainly suitable for searching high-quality web pages and network links on the Internet. For enterprise information systems, the information content is complex and diverse, and the search results should not only consider web pages, but also the individual's business attributes, job attributes, and personal habits. Therefore, the application effect of existing technologies in enterprise information systems is not good. Summary of the Invention

[0005] To address the problems existing in the prior art, embodiments of the present invention provide a sorting method, apparatus, electronic device, and storage medium for document search.

[0006] In a first aspect, embodiments of the present invention provide a sorting method for document search, comprising:

[0007] Get the user's search query;

[0008] Based on the user's search query, document search results corresponding to the user's search query are obtained, and the document search results are sorted according to the similarity results determined by a preset first model; wherein, the preset first model includes one or more of the following: a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset document topic weighted TF-IDF model, and a preset clustering model; the preset first model determines the similarity results based on the output results of one or more of the following: the preset document type preference weight model, the preset document content prediction model, the preset document topic prediction model, the preset weighted TF-IDF model, and the preset clustering model;

[0009] The document type preference weight model is a model trained by machine learning that takes user search statements, user log behavior features, and all documents in the document database as input and user document type preference weights as output, and is used to predict user document type preferences.

[0010] The preset document content prediction model is a model trained by machine learning that takes user search statements, user log behavior features, and user-selected documents corresponding to the user's historical search statements and user log behavior features as inputs and user search document content as outputs.

[0011] The preset document topic prediction model is a model trained by machine learning that takes the user-selected document corresponding to the user's historical search statement and the user's log behavior features as input, and the selected document topic corresponding to the user-selected document as output.

[0012] The preset weighted TF-IDF model is: a model trained by machine learning that takes the keywords corresponding to the user's search query as input data and the TF-IDF weight values ​​corresponding to the keywords as output, and is used to predict the TF-IDF weight values ​​of the document topics selected by the user.

[0013] The preset clustering model is a model trained by machine learning, which takes the business attributes associated with the user as input and the probability distribution value of the document topic as output, and is used to predict the probability distribution value of the document topic selected by the user; the business attributes include the user's department, the user's job responsibilities, and the user's business permissions.

[0014] Furthermore, the preset first model determines the similarity result based on the output of one or more of the following: a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset weighted TF-IDF model, and a preset clustering model. Specifically, this includes:

[0015] The preset first model uses the cosine similarity algorithm to determine the similarity result based on the output results of one or more of the preset document type preference weight model, preset document content prediction model, preset document topic prediction model, preset weighted TF-IDF model and preset clustering model.

[0016] Furthermore, the document type preference weight model is a model trained through machine learning to predict users' document type preferences, taking user search statements, user log behavior features, and all documents in the document database as input and user document type preference weights as output. It further includes:

[0017] Transform user search statements into word vectors f1, transform user log behavior features into numerical vectors f2, and classify all documents in the document database.

[0018] Feature x1 is formed by transforming user search statements into word vectors f1 and user log behavior features into numerical vectors f2, and is used as the feature of the training data.

[0019] The classified document is represented by y1, which serves as the label for the training data.

[0020] Based on x1 and y1, the document type preference weight model is constructed using the XGBOOST machine learning algorithm, and the user's document type preference weight is output.

[0021] Furthermore, the preset document content prediction model is a model trained through machine learning to predict the user's search document content, taking the user's search query, user log behavior features, and user-selected documents corresponding to the user's historical search query and user log behavior features as input, and the user's search document content as output. It further includes:

[0022] The user's search statement is transformed into a word vector f3, and the user's selected document corresponding to the user's historical search statement and the user's log behavior features is transformed into a word vector f4, and the user's log behavior features are transformed into numerical features f5.

[0023] Training sample x2 is formed based on word vectors f3 and f4;

[0024] Training sample y2 is formed based on digital feature f5;

[0025] Based on x2 and y2, a bidirectional LSTM algorithm for machine learning is used to construct the document content prediction model and output the user's search document content.

[0026] Further, the preset document topic prediction model is a model trained through machine learning to predict the topic of user-selected documents, taking the user-selected documents corresponding to the user's historical search statements and the user's log behavior features as input and the selected document topic corresponding to the user-selected documents as output. It further includes:

[0027] The document topic prediction model is constructed using the LDA algorithm of machine learning, and the topic of the document selected by the user is output.

[0028] Furthermore, the preset weighted TF-IDF model is a model trained through machine learning, which uses keywords corresponding to the user's search query as input data and TF-IDF weight values ​​corresponding to the keywords as output, to predict the TF-IDF weight values ​​of the user's selected document topic. It further includes:

[0029] In the weighted TF-IDF model, the user's search statement is processed by word segmentation and keyword extraction to determine the keywords corresponding to the user's search statement;

[0030] The relevance algorithm is used to calculate the degree of matching between the keywords and the search target domain based on the keywords corresponding to the user's search query; the relevance algorithm includes the term frequency (TF) algorithm and the directional document frequency (IDF) algorithm;

[0031] The TF-IDF weight value of the topic of the document selected by the user is determined based on the degree of matching between the keywords and the search target domain.

[0032] Furthermore, the preset clustering model is: a model trained through machine learning, taking user-related business attributes as input and document topic probability distribution values ​​as output, used to predict the probability distribution values ​​of document topics selected by the user; the business attributes include the user's department, the user's job responsibilities, and the user's business permissions, and also includes:

[0033] Clustering is performed based on the user's department, job responsibilities, and business permissions according to the tag feature data to determine multiple major categories;

[0034] The probability distribution value of the document topic selected by the user is obtained by calculating the document topic distribution based on each major category.

[0035] Secondly, embodiments of the present invention provide a sorting apparatus for document search, comprising:

[0036] The acquisition module is used to acquire the user's search query.

[0037] A sorting module is used to obtain document search results corresponding to the user's search query, and sort the document search results according to similarity results determined by a preset first model; wherein, the preset first model includes one or more of a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset document topic weighted TF-IDF model, and a preset clustering model; the preset first model determines the similarity results based on the output results of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document topic prediction model, the preset weighted TF-IDF model, and the preset clustering model;

[0038] The document type preference weight model is a model trained by machine learning that takes user search statements, user log behavior features, and all documents in the document database as input and user document type preference weights as output, and is used to predict user document type preferences.

[0039] The preset document content prediction model is a model trained by machine learning that takes user search statements, user log behavior features, and user-selected documents corresponding to the user's historical search statements and user log behavior features as inputs and user search document content as outputs.

[0040] The preset document topic prediction model is a model trained by machine learning that takes the user-selected document corresponding to the user's historical search statement and the user's log behavior features as input, and the selected document topic corresponding to the user-selected document as output.

[0041] The preset weighted TF-IDF model is: a model trained by machine learning that takes the keywords corresponding to the user's search query as input data and the TF-IDF weight values ​​corresponding to the keywords as output, and is used to predict the TF-IDF weight values ​​of the document topics selected by the user.

[0042] The preset clustering model is a model trained by machine learning, which takes the business attributes associated with the user as input and the probability distribution value of the document topic as output, and is used to predict the probability distribution value of the document topic selected by the user; the business attributes include the user's department, the user's job responsibilities, and the user's business permissions.

[0043] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the sorting method for document search as described in the first aspect above.

[0044] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the sorting method for document search as described in the first aspect above.

[0045] As can be seen from the above technical solutions, the sorting method, apparatus, electronic device, and storage medium for document search provided in the embodiments of the present invention obtain user search statements; obtain document search results corresponding to user search statements based on user search statements, and sort the document search results according to similarity results determined by a preset first model; the preset first model determines similarity results based on the output results of one or more of a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset weighted TF-IDF model, and a preset clustering model. The sorting method provided in the embodiments of the present invention is more in line with the search needs of enterprise employees, enabling users to conveniently and quickly locate the documents needed in their work. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a sorting method for document search provided in an embodiment of the present invention.

[0048] Figure 2 A flowchart illustrating a sorting method for document search provided in another embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the process for determining similarity results according to an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the process for constructing a document type preference weight model according to an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of the process for constructing a document content prediction model according to an embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram of the process for constructing a document topic prediction model according to an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of the process for constructing a document topic-weighted TF-IDF model according to an embodiment of the present invention;

[0054] Figure 8 This is a schematic diagram of the process for constructing a clustering model according to an embodiment of the present invention;

[0055] Figure 9 A schematic diagram of a sorting device for document search provided in an embodiment of the present invention;

[0056] Figure 10 This is a schematic diagram of the physical structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The sorting method for document search provided by the present invention will be explained and described in detail below through specific embodiments.

[0058] Figure 1 This is a flowchart illustrating a sorting method for document searching according to an embodiment of the present invention; as shown below. Figure 1 As shown, the method includes:

[0059] Step 101: Obtain the user's search query.

[0060] In this step, it is understood that the user enters a search query in the search bar, such as ABCDE, or FGHJK, etc.

[0061] Step 102: Obtain document search results corresponding to the user's search query, and sort the document search results according to the similarity results determined by a preset first model; wherein, the preset first model includes one or more of a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset document topic weighted TF-IDF model, and a preset clustering model; the preset first model determines the similarity results based on the output results of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document topic prediction model, the preset weighted TF-IDF model, and the preset clustering model;

[0062] The document type preference weight model is a model trained by machine learning that takes user search statements, user log behavior features, and all documents in the document database as input and user document type preference weights as output, and is used to predict user document type preferences.

[0063] The preset document content prediction model is a model trained by machine learning that takes user search statements, user log behavior features, and user-selected documents corresponding to the user's historical search statements and user log behavior features as inputs and user search document content as outputs.

[0064] The preset document topic prediction model is a model trained by machine learning that takes the user-selected document corresponding to the user's historical search statement and the user's log behavior features as input, and the selected document topic corresponding to the user-selected document as output.

[0065] The preset weighted TF-IDF model is: a model trained by machine learning that takes the keywords corresponding to the user's search query as input data and the TF-IDF weight values ​​corresponding to the keywords as output, and is used to predict the TF-IDF weight values ​​of the document topics selected by the user.

[0066] The preset clustering model is a model trained by machine learning, which takes the business attributes associated with the user as input and the probability distribution value of the document topic as output, and is used to predict the probability distribution value of the document topic selected by the user; the business attributes include the user's department, the user's job responsibilities, and the user's business permissions.

[0067] In this embodiment, it should be noted that, in view of the shortcomings of the prior art, a more intelligent search and sorting technology for enterprise informatization is needed (i.e., the sorting method for document search provided in this embodiment), which supports a high-performance, full-featured full-text search engine that integrates system search and website search, and can more intelligently sort the information content in enterprise informatization, and more intelligently and effectively provide the content that users need quickly.

[0068] In this embodiment, it should be noted that the document search sorting method provided is a search result sorting technology applied to full-text search engines in enterprise information systems. Specifically, it is designed for sorting search results for enterprise documents, work orders, applications, and other information, taking into account employee responsibilities, permissions, and business characteristics within an enterprise information system. It provides intelligent and relatively accurate search sorting capabilities while ensuring that document queries within the internal system are subject to the same access control as the data source system. On top of existing enterprise application systems, this enables each employee to accurately and quickly find the documents they need that match their usage permissions and business characteristics from a large number of difficult-to-search documents, thus saving considerable time.

[0069] In this embodiment, it should be noted that the sorting method for document search provided in this embodiment is a further refinement and enhancement based on the word frequency position weighted sorting technology combined with the characteristics of enterprise documents. Through similarity matching and title weighting, and combined with artificial intelligence technology in the information system, it realizes intelligent and personalized search result sorting that conforms to the data permissions and business characteristics of enterprise employees.

[0070] In this embodiment, it should be noted that the preset first model includes one or more of the following: a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset document topic weighted TF-IDF model, and a preset clustering model; for example, if the preset first model includes the preset document type preference weight model, the preset document content prediction model, the preset document topic prediction model, the preset document topic weighted TF-IDF model, and the preset clustering model, see [link to relevant documentation]. Figure 2 For example:

[0071] Based on the document type preference weight model, the user's document type preference weight is calculated.

[0072] Generate the user's search document content based on the preset document content prediction model;

[0073] The selected document topic is obtained based on the document topic prediction model, which corresponds to the document selected by the user.

[0074] Predict the TF-IDF weight values ​​of the user-selected document topic based on the weighted TF-IDF model;

[0075] Further combining TF-IDF weight values, the content of the user's searched document, and the topic of the selected document corresponding to the user's selected document, the first similarity result is obtained;

[0076] The second similarity result, which is the final similarity result, is obtained by further combining the document topic probability distribution value obtained from the clustering model and the user's document type preference weight.

[0077] As can be seen from the above technical solution, the document search ranking method provided by the embodiments of the present invention obtains the user's search statement; obtains the document search results corresponding to the user's search statement based on the user's search statement, and ranks the document search results according to the similarity results determined by a preset first model; the preset first model determines the similarity results based on the output results of one or more of the following: a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset weighted TF-IDF model, and a preset clustering model. The ranking method provided by the embodiments of the present invention is more in line with the search needs of enterprise employees, enabling users to conveniently and quickly locate the documents needed in their work.

[0078] Based on the above embodiments, in this embodiment, the preset first model determines the similarity result based on the output results of one or more of the following: a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset weighted TF-IDF model, and a preset clustering model. Specifically, this includes:

[0079] The preset first model uses the cosine similarity algorithm to determine the similarity result based on the output results of one or more of the preset document type preference weight model, preset document content prediction model, preset document topic prediction model, preset weighted TF-IDF model and preset clustering model.

[0080] In this embodiment, for example: the similarity result is determined based on the output of one or more of the following: a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset weighted TF-IDF model, and a preset clustering model; and a document that meets the user's expectations is generated. See [link to relevant documentation]. Figure 3 :

[0081] S1. Based on the user's search text and behavioral characteristics, call model_c1 to calculate the user's document preference weight w1.

[0082] S2. Based on the user's search text and behavioral characteristics, call model_c2 to generate the content that the user wants to view, arti_pre. Based on arti_pre, call model_c3 to generate the topic distribution dist_pre01 of the search results.

[0083] S3. Filter all documents searched by the user, call model_c3 to generate the topic distribution of the content viewed by the user, and calculate the mean of the main distribution preference w2.

[0084] S4. Based on the user clustering model, determine the major category to which the user belongs and extract the mean w3 of the topic distribution for that major category.

[0085] S5. Based on the user's search keywords, output candidate documents according to formula (4). Call model_c3 to calculate the topic distribution of all documents and output the topic distribution set {dist_trues}.

[0086] S6. Calculate the mean of w2 and dist_pre01 as the topic distribution of the search text, and output dist_pre02.

[0087] S7. Using the cosine similarity algorithm, calculate the similarity between dist_pre02 and each element of the set {dist_trues}, and output the similarity set {cors}.

[0088] S8. Using the cosine similarity algorithm, calculate the similarity between w3 and each element of the set {dist_trues}, and output the similarity set {cors_w3}.

[0089] S9. Calculate the weighted sum of {cors} and {cors_w3}, and output {cors_weight}, where the weight of {cors} is 0.7 and the weight of {cors_w3} is 0.3. The weight values ​​are obtained through cross-validation.

[0090] S10. Calculate the product of w1 and {cors_w3}, calculate the final similarity {cors_final} for each document, sort them, and the document with the higher ranking is the document that best meets the user's expectations.

[0091] As can be seen from the above technical solution, the sorting method for document search provided in the embodiments of the present invention adopts cross-validation and weighted calculation of search relevance to obtain a more reliable document similarity, thereby accurately recommending documents to users.

[0092] Based on the above embodiments, in this embodiment, the document type preference weight model is: a model trained through machine learning to predict users' document type preferences, taking user search statements, user log behavior features, and all documents in the document database as input, and taking the user's document type preference weight as output; it further includes:

[0093] Transform user search statements into word vectors f1, transform user log behavior features into numerical vectors f2, and classify all documents in the document database.

[0094] Feature x1 is formed by transforming user search statements into word vectors f1 and user log behavior features into numerical vectors f2, and is used as the feature of the training data.

[0095] The classified document is represented by y1, which serves as the label for the training data.

[0096] Based on x1 and y1, the document type preference weight model is constructed using the XGBOOST machine learning algorithm, and the user's document type preference weight is output.

[0097] In this embodiment, for the construction of the document type preference weight model, see [link to documentation]. Figure 4 For example:

[0098] Users, based on their department and position, will search for documents within a certain content range. Therefore, to quickly pinpoint the user's search scope, it is necessary to build a user document type preference model. The specific process is as follows:

[0099] S11. First, collect the user's historical search statements, user log behavior characteristics, and the document finally selected by the user to form the raw data.

[0100] S12. Using the word2vec algorithm, historical search statements are transformed into word vectors, denoted as fea_1 (equivalent to f1); user log behavior features are abstracted to generate new numerical features, denoted as fea_2 (equivalent to f2); all documents are classified, and the document type after classification is denoted as y (equivalent to y1); fea_1 and fea_2 are concatenated to form a new feature x (equivalent to x1), where (x, y) represents the features and labels of the training data.

[0101] S13. Using the XGBOOST algorithm, construct a document type preference weight model, and output the probability w1 = (pro_1, pro_2, ..., pro_m) of a document type, where m is the number of document types, which can be used as the degree of preference for document types and as a basis for subsequent document type preferences.

[0102] As can be seen from the above technical solution, the sorting method for document search provided in the embodiments of the present invention generates a document type preference weight model based on the user's historical search records, which can quickly locate the user's document preferences and save the search scope when the user searches.

[0103] Based on the above embodiments, in this embodiment, the preset document content prediction model is: a model trained by machine learning to predict the user's search document content, taking the user's search statement, user log behavior features, and user-selected documents corresponding to the user's historical search statement and user log behavior features as input, and the user's search document content as output; further comprising:

[0104] The user's search statement is transformed into a word vector f3, and the user's selected document corresponding to the user's historical search statement and the user's log behavior features is transformed into a word vector f4, and the user's log behavior features are transformed into numerical features f5.

[0105] Training sample x2 is formed based on word vectors f3 and f4;

[0106] Training sample y2 is formed based on digital feature f5;

[0107] Based on x2 and y2, a bidirectional LSTM algorithm for machine learning is used to construct the document content prediction model and output the user's search document content.

[0108] In this embodiment, for the construction of the document content prediction model, see [link to documentation]. Figure 5 For example:

[0109] Based on a many-to-many model using recurrent neural networks (RNNs), a one-to-one correspondence between user search content and document content can be achieved. This allows for the prediction of content that a user intends to view based on their search query, providing text for the subsequent construction of topic distributions. The specific process is as follows:

[0110] S21. First, collect the user's historical search statements, user log behavior characteristics, and the document finally selected by the user to form the raw data.

[0111] S22. Using the word2vec algorithm, historical search statements and document content are transformed into word vectors, denoted as fea_1 (equivalent to f3) and fea_2 (equivalent to f4), respectively. User log behavior features are abstracted to generate new numerical features, denoted as fea_3 (equivalent to f5). fea_1 and fea_3 are merged to generate input x (equivalent to x2), fea_2 as y (equivalent to y2), and the sample is (x, y). A bidirectional LSTM algorithm is used to predict the document content, and the generated content is denoted as arti_pre.

[0112] As can be seen from the above technical solutions, the sorting method for document search provided in the embodiments of the present invention constructs a user document content prediction model, which can generate the desired document content based on the user's search terms, and provides more sufficient content information when matching historical documents.

[0113] Based on the above embodiments, in this embodiment, the preset document topic prediction model is: a model trained by machine learning to predict the topic of user-selected documents, taking the user-selected documents corresponding to the user's historical search statements and the user's log behavior features as input and the selected document topic corresponding to the user-selected documents as output; the model further includes:

[0114] The document topic prediction model is constructed using the LDA algorithm of machine learning, and the topic of the document selected by the user is output.

[0115] In this embodiment, for the construction of the document topic prediction model, see [link to documentation]. Figure 6 For example:

[0116] Each document searched by a user has an implicit topic, and these topics are not singular. Therefore, the LDA algorithm can be used to calculate the topic of each document, thereby calculating the user's average topic preference and the topic preference of the generated document. LDA is a three-layer Bayesian model, consisting of a document layer, a topic layer, and a vocabulary layer. Its purpose is to identify topics, that is, to transform the document-vocabulary matrix into a document-topic matrix and a topic-vocabulary matrix. Based on this model, the document generation process is as follows:

[0117] S31. For a document set M, sample the distribution parameters φ of the topic to generate the word from the Dirichlet distribution with parameter β.

[0118] S32. For each document m in M, sample the distribution parameters θ of the doc-topic pair from the Dirichlet distribution with parameter α.

[0119] S33. For the nth word W_mn in document m, first sample a latent topic Z_m of document m according to the θ distribution, and then sample a word W_mn of topic Z_m according to the φ distribution.

[0120] The generated topic model is model_c3, and the flowchart is as follows: Figure 6 As shown.

[0121] As can be seen from the above technical solutions, the sorting method for document search provided in the embodiments of the present invention constructs a document topic model, thereby realizing the abstraction of document content under the deep learning framework and saving similarity calculation time.

[0122] Based on the above embodiments, in this embodiment, the preset weighted TF-IDF model is: a model trained through machine learning, using keywords corresponding to the user's search query as input data and TF-IDF weight values ​​corresponding to the keywords as output, for predicting the TF-IDF weight values ​​of the user's selected document topic, and further comprising:

[0123] In the weighted TF-IDF model, the user's search statement is processed by word segmentation and keyword extraction to determine the keywords corresponding to the user's search statement;

[0124] The relevance algorithm is used to calculate the degree of matching between the keywords and the search target domain based on the keywords corresponding to the user's search query; the relevance algorithm includes the term frequency (TF) algorithm and the directional document frequency (IDF) algorithm;

[0125] The TF-IDF weight value of the topic of the document selected by the user is determined based on the degree of matching between the keywords and the search target domain.

[0126] In this embodiment, see Figure 7 It should be noted that:

[0127] 1. Application of relevance algorithms:

[0128] The relevance algorithm is used to calculate the matching degree, which is the degree of matching between the search keyword (query, q) and the search target domain (field, f). Two methods are introduced into the relevance algorithm: Term Frequency (TF) and Inverse Document Frequency (IDF).

[0129] Search term frequency (TF):

[0130] The frequency of search terms. The higher the frequency, the higher the relevance.

[0131] Reverse document frequency:

[0132] The frequency of search terms in the index. Higher frequency generally indicates lower relevance. Search terms appearing in the majority of documents carry less weight than those appearing in a minority.

[0133] The formula is as follows:

[0134]

[0135] q represents a collection of documents, d represents a specific document, and t represents a word.

[0136] score(t,q,d) represents the relevance score of document d containing query term t in document set q.

[0137] 2. Title weighting application:

[0138] Title-weighted scoring calculates different weights based on the position and importance of search terms, such as the document title field, during retrieval. The weight scores are calculated as follows: Title > Body > Attachment Name. Additionally, search results can be sorted in descending order by drafting time.

[0139] Initial weights are set for the search terms, w1, w2, and w3, representing their importance in the title, body text, and attachments, respectively. The weights are updated iteratively based on the frequency of each search term's appearance in these three areas, using the following formula:

[0140]

[0141]

[0142]

[0143] The weight of a word is calculated and multiplied by its TF-IDF value to obtain the weighted TF-IDF value.

[0144] Score weight =score*w i (4)

[0145] As can be seen from the above technical solution, the sorting method for document search provided by the embodiments of the present invention is based on the title-weighted TF-IDF model, thereby efficiently outputting documents to be sorted.

[0146] Based on the above embodiments, in this embodiment, the preset clustering model is: a model trained through machine learning that takes user-associated business attributes as input and document topic probability distribution values ​​as output, and is used to predict the probability distribution values ​​of document topics selected by the user; the business attributes include the user's department, the user's job responsibilities, and the user's business permissions, and also includes:

[0147] Clustering is performed based on the user's department, job responsibilities, and business permissions according to the tag feature data to determine multiple major categories;

[0148] The probability distribution value of the document topic selected by the user is obtained by calculating the document topic distribution based on each major category.

[0149] In this embodiment, see Figure 8 It should be noted that:

[0150] First, based on user department, job title, and business permissions, feature extraction is performed through processing and analysis to cluster users. The primary clustering algorithm used is K-means. Its main idea is to select K points as initial cluster centers, assign each object to the nearest center to form K clusters, recalculate the center of each cluster, and repeat the above iterative steps until the clusters no longer change or the specified number of iterations is reached.

[0151] The most crucial step is determining the K-point, which serves as the initial centroid and significantly impacts the clustering results. It can be verified that different initial centroids lead to varying clustering outcomes. In this implementation, the error square method is used to evaluate the quality of the clustering results, defined as follows:

[0152]

[0153] Generally, the larger k is, the smaller the SSE is. Assuming k = N = the number of samples, then each point forms its own class, and the center point of each class is the unique point in that class itself, so SSE = 0.

[0154] After user clustering, the topic distribution of the searched documents is calculated for users in the same category, and the mean w3 is used as the preference weight of users in that category for documents.

[0155] As can be seen from the above technical solution, the sorting method for document search provided by the embodiments of the present invention generates a large category topic distribution through user clustering, which provides weights for subsequent similarity calculation.

[0156] Based on the above embodiments, it should be noted that the sorting method for document search provided by the embodiments of the present invention has the following advantages:

[0157] This invention is more suitable for searching official documents, work orders, files and applications in enterprise information systems than the original sorting technology, and pays more attention to the data permissions and business characteristics of enterprise employees.

[0158] The relevance-based sorting function provided by this invention makes the search results sorting more closely match the search needs of enterprise employees, allowing users to easily and quickly locate the files they are looking for.

[0159] This invention combines artificial intelligence capabilities with machine learning technologies such as topic modeling, recurrent neural networks, and data clustering to make the search more intelligent and possess strong learning capabilities. As its accuracy and intelligence level accumulate with the time it is used, the accuracy of search result ranking will not decrease due to the increasing amount of data in the enterprise's information system.

[0160] Figure 9This is a schematic diagram of a sorting device for document searching provided in an embodiment of the present invention, as shown below. Figure 9 As shown, the device includes: an acquisition module 201 and a sorting module 202, wherein:

[0161] The acquisition module 201 is used to acquire the user's search query.

[0162] The sorting module 202 is used to obtain document search results corresponding to the user's search query based on the user's search query, and sort the document search results according to the similarity results determined by a preset first model; wherein, the preset first model includes one or more of a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset document topic weighted TF-IDF model, and a preset clustering model; the preset first model determines the similarity results based on the output results of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document topic prediction model, the preset weighted TF-IDF model, and the preset clustering model;

[0163] The document type preference weight model is a model trained by machine learning that takes user search statements, user log behavior features, and all documents in the document database as input and user document type preference weights as output, and is used to predict user document type preferences.

[0164] The preset document content prediction model is a model trained by machine learning that takes user search statements, user log behavior features, and user-selected documents corresponding to the user's historical search statements and user log behavior features as inputs and user search document content as outputs.

[0165] The preset document topic prediction model is a model trained by machine learning that takes the user-selected document corresponding to the user's historical search statement and the user's log behavior features as input, and the selected document topic corresponding to the user-selected document as output.

[0166] The preset weighted TF-IDF model is: a model trained by machine learning that takes the keywords corresponding to the user's search query as input data and the TF-IDF weight values ​​corresponding to the keywords as output, and is used to predict the TF-IDF weight values ​​of the document topics selected by the user.

[0167] The preset clustering model is a model trained by machine learning, which takes the business attributes associated with the user as input and the probability distribution value of the document topic as output, and is used to predict the probability distribution value of the document topic selected by the user; the business attributes include the user's department, the user's job responsibilities, and the user's business permissions.

[0168] The sorting device for document search provided in this embodiment of the invention can be used to execute the sorting method for document search in the above embodiments. Its technical principle and beneficial effects are similar, and can be found in the above embodiments. It will not be repeated here.

[0169] Based on the same inventive concept, embodiments of the present invention provide an electronic device, see [link to relevant documentation]. Figure 10 The electronic device specifically includes the following components: processor 301, communication interface 303, memory 302, and communication bus 304;

[0170] The processor 301, communication interface 303, and memory 302 communicate with each other via communication bus 304. Communication interface 303 is used to transmit information between various modeling software and related devices such as intelligent manufacturing equipment module libraries. The processor 301 calls the computer program in memory 302. When the processor executes the computer program, it implements the methods provided in the above-described method embodiments. For example, when the processor executes the computer program, it implements the following steps: obtaining a user search statement; obtaining document search results corresponding to the user search statement based on the user search statement, and sorting the document search results according to the similarity results determined by a preset first model. The model includes one or more of the following: a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset document topic weighted TF-IDF model, and a preset clustering model. The preset first model determines the similarity result based on the output of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document topic prediction model, the preset weighted TF-IDF model, and the preset clustering model. The document type preference weight model takes user search statements, user log behavior features, and all documents in the document database as input, and outputs the user's document type preference weights, through a machine... The document content prediction model is a model trained by machine learning to predict users' document type preferences. The preset document content prediction model is a model trained by machine learning to predict users' search document content, taking user search statements, user log behavior features, and user-selected documents corresponding to the user's historical search statements and user log behavior features as input, and the user's search document content as output. The preset document topic prediction model is a model trained by machine learning to predict users' search document content, taking user-selected documents corresponding to the user's historical search statements and user log behavior features as input, and the selected document topic corresponding to the user's selected document as output. The system includes a model for predicting the topic of a user-selected document; the preset weighted TF-IDF model is a model trained through machine learning that takes keywords corresponding to the user's search query as input data and TF-IDF weight values ​​corresponding to the keywords as output, and is used to predict the TF-IDF weight values ​​of the topic of the user-selected document; the preset clustering model is a model trained through machine learning that takes business attributes associated with the user as input and document topic probability distribution values ​​as output, and is used to predict the probability distribution values ​​of the topic of the user-selected document; the business attributes include the user's department, the user's job responsibilities, and the user's business permissions.

[0171] Based on the same inventive concept, another embodiment of the present invention provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program performs the methods provided in the above-described method embodiments, such as: obtaining a user search query; obtaining document search results corresponding to the user search query based on the user search query; and sorting the document search results according to similarity results determined by a preset first model. The preset first model includes one or more of a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset document topic weighted TF-IDF model, and a preset clustering model. The preset first model determines the similarity results based on the output results of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document topic prediction model, the preset weighted TF-IDF model, and the preset clustering model. The document type preference weight model is a model trained by machine learning, using user search queries, user log behavior features, and all documents in a document database as input, and the user's document type preference weight as output, for predicting the user's document type preference. The preset document type preference weight model is a non-transitory computer-readable storage medium storing a computer program. The content prediction model is as follows: It takes user search statements, user log behavior features, and user-selected documents corresponding to the user's historical search statements and log behavior features as input, and the content of the user's searched documents as output, and is trained through machine learning to predict the content of the user's searched documents. The preset document topic prediction model is as follows: It takes user-selected documents corresponding to the user's historical search statements and log behavior features as input, and the selected document topic corresponding to the user's selected document as output, and is trained through machine learning to predict the topic of the user's selected document. The preset weighted TF-IDF model is as follows: It takes keywords corresponding to the user's search statements as input data, and the TF-IDF weight values ​​corresponding to the keywords as output, and is trained through machine learning to predict the TF-IDF weight values ​​of the user's selected document topics. The preset clustering model is as follows: It takes business attributes associated with the user as input, and the document topic probability distribution value as output, and is trained through machine learning to predict the probability distribution value of the user's selected document topics. The business attributes include the user's department, job responsibilities, and business permissions.

[0172] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0174] Furthermore, in this invention, terms such as "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0175] Furthermore, in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0176] Furthermore, in the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A ranking method for document search, characterized in that, The method comprises: obtaining a user search statement; obtaining a document search result corresponding to the user search statement based on the user search statement, and sorting the document search result according to a similarity result determined by a preset first model; wherein the preset first model comprises one or more of a preset document type preference weight model, a preset document content prediction model, a preset document theme prediction model, a preset document theme weighted TF-IDF model, and a preset clustering model; and the preset first model determines the similarity result based on the output result of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document theme prediction model, the preset weighted TF-IDF model, and the preset clustering model; the document type preference weight model is a model obtained by machine learning training, taking a user search statement, user log behavior features, and all documents in a document database as inputs, and taking a user's document type preference weight as an output, and is used for predicting a user's document type preference; the preset document content prediction model is a model obtained by machine learning training, taking a user search statement, user log behavior features, and user-selected documents corresponding to the user historical search statement and the user log behavior features as inputs, and taking a user's search document content as an output, and is used for predicting a user's search document content; the preset document theme prediction model is a model obtained by machine learning training, taking user-selected documents corresponding to the user historical search statement and the user log behavior features as inputs, and taking selected document themes corresponding to the user-selected documents as an output, and is used for predicting the themes of user-selected documents; the preset weighted TF-IDF model is a model obtained by machine learning training, taking keywords corresponding to a user search statement as input data, and taking a TF-IDF weight value corresponding to the keywords as an output, and is used for predicting the TF-IDF weight value of a user-selected document theme; the preset clustering model is a model obtained by machine learning training, taking business attributes associated with a user as inputs, and taking a document theme probability distribution value as an output, and is used for predicting the probability distribution value of a user-selected document theme; the business attributes include a department to which the user belongs, a position and responsibility of the user, and a business authority of the user.

2. The ranking method for document search according to claim 1, wherein, The preset first model determines the similarity result based on the output result of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document theme prediction model, the preset weighted TF-IDF model, and the preset clustering model, and specifically comprises: The preset first model determines the similarity result based on the output result of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document theme prediction model, the preset weighted TF-IDF model, and the preset clustering model, and adopts a cosine similarity algorithm.

3. The ranking method for document search according to claim 1, wherein, The document type preference weight model is: taking the user search statement, the user log behavior feature, and all documents in the document database as input, taking the user's document type preference weight as output, and being a model obtained by machine learning training and used for predicting the user's document type preference, and further comprising: transforming the user search statement into a word vector f1, transforming the user log behavior feature into a digital vector f2, and classifying all documents in the document database; forming a feature x1 based on the word vector f1 obtained by transforming the user search statement and the digital vector f2 obtained by transforming the user log behavior feature, as a feature of training data; using y1 to represent the classified document, as a label of training data; based on x1 and y1, adopting a machine learning XGBOOST algorithm to construct the document type preference weight model, and outputting the user's document type preference weight.

4. The ranking method for document search according to claim 1, wherein, The preset document content prediction model is: taking the user search statement, the user log behavior feature, and the user-selected document corresponding to the user historical search statement and the user log behavior feature as input, taking the user search document content as output, and being a model obtained by machine learning training and used for predicting the user's search document content, and further comprising: transforming the user search statement into a word vector f3, transforming the user-selected document corresponding to the user historical search statement and the user log behavior feature into a word vector f4, and transforming the user log behavior feature into a digital feature f5; forming a training sample x2 based on the word vector f3 and the word vector f4; forming a training sample y2 based on the digital feature f5; based on x2 and y2, adopting a machine learning bidirectional LSTM algorithm to construct the document content prediction model, and outputting the user's search document content.

5. The ranking method for document search according to claim 1, wherein, The preset document topic prediction model is: taking the user-selected document corresponding to the user historical search statement and the user log behavior feature as input, taking the selected document topic corresponding to the user-selected document as output, and being a model obtained by machine learning training and used for predicting the topic of the user-selected document, and further comprising: using a machine learning LDA algorithm to construct the document topic prediction model, and outputting the topic of the user-selected document.

6. The ranking method for document search according to claim 1, wherein, The preset weighted TF-IDF model is: taking the keyword corresponding to the user search statement as input data, taking the TF-IDF weight value corresponding to the keyword as output, and being a model obtained by machine learning training and used for predicting the TF-IDF weight value of the user-selected document topic, and further comprising: performing word segmentation processing and keyword extraction processing on the user search statement in the weighted TF-IDF model to determine the keyword corresponding to the user search statement; based on the keyword corresponding to the user search statement, using a relevance algorithm to calculate the matching degree between the keyword and the search target domain; the relevance algorithm includes a term frequency TF algorithm and a direction document frequency IDF algorithm; based on the matching degree between the keyword and the search target domain, determining the TF-IDF weight value of the user-selected document topic.

7. The ranking method for document search according to claim 5, wherein, The preset clustering model is a model obtained by machine learning and used for predicting probability distribution values of document topics selected by a user, taking business attributes associated with the user as input and taking document topic probability distribution values as output; the business attributes include a department to which the user belongs, a position responsibility of the user, and a business authority of the user, and further include: Clustering is performed according to the department to which the user belongs, the position responsibility of the user, and the business authority of the user, to determine a plurality of large categories; Probability distribution values of document topics selected by a user are obtained through document topic distribution calculation based on each large category.

8. A ranking apparatus for document search, characterized by, The method comprises the following steps: An acquisition module is configured to acquire a user search statement; An ordering module is configured to obtain a document search result corresponding to the user search statement based on the user search statement, and to order the document search result according to a similarity result determined by a preset first model; wherein the preset first model comprises one or more of a preset document type preference weight model, a preset document content prediction model, a preset document topic prediction model, a preset document topic weighted TF-IDF model, and a preset clustering model; the preset first model determines the similarity result based on output results of one or more of the preset document type preference weight model, the preset document content prediction model, the preset document topic prediction model, the preset weighted TF-IDF model, and the preset clustering model; The document type preference weight model is a model obtained by machine learning and used for predicting a document type preference of a user, taking a user search statement, user log behavior features, and all documents in a document database as input and taking a document type preference weight of the user as output; The preset document content prediction model is a model obtained by machine learning and used for predicting a search document content of a user, taking a user search statement, user log behavior features, and user-selected documents corresponding to the user historical search statement and the user log behavior features as input and taking the search document content of the user as output; The preset document topic prediction model is a model obtained by machine learning and used for predicting a topic of a user-selected document, taking user-selected documents corresponding to the user historical search statement and the user log behavior features as input and taking a selected document topic corresponding to the user-selected document as output; The preset weighted TF-IDF model is a model obtained by machine learning and used for predicting a TF-IDF weight value of a user-selected document topic, taking a keyword corresponding to a user search statement as input data and taking a TF-IDF weight value corresponding to the keyword as output; The preset clustering model is a model obtained by machine learning and used for predicting probability distribution values of document topics selected by a user, taking business attributes associated with the user as input and taking document topic probability distribution values as output; the business attributes include a department to which the user belongs, a position responsibility of the user, and a business authority of the user.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the ranking method for document search according to any one of claims 1-7 when executing the program.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the ranking method for document search according to any one of claims 1-7 when executed by the processor.

Images