Literature retrieval method and device, medium and equipment
Through the introduction of target retrieval strategies based on retrieval information configuration in the literature retrieval system, including condition search and statement retrieval, the problem of single search mode in existing technologies has been solved, and the satisfaction and improvement of retrieval efficiency of various retrieval needs are achieved.
Patent Information
- Application Number
- CN202411637660.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-15
- Publication Date
- 2025-01-28
AI Technical Summary
In the existing literature search methods, the search mode is single, and it cannot support multiple search modes, which cannot meet the diverse search needs of users.
Based on the target configuration method of retrieval information, the target retrieval strategy corresponding to the target configuration method, including condition search strategies and statement retrieval strategies. The specific steps include: under the condition search strategy, the database is retrieved from the database based on the target retrieval conditions; under the sentence retrieval strategy, the query language or vector similarity retrieval method is used to match the target literature or knowledge fragment.
The accurate choice of multiple retrieval modes is achieved, supporting different particle size and type of knowledge fragment search, and improving the degree of recall of search results and the efficiency of the use of knowledge resources.
Smart Images
Figure CN119377385A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information processing technology, and in particular to a document retrieval method, device, medium and equipment. Background Art
[0002] In the existing literature retrieval methods, the corresponding literature data is usually retrieved based on the keyword search method. However, this retrieval method has the problem of a single recall path and cannot meet the diverse retrieval needs of users.
[0003] Therefore, there is an urgent need for a document retrieval method to solve the problem that the existing technology has a single retrieval mode, cannot support multiple retrieval modes, and cannot meet the diverse retrieval needs of users. Summary of the invention
[0004] In view of this, the present invention provides a document retrieval method, device, medium and equipment, the main purpose of which is to solve the problem that the current retrieval mode is single, cannot support multiple retrieval modes, and cannot meet users' diverse retrieval needs.
[0005] To solve the above problems, the present application provides a document retrieval method, comprising:
[0006] Based on the target configuration mode of the retrieval information, determining a target retrieval strategy corresponding to the target configuration mode, wherein the retrieval strategy includes: a conditional retrieval strategy and a statement retrieval strategy;
[0007] When the target search strategy is a conditional search strategy, based on the target search condition in the search information, a number of target documents meeting the target search condition are retrieved from the target database;
[0008] When the target retrieval strategy is a sentence retrieval strategy, based on the target retrieval sentence in the retrieval information, a query language retrieval method or a vector similarity retrieval method is determined as the target retrieval method, so as to retrieve a number of target documents or a number of target knowledge fragments matching the target retrieval sentence from the target database based on the target retrieval sentence using the target retrieval method.
[0009] Optionally, the configuration mode includes a first configuration mode in which a search condition is selected and a second configuration mode in which a search statement is input;
[0010] The configuration mode based on the search information, determining the target search strategy corresponding to the configuration mode, specifically includes:
[0011] Based on the first configuration mode of the search information, determining the conditional search strategy as the target search strategy;
[0012] Alternatively, based on the second configuration mode of the search information, the sentence search strategy is determined as the target search strategy.
[0013] Optionally, the target search condition includes any one or more of the following: subject condition, institution condition, research level condition, name condition;
[0014] The step of retrieving a plurality of target documents that meet the target search condition from a target database based on the target search condition in the search information specifically includes:
[0015] Constructing a query language based on any one or more of the search conditions of the subject condition, the institution condition, the research level condition and the name condition in the search information;
[0016] The documents in the target database are screened based on the query language to obtain the target documents.
[0017] Optionally, determining the query language retrieval method or the vector similarity retrieval method as the target retrieval method based on the target retrieval statement in the retrieval information specifically includes:
[0018] Recognize the search intent of the target search sentence to obtain conditional search intent or non-conditional search intent;
[0019] When obtaining the conditional search intention, determine the query language search method as the target search method;
[0020] When a non-conditional search intention is obtained, a vector similarity search method is determined as a target search method.
[0021] Optionally, when the target search method is a query language search method, the step of retrieving a plurality of target documents or a plurality of target knowledge fragments matching the target search statement from a target database using the target search method based on the target search statement specifically includes:
[0022] Determining a target search condition based on the keywords in the target search sentence, so as to screen the documents in the target database based on the target search condition to obtain the target documents;
[0023] When the target search method is a vector similarity search method, the step of retrieving a plurality of target documents or a plurality of target knowledge fragments matching the target search statement from a target database by using the target search method based on the target search statement specifically includes:
[0024] A target sentence vector is determined based on the target search sentence, so as to screen documents in a target database based on the target sentence vector to obtain a target document containing the target sentence vector or a target knowledge fragment matching the target sentence vector.
[0025] Optionally, determining a target sentence vector based on the target search sentence, screening documents in a target database based on the target sentence vector, and obtaining a target document containing the target sentence vector or a target knowledge fragment matching the target sentence vector specifically includes:
[0026] Determining the target sentence vector based on the key fragments in the target search sentence;
[0027] Based on the target sentence vector and the target retrieval level, determining, from a target vector library corresponding to the target database, target document segment vectors whose vector similarity meets a predetermined similarity condition and meets the target retrieval level;
[0028] Based on each of the target document segment vectors and the correspondence between each document segment vector and the document segment, the target document segment corresponding to each target document segment vector is determined, so as to determine the plurality of target knowledge segments or the plurality of target documents based on each of the target document segments.
[0029] Optionally, after obtaining a plurality of target documents or a plurality of target knowledge fragments, the method further includes:
[0030] Sorting each target document and target knowledge fragment to obtain a sorted target document list and a sorted target knowledge fragment list;
[0031] The target document list and the target knowledge fragment list are displayed.
[0032] In order to solve the above problems, the present application provides a document retrieval device, comprising:
[0033] A determination module, used to determine a target search strategy corresponding to the configuration mode based on the configuration mode of the search information, wherein the search strategy includes: a conditional search strategy and a statement search strategy;
[0034] A retrieval module, used for retrieving a number of target documents that meet the target retrieval conditions from a target database based on the target retrieval conditions in the retrieval information when the target retrieval strategy is a conditional retrieval strategy;
[0035] The retrieval module is also used to determine the query language retrieval method or the vector similarity retrieval method as the target retrieval method based on the target retrieval statement in the retrieval information when the target retrieval strategy is a sentence retrieval strategy, so as to retrieve a number of target documents or a number of target knowledge fragments matching the target retrieval statement from the target database using the target retrieval method based on the target retrieval statement.
[0036] In order to solve the above problems, the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned document retrieval methods are implemented.
[0037] To solve the above problems, the present application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above-mentioned document retrieval methods when executing the computer program on the memory.
[0038] The document retrieval method, device, medium and equipment in the present application can accurately determine the conditional retrieval strategy or sentence retrieval strategy as the target retrieval strategy according to the configuration mode by obtaining the configuration mode of the retrieval information, and realize the accurate selection of multiple retrieval modes / retrieval strategies. Moreover, when determining the target detection strategy as the sentence retrieval strategy, it can further accurately and quickly match the sentence vector / fragment vector of the target retrieval sentence in the retrieval information from the target database to obtain the target knowledge fragment or target document corresponding to the sentence vector / fragment vector, thereby realizing the accurate retrieval of knowledge fragments and solving the problem that the retrieval method / retrieval mode in the prior art is single and cannot meet the diverse retrieval needs of users. It realizes the problem of supporting the retrieval of knowledge fragments of different granularities and types such as long and short sentences, paragraphs, phrases, word units, entities, knowledge graphs, etc. in the document content, expands the retrieval path / retrieval mode, improves the recall degree of retrieval results, and improves the utilization efficiency of knowledge resources in HowNet.
[0039] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0041] Figure 1A flowchart of a document retrieval method according to an embodiment of the present application;
[0042] Figure 2 This is a flowchart of a document retrieval method according to another embodiment of the present application;
[0043] Figure 3 This is a structural block diagram of a document retrieval device according to another embodiment of the present application.
[0044] Figure 4 This is a structural block diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION
[0045] Various aspects and features of the present application are described herein with reference to the accompanying drawings.
[0046] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.
[0047] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0048] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.
[0049] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.
[0050] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.
[0051] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.
[0052] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.
[0053] The present application embodiment provides a retrieval method, which can be specifically applied to electronic devices such as terminals and servers, such as Figure 1 As shown, the document retrieval method in this embodiment includes the following steps:
[0054] Step S101, based on the configuration mode of the search information, determining a target search strategy corresponding to the configuration mode, wherein the search strategy includes: a conditional search strategy or a statement search strategy;
[0055] Step S102, when the target search strategy is a conditional search strategy, based on the target search condition in the search information, a number of target documents meeting the target search condition are retrieved from the target database;
[0056] Step S103, when the target retrieval strategy is a sentence retrieval strategy, based on the target retrieval sentence in the retrieval information, determine the query language retrieval method or the vector similarity retrieval method as the target retrieval method, so as to retrieve from the target database a number of target documents or a number of target knowledge fragments that match the target retrieval sentence based on the target retrieval sentence using the target retrieval method.
[0057] The document retrieval method in this embodiment can accurately determine the conditional retrieval strategy or sentence retrieval strategy as the target retrieval strategy according to the configuration mode by obtaining the configuration mode of the retrieval information, and realizes the accurate selection of multiple retrieval modes / retrieval strategies. Moreover, when determining the target detection strategy as the sentence retrieval strategy, it can further accurately and quickly match the sentence vector / fragment vector of the target retrieval sentence in the retrieval information from the target database to obtain the target knowledge fragment or target document corresponding to the sentence vector / fragment vector, thereby realizing the accurate retrieval of knowledge fragments and solving the problem that the retrieval method / retrieval mode in the prior art is single and cannot meet the diverse retrieval needs of users. It realizes the problem of supporting the retrieval of knowledge fragments of different granularities and types such as long and short sentences, paragraphs, phrases, word units, entities, knowledge graphs, etc. in the document content, expands the retrieval path / retrieval mode, improves the recall degree of retrieval results, and improves the utilization efficiency of knowledge resources in HowNet.
[0058] Another embodiment of the present application provides a document retrieval method, comprising the following steps:
[0059] Step S201, pre-establishing a target database and a target vector library corresponding to the target database;
[0060] In this embodiment, the target database may be a Kbase database, which contains document data types such as master's theses, doctoral theses, journals, conferences, patents, and the like.
[0061] In the specific implementation process of this embodiment, a vector library can be pre-built according to the target database. That is, for any document, the document can be split based on each search level to obtain document fragments / knowledge fragments corresponding to each search level, and then each document fragment / knowledge fragment is converted into a vector to obtain a number of document fragment vectors. Specifically, since the document includes multiple levels such as subtitles, title combinations, titles, abstracts, keywords, parent titles, and child titles, these levels can be used as search levels to split the document to obtain a number of document fragments / knowledge fragments corresponding to each search level. In this embodiment, since the same search level may contain multiple document fragments / knowledge fragments, for example, there are multiple words corresponding to the search level of "keywords", it contains multiple document fragments. In this embodiment, after splitting to obtain document fragments, the document fragments can be vectorized to obtain corresponding document fragment vectors / fragment vectors. In this embodiment, the BGE model can be specifically used to generate document fragment vectors / fragment vectors.
[0062] Step S201, based on the first configuration mode of the search information, determining the condition search strategy as the target search strategy; or based on the second configuration mode of the search information, determining the statement search strategy as the target search strategy;
[0063] In this step, the corresponding target search strategy is determined based on the target configuration mode of the search information, wherein the target configuration mode includes the first configuration mode selected by the search condition and the second configuration mode input by the search statement. That is, when the user inputs the target search statement through the statement input box, the target configuration mode of the search information can be determined to be the first configuration mode, and then the statement search strategy can be determined to be the target search strategy; when the user checks / selects the search condition through the condition box, the target configuration mode of the search information can be determined to be the second configuration mode, and then the condition search strategy can be determined to be the target search strategy.
[0064] In this step, the search conditions include any one or more of the following: subject conditions, institutional conditions, research level conditions, and name conditions. That is to say, the condition boxes of the above-mentioned search conditions can be displayed at a predetermined position on the display interface, and then the user can configure the search information using the first configuration method selected by the search condition. Specifically, the user can determine the target search conditions for subsequent document retrieval by checking / selecting the search conditions. For example, the user selects "subject conditions" and "institutional conditions" as target search conditions. After the user selects the two search conditions of "subject conditions" and "institutional conditions", the corresponding key fields can be further configured for the two conditions, such as configuring "artificial intelligence" in the condition box corresponding to the "subject conditions" and configuring "Tsinghua University" in the condition box corresponding to the "institutional conditions", thereby completing the configuration operation of the search information. At this time, since the target search condition is determined by the first configuration method selected by the search condition to complete the search information configuration, the first configuration method selected by the search condition can be determined as the target configuration method.
[0065] In this embodiment, when the user uses the second configuration mode of the search statement input mode to configure the search information, that is, when the user inputs the search statement in the predetermined statement input box, the second configuration mode of the search statement input can be determined as the target configuration mode.
[0066] Step S202, when the target search strategy is a conditional search strategy, execute step S203, and when the target search strategy is a statement search strategy, execute step S204;
[0067] Step S203, constructing a KSQL query language based on any one or several of the subject conditions, institution conditions, research level conditions and name conditions in the search information; and screening the documents in the target database based on the KSQL query language to obtain the target documents.
[0068] Step S204, performing search intent recognition on the target search sentence in the search information to obtain a conditional search intent or a non-conditional search intent; when obtaining a conditional search intent, determining the query language search method as the target search method, and executing step S205; when obtaining a non-conditional search intent, determining the vector similarity search method as the target search method, and executing step S206;
[0069] In this step, when determining the search intent, a large language model can be used to identify the search intent, or it can be determined based on whether the search information contains subject conditions, institutional conditions, research level conditions and / or name conditions. If any of the search conditions are included, the search intent is determined to be a conditional search intent, otherwise the search intent is determined to be a non-conditional search intent.
[0070] Step S205, determining a target search condition based on the keywords in the target search sentence, so as to screen the documents in the target database based on the target search condition to obtain the target documents;
[0071] Step S206, determining a target sentence vector based on the target search sentence, and screening documents in a target database based on the target sentence vector to obtain a target document containing the target sentence vector or a target knowledge fragment matching the target sentence vector;
[0072] In the specific implementation of this step, the target sentence vector can be first determined based on the key fragments in the target search sentence; then, based on the target sentence vector and the target search level, each target document fragment vector whose vector similarity meets the predetermined similarity condition and meets the target search level is determined from the target vector library corresponding to the target database; based on each of the target document fragment vectors and the correspondence between each document fragment vector and the document fragment, the target document fragment corresponding to each target document fragment vector is determined, so as to determine the several target knowledge fragments or several target documents based on each of the target document fragments. In this embodiment, the similarity cosine coefficient between the target sentence vector and the document fragment vector can be specifically calculated; then the documents and document fragments corresponding to the top n document fragment vectors with similarity are returned, where n is between 5 and 10; or based on the vector similarity of each document fragment vector and the predetermined similarity threshold, the document fragment vector whose vector similarity is greater than the similarity threshold is determined as the target document fragment vector.
[0073] In this embodiment, the target search level includes any one or more of the following: subheading, title combination, article title, abstract, keyword, parent title, and child title. Specifically, before performing a search, the user can set and adjust the target search level according to actual needs, for example, from the search levels of "subheading", "title combination", "article title", "abstract", "keyword", "parent title", and "child title", determine the two search levels of "subheading" and "article title" as the target search levels, or determine the three search levels of "article title", "abstract", and "keyword" as the target search levels.
[0074] After determining the target retrieval level, the document fragment vectors corresponding to the target retrieval level and used for similarity calculation can be screened out from the document fragment vectors in the vector library. Then, the screened document fragment vectors are subjected to similarity calculation with the target sentence vector to obtain the similarity of each document fragment vector. Finally, based on the similarity of each document fragment vector, the target document fragment vector can be screened out from each document fragment vector to obtain the target document fragment vector.
[0075] In the embodiment of the present application, after obtaining a number of target documents or a number of target knowledge fragments / target document fragments, the target documents and target knowledge fragments / target document fragments can also be sorted to obtain a sorted target document list and a target knowledge fragment list. Specifically, Kbase sorting can be used to return a list of documents with high relevance; finally, the target document list and the target knowledge fragment list are displayed.
[0076] The document retrieval method in this embodiment can accurately determine the conditional retrieval strategy or sentence retrieval strategy as the target retrieval strategy according to the configuration mode by obtaining the configuration mode of the retrieval information, and realizes the accurate selection of multiple retrieval modes / retrieval strategies. Moreover, when determining the target detection strategy as the sentence retrieval strategy, it can further accurately and quickly match the sentence vector / fragment vector of the target retrieval sentence in the retrieval information from the target database to obtain the target knowledge fragment or target document corresponding to the sentence vector / fragment vector, thereby realizing the accurate retrieval of knowledge fragments and solving the problem that the retrieval method / retrieval mode in the prior art is single and cannot meet the diverse retrieval needs of users. It realizes the problem of supporting the retrieval of knowledge fragments of different granularities and types such as long and short sentences, paragraphs, phrases, word units, entities, knowledge graphs, etc. in the document content, expands the retrieval path / retrieval mode, improves the recall degree of retrieval results, and improves the utilization efficiency of knowledge resources in HowNet.
[0077] The document retrieval method in this application is complementary to the traditional document retrieval method in terms of retrieval path / retrieval mode. Compared with a single retrieval mode, vector retrieval combined with keyword retrieval can expand the retrieval path, improve the recall of retrieval results, and improve the utilization efficiency of CNKI knowledge resources.
[0078] Moreover, compared with generative retrieval, the retrieval results of this application are stable and controllable. The document retrieval based on retrieval enhancement does not have a large language model to generate content, and the output content does not have the problem of hallucination. The retrieval results are objective, accurate, and safe, and take into account both recall rate and precision rate.
[0079] In this application, by establishing a vector database embedded in the HowNet database, the search granularity can be flexibly matched, such as single keyword, multiple keywords, long and short sentences, search formulas, document paragraphs, etc. Compared with vector searches of fixed types and lengths, the present invention provides more granularity options, more dense searches, and higher results relevance.
[0080] This application can realize fragment retrieval, locate the specific source and position in the original document, and achieve better traceability verification effect.
[0081] This application uses natural language conversion to adapt to the Kbase database, which can be converted into search formulas and KSQL statements, laying the foundation for accurate conditional retrieval.
[0082] Another embodiment of the present application provides a document retrieval method, such as Figure 2 As shown, the specific steps include:
[0083] Step 1: Responding to the user's selection operation on the condition box and the key field configured by the user on the condition box, obtaining search information; or responding to the user's input of a target search statement in a predetermined input box, obtaining search information;
[0084] This step specifically includes the following three situations:
[0085] Case 1: The user selects the three conditions of "subject condition", "institutional condition" and "research level" from the several subject conditions displayed on the display interface, and the user configures the corresponding key fields for these two conditions, such as configuring the key field "artificial intelligence" for the "subject condition", configuring the key field "Tsinghua University" for the "institutional condition", and configuring the key field "technical research" for the "research level condition". The user completes the configuration operation of the retrieval information, and the retrieval device can obtain the retrieval information.
[0086] Case 2: The user enters "Help me find doctoral dissertations on artificial intelligence from the School of Computer Science, Tsinghua University" in the input box. This will generate / obtain the search information "Help me find doctoral dissertations on artificial intelligence from the School of Computer Science, Tsinghua University".
[0087] Case three: The user enters "The key points for developing new quality productivity in health and wellness" in the input box, thereby generating / obtaining the target search information "The key points for developing new quality productivity in health and wellness".
[0088] Step 2: When the target configuration mode is the first configuration mode selected by the search condition, the condition search strategy is determined as the target search strategy, and step 3 is executed; when the target configuration mode is the second configuration mode of the search statement input mode, the statement search strategy is determined as the target search strategy, and step 4 is executed;
[0089] In this part, when the user performs a selection operation on the condition box and configures the key field, the condition search strategy can be determined as the target search strategy, and step three is executed; when the user enters a search statement in the input box, the statement search strategy can be determined as the target search strategy, and step four is executed;
[0090] Step 3: Based on the target search condition in the search information, a number of target documents meeting the target search condition are retrieved from the target database;
[0091] In the specific implementation process of this step, after it is determined that the user has selected the three conditions of "subject condition", "institutional condition" and "research level", and the key fields configured by the user for each condition are obtained, a KSQL query language can be constructed based on these search conditions and corresponding key fields; based on the KSQL query language, the documents in the target database are screened to obtain the target documents.
[0092] Step 4: Recognize the search intent of the target search sentence in the search information to obtain a conditional search intent or a non-conditional search intent;
[0093] In this step, it is specifically possible to determine whether the keywords in the target search statement meet the predetermined conditional attributes based on the keywords in the target search statement, so as to determine whether the target search statement contains any one or more of the search conditions of the subject condition, the institution condition, the research level condition, and the name condition / author condition. If any one or more of the search conditions are contained, the search intent can be determined to be a conditional search intent. For example, when the keywords involve conditional attributes such as school name, person name, field / topic, research level, etc., it can be determined that the target search statement corresponds to a conditional search intent. Otherwise, it can be determined that the target search statement corresponds to a non-conditional search intent.
[0094] In this embodiment, when performing search intent recognition, the search intent can be recognized by using a pre-trained large language model LLM. The search intent can also be recognized based on the attributes of keywords.
[0095] For example, taking the target search sentence "Help me find doctoral dissertations on artificial intelligence from the School of Computer Science at Tsinghua University" as an example, since the target search sentence contains keywords such as "Tsinghua University", "artificial intelligence", and "doctoral dissertation", these keywords involve several conditional attributes such as institutional conditions, subject conditions, and research level conditions ("Tsinghua University" corresponds to institutional conditions, "artificial intelligence" subject conditions, and "doctoral dissertation" corresponds to research level conditions), it can be determined that the search intent of the target search sentence is a conditional search intent.
[0096] For example, taking the target search statement "key points for developing new quality productivity in health and medical care" as an example, since the keywords in the target search statement do not involve subject conditions, institutional conditions, research level conditions, name conditions / author conditions, etc., it can be determined that the search intent of the target search statement is a non-conditional search intent.
[0097] Step 5: When a conditional search intent is obtained, the query language search method is determined as the target search method, and step 6 is executed; when a non-conditional search intent is obtained, the vector similarity search method is determined as the target search method, and step 7 is executed;
[0098] Step 6: Determine a target search condition based on the keywords in the target search sentence, and screen the documents in the target database based on the target search condition to obtain the target documents;
[0099] For example, when "Help me search for doctoral dissertations on artificial intelligence from the School of Computer Science at Tsinghua University" is determined as a conditional search intent, the KSQL query language can be constructed based on the keywords in the search statement and the subject conditions corresponding to each keyword. For example, after determining the subject condition, the condition corresponding to the keyword "artificial intelligence" is the subject condition; determining the condition corresponding to "Tsinghua University" is the institution condition; and determining the condition corresponding to "doctoral dissertation" is the research level condition, the KSQL query language can be constructed based on the above information, and then the documents in the target database can be screened based on the KSQL query language to obtain the target documents.
[0100] Step seven: determine a target sentence vector based on the target search sentence, and screen documents in a target database based on the target sentence vector to obtain a target document containing the target sentence vector or a target knowledge fragment matching the target sentence vector.
[0101] In this step, after determining that "the key points for developing new quality productivity in health and wellness" is a non-conditional search intent, it can be further vectorized to obtain the target sentence vector. Then, according to the search level set by the user (for example, the search level is the title), the document fragment vector corresponding to the search level is obtained from the pre-built vector library corresponding to the target database. Then, the similarity between these document fragment vectors and the target sentence vector is calculated respectively to obtain the similarity corresponding to each document fragment vector. Finally, a number of target document fragment vectors can be screened out from these document fragment vectors according to the similarity, and the corresponding target document can be determined from the target database based on these target document fragment vectors.
[0102] In this embodiment, when constructing a vector library based on the target database, a correspondence between each document segment vector and a document identifier may be pre-established, so as to facilitate subsequent rapid determination of a target document from the target database based on the document identifier.
[0103] In this embodiment, by constructing a vector library of knowledge fragment vectors / document fragment vectors containing several retrieval levels (different granularities), it is convenient to subsequently search for the target sentence vector according to different retrieval levels, so that the retrieval results are more in line with user needs and document retrieval according to different retrieval levels (different granularities) is realized.
[0104] In the specific implementation of this embodiment, the routing module can be used to determine the target configuration mode, determine the target search strategy, and when the target search strategy is a statement search strategy, determine the search mode corresponding to the target search strategy, so that the corresponding search module can perform the search operation.
[0105] In this embodiment, a foreign language database corresponding to the language type and a foreign language vector library corresponding to the foreign language database can also be pre-constructed. Thus, when a user wants to search for foreign language documents, the target language type of the question can be identified based on a predetermined token identifier, and then the vector library corresponding to the target language type / language type is determined. Then the input search statement / question (query) is translated into a natural language and converted into a foreign language semantic vector, and a search is performed in the foreign language vector library based on the foreign language semantic vector to obtain a vectorized knowledge fragment / foreign language fragment vector that is closer to the foreign language semantic vector / question vector. The specific search process is the same as the principle of the vector similarity search method in the above embodiment, which also calculates the similarity cosine coefficient, inverts the sequence, samples top_k, and returns multiple specific foreign language paragraph contents, positions, and the full text of the document.
[0106] The present application embodiment provides a retrieval device, such as Figure 3 As shown, including:
[0107] The determination module 11 is used to determine a target search strategy corresponding to the configuration mode based on the configuration mode of the search information, wherein the search strategy includes: a conditional search strategy and a statement search strategy;
[0108] A search module 12, configured to retrieve a number of target documents that meet the target search condition from a target database based on the target search condition in the search information when the target search strategy is a conditional search strategy;
[0109] The retrieval module 12 is also used to determine the query language retrieval method or the vector similarity retrieval method as the target retrieval method based on the target retrieval statement in the retrieval information when the target retrieval strategy is a sentence retrieval strategy, so as to retrieve a number of target documents or a number of target knowledge fragments matching the target retrieval statement from the target database using the target retrieval method based on the target retrieval statement.
[0110] In the specific implementation process of this embodiment, the configuration method includes a first configuration method for selecting the search conditions and a second configuration method for inputting the search statement; the determination module is specifically used to: determine the conditional search strategy as the target search strategy based on the first configuration method of the search information; or determine the statement search strategy as the target search strategy based on the second configuration method of the search information.
[0111] In the specific implementation process of this embodiment, the target search conditions include any one or more of the following: subject conditions, institutional conditions, research level conditions, and name conditions; the search module includes a search unit, and the search unit is used to: construct a query language based on any one or more of the subject conditions, institutional conditions, research level conditions, and name conditions in the search information; and screen the documents in the target database based on the query language to obtain the several target documents.
[0112] In the specific implementation process of this embodiment, the retrieval module also includes an intention recognition module and a method determination module. The intention recognition module is used to: identify the retrieval intention of the target retrieval statement to obtain a conditional retrieval intention or a non-conditional retrieval intention; the method determination module is used to: when obtaining a conditional retrieval intention, determine the query language retrieval method as the target retrieval method; when obtaining a non-conditional retrieval intention, determine the vector similarity retrieval method as the target retrieval method.
[0113] In the specific implementation process of this embodiment, the search module further includes a second search unit and a third search unit. When the target search method is a query language search method, the second search unit is used to: determine the target search condition based on the keywords in the target search sentence, so as to screen the documents in the target database based on the target search condition to obtain the target documents;
[0114] When the target retrieval method is a vector similarity retrieval method, the third retrieval unit is used to: determine a target sentence vector based on the target retrieval statement, so as to screen documents in a target database based on the target sentence vector, and obtain a target document containing the target sentence vector or a target knowledge fragment matching the target sentence vector.
[0115] In the specific implementation process of this embodiment, the third retrieval unit is specifically used to: determine the target sentence vector based on the key fragments in the target search sentence; determine each target document fragment vector whose vector similarity meets the predetermined similarity condition and meets the target retrieval level from the target vector library corresponding to the target database based on the target sentence vector and the target retrieval level; determine the target document fragment corresponding to each target document fragment vector based on each target document fragment vector and the correspondence between each document fragment vector and the document fragment, so as to determine the several target knowledge fragments or several target documents based on each target document fragment.
[0116] In the specific implementation process of this embodiment, the retrieval device also includes a display module, which is used to: sort each target document and target knowledge fragment to obtain a sorted target document list and target knowledge fragment list; and display the target document list and target knowledge fragment list.
[0117] The document retrieval device in the present application can accurately determine the conditional retrieval strategy or sentence retrieval strategy as the target retrieval strategy according to the configuration mode by obtaining the configuration mode of the retrieval information, and realizes the accurate selection of multiple retrieval modes / retrieval strategies. Moreover, when determining the target detection strategy as the sentence retrieval strategy, it can further accurately and quickly match the sentence vector / fragment vector of the target retrieval sentence in the retrieval information from the target database to obtain the target knowledge fragment or target document corresponding to the sentence vector / fragment vector, thereby realizing the accurate retrieval of knowledge fragments and solving the problem that the retrieval method / retrieval mode in the prior art is single and cannot meet the diverse retrieval needs of users. It realizes the problem of supporting the retrieval of knowledge fragments of different granularities and types such as long and short sentences, paragraphs, phrases, word units, entities, knowledge graphs, etc. in the document content, expands the retrieval path / retrieval mode, improves the recall degree of retrieval results, and improves the utilization efficiency of knowledge resources in HowNet.
[0118] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the following method steps are implemented:
[0119] Step 1: based on the target configuration mode of the retrieval information, determine the target retrieval strategy corresponding to the target configuration mode, wherein the retrieval strategy includes: a conditional retrieval strategy and a statement retrieval strategy;
[0120] Step 2: when the target search strategy is a conditional search strategy, based on the target search condition in the search information, a number of target documents meeting the target search condition are retrieved from the target database;
[0121] Step three, when the target retrieval strategy is a sentence retrieval strategy, based on the target retrieval sentence in the retrieval information, determine the query language retrieval method or the vector similarity retrieval method as the target retrieval method, so as to retrieve a number of target documents or a number of target knowledge fragments matching the target retrieval sentence from the target database based on the target retrieval sentence using the target retrieval method.
[0122] The specific implementation process of the above method steps can be found in the embodiments of any of the above document retrieval methods, and this embodiment will not be repeated here.
[0123] The device in this application can accurately determine the conditional search strategy or sentence search strategy as the target search strategy according to the configuration mode by obtaining the configuration mode of the search information, and realizes the accurate selection of multiple search modes / search strategies. Moreover, when determining the target detection strategy as the sentence search strategy, it can further accurately and quickly match the sentence vector / fragment vector of the target search sentence in the search information from the target database to obtain the target knowledge fragment or target document corresponding to the sentence vector / fragment vector, thereby realizing the accurate search for knowledge fragments and solving the problem that the search mode / search mode in the prior art is single and cannot meet the diverse search needs of users. It realizes the problem of supporting the search of knowledge fragments of different granularities and types such as long and short sentences, paragraphs, phrases, word units, entities, knowledge graphs, etc. in the document content, expands the search path / search mode, improves the recall degree of search results, and improves the utilization efficiency of knowledge resources in HowNet.
[0124] Another embodiment of the present application provides an electronic device, such as Figure 4 As shown, at least a memory 1 and a processor 2 are included, the memory 1 stores a computer program, and the processor 2 implements the following method steps when executing the computer program on the memory 1:
[0125] Step 1: based on the target configuration mode of the retrieval information, determine the target retrieval strategy corresponding to the target configuration mode, wherein the retrieval strategy includes: a conditional retrieval strategy and a statement retrieval strategy;
[0126] Step 2: when the target search strategy is a conditional search strategy, based on the target search condition in the search information, a number of target documents meeting the target search condition are retrieved from the target database;
[0127] Step three, when the target retrieval strategy is a sentence retrieval strategy, based on the target retrieval sentence in the retrieval information, determine the query language retrieval method or the vector similarity retrieval method as the target retrieval method, so as to retrieve a number of target documents or a number of target knowledge fragments matching the target retrieval sentence from the target database based on the target retrieval sentence using the target retrieval method.
[0128] The specific implementation process of the above method steps can be found in the embodiments of any of the above document retrieval methods, and this embodiment will not be repeated here.
[0129] The device in this application can accurately determine the conditional search strategy or sentence search strategy as the target search strategy according to the configuration mode by obtaining the configuration mode of the search information, and realizes the accurate selection of multiple search modes / search strategies. Moreover, when determining the target detection strategy as the sentence search strategy, it can further accurately and quickly match the sentence vector / fragment vector of the target search sentence in the search information from the target database to obtain the target knowledge fragment or target document corresponding to the sentence vector / fragment vector, thereby realizing the accurate search for knowledge fragments and solving the problem that the search mode / search mode in the prior art is single and cannot meet the diverse search needs of users. It realizes the problem of supporting the search of knowledge fragments of different granularities and types such as long and short sentences, paragraphs, phrases, word units, entities, knowledge graphs, etc. in the document content, expands the search path / search mode, improves the recall degree of search results, and improves the utilization efficiency of knowledge resources in HowNet.
[0130] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.
Claims
1. A document retrieval method, characterized in that: include: Based on the target configuration mode of the retrieval information, determining a target retrieval strategy corresponding to the target configuration mode, wherein the retrieval strategy includes: a conditional retrieval strategy and a statement retrieval strategy; When the target search strategy is a conditional search strategy, based on the target search condition in the search information, a number of target documents meeting the target search condition are retrieved from the target database; When the target retrieval strategy is a sentence retrieval strategy, based on the target retrieval sentence in the retrieval information, a query language retrieval method or a vector similarity retrieval method is determined as the target retrieval method, so as to retrieve a number of target documents or a number of target knowledge fragments matching the target retrieval sentence from the target database based on the target retrieval sentence using the target retrieval method.
2. The method according to claim 1, characterized in that The configuration mode includes a first configuration mode in which a search condition is selected and a second configuration mode in which a search statement is input; The configuration mode based on the search information, determining the target search strategy corresponding to the configuration mode, specifically includes: Based on the first configuration mode of the search information, determining the conditional search strategy as the target search strategy; Alternatively, based on the second configuration mode of the search information, the sentence search strategy is determined as the target search strategy.
3. The method according to claim 1, characterized in that The target search conditions include any one or more of the following: subject conditions, institution conditions, research level conditions, and name conditions; The step of retrieving a plurality of target documents that meet the target search condition from a target database based on the target search condition in the search information specifically includes: Constructing a query language based on any one or more of the search conditions of the subject condition, the institution condition, the research level condition and the name condition in the search information; The documents in the target database are screened based on the query language to obtain the target documents.
4. The method according to claim 1, characterized in that The step of determining the query language retrieval method or the vector similarity retrieval method as the target retrieval method based on the target retrieval sentence in the retrieval information specifically includes: Recognize the search intent of the target search sentence to obtain conditional search intent or non-conditional search intent; When obtaining the conditional search intention, determine the query language search method as the target search method; When a non-conditional search intention is obtained, a vector similarity search method is determined as a target search method.
5. The method according to claim 1, characterized in that When the target search method is a query language search method, the step of retrieving a plurality of target documents or a plurality of target knowledge fragments matching the target search statement from a target database using the target search method based on the target search statement specifically includes: Determining a target search condition based on the keywords in the target search sentence, so as to screen the documents in the target database based on the target search condition to obtain the target documents; When the target search method is a vector similarity search method, the step of retrieving a plurality of target documents or a plurality of target knowledge fragments matching the target search statement from a target database by using the target search method based on the target search statement specifically includes: A target sentence vector is determined based on the target search sentence, so as to screen documents in a target database based on the target sentence vector to obtain a target document containing the target sentence vector or a target knowledge fragment matching the target sentence vector.
6. The method according to claim 5, characterized in that The step of determining a target sentence vector based on the target search sentence, screening documents in a target database based on the target sentence vector, and obtaining a target document containing the target sentence vector or a target knowledge fragment matching the target sentence vector, specifically includes: Determining the target sentence vector based on the key fragments in the target search sentence; Based on the target sentence vector and the target retrieval level, determining, from a target vector library corresponding to the target database, target document segment vectors whose vector similarity meets a predetermined similarity condition and meets the target retrieval level; Based on each of the target document segment vectors and the correspondence between each document segment vector and the document segment, the target document segment corresponding to each target document segment vector is determined, so as to determine the plurality of target knowledge segments or the plurality of target documents based on each of the target document segments.
7. The method according to any one of claims 1 to 6, characterized in that: After obtaining a plurality of target documents or a plurality of target knowledge fragments, the method further comprises: Sorting each target document and target knowledge fragment to obtain a sorted target document list and a sorted target knowledge fragment list; The target document list and the target knowledge fragment list are displayed.
8. A document retrieval device, characterized in that: include: A determination module, used to determine a target search strategy corresponding to the configuration mode based on the configuration mode of the search information, wherein the search strategy includes: a conditional search strategy and a statement search strategy; A retrieval module, used for retrieving a number of target documents that meet the target retrieval conditions from a target database based on the target retrieval conditions in the retrieval information when the target retrieval strategy is a conditional retrieval strategy; The retrieval module is also used to determine the query language retrieval method or the vector similarity retrieval method as the target retrieval method based on the target retrieval statement in the retrieval information when the target retrieval strategy is a sentence retrieval strategy, so as to retrieve a number of target documents or a number of target knowledge fragments matching the target retrieval statement from the target database using the target retrieval method based on the target retrieval statement.
9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the document retrieval method according to any one of claims 1 to 7 are implemented.
10. An electronic device, characterized in that: The method comprises at least a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the document retrieval method according to any one of claims 1 to 7 when executing the computer program on the memory.