Literature retrieval result determination method and device, electronic equipment and storage medium

Abstract

The invention provides a determination method and device of a literature retrieval result, electronic equipment and a storage medium. The method for determining the document retrieval result comprises the steps of determining a plurality of target documents corresponding to a retrieval text based on the retrieval text; for each target document in the plurality of target documents, inputting a target feature set corresponding to the target document into the trained correlation detection model to obtain a target correlation score corresponding to the target document; sorting the plurality of target literatures based on the target correlation scores corresponding to the plurality of target literatures; and determining a literature retrieval result corresponding to the retrieval text based on the plurality of sorted target literatures. According to the scheme, the cognitive load and the retrieval accuracy of the user are comprehensively considered, the understanding difficulty of the user can be reduced under the condition that the accuracy of the document retrieval result is maintained, and the retrieval efficiency of the user can be improved.

Description

Technical Field

[0001] This invention relates to the field of document retrieval, and more specifically to a method for determining document retrieval results, a device for determining document retrieval results, an electronic device, a storage medium, and a computer program product. Background Technology

[0002] With the continuous development of the digital age, the number of academic documents is increasing day by day. Therefore, how to quickly and accurately filter out documents that are highly relevant to the search text from a large number of documents has become a core requirement in academic research, technology development and various information query scenarios.

[0003] Existing literature retrieval technologies typically rely on keyword matching as the basis for searching. They filter relevant literature by comparing the degree of overlap between keywords in the search text and fields such as document titles, abstracts, and keywords. These are then sorted by publication date or citation count. This approach is too limited and inefficient for users. Therefore, a method to improve user retrieval efficiency is urgently needed. Summary of the Invention

[0004] The present invention was proposed in view of the above-mentioned problems. The present invention provides a method for determining document retrieval results, a device for determining document retrieval results, an electronic device, a storage medium, and a computer program product.

[0005] According to one aspect of the present invention, a method for determining document retrieval results is provided. The method includes: determining multiple target documents corresponding to the retrieval text based on the retrieval text; for each target document, inputting a set of target features corresponding to that target document into a trained relevance detection model to obtain a target relevance score for that target document, wherein the set of target features includes a document type feature, a target word diversity feature, a target word density feature, and a retrieval text relevance feature corresponding to the target document; the document type feature is determined based on the document type of the target document; the target word diversity feature is determined based on the total number of non-repeating target words in the target text corresponding to the target document; the target text includes a title, keywords, and an abstract; the target words have entity semantics; the target word density feature is determined based on the density of target words in the target text; and the retrieval text relevance feature is determined based on the relevance between the target text and the retrieval text; ranking the multiple target documents based on their respective target relevance scores; and determining the document retrieval result corresponding to the retrieval text based on the ranked multiple target documents.

[0006] For example, the method for determining the above-mentioned literature search results also includes: For each of the multiple preset documents, Based on the target text corresponding to the preset document, the document type corresponding to the preset document is determined by the document type detection model after training. Multiple preset documents are used to match with the search text to determine multiple target documents among the multiple preset documents, or multiple preset documents as multiple target documents. Based on the document type, determine the document type characteristics corresponding to the preset document.

[0007] For example, the method for determining the above-mentioned literature search results also includes: Based on the target texts corresponding to multiple first training documents, the predicted types corresponding to each of the multiple first training documents are determined by a pre-trained document type detection model. Based on the predicted types and document type labels corresponding to each of the multiple first training documents, the model parameters of the pre-trained document type detection model are adjusted. Based on the target texts corresponding to multiple second training documents, the predicted types corresponding to multiple second training documents are determined by the adjusted pre-trained document type detection model. The total number of multiple second training documents is greater than the total number of multiple first training documents. Based on the predicted types and document type labels corresponding to each of the multiple second training documents, the model parameters of the pre-trained document type detection model are adjusted to obtain the document type detection model after training.

[0008] For example, based on the target text corresponding to the preset document, the document type corresponding to the preset document is determined by the trained document type detection model, including: Based on the target text corresponding to the preset document and the preset prompt words, the document type corresponding to the preset document is determined by the document type detection model after training. The preset prompt words are used to prompt the document type detection model after training to determine the document type corresponding to the preset document based on the research question, research method, research object, research conclusion and interdisciplinary degree of the preset document.

[0009] For example, the method for determining the above-mentioned literature search results also includes: For each of the multiple preset documents, Using a predefined vocabulary, the target text corresponding to the predefined document is segmented to obtain the segmented vocabulary set corresponding to the predefined document. Multiple predefined documents are used to match with the search text to determine multiple target documents among the multiple predefined documents, or multiple predefined documents as multiple target documents. The word segmentation word set is filtered by a filter word list to obtain the target word set corresponding to the preset document. The filter word list includes words that do not have entity semantics, and the target word set includes at least one target word corresponding to the preset document. The total number of the same target word in the target word set is greater than or equal to 1. Based on the target word set, the target word diversity feature and the target word density feature corresponding to the preset document are determined. The target word density feature corresponding to the preset document is determined based on the total number of target words in the target word set and the total number of sentences in the target text.

[0010] For example, the method for determining the above-mentioned literature search results also includes: For each of the multiple third training documents, the training feature set corresponding to that third training document is input into the untrained relevance detection model to obtain the predicted relevance score for that third training document. The multiple third training documents are determined based on the same retrieval training text. The training feature set includes the document type feature, the target word diversity feature, the target word density feature, and the retrieval training text relevance feature corresponding to that third training document. The document type feature is determined based on the document type of the third training document. The target word diversity feature is determined based on the total number of non-repeating target words in the target text of the third training document. The target word density feature is determined based on the density of target words in the target text. The retrieval training text relevance feature is determined based on the relevance between the retrieval training text and the third training document. Based on the prediction relevance scores of each of the multiple third training documents, the prediction order of the multiple third training documents is determined. Based on the differences between the predicted ranking order and the labeled ranking order of multiple third training documents, the model parameters of the incomplete relevance detection model are adjusted.

[0011] For example, the document type is one of the following: declarative, procedural, in-depth, and related types. The declarative type describes conceptual knowledge, the procedural type describes methodological knowledge, the in-depth type describes a deeper extension of conceptual knowledge, and the related type describes a horizontal extension of conceptual knowledge. The annotation order is determined based on the annotation relevance scores of each of the multiple third training documents. For each third training document, the annotation relevance scores of the third training document, in descending order, are as follows: when the document type of the third training document is declarative, when the document type of the third training document is procedural, when the document type of the third training document is in-depth, and when the document type of the third training document is related.

[0012] According to another aspect of the present invention, a document retrieval result determination device is also provided. The device comprises: a target document determination module, used to determine multiple target documents corresponding to the retrieval text based on the retrieval text; a target relevance score determination module, used to input a target feature set corresponding to each target document into a trained relevance detection model to obtain a target relevance score for that target document, wherein the target feature set includes a document type feature, a target word diversity feature, a target word density feature, and a retrieval text relevance feature corresponding to the target document; the document type feature is determined based on the document type of the target document; the target word diversity feature is determined based on the total number of non-repeating target words in the target text corresponding to the target document; the target text includes a title, keywords, and an abstract; the target words have entity semantics; the target word density feature is determined based on the density of target words in the target text; and the retrieval text relevance feature is determined based on the relevance between the target text and the retrieval text; a target document ranking module, used to rank the multiple target documents based on their respective target relevance scores; and a document retrieval result determination module, used to determine the document retrieval result corresponding to the retrieval text based on the ranked multiple target documents.

[0013] According to another aspect of the present invention, an electronic device is also provided, comprising a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-described method for determining document retrieval results.

[0014] According to another aspect of the present invention, a storage medium is also provided, which stores computer program instructions that, when executed, are used to perform the above-described method for determining document retrieval results.

[0015] According to another aspect of the present invention, a computer program product is also provided, the computer program product comprising computer program instructions, which, when executed by a processor, are used to perform the above-described method for determining document retrieval results.

[0016] According to the above-described scheme of the present invention, multiple target documents corresponding to the search text can be determined based on the search text. Then, for each target document, the set of target features corresponding to that target document is input into a trained relevance detection model to obtain a target relevance score for that target document. Next, based on the target relevance scores of each target document, the multiple target documents are ranked. Finally, based on the ranked multiple target documents, the document retrieval results corresponding to the search text are determined. This scheme determines document retrieval results based on the document type features, target word diversity features, target word density features, and search text relevance features of the target documents. It comprehensively considers the user's cognitive load and retrieval accuracy, reducing the user's understanding difficulty while maintaining the accuracy of the document retrieval results, thus improving the user's retrieval efficiency. Attached Figure Description

[0017] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0018] Figure 1 A schematic flowchart illustrating a method for determining document retrieval results according to an embodiment of the present invention is shown; Figure 2 A schematic block diagram of a device for determining document retrieval results according to an embodiment of the present invention is shown; Figure 3 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0020] Current academic search engines typically rank search results for academic documents and other textual resources based on factors such as text similarity, publication date, citation count, and download count. These existing solutions focus on the explicit attributes of academic texts, neglecting the implicit attribute of cognitive load on users. When users need to read, understand, and utilize an academic text, they must expend mental effort to recognize and learn the knowledge it contains. Therefore, the difficulty of understanding, breadth of knowledge, and depth of the subject matter of an academic text will affect the user's comprehension efficiency. Consequently, the literature search results output by current academic search engines are not necessarily the most efficient.

[0021] To at least partially solve the above problems, embodiments of the present invention provide a method for determining literature retrieval results. Figure 1 A schematic flowchart illustrating a method for determining document retrieval results according to an embodiment of the present invention is shown. Figure 1 As shown, the method for determining the literature search results may include steps S110 to S140.

[0022] In step S110, based on the search text, multiple target documents corresponding to the search text are determined.

[0023] The search text may include one or more paragraphs of text. In some embodiments, the search text may include multiple words. In still other embodiments, the search text may include multiple phrases. It is understood that the search text may also be composed of a combination of words, phrases, sentences, etc.

[0024] In some embodiments, a set of candidate documents for retrieval can be pre-built, and a subset of candidate documents that match the search text can be identified within this set. In one example, this subset of candidate documents can be designated as the target documents corresponding to the search text. In another example, this subset of candidate documents can be filtered using preset filtering criteria to determine the target documents. It is understood that the set of candidate documents can also be updated to include new candidate documents subsequently entered by the developers or users of the search page. Users can interact with the text boxes on the search page to input search text. It is understood that the search page may also include other functions, such as uploading and deleting candidate documents, depending on the actual needs of the developers. The search page provides access to the set of candidate documents for adding, deleting, modifying, and querying them.

[0025] In some embodiments, the matching relationship between the target document and the search text can be expressed as a matching relationship between the text content of the target document and the search text. In one example, the text content may include at least one of the following: title, keywords, and abstract. In another example, the text content may also include other existing document structures of the target document, such as the main text and references; this embodiment of the invention does not impose limitations here. Specifically, for example, if the text content of the candidate document includes part or all of the content of the search text, the candidate document and the search text have a matching relationship. More specifically, for example, if there are synonyms in the search text, then if the text content of the candidate document includes the synonym, it can be determined that the candidate document and the search text have a matching relationship.

[0026] In step S120, for each target document among multiple target documents, the target feature set corresponding to the target document is input into the trained relevance detection model to obtain the target relevance score corresponding to the target document.

[0027] The target feature set includes the document type feature, the target word diversity feature, the target word density feature, and the search text relevance feature corresponding to the target document.

[0028] The document type feature corresponding to the target document is determined based on the document type of the target document. It can be understood that the document type feature corresponding to the target document is an inherent feature of the target document and is unrelated to the search text. In some embodiments, the document type of each candidate document can be determined before determining the target document based on the search text. Then, based on the document type, the document type feature corresponding to the candidate document is determined. The above scheme can reduce the real-time computing power requirement when using the search text, which is beneficial to improving search efficiency. In some other embodiments, the document type feature corresponding to the target document can also be determined after determining the target document. This embodiment of the invention does not limit this and can be determined according to the actual needs of the developers. In one example, the document type can be determined manually or by a trained document type detection model. Different document types can be preset with different values ​​or numbers for differentiation. For each document type, the value or number corresponding to that document type can be used as the document type feature corresponding to that document type. The document type can be preset by the developers to represent the type of candidate document. The specific settings can be determined according to the actual needs of the developers, and this embodiment of the invention does not limit this. Different document types impose different cognitive loads on users. Therefore, this embodiment of the invention uses document type features as an element in the target feature set, thereby reducing the difficulty of understanding for users while maintaining the accuracy of document retrieval results and improving users' retrieval efficiency.

[0029] The target word diversity features corresponding to the target document are determined based on the total number of non-repeating target words in the target text corresponding to the target document. It can be understood that the target word diversity features are inherent features of the target document and are unrelated to the search text. Determining the target word diversity features of each candidate document before obtaining the search text can reduce the computational requirements when using the search text, thus improving search efficiency. The target text includes the title, keywords, and abstract. Target words have entity semantics. Target words can be determined in the target text through target word matching algorithms or models; this embodiment of the invention does not impose limitations. The title, keywords, and abstract included in the target text can accurately reflect the relevant content of the target document. Even without the main text, it can improve the efficiency of determining the target feature set, especially in the field of document retrieval where there are a large number of candidate documents and the main texts of these candidate documents are mostly long.

[0030] In one example, the total number of unique target words in the target text can be directly used as the target word diversity feature. In another example, the total number of unique target words can be normalized (the normalization benchmark can be the maximum value among the total number of unique target words in all candidate documents), and the normalized total can be used as the target word diversity feature. In a specific example, the target word diversity feature can be determined by the following formula: (word i -word min ) / (word max -word min (Among them, word) i The total number of unique target words in the target text corresponding to the target document. min The minimum number of unique target words in the target text corresponding to all candidate documents. max This refers to the maximum value among the total number of non-repeating target words in the target text corresponding to all candidate documents. It is understood that those skilled in the art can adjust the determination process of target word diversity features according to actual needs, and this embodiment of the invention does not impose any limitations. Different total numbers of target words impose different cognitive loads on users; therefore, this embodiment of the invention uses target word diversity features as one element in the target feature set, thereby reducing the difficulty of understanding for users while maintaining the accuracy of document retrieval results, and improving user retrieval efficiency.

[0031] The target word density feature for the target document is determined based on the density of target words in the target text. It can be understood that the target word density feature is an inherent feature of the target document and is independent of the search text. The target word density feature of each candidate document can be determined before obtaining the search text, thereby reducing the real-time computing power requirements when using the search text and improving search efficiency. In one example, the total number of non-repeating target words can be divided by the total number of sentences in the target text corresponding to the target document, and the resulting value (hereinafter referred to as the first mean) can be used as the target word density feature. In another example, the total number of target words (which can be repeated) can be divided by the total number of sentences, and the resulting value (hereinafter referred to as the second mean) can be used as the target word density feature. In yet another example, the first mean or the second mean can also be normalized to obtain the target word density feature. In a specific example, the target word density feature can be determined by the following formula: (density) i -density min ) / (density max -density min Density i The density is the second mean corresponding to the target literature.min Density is the minimum of the second mean among all candidate documents. max The value is the maximum of the second mean among all candidate documents. It is understood that those skilled in the art can adjust the determination process of the target word density feature according to actual needs, and this embodiment of the invention does not impose any limitations. Different target word densities impose different cognitive loads on users; therefore, this embodiment of the invention uses the target word density feature as an element in the target feature set, thereby reducing the user's understanding difficulty while maintaining the accuracy of document retrieval results, and improving the user's retrieval efficiency.

[0032] The relevance features of the search text corresponding to the target document are determined based on the relevance between the target text and the search text. These relevance features are dynamically quantifiable and correlate with the search text. Specifically, the relevance can be determined using ranking models or algorithms such as vector space models, latent semantic models, probabilistic retrieval models, and the BM25 (Best Matching 25) algorithm. In one example, the relevance can be directly used as the search text relevance feature. In another example, the relevance can be normalized to obtain the search text relevance feature.

[0033] When the document type feature is the numerical value or number corresponding to the document type, the target word diversity feature is the total number of non-repeating target words after normalization, the target word density feature is the first or second mean after normalization, and the retrieval text relevance feature is the relevance after normalization, the trained relevance detection model has a lower detection difficulty and is conducive to improving the representativeness of the target relevance score.

[0034] This invention provides a training process for a relevance detection model for reference: For each of the multiple third training documents, the training feature set corresponding to that third training document is input into the untrained relevance detection model to obtain the predicted relevance score corresponding to that third training document. Some of the candidate documents in the aforementioned candidate document set may be the same as the third training documents; it is understood that the total number of third training documents may be less than the total number of candidate documents. Multiple third training documents are determined based on the same retrieval training text. The retrieval training text may be the retrieval text used during the training phase. The training feature set includes the document type feature, the target word diversity feature, the target word density feature, and the retrieval training text relevance feature corresponding to the third training document. The document type feature is determined based on the document type of the third training document; the target word diversity feature is determined based on the total number of non-repeating target words in the target text of the third training document; the target word density feature is determined based on the density of target words in the target text; and the retrieval training text relevance feature is determined based on the relevance between the retrieval training text and the third training document. The specific generation method of the above-mentioned retrieval training text relevance features can be referred to the relevant content of retrieval text relevance features above, which is determined based on the third training document. This embodiment of the invention will not elaborate further here. The model parameters of the relevance detection model can be adjusted based on the predicted relevance score corresponding to the third training document. For example, the third training document can correspond to a relevance score label, and the loss value can be determined by comparing it with the predicted relevance score. Combining the above example, for the third training document, the numerical value or number corresponding to the document type can be used as the document type feature corresponding to the third training document; the total number of normalized non-repeating target words can be used as the target word diversity feature corresponding to the third training document; the normalized first mean or second mean can be used as the target word density feature corresponding to the third training document; and the normalized relevance can be used as the retrieval training text relevance feature corresponding to the third training document. The above scheme can reduce the learning difficulty of the relevance detection model and help improve the accuracy of the target relevance score output by the trained relevance detection model. Combined with the scheme in the previous example, the accuracy of the target relevance score can be further improved.

[0035] In step S130, the multiple target documents are sorted based on their respective target relevance scores.

[0036] In one example, the higher the target relevance score of a target document, the higher its ranking. In some embodiments, the sequence number and / or target relevance score of each target document may also be displayed on the search page; this is not a limitation of the present invention. In another example, target documents may be sorted in conjunction with user-selected filtering rules. For example, if a user selects a time range on the search page (i.e., the filtering rule in this example), target documents within that time range can be prioritized according to their target relevance scores. Then, target documents outside the time range are sorted according to their target relevance scores. In other words, the sorted target documents are, in order: target documents within the time range sorted by their target relevance scores, and target documents outside the time range sorted by their target relevance scores. It is understood that the specific content of the above filtering rules is not limited in the present invention and can be determined according to the user's actual needs.

[0037] In step S140, based on the sorted target documents, the document retrieval results corresponding to the retrieval text are determined.

[0038] In some embodiments, the sorted target documents can be directly determined as the document retrieval results corresponding to the search text. In other embodiments, the sorted target documents with serial numbers less than a preset number of target documents can be determined as the document retrieval results corresponding to the search text. In still other embodiments, the sorted target documents with target relevance scores higher than a preset target relevance score threshold can be determined as the document retrieval results corresponding to the search text. The specific values ​​of the preset number of target documents and the preset target relevance score threshold are not limited in this embodiment of the invention and can be determined according to the actual needs of the developers.

[0039] According to the above-described scheme of the present invention, multiple target documents corresponding to the search text can be determined based on the search text. Then, for each target document, the set of target features corresponding to that target document is input into a trained relevance detection model to obtain a target relevance score for that target document. Next, based on the target relevance scores of each target document, the multiple target documents are ranked. Finally, based on the ranked multiple target documents, the document retrieval results corresponding to the search text are determined. This scheme determines document retrieval results based on the document type features, target word diversity features, target word density features, and search text relevance features of the target documents. It comprehensively considers the user's cognitive load and retrieval accuracy, reducing the user's understanding difficulty while maintaining the accuracy of the document retrieval results, thus improving the user's retrieval efficiency.

[0040] For example, the method for determining the above-mentioned document retrieval results further includes steps S210 to S220.

[0041] In step S210, for each of the multiple preset documents, based on the target text corresponding to the preset document, the document type corresponding to the preset document is determined by the trained document type detection model.

[0042] Multiple preset documents are used to match the search text to identify multiple target documents among the multiple preset documents, or multiple preset documents as multiple target documents.

[0043] In one example, the document type of the candidate document can be determined before the user performs a search. This candidate document can be set as a preset document, and the document type can be set as the document type corresponding to the preset document. In another example, the document type of the target document can be determined after the user performs a search. This target document can be set as a preset document, and the document type can be set as the document type corresponding to the preset document.

[0044] In some embodiments, the target text corresponding to a preset document can be directly input into the trained document type detection model to obtain the document type corresponding to the preset document. In other embodiments, a portion of the target text can also be input into the trained document type detection model. For example, at least one of the title, keywords, and abstract included in the target text can be input into the trained document type detection model. As another example, non-repeating target words in the target text can be input as a text sequence into the trained document type detection model.

[0045] In step S220, for each of the multiple preset documents, the document type characteristics corresponding to the preset document are determined based on the document type.

[0046] In some embodiments, document types may have multi-level classifications. Taking a two-level classification as an example, each primary classification includes multiple secondary classifications. If the document type corresponding to a preset document is a secondary classification, the document type characteristics corresponding to the primary classification of that secondary classification can be determined as the document type characteristics corresponding to the preset document. In one example, the document type characteristics corresponding to different primary classifications may be different, and the document type characteristics corresponding to multiple secondary classifications of the same primary classification may be the same.

[0047] According to the above-described scheme of the present invention, for each of a plurality of preset documents, the document type corresponding to the preset document can be determined based on the target text corresponding to the preset document using a trained document type detection model. Then, based on the document type, the document type features corresponding to the preset document are determined. The above scheme improves the accuracy and efficiency of determining document type features through the trained document type detection model, thereby quickly and accurately determining the document retrieval results corresponding to the search text, which helps to improve the user's retrieval efficiency while maintaining the accuracy of the document retrieval results.

[0048] For example, the method for determining the above-mentioned literature search results further includes steps S310 to S340.

[0049] In step S310, based on the target texts corresponding to each of the multiple first training documents, the predicted type corresponding to each of the multiple first training documents is determined by a pre-trained document type detection model.

[0050] The first training document can be a document containing text content used to train a pre-trained document type detection model. This text content includes at least a title, keywords, and an abstract. In one example, some candidate documents in the candidate document set may also be the same as the first training document. The pre-trained document type detection model can be pre-trained based on natural language, and the pre-training process for large language models can be referenced; details will not be elaborated here. When the text content does not include the main body, training efficiency can be improved while maintaining output accuracy, especially when the total number of first training documents is large and the main body of the first training documents is usually long.

[0051] In one example, the target text corresponding to the first training document can be directly input into the pre-trained document type detection model. In another example, a portion of the target text can also be input into the pre-trained document type detection model. Specifically, refer to step S210 above. For each of the multiple preset documents, based on the target text corresponding to that preset document, the relevant content of the document type corresponding to that preset document is determined through the trained document type detection model. The specific details of this embodiment are not elaborated here.

[0052] In step S320, the model parameters of the pre-trained document type detection model are adjusted based on the predicted types and document type labels corresponding to each of the multiple first training documents.

[0053] In some embodiments, the document type label corresponding to the first training document is obtained based on the user's annotation operation. In other embodiments, the first training document and the document type-related prompt words obtained from the user's annotation operation can be input into a large language model to obtain the document type label corresponding to the first training document.

[0054] For multiple initial training documents, a first loss value can be determined based on the degree of difference between the predicted type and the document type label corresponding to the first training document. The model parameters of the pre-trained document type detection model can then be adjusted based on this first loss value. The magnitude of this first loss value can be positively correlated with the aforementioned degree of difference. It is understood that developers can also determine other loss values ​​based on actual needs, and then determine the overall loss value based on these other loss values ​​and the first loss value. The training objective of the pre-trained document type detection model can be to obtain the minimum possible overall loss value.

[0055] In step S330, based on the target texts corresponding to each of the multiple second training documents, the predicted type corresponding to each of the multiple second training documents is determined by the adjusted pre-trained document type detection model.

[0056] The total number of multiple second training documents is greater than the total number of multiple first training documents. This approach can reduce the probability of overfitting.

[0057] The second training document can be a document containing text content used to train the adjusted pre-trained document type detection model. This text content must include at least a title, keywords, and an abstract. In cases where the text content does not include the main body, training efficiency can be improved while maintaining output accuracy, especially when the total number of second training documents is large and the main body of the second training documents is typically long.

[0058] In one example, some of the candidate documents in the candidate document set may be the same as the second training documents mentioned above.

[0059] In some embodiments, the second training documents are different from all of the first training documents. In some alternative embodiments, some or all of the documents in the first training documents may be identified as part of the second training documents.

[0060] In one example, the target text corresponding to the second training document can be directly input into the adjusted pre-trained document type detection model. In another example, a portion of the target text can also be input into the adjusted pre-trained document type detection model. Specifically, refer to step S210 above. For each of the multiple preset documents, based on the target text corresponding to that preset document, the relevant content of the document type corresponding to that preset document is determined through the trained document type detection model. The specific details of this embodiment are not elaborated here.

[0061] In step S340, based on the predicted types and document type labels corresponding to each of the multiple second training documents, the model parameters of the pre-trained document type detection model are adjusted to obtain the document type detection model after training.

[0062] Referring to step S320, the document type label corresponding to the second training document can be obtained based on the second training document. Further details of this embodiment are omitted here.

[0063] For multiple second training documents, a second loss value can be determined based on the degree of difference between the predicted type and the document type label corresponding to the second training document. The model parameters of the adjusted pre-trained document type detection model can then be adjusted based on this second loss value. The magnitude of the second loss value can be positively correlated with the aforementioned degree of difference. It is understood that developers can also determine other loss values ​​according to actual needs, and then determine the overall loss value based on these other loss values ​​and the second loss value. The training objective of the adjusted pre-trained document type detection model can be to obtain the minimum possible overall loss value.

[0064] According to the above-described scheme of the present invention, based on the target text corresponding to each of the multiple first training documents, a pre-trained document type detection model can be used to determine the predicted type corresponding to each of the multiple first training documents. Then, based on the predicted types and corresponding document type labels of the multiple first training documents, the model parameters of the pre-trained document type detection model are adjusted. Next, based on the target text corresponding to each of the multiple second training documents, the adjusted pre-trained document type detection model is used to determine the predicted type corresponding to each of the multiple second training documents. Finally, based on the predicted types and corresponding document type labels of the multiple second training documents, the model parameters of the adjusted pre-trained document type detection model are adjusted to obtain the trained document type detection model. This scheme, through two rounds of parameter adjustment training of the document type detection model, can improve the accuracy of document type detection for target or candidate documents, thereby quickly and accurately determining the document retrieval results corresponding to the search text, which is beneficial for improving the user's retrieval efficiency while maintaining the accuracy of the document retrieval results.

[0065] For example, step S210 involves determining the document type corresponding to the preset document based on the target text corresponding to the preset document using the trained document type detection model, including step S211.

[0066] In step S211, based on the target text corresponding to the preset document and the preset prompt words, the document type corresponding to the preset document is determined by the document type detection model after training.

[0067] Preset prompts are used to guide the document type detection model after training. Based on the research question, research method, research object, research conclusion, and interdisciplinary degree of the preset document, the model determines the document type corresponding to the preset document.

[0068] Preset prompt words can be input into the trained document type detection model. Then, based on the target text corresponding to the preset document, the document type corresponding to the preset document can be determined by the trained document type detection model after inputting the preset prompt words. For details, please refer to the relevant content of step S210, which will not be elaborated here in this embodiment of the invention.

[0069] It is understandable that, in addition to first inputting the preset prompt words into the trained document type detection model and then inputting the relevant content of the target text, one can also first input the relevant content of the target text into the trained document type detection model and then input the preset prompt words, or input both simultaneously. This embodiment of the invention does not impose any limitations.

[0070] This embodiment of the invention provides the following preset prompt for reference: "Please determine the document type based on the title, keywords, and abstract. The determination should comprehensively consider five attributes: research question, research method, research object, research conclusion, and whether it is interdisciplinary." It can be understood that the document type detection model can also be used to output multi-level document types, such as the primary and secondary categories mentioned above. Correspondingly, the preset prompt may include specific definitions of the primary and secondary categories to facilitate the generation of accurate document types by the trained document type detection model.

[0071] According to the above-described solution of the present invention, the document type corresponding to the preset document can be determined based on the target text corresponding to the preset document and preset prompt words, using a trained document type detection model. The above solution, through preset prompt words, can improve the accuracy of the document type generated by the trained document type detection model. It can reduce the difficulty of understanding for users while maintaining the accuracy of document retrieval results, thus improving user retrieval efficiency.

[0072] For example, the method for determining the above-mentioned literature search results further includes steps S410 to S430.

[0073] In step S410, for each of the multiple preset documents, the target text corresponding to the preset document is segmented using a preset vocabulary to obtain the segmented word set corresponding to the preset document.

[0074] Multiple preset documents are used to match the search text to determine multiple target documents among the multiple preset documents, or multiple preset documents as multiple target documents. The specific definition of preset documents can be found in step S210. For each preset document among the multiple preset documents, based on the target text corresponding to the preset document, the relevant content in the document type corresponding to the preset document is determined by the trained document type detection model. The specific details of this embodiment are not elaborated here.

[0075] In some embodiments, different preset thesaurus exist for different search texts. This preset thesaurus is the same as the specified domain of the search text. In still other embodiments, the same general preset thesaurus is used for different search texts. The specific content of the preset thesaurus is not limited in this embodiment of the invention and can be determined according to the actual needs of the developers.

[0076] The target text corresponding to the preset document may include new words not found in the preset thesaurus. In some embodiments, the word segmentation set only includes preset words found in the preset thesaurus, and the number of these preset words in the word segmentation set is the same as the number of these preset words in the target text corresponding to the preset document.

[0077] In step S420, for each of the multiple preset documents, the word segmentation word set is filtered by a filtering word list to obtain the target word set corresponding to that preset document.

[0078] The filter word list includes words that do not have entity semantics, and the target word set includes at least one target word corresponding to the preset document. The total number of the same target word in the target word set is greater than or equal to 1.

[0079] It is understandable that the target word set does not include the filter words that exist in the filter word list, and the number of target words in the target word set is the same as the number of target words in the target text corresponding to the preset document.

[0080] In step S430, for each of the multiple preset documents, based on the target word set, the target word diversity feature and the target word density feature corresponding to the preset document are determined.

[0081] The target word density feature corresponding to the preset document is determined based on the total number of target words in the target word set and the total number of sentences in the target text.

[0082] The total number of target words in the target word set is the same as the total number of target words in the target text corresponding to the preset document. Sentences in the target text corresponding to the preset document can be counted based on punctuation marks to obtain the total number of sentences, and thus the target word density in the target text corresponding to the preset document.

[0083] Based on the total number of target words in the target text corresponding to the preset document and the target word density in the target text corresponding to the preset document, the target word diversity feature and the target word density feature corresponding to the preset document can be determined.

[0084] For details on the target word multivariate features and target word density features, please refer to step S120. For each target document among multiple target documents, the target feature set corresponding to the target document is input into the trained relevance detection model to obtain the relevant content of the target relevance score corresponding to the target document. The embodiments of the present invention will not be elaborated here.

[0085] According to the above-described scheme of the present invention, for each of a plurality of preset documents, the target text corresponding to that preset document is segmented using a preset thesaurus to obtain a segmented word set corresponding to that preset document. Then, for each of the plurality of preset documents, the segmented word set is filtered using a filtering thesaurus to obtain a target word set corresponding to that preset document. Finally, for each of the plurality of preset documents, based on the target word set, the target word diversity features and target word density features corresponding to that preset document are determined. The above scheme effectively determines the target word set through segmentation and filtering, thereby improving the accuracy of the determined target word diversity features and target word density features corresponding to the preset documents. This helps to reduce the difficulty of understanding for users while maintaining the accuracy of document retrieval results, and improves the user's retrieval efficiency.

[0086] For example, the method for determining the above-mentioned literature search results further includes steps S510 to S530.

[0087] In step S510, for each of the multiple third training documents, the training feature set corresponding to the third training document is input into the untrained relevance detection model to obtain the predicted relevance score corresponding to the third training document.

[0088] Multiple third training documents are determined based on the same retrieval training text. The training feature set includes the document type feature, the target word diversity feature, the target word density feature, and the retrieval training text relevance feature corresponding to the third training document. The document type feature is determined based on the document type of the third training document. The target word diversity feature is determined based on the total number of non-repeating target words in the target text of the third training document. The target word density feature is determined based on the density of target words in the target text. The retrieval training text relevance feature is determined based on the relevance between the retrieval training text and the third training document.

[0089] The third training document can be a document that matches the retrieval training text, contains text content, and is used to train the relevance detection model. This text content includes at least a title, keywords, and an abstract. The specific content of the third training document and model training can be found in step S120. For each target document among multiple target documents, the target feature set corresponding to that target document is input into the trained relevance detection model to obtain the relevant content of the target relevance score corresponding to that target document. This embodiment of the invention will not elaborate further here.

[0090] In step S520, the prediction order of the multiple third training documents is determined based on the prediction relevance scores corresponding to each of the multiple third training documents.

[0091] The multiple third training documents can be ranked based on their respective predicted relevance scores. Specifically, the higher the predicted relevance score of a third training document, the higher its ranking.

[0092] In step S530, the model parameters of the incomplete relevance detection model are adjusted based on the difference between the predicted ranking order and the labeled ranking order of multiple third training documents.

[0093] The annotation relevance scores of each third training document can be determined by referring to the training feature set of each document, thereby determining the annotation order of these documents. Specifically, for each third training document, the feature weights of the retrieval training text relevance features are higher than the feature weights of the document type features, target word diversity features, and target word density features, thus maintaining retrieval accuracy. In some embodiments, the annotation relevance scores of each third training document can be determined based on the user's annotation operations.

[0094] The following describes the steps for determining the annotation relevance scores of multiple third training documents corresponding to a retrieval training text according to an embodiment of the present invention: For each third training document, firstly, a first feature score, a second feature score, a third feature score, and a fourth feature score are determined respectively. Then, a feature matrix is ​​determined based on the first feature score (e.g., the numerical value corresponding to the document type in step S120 above), the second feature score (e.g., the normalized total in step S120 above), the third feature score (e.g., the normalized first mean or second mean in step S120 above), and the fourth feature score (e.g., the normalized relevance in step S120 above; see above, the weight of the fourth feature score may be higher than that of other feature scores). Finally, the annotation relevance score corresponding to the third training document is determined based on the Euclidean distance between the feature matrix and the matrix (0, 1, 0, 1). Based on the annotation scores corresponding to each third training document, the annotation order of the third training documents is determined.

[0095] A third loss value can be determined based on the varying degrees of difference between the predicted ranking order and the labeled ranking order of multiple third training documents. The model parameters of the incomplete relevance detection model can then be adjusted based on this third loss value. The magnitude of this third loss value can be positively correlated with the aforementioned varying degrees of difference. It is understood that developers can also determine other loss values ​​based on actual needs, and then determine the overall loss value based on these other loss values ​​and the third loss value. The training objective of the incomplete relevance detection model can be to obtain the minimum possible overall loss value.

[0096] According to the above-described scheme of the present invention, for each of the multiple third training documents, the training feature set corresponding to that third training document can be input into an untrained relevance detection model to obtain a predicted relevance score for that third training document. Then, based on the predicted relevance scores of each of the multiple third training documents, the predicted ranking order of the multiple third training documents is determined. Finally, based on the differences between the predicted ranking order and the labeled ranking order of the multiple third training documents, the model parameters of the untrained relevance detection model are adjusted. This scheme trains the relevance detection model using third training documents, which helps the relevance detection model to primarily focus on how to accurately rank the target documents, and secondarily focus on determining accurate target relevance scores, which is more in line with the actual needs of retrieval.

[0097] For example, the document type is one of the following: declarative, procedural, in-depth, or relevance type. The annotation order is determined based on the annotation relevance scores corresponding to each of the multiple third training documents. For each third training document, the annotation relevance scores corresponding to the third training document, in descending order, are as follows: when the document type of the third training document is declarative, when the document type of the third training document is procedural, when the document type of the third training document is in-depth, and when the document type of the third training document is relevance.

[0098] In some embodiments, declarative types, procedural types, in-depth types, and related types are primary categories. Each primary category may include multiple secondary categories.

[0099] Declarative types can describe conceptual knowledge. These can include literature reviews, overview studies, and conceptual studies. Procedural types can describe methodological knowledge. These can include practical research, theoretical research, and applied research. In-depth types can describe the vertical expansion of conceptual knowledge. These can include technological research, mechanism research, model research, algorithm research, relationship research, and refinement research. Related types can describe the horizontal expansion of conceptual knowledge. These can include interdisciplinary research and future prospects. Given a defined secondary classification, the corresponding primary classification can be determined based on that secondary classification.

[0100] In one example, for multiple second-level categories under the same primary category, the annotation relevance scores corresponding to these multiple second-level categories are the same. In another example, for multiple second-level categories under the same primary category, the annotation relevance scores corresponding to these multiple second-level categories are different. For example, the annotation relevance score corresponding to theoretical research can be higher than that corresponding to applied research. The specific numerical values ​​of the annotation relevance scores corresponding to the above-mentioned second-level categories are not limited in this embodiment of the invention and can be determined by the developers according to actual needs.

[0101] It's understandable that the content depth of declarative, procedural, and in-depth types gradually increases, making it more difficult for users to understand. Lower labeled relevance scores can cause these types to rank lower, leading to higher user utilization of preceding documents. Conversely, related types, as horizontal extensions of knowledge, are not frequently used in actual searches, hence they have the lowest labeled relevance scores.

[0102] According to the above-described scheme of the present invention, the document type is one of the following: declarative, procedural, in-depth, or relevance type. The labeled relevance score of the third training document differs for different document types. In the above scheme, the labeled relevance score decreases in a stepwise manner for different document types, fully considering the user's cognitive attributes, optimizing the internal ranking of document retrieval results, reducing the difficulty of understanding for users, and improving the user's retrieval efficiency.

[0103] This invention also provides a device for determining literature retrieval results. Figure 2 A schematic block diagram of a document retrieval result determination device 600 according to an embodiment of the present invention is shown. (In conjunction with...) Figure 2 As shown, the processing device 600 may include a target document determination module 610, a target relevance score determination module 620, a target document ranking module 630, and a document retrieval result determination module 640.

[0104] The target document determination module 610 is used to determine multiple target documents corresponding to the search text based on the search text. The target relevance score determination module 620, for each target document among the multiple target documents, inputs the target feature set corresponding to that target document into the trained relevance detection model to obtain the target relevance score for that target document. The target feature set includes the document type feature, the target word diversity feature, the target word density feature, and the search text relevance feature. The document type feature is determined based on the document type of the target document; the target word diversity feature is determined based on the total number of non-repeating target words in the target text corresponding to the target document; the target text includes a title, keywords, and an abstract; the target words have entity semantics; the target word density feature is determined based on the density of target words in the target text; and the search text relevance feature is determined based on the relevance between the target text and the search text. The target document ranking module 630 is used to rank the multiple target documents based on their respective target relevance scores. The document retrieval result determination module 640 is used to determine the document retrieval result corresponding to the search text based on the ranked multiple target documents.

[0105] For example, the document retrieval result determination device 600 further includes a document type determination module and a document type feature determination module.

[0106] The document type determination module, for each of a plurality of preset documents, determines the document type corresponding to that preset document based on the target text it corresponds to, using a trained document type detection model. The plurality of preset documents are used to match the search text to identify multiple target documents among the multiple preset documents, or multiple preset documents as multiple target documents. The document type feature determination module, for each of the plurality of preset documents, determines the document type features corresponding to that preset document based on its document type.

[0107] For example, the document retrieval result determination device 600 further includes a first prediction type determination module, a first loss value adjustment module, a second prediction type determination module, and a second loss value adjustment module.

[0108] The first prediction type determination module determines the predicted type of each of the multiple first training documents based on their respective target texts using a pre-trained document type detection model. The first loss value adjustment module adjusts the model parameters of the pre-trained document type detection model based on the predicted types and corresponding document type labels of each of the multiple first training documents. The second prediction type determination module determines the predicted type of each of the multiple second training documents based on their respective target texts using the adjusted pre-trained document type detection model, where the total number of the multiple second training documents is greater than the total number of the multiple first training documents. The second loss value adjustment module adjusts the model parameters of the adjusted pre-trained document type detection model based on the predicted types and corresponding document type labels of each of the multiple second training documents to obtain the trained document type detection model.

[0109] The document type determination module includes a document type determination sub-module.

[0110] The document type determination submodule is used to determine the document type corresponding to the preset document based on the target text corresponding to the preset document and the preset prompt words, through the trained document type detection model. The preset prompt words are used to prompt the trained document type detection model to determine the document type corresponding to the preset document based on the research question, research method, research object, research conclusion, and interdisciplinary degree of the preset document.

[0111] For example, the document retrieval result determination device 600 further includes a word segmentation module, a filtering module, and a target word density feature determination module.

[0112] The word segmentation module is used to segment the target text corresponding to each of a plurality of preset documents using a preset vocabulary list to obtain a segmented word set for that preset document. The multiple preset documents are used to match the search text to identify multiple target documents among the preset documents, or multiple preset documents as multiple target documents. The filtering module is used to filter the segmented word set for each of the plurality of preset documents using a filtering vocabulary list to obtain a target word set corresponding to that preset document. The filtering vocabulary list includes words without entity semantics, and the target word set includes at least one target word corresponding to that preset document. The total number of times the same target word appears in the target word set is greater than or equal to 1. The target word density feature determination module is used to determine the target word diversity feature and the target word density feature corresponding to each of the plurality of preset documents based on the target word set. The target word density feature is determined based on the total number of target words in the target word set and the total number of sentences in the target text.

[0113] For example, the document retrieval result determination device 600 further includes a prediction relevance score determination module, a prediction ranking order determination module, and a third loss value adjustment module.

[0114] The predicted relevance score determination module is used to input the training feature set corresponding to each of the multiple third training documents into the untrained relevance detection model to obtain the predicted relevance score for that third training document. Multiple third training documents are determined based on the same retrieval training text. The training feature set includes the document type feature, target word diversity feature, target word density feature, and retrieval training text relevance feature corresponding to each third training document. The document type feature is determined based on the document type of the third training document; the target word diversity feature is determined based on the total number of non-repeating target words in the target text of the third training document; the target word density feature is determined based on the density of target words in the target text; and the retrieval training text relevance feature is determined based on the relevance between the retrieval training text and the third training document. The predicted ranking order determination module is used to determine the predicted ranking order of the multiple third training documents based on their respective predicted relevance scores. The third loss value adjustment module is used to adjust the model parameters of the untrained relevance detection model based on the differences between the predicted ranking order and the labeled ranking order of the multiple third training documents.

[0115] For example, the document type is one of the following: declarative, procedural, in-depth, and related types. The declarative type describes conceptual knowledge, the procedural type describes methodological knowledge, the in-depth type describes a deeper extension of conceptual knowledge, and the related type describes a horizontal extension of conceptual knowledge. The annotation order is determined based on the annotation relevance scores of each of the multiple third training documents. For each third training document, the annotation relevance scores of the third training document, in descending order, are as follows: when the document type of the third training document is declarative, when the document type of the third training document is procedural, when the document type of the third training document is in-depth, and when the document type of the third training document is related.

[0116] According to another aspect of the present invention, an electronic device is also provided. Figure 3 A schematic block diagram of an electronic device 700 according to an embodiment of the present invention is shown. Figure 3 As shown, the electronic device 700 includes a processor 710 and a memory 720. The memory 720 stores a computer program, and the computer program instructions are executed by the processor 710 to perform the above-mentioned method for determining the document retrieval results.

[0117] Furthermore, according to another aspect of the present invention, a storage medium is provided, on which program instructions are stored. When the program instructions are executed by a computer or processor, the computer or processor performs corresponding steps of the document retrieval result determination method described above in the embodiments of the present invention, and is used to implement corresponding modules in the document retrieval result determination apparatus or the electronic device described above in the embodiments of the present invention. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media. According to yet another aspect of the present invention, a computer program product is also provided, including computer program instructions. When the computer program instructions are executed by a computer or processor, the computer or processor performs corresponding steps of the document retrieval result determination method described above.

[0118] This invention provides a practical scenario for reference, which involves converting a computer program product's TXT file into JSON format using an open-source script. The configuration file in the Solr search engine (Apache Solr, hereinafter referred to as Solr) is modified to enable the ranking learning function, specifically by adding a query parser (such as a Left-to-Right, LTR query parser) to solrconfig.xml. Simultaneously, the fields corresponding to the features are explicitly defined in schema.xml. The feature definitions and data are then uploaded separately. First, the feature configuration file features.json is prepared based on the correspondence between ranking features and ranking fields. This file is then uploaded to Solr, generating the feature configuration file _schema_feature-store.json. Next, the converted model.json file is uploaded to Solr's system collection, generating the model file _schema_model-store.json. This model file, combined with the search page described above, provides search functionality.

[0119] Those skilled in the art can understand the specific implementation schemes of the above-mentioned electronic devices and storage media by reading the relevant descriptions of the methods for determining the literature search results. For the sake of brevity, they will not be elaborated here.

[0120] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention thereto. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0121] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0122] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0123] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0124] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0125] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or elements of any method or apparatus so disclosed may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0126] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0127] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the document retrieval result determination device according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0128] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0129] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for determining literature retrieval results, characterized in that, The method includes: Based on the search text, multiple target documents corresponding to the search text are identified; For each of the multiple target documents, the target feature set corresponding to that target document is input into the trained relevance detection model to obtain the target relevance score for that target document. The target feature set includes the document type feature, the target word diversity feature, the target word density feature, and the retrieval text relevance feature corresponding to that target document. The document type feature is determined based on the document type of the target document. The target word diversity feature is determined based on the total number of non-repeating target words in the target text corresponding to the target document. The target text includes a title, keywords, and an abstract. The target words have entity semantics. The target word density feature is determined based on the density of target words in the target text. The retrieval text relevance feature is determined based on the relevance between the target text and the retrieval text. Based on the target relevance scores corresponding to each of the target documents, the target documents are ranked. Based on the sorted target documents, the document retrieval results corresponding to the search text are determined.

2. The method as described in claim 1, characterized in that, The method further includes: For each of the multiple preset documents, Based on the target text corresponding to the preset document, the document type corresponding to the preset document is determined by the document type detection model after training. The multiple preset documents are used to match with the search text to determine the multiple target documents among the multiple preset documents, or the multiple preset documents are the multiple target documents. Based on the document type, determine the document type characteristics corresponding to the preset document.

3. The method as described in claim 2, characterized in that, The method further includes: Based on the target texts corresponding to each of the multiple first training documents, the predicted type corresponding to each of the multiple first training documents is determined by a pre-trained document type detection model. Based on the predicted type and document type label corresponding to each of the multiple first training documents, the model parameters of the pre-trained document type detection model are adjusted. Based on the target text corresponding to each of the multiple second training documents, the predicted type corresponding to each of the multiple second training documents is determined by the adjusted pre-trained document type detection model, wherein the total number of the multiple second training documents is greater than the total number of the multiple first training documents. Based on the predicted type and document type label corresponding to each of the multiple second training documents, the model parameters of the pre-trained document type detection model are adjusted to obtain the document type detection model after training.

4. The method as described in claim 2, characterized in that, The process of determining the document type corresponding to the preset document based on the target text and through a trained document type detection model includes: Based on the target text corresponding to the preset document and the preset prompt words, the document type corresponding to the preset document is determined by the document type detection model after training. The preset prompt words are used to prompt the document type detection model after training to determine the document type corresponding to the preset document based on the research question, research method, research object, research conclusion and interdisciplinary degree of the preset document.

5. The method as described in claim 1, characterized in that, The method further includes: For each of the multiple preset documents, Using a preset vocabulary list, the target text corresponding to the preset document is segmented to obtain the segmented word set corresponding to the preset document. The multiple preset documents are used to match with the search text to determine the multiple target documents among the multiple preset documents, or the multiple preset documents are the multiple target documents. The word segmentation set is filtered by a filter word list to obtain the target word set corresponding to the preset document. The filter word list includes words without entity semantics, and the target word set includes at least one target word corresponding to the preset document. The total number of the same target word in the target word set is greater than or equal to 1. Based on the target word set, the target word diversity feature and the target word density feature corresponding to the preset document are determined, wherein the target word density feature corresponding to the preset document is determined based on the total number of target words in the target word set and the total number of sentences in the target text.

6. The method as described in claim 1, characterized in that, The method further includes: For each of the multiple third training documents, the training feature set corresponding to that third training document is input into an untrained relevance detection model to obtain the predicted relevance score for that third training document. The multiple third training documents are determined based on the same retrieval training text. The training feature set includes the document type feature, target word diversity feature, target word density feature, and retrieval training text relevance feature corresponding to that third training document. The document type feature is determined based on the document type of the third training document. The target word diversity feature is determined based on the total number of non-repeating target words in the target text of the third training document. The target word density feature is determined based on the density of target words in the target text. The retrieval training text relevance feature is determined based on the relevance between the retrieval training text and the third training document. Based on the prediction relevance scores corresponding to each of the multiple third training documents, the prediction order of the multiple third training documents is determined. Based on the difference between the predicted order and the labeled order of the multiple third training documents, the model parameters of the incomplete relevance detection model are adjusted.

7. The method as described in claim 6, characterized in that, The document type is one of the following: declarative, procedural, in-depth, and related. The declarative type describes conceptual knowledge; the procedural type describes methodological knowledge; the in-depth type describes a deeper extension of conceptual knowledge; and the related type describes a horizontal extension of conceptual knowledge. The annotation order is determined based on the annotation relevance scores of each of the multiple third training documents. For each third training document, the annotation relevance scores, from largest to smallest, are as follows: when the document type is declarative, when the document type is procedural, when the document type is in-depth, and when the document type is related.

8. A device for determining document retrieval results, characterized in that, The device includes: The target document determination module is used to determine multiple target documents corresponding to the search text based on the search text. The target relevance score determination module is used to input the target feature set corresponding to each target document among the plurality of target documents into the trained relevance detection model to obtain the target relevance score corresponding to the target document. The target feature set includes the document type feature, the target word diversity feature, the target word density feature, and the retrieval text relevance feature corresponding to the target document. The document type feature is determined based on the document type of the target document. The target word diversity feature is determined based on the total number of non-repeating target words in the target text corresponding to the target document. The target text includes the title, keywords, and abstract. The target words have entity semantics. The target word density feature is determined based on the density of target words in the target text. The retrieval text relevance feature is determined based on the relevance between the target text and the retrieval text. The target document ranking module is used to rank the multiple target documents based on their respective target relevance scores. The document retrieval result determination module is used to determine the document retrieval result corresponding to the retrieval text based on the sorted multiple target documents.

9. An electronic device, characterized in that, The system includes a memory and a processor, wherein: the memory is used to store a computer program; and the processor is used to execute the computer program to implement the method for determining document retrieval results as described in any one of claims 1-7.

10. A storage medium storing computer program instructions, characterized in that, The computer program instructions, when executed, are used to perform the method for determining the document retrieval results as described in any one of claims 1-7.

11. A computer program product comprising computer program instructions, characterized in that, The computer program instructions, when executed by a processor, are used to perform the method for determining the document retrieval results as described in any one of claims 1-7.

Images