Model construction and literature recommendation method and device, equipment and medium

Abstract

The embodiment of the invention provides a method, a device, equipment and a medium for constructing a model and recommending literatures. The method comprises the following steps: acquiring first document data and first quality evaluation information of each first document; analyzing the first literature data of each first literature through a large language model to obtain literature analysis information conforming to the first quality evaluation information of each first literature; taking the first literature data of each first literature as input, taking the first quality evaluation information and literature analysis information of each first literature as tags, and retraining the large language model to obtain a literature evaluation model. Thus, a user can provide the first document data and the first quality evaluation information of the first document, and retrain the large language model to obtain a document evaluation model conforming to an evaluation system thereof. When literature recommendation is carried out on the user subsequently, the literature evaluation model is used for carrying out literature quality evaluation and recommendation on the literature, and the problem that the quality of the screened literature is uneven can be solved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment, and medium for building a model and recommending documents. Background Art

[0002] With the increase in the number of academic publications and the rapid iteration of knowledge, it is difficult for users to easily find references that meet their needs. In order to help users find references that meet their needs more quickly, the current common solution is to search and recommend literature based on keywords. However, the quality of literature screened in this way is uneven and it is difficult to meet user needs.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0005] The embodiments of the present application provide a method, apparatus, device, and medium for constructing a model and recommending documents, which can solve the problem of uneven quality of screened documents to a certain extent.

[0006] In a first aspect, an embodiment of the present application provides a method for constructing a literature evaluation model, comprising:

[0007] Obtaining first document data and first quality evaluation information of each first document;

[0008] Analyzing the first document data of each of the first documents by using a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each of the first documents;

[0009] The first document data of each of the first documents is used as input, and the first quality evaluation information of each of the first documents and the document analysis information of each of the first documents are used as labels, so as to retrain the large language model and obtain the document evaluation model.

[0010] The large language model is a language model with information analysis and certain reasoning capabilities, that is, it can perform certain analysis and summary based on the input information to give analysis data, and can make analysis that meets the target under the given target. Based on this, in the above implementation scheme, by using the preset large language model to analyze the first document data of the first document when the first document data and the first quality evaluation information of the first document are obtained, the document analysis information that meets the first quality evaluation information of the first document is obtained. At this time, since the document analysis information is information that meets the first quality evaluation information of the first document, it can be considered that the document analysis information is data that can be trusted. On this basis, the first document data of the first document is used as input, and the first quality evaluation information and document analysis information of the first document are used as labels to retrain the large language model. The document evaluation model obtained in this way is in line with the first document data and the first quality evaluation information of the first document. Therefore, the user can retrain the large language model by providing the first document data and the first quality evaluation information of the first document, thereby obtaining a document evaluation model that meets their personalized needs. When recommending literature to users in the future, the literature evaluation model can be used to achieve literature quality evaluation that conforms to the user evaluation system, so that literature recommendations can be made according to the literature quality evaluation, which can solve the problem of uneven quality of screened literature to a certain extent and make it easier to recommend literature that is more likely to meet users' needs.

[0011] Furthermore, the first document data of each of the first documents is analyzed by a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each of the first documents, including: inputting the first document data of each of the first documents into the large language model to obtain document analysis information and second quality evaluation information of each of the first documents; determining from each of the first documents a target first document whose difference between the second quality evaluation information and the first quality evaluation information exceeds a preset allowable range; and inputting the first document data and the first quality evaluation information of the target first document into the large language model to obtain document analysis information that conforms to the first quality evaluation information of the target first document.

[0012] In the above implementation scheme, when the first document data and the first quality evaluation information of the first document are obtained, the first document data of the first document is input into the large language model to obtain the document analysis information and the second quality evaluation information of the first document. Then, the target first document whose difference between the second quality evaluation information and the first quality evaluation information exceeds the preset allowable range is determined from each of the first documents. Since the large language model has not been retrained at this time, when only the first document data is input, the large language model is analyzed in a normal way without target restrictions, and the quality evaluation outputted by it may not meet expectations at this time. Based on this, the gap between the second quality evaluation information obtained by analyzing the first document data of the first document using the large language model and the first quality evaluation information may exceed the preset allowable range, and at this time the large language model cannot accurately predict the quality evaluation information and document analysis information of the first document based on the first document data of the first document. To this end, by inputting the first document data of the target first document whose gap between the second quality evaluation information and the first quality evaluation information exceeds the preset allowable range together with the first quality evaluation information into the large language model, the large language model can reversely infer the document analysis information of the target first document based on the first quality evaluation information of the given target first document, thereby obtaining document analysis information that conforms to the first quality evaluation information of the target first document. The document analysis information that conforms to the first quality evaluation information of the target first document determined in this way is trustworthy data, and the subsequent use of the data for retraining the large language model can make the trained document evaluation model more consistent with the evaluation system of the first document used.

[0013] Furthermore, taking the first document data of each of the first documents as input, taking the first quality evaluation information of each of the first documents and the document analysis information of each of the first documents as labels, the large language model is retrained to obtain the document evaluation model, including: taking the first document data of all of the first documents as input, taking the first quality evaluation information and document analysis information of all of the first documents as labels, the large language model is retrained to obtain the document evaluation model; wherein, for a first document whose difference between the second quality evaluation information and the first quality evaluation information does not exceed a preset allowable range: the document analysis information obtained after the first document data of the first document is input into the large language model is the document analysis information that conforms to the first quality evaluation information of the first document.

[0014] In the above implementation process, the first document data of all first documents are used as input, and the first quality evaluation information and document analysis information of all first documents are used as labels to retrain the large language model. Since the document analysis information of the first document is consistent with the first quality evaluation information, the document analysis information of the first document is equivalent to a supplement to the first quality evaluation information, providing more subtle document features. In this way, the first quality evaluation information and document analysis information of the first document are used as labels at the same time to retrain the large language model, so that the large language model can learn more subtle document features and achieve a more accurate document evaluation effect when the amount of data of the first document is limited.

[0015] Furthermore, taking the first document data of each of the first documents as input, taking the first quality evaluation information of each of the first documents and the document analysis information of each of the first documents as labels, the large language model is retrained to obtain the document evaluation model, including: taking the first document data of the target first document as input, taking the first quality evaluation information and document analysis information of the target first document as labels, and retraining the large language model to obtain the document evaluation model.

[0016] In the above implementation process, the first document data of the target first document is used as input, and the first quality evaluation information and document analysis information of the target first document are used as labels to retrain the large language model. Since the target first document is the first document whose difference between the second quality evaluation information and the first quality evaluation information exceeds the preset allowable range, the large language model that has not been retrained cannot accurately analyze the target first document. Therefore, the first document data of the target first document is used as input, and the first quality evaluation information and document analysis information of the target first document are used as labels to retrain the large language model, so that the large language model can achieve better training results in limited training rounds.

[0017] Furthermore, the first document data of each of the first documents is analyzed by a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each of the first documents, including: inputting the first document data and first quality evaluation information of all the first documents into the large language model to obtain document analysis information that conforms to the first quality evaluation information of each of the first documents.

[0018] In the above implementation process, the first document data and the first quality evaluation information of all the first documents are input into the large language model. Under the condition of given first quality evaluation information, the large language model is enabled to infer the document analysis information that conforms to the first quality evaluation information. In this way, accurate document analysis information can be obtained, which facilitates retraining of the large language model using the document analysis information.

[0019] Furthermore, the document analysis information includes evaluation information of multiple preset evaluation dimensions.

[0020] In a second aspect, an embodiment of the present application provides a method for document recommendation, comprising: obtaining quality evaluation information of a second document; recommending the second document based on the quality evaluation information; wherein the quality evaluation information of the second document is obtained by predicting the second document data of the second document using a preset document evaluation model, and the document evaluation model is obtained according to the above-mentioned method for constructing a document evaluation model.

[0021] In the above implementation scheme, the document evaluation model is used to obtain the quality evaluation information of the second document, and then the second document is recommended according to the quality evaluation information. Since the document evaluation model is obtained according to the aforementioned method for constructing the document evaluation model. Therefore, the document evaluation model is fitted with the first document data and the first quality evaluation information of the first document. Therefore, the user can retrain the large language model by providing the first document data and the first quality evaluation information of the first document, thereby obtaining a document evaluation model that meets their personalized needs. When the document evaluation model is used to obtain the quality evaluation information of the second document, the document quality evaluation that meets the user evaluation system can be achieved, so that the document recommendation is performed according to the document quality evaluation, which solves the problem of uneven quality of the screened documents to a certain extent, and can more easily recommend documents that are more likely to meet their needs to users.

[0022] Furthermore, before recommending the second document based on the quality evaluation information, it also includes: obtaining document display information of the second document; the document display information is obtained by performing text analysis on the document content of the second document using the document evaluation model; correspondingly, recommending the second document includes: recommending the document display information.

[0023] In the above implementation process, the document evaluation model is used to perform text analysis on the document content of the second document to obtain document display information, which can simplify the document content of the second document. Then, the document display information is recommended so that the user can quickly understand the document content of the second document based on the document display information.

[0024] Furthermore, the document display information includes text information of at least one subject type, the document content of the second document includes the text content of the second document, and the document display information is obtained by performing text analysis on the document content of the second document using the document evaluation model in the following manner: the document content of the second document is input into the document evaluation model, so that the document evaluation model analyzes the text content in the document content to obtain text information of each subject type.

[0025] Through the above implementation process, the text content in the document content of the second document can be analyzed to obtain text information of each subject type, thereby simplifying the text content of the second document.

[0026] Furthermore, the document content of the second document also includes charts of the second document and description information of the charts, and the document display information also includes chart information of each of the subject types; the document evaluation model is also used to: for each subject type: when the description information of the chart and the text information of the subject type meet a preset correlation, determine the chart as the chart information of the subject type.

[0027] Through the above implementation process, the chart information of the second document can be simplified, and the simplified chart can be matched with the subject type. In this way, after making a recommendation, the user can simultaneously obtain key graphic information, which makes it easier for the user to quickly understand the core content of the document, thereby helping the user to determine whether the recommended document is the document the user actually needs.

[0028] Furthermore, the method for document recommendation also includes: obtaining a prompt word; and determining a subject type in the document display information according to the prompt word.

[0029] In the above implementation, the selection of the subject type to be displayed can be achieved through prompt words, so that the user can freely choose the subject type to be displayed by inputting different prompt words, so that the document display information can display the content of the required subject on demand, so that the displayed content can better meet user needs.

[0030] In a third aspect, an embodiment of the present application provides a device for constructing a document evaluation model, comprising: a first acquisition module, an analysis module and a training module; the first acquisition module is used to acquire first document data and first quality evaluation information of each first document; the analysis module is used to analyze the first document data of each first document through a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each first document; the training module is used to use the first document data of each first document as input, and the first quality evaluation information of each first document and the document analysis information of each first document as labels to retrain the large language model to obtain the document evaluation model.

[0031] In a fourth aspect, an embodiment of the present application provides a device for document recommendation, comprising: a second acquisition module and a recommendation module; the second acquisition module is used to acquire quality evaluation information of a second document; the recommendation module is used to recommend the second document based on the quality evaluation information; wherein the quality evaluation information of the second document is obtained by predicting the second document data of the second document using a preset document evaluation model, and the document evaluation model is obtained according to the above-mentioned method for constructing a document evaluation model.

[0032] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-mentioned method for constructing a literature evaluation model or the method for literature recommendation.

[0033] In a sixth aspect, an embodiment of the present application provides a storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions enable the processor to implement the above-mentioned method for constructing a literature evaluation model or the method for literature recommendation.

[0034] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 It is a flowchart of a method for constructing a literature evaluation model provided in an embodiment of the present application;

[0037] Figure 2 It is a flowchart of a method for document recommendation provided in an embodiment of the present application;

[0038] FIG3( a ) is a schematic diagram of a document display template provided in an embodiment of the present application;

[0039] FIG3( b ) is a schematic diagram of document display information provided in an embodiment of the present application;

[0040] Figure 4 It is a structural schematic diagram of a device for constructing a literature evaluation model provided in an embodiment of the present application;

[0041] Figure 5 is a structural schematic diagram of a device for document recommendation provided in an embodiment of the present application;

[0042] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to be able to understand the features and technical contents of the embodiments of the present application in more detail, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0044] The terms "first", "second", etc. in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present application described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0045] Unless otherwise stated, the term "plurality" means two or more.

[0046] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0047] Embodiment 1

[0048] The present application provides a method for constructing a literature evaluation model. Figure 1 As shown, Figure 1 A schematic diagram of a method for constructing a literature evaluation model provided in an embodiment of the present application includes:

[0049] Step S101, obtaining first document data and first quality evaluation information of each first document.

[0050] The first document data may include the entire document content of the first document, or may include a portion of the document content of the first document, such as the document title and document abstract.

[0051] Exemplarily, the first document data and the first quality evaluation information of the first document may be obtained in the following manner:

[0052] In an optional implementation of an embodiment of the present application, a document providing instruction sent by a user can be obtained, and then the first document data of the first document can be obtained according to the document providing instruction. Wherein, the document providing instruction can directly include the first document data and the first quality evaluation information of the first document. The document providing instruction can also include the storage address and the first quality evaluation information of the first document or the unique identification number and the first quality evaluation information used to identify the first document. In this way, the first quality evaluation information of the first document can be directly obtained from the document providing instruction, and then the first document data of the first document can be indirectly obtained by obtaining the unique identification number or storage address of the first document.

[0053] In another optional implementation, the document that the user has read and given quality evaluation information can be directly obtained from a preset document storage database as the first document, and then the user's quality evaluation information on the document can be used as the first quality evaluation information of the first document, and the document data of the document can be used as the first document data of the first document.

[0054] Step S102 , analyzing the first document data of each first document by using a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each first document.

[0055] In the embodiment of the present application, the preset large language model is trained in advance and is used to predict the user's quality evaluation information and document analysis information for the document based on the document data of the document. That is, the first document data of the first document is input into the large language model, and the document analysis information and the second quality evaluation information of the first document can be obtained. However, the second quality evaluation information and the document analysis information at this time are obtained divergently, that is, the second quality evaluation information may be the same as the real quality evaluation information, that is, the first quality evaluation information, or it may be different from the first quality evaluation information. Similarly, the document analysis information may also be the same as the real document analysis information, or it may be different from the real document analysis information. At the same time, since the large language model is a language model with information analysis and certain reasoning capabilities, that is, it can perform certain analysis and summary based on the input information to give analysis data, and can make analysis that meets the target under the given target. Therefore, on the basis of given quality evaluation information and document data, the first document data and the first quality evaluation information of the first document are simultaneously input into the large language model, and the large language model can reverse the document analysis information of the first document, thereby obtaining document analysis information that meets the first quality evaluation information of the first document. Therefore, the document analysis information that meets the first quality evaluation information can be determined as trusted document analysis information, and the data can be subsequently used to retrain the large language model, so that the trained document evaluation model can be more consistent with the evaluation system of the first document used.

[0056] In an optional implementation of the embodiment of the present application, the first document data and first quality evaluation information of all first documents may be input into a large language model to obtain document analysis information that conforms to the first quality evaluation information of each first document.

[0057] In another optional implementation, the first document data of each first document can be input into a large language model to obtain document analysis information and second quality evaluation information of each first document. A target first document whose difference between the second quality evaluation information and the first quality evaluation information exceeds a preset allowable range is determined from each first document. The first document data and the first quality evaluation information of the target first document are input into the large language model to obtain document analysis information that conforms to the first quality evaluation information of the target first document. Since the document analysis information that conforms to the first quality evaluation information of the target first document is obtained by the large language model based on the first document data and the first quality evaluation information of the target first document, that is, based on the first quality evaluation information of the given target first document, the document analysis information of the target first document is reversed to obtain the document analysis information that conforms to the first quality evaluation information of the target first document. The document analysis information that is determined to conform to the first quality evaluation information of the target first document in this way is data that can be trusted.

[0058] The document analysis information includes evaluation information of multiple preset evaluation dimensions. The document analysis information may also include document problems, document methods and / or document results. Document problems represent the problems to be solved in the document content. Document methods represent the methods or means used in the document content to solve the problems to be solved. Document results represent the effects that can be achieved when the document methods are used to solve the problems to be solved. Preset evaluation dimensions include importance, universality and innovation.

[0059] Step S103 , taking the first document data of each first document as input, and taking the first quality evaluation information of each first document and the document analysis information of each first document as labels, retraining the large language model to obtain a document evaluation model.

[0060] In an optional implementation method in the embodiment of the present application, the first document data of the target first document can be used as input, and the first quality evaluation information and document analysis information of the target first document can be used as labels to retrain the large language model to obtain a document evaluation model.

[0061] In another optional implementation, the first document data of all first documents can be used as input, and the first quality evaluation information and document analysis information of all first documents can be used as labels to retrain the large language model to obtain a document evaluation model; wherein, for the first document whose difference between the second quality evaluation information and the first quality evaluation information does not exceed a preset allowable range: the document analysis information obtained after inputting the first document data of the first document into the large language model is the document analysis information that conforms to the first quality evaluation information of the first document.

[0062] In the embodiment of the present application, the large language model is trained in advance, and the quality evaluation information and document analysis information of the document can be predicted based on the document data of the document. However, when the first document data of the first document is analyzed using the large language model, erroneous document analysis information and quality evaluation information may be obtained. In order to improve the accuracy of the large language model in analyzing the first document data of the first document, the large language model is retrained using the first document data and the first quality evaluation information of the first document, so that the large language model can learn the document features of the first document, thereby achieving the effect of improving the accuracy of the large language model in analyzing the first document data of the first document.

[0063] The method for constructing a document evaluation model provided in the embodiment of the present application, by analyzing the first document data of the first document and the first quality evaluation information of the first document using a preset large language model, obtains document analysis information that conforms to the first quality evaluation information of the first document, at this time, because the document analysis information is information that conforms to the first quality evaluation information of the first document, it can be considered that the document analysis information is data that can be trusted. On this basis, the first document data of the first document is used as input, and the first quality evaluation information and document analysis information of the first document are used as labels to retrain the large language model, so that the document evaluation model obtained is the first document data and the first quality evaluation information of the first document. Therefore, the user can retrain the large language model by providing the first document data and the first quality evaluation information of the first document, so as to obtain a document evaluation model that meets its personalized needs. Later, when recommending documents for users, the document evaluation model can be used to achieve document quality evaluation that conforms to the user evaluation system, so as to recommend documents according to the document quality evaluation, to a certain extent, solve the problem of uneven quality of the selected documents, and can more easily recommend documents that are more likely to meet their needs to users.

[0064] Embodiment 2

[0065] This embodiment provides a method for document recommendation based on the first embodiment. Figure 2 As shown, the methods recommended by the literature include:

[0066] Step S201, obtaining quality evaluation information of the second document, wherein the quality evaluation information of the second document is obtained by predicting the second document data of the second document using a preset document evaluation model, and the document evaluation model is obtained according to the method for constructing a document evaluation model provided in the first embodiment.

[0067] In an optional implementation of the embodiment of the present application, the document evaluation model can be deployed in a preset database to perform quality evaluation on each document in the database. At this time, each document in the database can be used as a second document. The document data of all documents in the database can be input into the document evaluation model to obtain quality evaluation information of all documents in the database.

[0068] In another optional implementation, the document evaluation model can also be deployed in a database to perform quality evaluation on the newly added documents and / or updated documents of the document data update in the database. The second document is the newly added document and / or the updated document. The document data of the newly added document and / or the document data of the updated document are input into the document evaluation model to obtain the quality evaluation information of the newly added document and / or the quality evaluation information of the updated document.

[0069] In another optional implementation, a document selection instruction can be obtained, and the document data of the document corresponding to the document selection instruction is input into the document evaluation model to obtain the quality evaluation information of the document corresponding to the document selection instruction. Wherein, the document corresponding to the document selection instruction is the second document. If the document selection instruction is used to indicate that the document that meets the preset screening conditions is obtained from the database, then the document corresponding to the document selection instruction is all the documents that meet the preset screening conditions. The preset screening conditions at least include one of the conditions such as keywords, document publication time, and the field to which the document belongs. If the document selection instruction is used to indicate the storage address of the document, then the document corresponding to the document selection instruction is the document stored in the storage address. If the document selection instruction is used to indicate the unique identification number of the document. Then the document corresponding to the document selection instruction is the document corresponding to the unique identification number. Of course, the document selection instruction can also directly include the document content of the document.

[0070] Step S203: recommend a second document based on the quality evaluation information.

[0071] Exemplarily, the document content of the second document may be recommended according to the quality evaluation information, or the document analysis information of the second document may be recommended according to the quality evaluation information. Similarly, the document analysis information of the second document is also obtained by predicting the second document data of the second document using the document evaluation model.

[0072] In an optional implementation method of the embodiment of the present application, if there are multiple second documents, a document screening instruction can be obtained, a preset screening condition can be determined according to the document screening instruction, and a third document that meets the preset screening condition can be screened out from all second documents according to the preset screening condition, and then the third document can be recommended based on the quality evaluation information.

[0073] Optionally, recommending the third document according to the quality evaluation information may include: directly recommending the third document whose quality evaluation information meets a preset evaluation condition; or, sorting the third document according to the quality evaluation information, and then recommending the third document according to the sorting.

[0074] In another optional implementation, there are still multiple second documents, and recommending the second documents based on the quality evaluation information may include: directly recommending the second documents whose quality evaluation information meets the preset evaluation conditions; or, sorting the second documents according to the quality evaluation information, and then recommending the second documents according to the sorting.

[0075] In an optional implementation of the embodiment of the present application, in the case where there is only one second document, whether to recommend the second document can be determined directly based on the quality evaluation information. That is, if the quality evaluation of the second document meets the preset evaluation condition, the second document is directly recommended. Otherwise, the second document is not recommended.

[0076] In another optional implementation of the embodiment of the present application, in the case where there is only one second document, a recommendation may be made directly, and in this case, the quality evaluation information of the second document may be displayed at the same time for user reference.

[0077] In some embodiments, when there are new documents in the database, the new documents can be used as second documents, and the document data of the new documents can be predicted using the document evaluation model to obtain quality evaluation information of the new documents, and then it is determined whether the quality evaluation information of the new documents meets the preset evaluation conditions. If the quality evaluation information of the new documents meets the preset evaluation conditions, the new documents are directly recommended.

[0078] The quality evaluation information is used to characterize the quality of the document. Optionally, the quality evaluation information can be expressed as a score or a grade. If the quality of the document is proportional to the score or grade of the quality evaluation information. Correspondingly, the second document is recommended according to the ranking, including: recommending the second document in order from high to low scores or grades. If the quality of the document is inversely proportional to the score or grade of the quality evaluation information, correspondingly, the second document is recommended according to the ranking, including: recommending the second document in order from low to high scores.

[0079] The method for document recommendation provided by the embodiment of the present application obtains the quality evaluation information of the second document by utilizing the document evaluation model, and then recommends the second document according to the quality evaluation information. Since the document evaluation model is obtained according to the aforementioned method for constructing the document evaluation model. Therefore, the document evaluation model is fitted with the first document data and the first quality evaluation information of the first document. Therefore, the user can achieve retraining of the large language model by providing the first document data and the first quality evaluation information of the first document, thereby obtaining a document evaluation model that meets its personalized needs. When the document evaluation model is used to obtain the quality evaluation information of the second document, the document quality evaluation that meets the user evaluation system can be achieved, thereby recommending documents according to the document quality evaluation, solving the problem of uneven quality of the screened documents to a certain extent, and being able to more easily recommend documents that are more likely to meet their needs to users.

[0080] Before step S201, in order for the user to quickly understand the specific document content of the second document, before recommending the second document according to the quality evaluation information, the document display information of the second document may also be obtained. The document display information is obtained by performing text analysis on the document content of the second document using the document evaluation model. Correspondingly, when recommending the second document according to the quality evaluation information, the document display information of the second document may be recommended at the same time. In this way, the user can understand the specific document content of the second document by reading the document display information of the second document, without having to read the entire document content of the second document.

[0081] Exemplarily, the document display information may include text information of at least one subject type, the document content of the second document includes the text content of the second document, and the document display information is obtained by performing text analysis on the document content of the second document using a document evaluation model in the following manner: the document content of the second document is input into the document evaluation model, so that the document evaluation model analyzes the text content in the document content to obtain text information of each subject type.

[0082] Optionally, the document content of the second document may also include charts and description information of the charts, and the document display information may also include chart information of each subject type; the document evaluation model may also be used for: for each subject type: when the description information of the chart and the text information of the subject type meet a preset correlation, the chart is determined as the chart information of the subject type.

[0083] In an optional implementation of the embodiment of the present application, after obtaining the text information of each subject type and the chart information of each subject type, the text information and chart information of each subject type can be arranged and stored according to a preset subject type arrangement order to obtain document display information.

[0084] In some embodiments, the preset topic types include methods, background, and results. Among them, the topic type "background" corresponds to the first arrangement position; the topic type "method" corresponds to the second arrangement position; and the topic type "result" corresponds to the third arrangement position. Therefore, the text information and chart information of the topic type "background" are stored in the first arrangement position, the text information and chart information of the topic type "method" are stored in the second arrangement position, and the text information and chart information of the topic type "result" are stored in the third arrangement position. Among them, adjacent arrangement positions are separated by preset separators. Preset separators are, for example, comma ","; semicolon ";"; space; slash " / " and blank line.

[0085] In another optional implementation, the document display information corresponds to a document display template, and the document display template includes a text filling position and a chart filling position corresponding to the subject type. After obtaining the text information of each subject type and the chart information of each subject type, the text information of each subject type can be filled into the text filling position of the corresponding subject type. And the chart information of each subject type can be filled into the chart filling position of the corresponding subject type, thereby obtaining the document display information.

[0086] Combined with the document display template shown in Figure 3(a), the subject types include background, method, result and conclusion. Correspondingly, the text filling position corresponding to the subject type "background" is the first blank area; the text filling position corresponding to the subject type "method" is the second blank area; the text filling position corresponding to the subject type "result" is the third blank area; the text filling position corresponding to the subject type "conclusion" is the fourth blank area. The chart filling position corresponding to the subject type "background" is the fifth blank area; the chart filling position corresponding to the subject type "method" is the sixth blank area; the chart filling position corresponding to the subject type "result" is the seventh blank area; the chart filling position corresponding to the subject type "conclusion" is the eighth blank area.

[0087] On this basis, the text information of the theme type "background" can be filled into the first blank area, and the chart information of the theme type "background" can be filled into the fifth blank area. At the same time, the text information of the theme type "method" can be filled into the second blank area, and the chart information of the theme type "background" can be filled into the sixth blank area. The text information of the theme type "result" can be filled into the third blank area, and the chart information of the theme type "background" can be filled into the seventh blank area. The text information of the theme type "conclusion" can also be filled into the fourth blank area, and the chart information of the theme type "background" can be filled into the eighth blank area.

[0088] It can be understood that the various embodiments described in the first embodiment are also applicable to the second embodiment if there is no conflict. Based on the consideration of brevity of description, they will not be repeated here.

[0089] Embodiment 3

[0090] This embodiment is based on Embodiment 1 and Embodiment 2 and is used to further illustrate the present application.

[0091] User A provides the first document data and first quality evaluation information of 300 first documents, the first document data includes the document title and document abstract of the first document, and the first quality evaluation information is such as the first quality score. The document titles and document abstracts of all first documents are input into the preset large language model to obtain the document analysis information and the second quality evaluation information of all first documents. The large language model is, for example, the Mistral 7B v0.2 model. The document analysis information includes document problems, document methods, document results, evaluation of importance dimensions, evaluation of universal dimensions, and evaluation of innovation dimensions. The second quality evaluation information is such as the second quality score. For each first document, when the difference between the first quality score and the second quality score exceeds the preset allowable range, for example, when the first quality score is not equal to the second quality score, the first document is determined as a document with an evaluation prediction error, that is, the target first document. When the difference between the first quality score and the second quality score is within the preset allowable range, for example, when the first quality score is equal to the second quality score, the first document is determined as a document with a correct evaluation prediction.

[0092] Then, the title, abstract and first quality score of the target first document are input into the Mistral 7B v0.2 model to obtain document analysis information that meets the first quality score of the target first document. For documents with correct evaluation prediction, the first document data of the document with correct evaluation prediction is directly input into the Mistral 7B v0.2 model to obtain document analysis information that meets its first quality score.

[0093] Finally, the Mistral 7B v0.2 model is retrained with the document titles and abstracts of all first documents as input and the first quality scores and document analysis information of all first documents as labels to obtain a document evaluation model that can reflect the quality requirements of user A.

[0094] After obtaining the literature evaluation model, it is also necessary to test the analysis accuracy of the literature evaluation model.

[0095] The test document data and test quality scores of 100 test documents are obtained. The test document data of the 100 test documents are all input into the Mistral 7B v0.2 model to obtain the first predicted quality score of each test document. Then, from all the test documents, the number of test documents whose first predicted quality score is not equal to the test quality score is determined to be 33. Therefore, it can be determined that the Mistral 7B v0.2 model has an evaluation accuracy of 33% for the test documents. At the same time, the test document data of the 100 test documents are all input into the document evaluation model to obtain the second predicted quality score of each test document. Then, from all the test documents, the number of test documents whose second predicted quality score is not equal to the test quality score is determined to be 44. Therefore, it can be determined that the document evaluation model has an evaluation accuracy of 44% for the test documents. It can be seen that after the Mistral 7B v0.2 model is retrained using the first document data and the first quality score of the first document, the obtained document evaluation model can more accurately evaluate the quality of the test documents.

[0096] At the same time, after using the Mistral 7B v0.2 model to obtain the first predicted quality score of the test document, the test document with the first predicted quality score greater than the preset score threshold is recommended to user A, so there are only 21 test documents that meet the user's needs. That is, the recommendation accuracy is 21%. However, after using the document evaluation model to obtain the second predicted quality score of the test document, the test document with the second predicted quality score greater than the preset score threshold is recommended to user A, so there are 43 test documents that meet the user's needs. That is, the recommendation accuracy is 43%. It can be seen that the document evaluation model can be used to achieve document quality evaluation that meets the evaluation system of user A, so that document recommendations can be made according to the document quality evaluation, which can solve the problem of uneven quality of the screened documents to a certain extent, and can more easily recommend documents that are more likely to meet the user's needs. The preset score threshold is, for example, 8 points.

[0097] When user A inputs the keyword "large language model" in the paper database, all papers including the keyword "large language model" are determined as second documents.

[0098] The second document is structured, that is, the second document is divided according to the paper structure to obtain the document title, document abstract, document author and institution information, document publication date, document text content, document charts and their titles. Then the document title and document abstract of the second document are input into the document evaluation model to obtain the quality score, document problems, document methods and document results of the second document.

[0099] For the second document with a quality score higher than 8, the text content of the second document is input into the document evaluation model to obtain the text information of the subject type "background", the text information of the subject type "method", the text information of the subject type "result", and the text information of the subject type "conclusion". At the same time, the charts and titles of the second document are also input into the document evaluation model, so that the document evaluation model analyzes the charts and titles of the second document and connects them with the text information of each subject type, and selects the chart information of the subject type "background" as Chart 1. The chart information of the subject type "method" is Chart 2. The chart information of the subject type "result" is Chart 3. Finally, the database address, document author, institution information, document publication date, document question, document method and document result of the second document, as well as the text information and chart information of each subject type are filled into the corresponding position of the preset document display template to obtain the document display information as shown in Figure 3 (b).

[0100] Embodiment 4

[0101] Based on the same inventive concept, the embodiment of the present application also provides a device 400 for constructing a literature evaluation model. The device 400 includes at least one software function module that can be stored in a memory in the form of software or firmware or fixed in the operating system of the device 400. Figure 4 As shown, the device 400 for constructing a literature evaluation model includes a first acquisition module 401 , an analysis module 402 and a training module 403 .

[0102] A first acquisition module 401 is used to acquire first document data and first quality evaluation information of each first document;

[0103] An analysis module 402 is used to analyze the first document data of each first document by using a preset large language model to obtain document analysis information that meets the first quality evaluation information of each first document;

[0104] The training module 403 is used to retrain the large language model using the first document data of each first document as input and the first quality evaluation information of each first document and the document analysis information of each first document as labels to obtain a document evaluation model.

[0105] In a feasible implementation of the embodiment of the present application, the analysis module 402 is specifically used to input the first document data of each first document into the large language model to obtain document analysis information and second quality evaluation information of each first document; determine from each first document the target first document whose difference between the second quality evaluation information and the first quality evaluation information exceeds a preset allowable range; input the first document data and the first quality evaluation information of the target first document into the large language model to obtain document analysis information that meets the first quality evaluation information of the target first document.

[0106] In a feasible implementation of the embodiment of the present application, the training module 403 is specifically used to take the first document data of all first documents as input, and take the first quality evaluation information and document analysis information of all first documents as labels, to retrain the large language model to obtain a document evaluation model;

[0107] Among them, for the first document whose difference between the second quality evaluation information and the first quality evaluation information does not exceed the preset allowable range: the document analysis information obtained after the first document data of the first document is input into the large language model is the document analysis information that conforms to the first quality evaluation information of the first document.

[0108] In a feasible implementation of an embodiment of the present application, the training module 403 is specifically used to take the first document data of the target first document as input, and the first quality evaluation information and document analysis information of the target first document as labels, to retrain the large language model to obtain a document evaluation model.

[0109] In a feasible implementation of the embodiment of the present application, the analysis module 402 is specifically used to input the first document data and first quality evaluation information of all first documents into the large language model to obtain document analysis information that conforms to the first quality evaluation information of each first document.

[0110] In this feasible implementation manner, the document analysis information includes evaluation information of multiple preset evaluation dimensions.

[0111] It should be understood that, for the sake of brevity, some of the contents described in the first embodiment will not be repeated in this embodiment.

[0112] Embodiment 5

[0113] Based on the same inventive concept, the present application embodiment provides a device 500 for document recommendation. Figure 5 As shown, the device 500 for document recommendation includes a second acquisition module 501 and a recommendation module 502. The device 500 includes at least one software function module that can be stored in a memory in the form of software or firmware or fixed in the operating system of the device 500.

[0114] The second acquisition module 501 is used to acquire the quality evaluation information of the second document; the recommendation module 502 is used to recommend the second document according to the quality evaluation information;

[0115] The quality evaluation information of the second document is obtained by predicting the second document data of the second document using a preset document evaluation model, and the document evaluation model is obtained according to the method for constructing a document evaluation model in the first embodiment.

[0116] In a feasible implementation of the embodiment of the present application, the second acquisition module 501 is also used to obtain document display information of the second document; the document display information is obtained by performing text analysis on the document content of the second document using a document evaluation model; the recommendation module 502 is also used to recommend the document display information.

[0117] In this feasible implementation mode, the document display information is obtained by performing text analysis on the document content of the second document using a document evaluation model in the following manner: the document content of the second document is input into the document evaluation model so that the document evaluation model analyzes the text content in the document content to obtain text information of each subject type.

[0118] In this feasible implementation mode, the literature evaluation model is also used to: for each subject type: when the description information of the chart and the text information of the subject type meet the preset correlation, determine the chart as the chart information of the subject type.

[0119] In this feasible implementation manner, the second acquisition module 501 is further used to acquire prompt words; and determine the subject type in the document display information according to the prompt words.

[0120] It should be understood that, for the sake of brevity, some of the contents described in the second embodiment will not be repeated in this embodiment.

[0121] Combination Figure 6 As shown, an embodiment of the present application provides an electronic device, including a processor 601 and a memory 602. Optionally, the device may also include a communication interface 603 and a bus 604. The processor 601, the communication interface 603, and the memory 602 may communicate with each other through the bus 604. The communication interface 603 may be used for information transmission. The processor 601 may call the logic instructions in the memory 602 to execute the method for constructing a document evaluation model or the method for document recommendation of the above embodiment.

[0122] In addition, the logic instructions in the memory 602 described above may be implemented in the form of software functional units and when sold or used as independent products, may be stored in a computer-readable storage medium.

[0123] The memory 602 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the methods in the embodiments of the present application. The processor 601 executes the function application and data processing by running the program instructions / modules stored in the memory 602, that is, the method for constructing a literature evaluation model or the method for literature recommendation in the above-mentioned embodiments is implemented.

[0124] The memory 602 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 602 may include a high-speed random access memory and may also include a non-volatile memory.

[0125] The electronic device may be a computer or a server, etc.

[0126] An embodiment of the present application provides a storage medium storing computer executable instructions, wherein the computer executable instructions are configured to execute the above-mentioned method for constructing a literature evaluation model or the method for literature recommendation.

[0127] An embodiment of the present application provides a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for constructing a document evaluation model or the method for document recommendation.

[0128] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0129] The technical solution of the embodiment of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present application. The aforementioned storage medium can be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., which can store program codes, or a transient storage medium.

[0130] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0131] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. At the same time, the above embodiments can be combined with each other to form new embodiments without conflict.

Claims

1. A method for constructing a literature evaluation model, characterized in that: include: Obtaining first document data and first quality evaluation information of each first document; Analyzing the first document data of each of the first documents by using a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each of the first documents; The first document data of each of the first documents is used as input, and the first quality evaluation information of each of the first documents and the document analysis information of each of the first documents are used as labels, so as to retrain the large language model and obtain the document evaluation model.

2. The method according to claim 1, characterized in that The first document data of each of the first documents is analyzed by a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each of the first documents, including: Inputting the first document data of each of the first documents into the large language model to obtain document analysis information and second quality evaluation information of each of the first documents; Determining, from each of the first documents, a target first document whose difference between the second quality evaluation information and the first quality evaluation information exceeds a preset allowable range; The first document data and the first quality evaluation information of the target first document are input into the large language model to obtain document analysis information that matches the first quality evaluation information of the target first document.

3. The method according to claim 2, characterized in that Taking the first document data of each of the first documents as input, taking the first quality evaluation information of each of the first documents and the document analysis information of each of the first documents as labels, retraining the large language model to obtain the document evaluation model, including: Taking the first document data of all the first documents as input and the first quality evaluation information and document analysis information of all the first documents as labels, retraining the large language model to obtain the document evaluation model; Among them, for the first document whose difference between the second quality evaluation information and the first quality evaluation information does not exceed the preset allowable range: the document analysis information obtained after the first document data of the first document is input into the large language model is the document analysis information that conforms to the first quality evaluation information of the first document.

4. The method according to claim 2, characterized in that: Taking the first document data of each of the first documents as input, taking the first quality evaluation information of each of the first documents and the document analysis information of each of the first documents as labels, retraining the large language model to obtain the document evaluation model, including: The first document data of the target first document is used as input, and the first quality evaluation information and document analysis information of the target first document are used as labels, and the large language model is retrained to obtain the document evaluation model.

5. The method according to claim 1, characterized in that The first document data of each of the first documents is analyzed by a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each of the first documents, including: The first document data and first quality evaluation information of all the first documents are input into the large language model to obtain document analysis information that matches the first quality evaluation information of each of the first documents.

6. The method according to any one of claims 1 to 5, characterized in that: The document analysis information includes evaluation information of multiple preset evaluation dimensions.

7. A method for document recommendation, characterized in that: include: Obtain quality evaluation information of the second document; recommending the second document according to the quality evaluation information; The quality evaluation information of the second document is obtained by predicting the second document data of the second document using a preset document evaluation model, and the document evaluation model is obtained according to the method for constructing a document evaluation model according to any one of claims 1 to 6.

8. The method according to claim 7, characterized in that Before recommending the second document according to the quality evaluation information, the method further includes: obtaining document display information of the second document; the document display information is obtained by performing text analysis on the document content of the second document using the document evaluation model; Correspondingly, recommending the second document includes: recommending display information of the document.

9. The method according to claim 8, characterized in that The document display information includes text information of at least one subject type, the document content of the second document includes the text content of the second document, and the document display information is obtained by performing text analysis on the document content of the second document using the document evaluation model in the following manner: The document content of the second document is input into the document evaluation model so that the document evaluation model analyzes the text content in the document content to obtain text information of each of the subject types.

10. The method according to claim 9, characterized in that The document content of the second document also includes the diagrams of the second document and description information of the diagrams, and the document display information also includes diagram information of each subject type; the document evaluation model is also used for: For each topic type: when the description information of the chart and the text information of the topic type meet the preset correlation, the chart is determined as the chart information of the topic type.

11. The method according to claim 9 or 10, characterized in that: Also includes: Get the prompt word; The subject type in the document display information is determined according to the prompt word.

12. A device for constructing a literature evaluation model, characterized in that: include: A first acquisition module, an analysis module and a training module; The first acquisition module is used to acquire the first document data and the first quality evaluation information of each first document; The analysis module is used to analyze the first document data of each of the first documents through a preset large language model to obtain document analysis information that conforms to the first quality evaluation information of each of the first documents; The training module is used to take the first document data of each of the first documents as input, and take the first quality evaluation information of each of the first documents and the document analysis information of each of the first documents as labels, to retrain the large language model to obtain the document evaluation model.

13. A device for document recommendation, characterized in that: include: Second acquisition module and recommendation module; The second acquisition module is used to acquire quality evaluation information of the second document; The recommendation module is used to recommend the second document according to the quality evaluation information; The quality evaluation information of the second document is obtained by predicting the second document data of the second document using a preset document evaluation model, and the document evaluation model is obtained according to the method for constructing a document evaluation model according to any one of claims 1 to 6.

14. An electronic device, characterized in that: It includes a processor and a memory, the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method for constructing a document evaluation model as described in any one of claims 1 to 6, or the method for document recommendation as described in any one of claims 7 to 11.

15. A storage medium, characterized in that: The storage medium stores computer executable instructions, and when the computer executable instructions are called and executed by the processor, the computer executable instructions enable the processor to implement the method for constructing a document evaluation model as described in any one of claims 1 to 6, or the method for document recommendation as described in any one of claims 7 to 11.

Images