An evaluation review content generation method based on search enhancement

Abstract

The application discloses a kind of based on search enhancement's evaluation review content generation method, comprising the following steps: S1: the document to be evaluated review is converted to format, cut piece processing and then input to text embedding model, obtain the vector library to be searched;The question to be searched is input to the text embedding model, and obtain the semantic vector to be searched;S2: using semantic similarity algorithm and the vector library to be searched, filter out several document fragments with the highest semantic similarity with the semantic vector to be searched;S3: several document fragments with the highest semantic similarity, the semantic vector to be searched is input to large language model, obtains the to be evaluated review content of the document to be evaluated review.This application can improve the efficiency, accuracy and traceability of evaluation review content generation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of evaluation review, and more particularly to an evaluation review content generation method based on retrieval enhancement. BACKGROUND

[0002] Currently, the generation of evaluation review content (i.e., obtaining the key content that needs to be evaluated and reviewed in the evaluation review document) is a key link in the asset evaluation review industry, aiming to ensure the quality and accuracy of the evaluation report. The traditional generation method relies on manual methods, which has the problems of high cost, low generation efficiency and low generation accuracy. With the development of technology, large language models are introduced into this field, which can answer questions based on prompt words and improve generation efficiency.

[0003] However, the existing method of directly generating evaluation review content using a large language model has significant problems. On the one hand, the accuracy of the generated results is low, as the large language model often cannot accurately identify and extract key information when processing complex professional documents, resulting in generated content that may deviate from the actual requirements. On the other hand, the traceability of the generated results is low, and users have difficulty tracing the processing process of the model, which is a major challenge for evaluation review tasks that require high accuracy and verifiability.

[0004] Therefore, how to provide an evaluation review content generation method that can improve the efficiency, accuracy and traceability of evaluation review content generation is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Therefore, the purpose of the present application is to provide an evaluation review content generation method based on retrieval enhancement.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, an evaluation review content generation method based on retrieval enhancement is provided, comprising the following steps:

[0008] S1: inputting the evaluation review document after format conversion and cutting processing into a text embedding model to obtain a retrieval vector library;

[0009] inputting the retrieval question into the text embedding model to obtain a retrieval semantic vector;

[0010] S2: using a semantic similarity algorithm and the retrieval vector library to select a number of document segments with the highest semantic similarity to the retrieval semantic vector;

[0011] S3: inputting the number of document segments with the highest semantic similarity and the retrieval semantic vector into a large language model to obtain the evaluation review content of the evaluation review document.

[0012] Preferably, S1 specifically comprises the following steps:

[0013] Converting the to-be-evaluated review document into a format;

[0014] Performing page segmentation processing on the converted to-be-evaluated review document to obtain a plurality of page segmentation segments; and further performing context segmentation processing on each page segmentation segment to obtain a plurality of context segmentation segments;

[0015] Inputting the plurality of page segmentation segments into the text embedding model to obtain a plurality of coarse-grained semantic vectors; and inputting the plurality of context segmentation segments into the text embedding model to obtain a plurality of fine-grained semantic vectors; wherein the plurality of coarse-grained semantic vectors and the plurality of fine-grained semantic vectors constitute the to-be-searched vector library;

[0016] Inputting the coarse-grained to-be-searched question into the text embedding model to obtain a coarse-grained to-be-searched semantic vector; and inputting the fine-grained to-be-searched question into the text embedding model to obtain a fine-grained to-be-searched semantic vector; wherein the coarse-grained to-be-searched question and the fine-grained to-be-searched question constitute the to-be-searched question in S1.

[0017] Preferably, S2 specifically comprises the following steps:

[0018] Selecting M coarse-grained semantic vectors with the highest semantic similarity to the coarse-grained to-be-searched semantic vector from the plurality of coarse-grained semantic vectors;

[0019] Selecting N fine-grained semantic vectors with the highest semantic similarity to the fine-grained to-be-searched semantic vector from the fine-grained semantic vectors corresponding to the M coarse-grained semantic vectors; wherein the document content corresponding to the N fine-grained semantic vectors is the plurality of document segments with the highest semantic similarity in S2; M and N are positive integers.

[0020] Preferably, S3 specifically comprises the following steps:

[0021] Inputting the document content corresponding to the N fine-grained semantic vectors, the coarse-grained to-be-searched question, and the fine-grained to-be-searched question into a large language model to obtain the to-be-evaluated review content of the to-be-evaluated review document.

[0022] Preferably, the to-be-evaluated review document in S1 is converted into a standardized Markdown format.

[0023] Preferably, the coarse-grained to-be-searched question and the fine-grained to-be-searched question are obtained based on the same to-be-searched item; and the coarse-grained to-be-searched question and the fine-grained to-be-searched question are target identical to-be-searched questions.

[0024] Preferably, the text embedding model is BERT; and the large language model is GPT or llama.

[0025] In a second aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for generating review content based on retrieval enhancement when executing the computer program.

[0026] In a third aspect, a non-transitory computer-readable storage medium is provided, having a computer program stored thereon, wherein the computer program is executable by a processor to implement the method for generating review content based on retrieval enhancement when executed by the processor.

[0027] In a fourth aspect, a computer program product is provided, comprising a computer program, wherein the computer program is executable by a processor to implement the method for generating review content based on retrieval enhancement when executed by the processor.

[0028] According to the above technical solution, compared with the prior art, the present disclosure provides a method for generating review content based on retrieval enhancement. The following technical effects can be obtained:

[0029] 1) Efficiency improvement and cost reduction: The present disclosure uses a large language model, semantic embedding, and retrieval enhancement generation technology to automatically output the review content of the document to be evaluated, which greatly shortens the generation time, especially when dealing with large quantities of documents or peak tasks, supports parallel processing, and greatly reduces manpower investment and costs.

[0030] 2) Avoiding human subjectivity: The review content generation method of the present disclosure is based on a large language model, semantic embedding, and retrieval enhancement technology, which establishes a standardized process for the evaluation and review industry, unifies the quality benchmark, reduces human error interference, and avoids human subjective operation.

[0031] 3) Optimization of retrieval efficiency: The multi-scale text chunking strategy and progressive retrieval strategy provided by the present disclosure significantly improve the retrieval efficiency and can quickly match the most relevant document fragments to the query items.

[0032] 4) Accuracy and relevance enhancement: The present disclosure inputs the most relevant document fragments matched into the large language model, which significantly improves the accuracy and relevance of the generated results.

[0033] 5) Improved traceability: The "retrieval-generation" combined method provided by the present disclosure avoids the black box problem, making the generation process more transparent and having stronger traceability.

[0034] 6) Adaptability is improved: the application can be applied to different review field scenarios, improving the applicability of large language models. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0036] Figure 1 A flowchart of an evaluation review content generation method based on retrieval enhancement provided by the present application;

[0037] Figure 2 A structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0039] On the one hand, as Figure 1 shown, the embodiments of the present application disclose an evaluation review content generation method based on retrieval enhancement, comprising the following steps:

[0040] Step 1: format conversion is performed on the document to be evaluated and reviewed;

[0041] In an embodiment, the present application converts the document to be evaluated and reviewed into a standardized Markdown format.

[0042] In an embodiment, the document to be evaluated and reviewed includes multiple types of documents such as evaluation reports, evaluation instructions and evaluation result tables.

[0043] In an embodiment, the document to be evaluated and reviewed includes documents to be evaluated and reviewed in the fields of finance and accounting, auditing, etc.

[0044] Step 2: the document to be evaluated and reviewed after format conversion is subjected to page segmentation processing to obtain a plurality of page segmentation segments; each page segmentation segment is further subjected to context segmentation processing to obtain a plurality of context segmentation segments;

[0045] Step 3: inputting the plurality of page segmentation segments into the text embedding model to obtain a plurality of coarse-grained semantic vectors; inputting the plurality of context segmentation segments into the text embedding model to obtain a plurality of fine-grained semantic vectors; wherein the plurality of coarse-grained semantic vectors and the plurality of fine-grained semantic vectors constitute the vector library to be searched;

[0046] In an embodiment, the text embedding model is BERT or T5.

[0047] It can be understood that the present application designs a multi-scale text segmentation strategy, which divides the to-be-evaluated review document into two segment formats of page segmentation and context segmentation according to different granularities. The page segmentation is segmented according to the natural page rule of the document, which is suitable for a larger range of content query; the context segmentation is based on semantic analysis, which divides the document into smaller segments related to the context, which is suitable for more accurate semantic matching. The segmented document segments are all generated by the text embedding model to generate their corresponding semantic vectors, and are stored in the vector library in layers to obtain the vector library to be searched.

[0048] The following assumes that the to-be-evaluated review document is a 4-page evaluation report A, and the page segmentation processing divides the evaluation report A into 4 page segmentation segments A1, A2, A3, and A4 according to the number of pages. Next, the context segmentation processing is performed on the page segmentation segment A1 to obtain B1 context segmentation segments; the context segmentation processing is performed on the page segmentation segment A2 to obtain B2 context segmentation segments; the context segmentation processing is performed on the page segmentation segment A3 to obtain B3 context segmentation segments; the context segmentation processing is performed on the page segmentation segment A4 to obtain B4 context segmentation segments; the page segmentation segment A1, A2, A3, and A4, B1 context segmentation segments, B2 context segmentation segments, B3 context segmentation segments, and B4 context segmentation segments are stored in layers (specifically, the page segmentation segments A1, A2, A3, and A4 are stored in one layer, and the B1 context segmentation segments, B2 context segmentation segments, B3 context segmentation segments, and B4 context segmentation segments are stored in another layer) to obtain the vector library to be searched. The hierarchical vector library to be searched constructed by the present application contains a large range of page semantic vectors and a small range of context semantic vectors, which provides flexible search support for different search tasks.

[0049] Step 4: inputting the coarse-grained to-be-searched question into the text embedding model to obtain a coarse-grained to-be-searched semantic vector; inputting the fine-grained to-be-searched question into the text embedding model to obtain a fine-grained to-be-searched semantic vector; wherein the coarse-grained to-be-searched question and the fine-grained to-be-searched question constitute the to-be-searched question in S1.

[0050] It is understood that the coarse-grained search question and the fine-grained search question are obtained based on the same search entry; the coarse-grained search question and the fine-grained search question are search questions with the same target.

[0051] It is understood that the items to be searched include: risk-free rate Rf, risk premium, etc.

[0052] It is understood that: based on the review target, the present invention uses a manual method to convert several items to be retrieved into search questions; specifically: the present invention first converts several items to be retrieved into coarse-grained search questions with a larger semantic range, and then converts these several items to be retrieved into fine-grained search questions with a smaller semantic range.

[0053] The coarse-grained search question is used for subsequent pagination segmentation retrieval; the fine-grained search question is used for subsequent context segmentation retrieval.

[0054] Step 5: Select the M coarse-grained semantic vectors with the highest semantic similarity to the coarse-grained semantic vector to be retrieved from the plurality of coarse-grained semantic vectors;

[0055] Step 6: Select the N fine-grained semantic vectors with the highest semantic similarity to the fine-grained semantic vector to be retrieved from the fine-grained semantic vectors corresponding to the M coarse-grained semantic vectors; wherein, the document content corresponding to the N fine-grained semantic vectors is the document fragment with the highest semantic similarity in S2; M and N are both positive integers.

[0056] Step 7: Input the document content corresponding to the N fine-grained semantic vectors, the coarse-grained retrieval question, and the fine-grained retrieval question into the large language model to obtain the content to be evaluated and reviewed of the document to be evaluated and reviewed.

[0057] The progressive retrieval strategy provided by this invention can balance retrieval efficiency and generation accuracy, ensuring that the final matched content is accurate and complete.

[0058] It is understood that the content to be evaluated and reviewed in the document to be evaluated and reviewed by the present invention is text that has been adjusted by the large language model and is highly related to the N fine-grained semantic vectors and highly corresponds to the coarse-grained retrieval question and the fine-grained retrieval question.

[0059] Furthermore, the large language model is GPT or llama.

[0060] On the other hand, the present invention also provides an electronic device, such as Figure 2As shown, the electronic device can include a processor 201, a communications interface 202, a memory 203 and a communications bus 204, wherein the processor 201, the communications interface 202 and the memory 203 complete the communication with each other through the communications bus 204. The processor 201 can invoke the logic instructions in the memory 203 to execute the search enhancement-based evaluation review content generation method. In addition, the logic instructions in the memory 203 described above can be realized in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0061] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the search enhancement-based evaluation review content generation method provided by the above-mentioned methods.

[0062] In yet another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the search enhancement-based evaluation review content generation method provided by the above-mentioned methods.

[0063] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement it without creative labor.

[0064] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0065] The various embodiments are described in a progressive manner in the specification, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0066] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating evaluation and review content based on retrieval enhancement, characterized in that, Includes the following steps: S1: After converting the format and segmenting the document to be evaluated and reviewed, input it into the text embedding model to obtain the vector library to be retrieved; The question to be retrieved is input into the text embedding model to obtain the semantic vector to be retrieved; S1 specifically includes the following steps: Convert the format of the document to be evaluated and reviewed; The document to be evaluated and reviewed after format conversion is paginated and segmented to obtain several paginated segment fragments; each paginated segment fragment is further processed into context segments to obtain several context segments. The paginated segment fragments are input into the text embedding model to obtain several coarse-grained semantic vectors; the context segment fragments are input into the text embedding model to obtain several fine-grained semantic vectors; wherein, the coarse-grained semantic vectors and the fine-grained semantic vectors constitute the vector library to be retrieved; The coarse-grained retrieval question is input into the text embedding model to obtain a coarse-grained retrieval semantic vector; the fine-grained retrieval question is input into the text embedding model to obtain a fine-grained retrieval semantic vector; wherein, the coarse-grained retrieval question and the fine-grained retrieval question constitute the retrieval question in S1; S2: Using a semantic similarity algorithm and the target vector library, select several document fragments with the highest semantic similarity to the target semantic vector; S2 specifically includes the following steps: From the aforementioned coarse-grained semantic vectors, select the M coarse-grained semantic vectors with the highest semantic similarity to the coarse-grained semantic vector to be retrieved; From the fine-grained semantic vectors corresponding to the M coarse-grained semantic vectors, select the N fine-grained semantic vectors with the highest semantic similarity to the fine-grained semantic vector to be retrieved; wherein, the document content corresponding to the N fine-grained semantic vectors is the document fragment with the highest semantic similarity in S2; M and N are both positive integers; S3: Input the document fragments with the highest semantic similarity and the semantic vector to be retrieved into the large language model to obtain the content to be evaluated and reviewed of the document to be evaluated and reviewed; S3 specifically includes the following steps: The document content corresponding to the N fine-grained semantic vectors, the coarse-grained retrieval question, and the fine-grained retrieval question are input into the large language model to obtain the content to be evaluated and reviewed of the document to be evaluated and reviewed.

2. The method for generating evaluation and review content based on retrieval enhancement according to claim 1, characterized in that, In S1, the document to be evaluated and reviewed is converted into a standardized Markdown format.

3. The method for generating evaluation and review content based on retrieval enhancement according to claim 1, characterized in that, The coarse-grained search question and the fine-grained search question are obtained based on the same search entry; the coarse-grained search question and the fine-grained search question are search questions with the same target.

4. The method for generating evaluation and review content based on retrieval enhancement according to claim 1, characterized in that, The text embedding model is BERT; the large language model is GPT or Illama.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the retrieval-enhanced evaluation review content generation method as described in any one of claims 1 to 4.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the retrieval-enhanced evaluation review content generation method as described in any one of claims 1 to 4.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the retrieval-enhanced evaluation review content generation method as described in any one of claims 1 to 4.

Images