A method for evaluating document segmentation effects in large-scale model retrieval enhancement generation

Abstract

The present invention discloses a method for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation, which relates to the technical field of document segmentation. The method includes: obtaining segmentation pairs obtained after segmentation processing of a document to be evaluated, inputting the segmentation pairs into a general semantic model in order, obtaining a target evaluation value corresponding to each segmentation pair, and determining the target effect level corresponding to all target evaluation values ​​based on the corresponding relationship between the evaluation value and the effect level; the training process of the general semantic model is specifically as follows: segmenting the training document to obtain at least two original segments; randomly segmenting any original segment to obtain a preset number of slices; labeling according to whether there is semantic relevance to obtain n groups of training samples; calculating the target relevance score and target separability score corresponding to any group of training samples, and determining the evaluation value corresponding to the training sample. The present invention can feedback the score of the segmentation effect corresponding to each document and can also help assist in document segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of document segmentation, and in particular to a method for evaluating document segmentation effects in large-scale model retrieval enhancement generation. Background Art

[0002] Retrieval Augmented Generation (RAG) is a technology applied to knowledge content retrieval. It combines a vector database with a large language model, aiming to enhance the generation capabilities of the large model through retrieval technology. The vector database construction process is as follows: document preparation → document segmentation → slice vectorization → storage. Document segmentation is designed to optimize the construction of the vector database and improve the generation quality of the large language model. In this technology, the large model provides reference answers based on the segmented content of the retrieved and ranked documents. Therefore, document segmentation is particularly important for the quality of the large model's answers, and how to evaluate its segmentation effectiveness is also a problem that needs to be solved. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and specifically provide a method for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation, as follows:

[0004] 1) In a first aspect, the present invention provides a method for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation. The specific technical solution is as follows:

[0005] Obtain segmentation pairs obtained after segmentation processing of the document to be evaluated, input the segmentation pairs into the general semantic model in order, obtain the target evaluation value corresponding to each segmentation pair, and determine the target effect level corresponding to all target evaluation values ​​based on the correspondence between the evaluation value and the effect level;

[0006] The training process of the general semantic model is specifically as follows:

[0007] Obtaining a training document, and segmenting the training document into at least two original segments;

[0008] Randomly split any original block to obtain a preset number of slices;

[0009] According to whether there is semantic correlation, any two original blocks, any two slices, and any combination of an original block and any slice are labeled to obtain n groups of training samples;

[0010] Calculate the target relevance score and target separability score corresponding to any set of training samples, and determine the evaluation value corresponding to the training sample based on the relevance score weight and the separability score weight.

[0011] The beneficial effects of the method for evaluating the document segmentation effect in large-scale model retrieval enhancement generation provided by the present invention are as follows:

[0012] By expanding and adjusting the training set during the training process of the general semantic model, the number of training sets can be increased while also controlling the balance of labeled categories. In addition, providing intuitive feedback on the score of the segmentation effect corresponding to each document can also help assist in document segmentation.

[0013] Based on the above solution, the present invention can also be improved as follows.

[0014] Furthermore, based on the correspondence between evaluation values ​​and effect levels, the process of determining the target effect levels corresponding to all target evaluation values ​​is specifically as follows:

[0015] According to all target evaluation values ​​and the target relevance scores corresponding to each segmentation pair, the sum of all target evaluation values, i.e. the total evaluation value, is determined, and the target effect level corresponding to the total evaluation value in the correspondence between the evaluation value and the effect level is determined.

[0016] Furthermore, the process of determining the sum of all target evaluation values ​​is as follows:

[0017]

[0018] Among them, score is the total evaluation value, rel_score(i) is the target relevance score corresponding to the i-th group of training samples, and weight_score(i) is the evaluation value corresponding to the i-th group of training samples.

[0019] Furthermore, the method for determining the evaluation value corresponding to the training sample is:

[0020] weight_score=0.7*sep_score+0.3*rel_score;

[0021] Among them, weight_score is the evaluation value, sep_score is the target separation score, and rel_score is the target relevance score.

[0022] 2) In a second aspect, the present invention further provides a system for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation. The specific technical solution is as follows:

[0023] The evaluation module is used to: obtain segmentation pairs obtained after segmentation processing of the document to be evaluated, input the segmentation pairs into the general semantic model in order, obtain the target evaluation value corresponding to each segmentation pair, and determine the target effect level corresponding to all target evaluation values ​​based on the corresponding relationship between the evaluation value and the effect level;

[0024] The training process of the general semantic model is specifically as follows:

[0025] Obtaining a training document, and segmenting the training document into at least two original segments;

[0026] Randomly split any original block to obtain a preset number of slices;

[0027] According to whether there is semantic correlation, any two original blocks, any two slices, and any combination of an original block and any slice are labeled to obtain n groups of training samples;

[0028] Calculate the target relevance score and target separability score corresponding to any set of training samples, and determine the evaluation value corresponding to the training sample based on the relevance score weight and the separability score weight.

[0029] 3) In a third aspect, the present invention further provides an electronic device, comprising a processor, wherein the processor is coupled to a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the electronic device implements any of the above methods.

[0030] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor to enable a computer to implement any of the above methods.

[0031] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0033] Figure 1 A flowchart of a method for evaluating document segmentation effects in large-scale model retrieval enhancement generation according to an embodiment of the present invention;

[0034] Figure 2 This is a structural framework diagram of an electronic device. DETAILED DESCRIPTION

[0035] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0036] like Figure 1As shown, a method for evaluating the effect of document segmentation in a large-scale model retrieval enhancement generation according to an embodiment of the present invention has the following specific technical solutions:

[0037] S1, obtaining segmentation pairs obtained after segmentation processing of the document to be evaluated, inputting the segmentation pairs into the general semantic model in order, obtaining the target evaluation value corresponding to each segmentation pair, and determining the target effect level corresponding to all target evaluation values ​​based on the correspondence between the evaluation value and the effect level;

[0038] The training process of the general semantic model is specifically as follows:

[0039] Obtaining a training document, and segmenting the training document into at least two original segments;

[0040] Randomly split any original block to obtain a preset number of slices;

[0041] According to whether there is semantic correlation, any two original blocks, any two slices, and any combination of an original block and any slice are labeled to obtain n groups of training samples;

[0042] Calculate the target relevance score and target separability score corresponding to any set of training samples, and determine the evaluation value corresponding to the training sample based on the relevance score weight and the separability score weight.

[0043] The beneficial effects of the method for evaluating the document segmentation effect in large-scale model retrieval enhancement generation provided by the present invention are as follows:

[0044] By expanding and adjusting the training set during the training process of the general semantic model, the number of training sets can be increased while also controlling the balance of labeled categories. In addition, providing intuitive feedback on the score of the segmentation effect corresponding to each document can also help assist in document segmentation.

[0045] The cutting pair can be composed of two cut blocks or two slices or one cut block and one slice.

[0046] The segments are obtained by segmenting the sentences according to commas.

[0047] Slices are obtained by cutting each block again.

[0048] The specific process of segmenting the document to be evaluated is as follows:

[0049] The first method is to segment the document to be evaluated into paragraphs. For any paragraph obtained after segmentation, segment the paragraph into sentences according to periods or semicolons. The sentence length of the segmented sentences is determined in real time, and the sentences are associated with the serial number identifiers corresponding to the sentences to generate a sentence length record table. The length segmentation range corresponding to documents of different categories is determined. Based on the category of the current document to be evaluated, the target length segmentation range is determined. According to the target length segmentation range, the sentence lengths in the sentence length record table are graded. The serial number identifier corresponding to each grade is determined. According to the number of divisions corresponding to the grade, the sentences corresponding to each serial number identifier at the grade are sliced ​​according to the number of divisions, and the slicing results corresponding to each serial number identifier are obtained.

[0050] The process of determining the length segmentation range corresponding to different categories of documents is as follows:

[0051] By retrieving historical documents of different categories from the historical database, we determine the corresponding slicing results for each category of historical documents, integrate the slicing results, and determine the length of historical sentences corresponding to two slicing results, three slicing results, four slicing results, etc. Based on the statistical results, we determine the corresponding table of segmentation lengths for each category of documents.

[0052] Regarding the use of the segmentation length correspondence table, by determining the category of the document to be evaluated, the segmentation length correspondence table of the corresponding category is called. In the segmentation length correspondence table, the length range corresponding to different numbers of slices can be obtained, that is, the target length segmentation range mentioned above. The sentence length is compared in real time to see which target length segmentation range it falls within, and the segmentation of the sentence corresponding to the sentence length is completed according to the corresponding number of segments.

[0053] The second method is to segment the document to be evaluated into paragraphs, determine a fixed segmentation length, and segment any paragraph according to the fixed segmentation length to obtain at least two segments. It should be further explained that in the process of segmentation according to the fixed segmentation length, if there are commas or semicolons in the segments obtained during the segmentation process, it is necessary to re-segment, that is, segment according to the commas or semicolons, and after segmentation, re-segment according to the fixed segmentation length starting from the first text content after the comma or semicolon. For each result obtained by segmentation, the slicing process is performed according to the slicing method in the first method.

[0054] Furthermore, based on the correspondence between evaluation values ​​and effect levels, the process of determining the target effect levels corresponding to all target evaluation values ​​is specifically as follows:

[0055] According to all target evaluation values ​​and the target relevance scores corresponding to each segmentation pair, the sum of all target evaluation values, i.e. the total evaluation value, is determined, and the target effect level corresponding to the total evaluation value in the correspondence between the evaluation value and the effect level is determined.

[0056] Furthermore, the process of determining the sum of all target evaluation values ​​is as follows:

[0057]

[0058] Among them, score is the total evaluation value, rel_score(i) is the target relevance score corresponding to the i-th group of training samples, and weight_score(i) is the evaluation value corresponding to the i-th group of training samples.

[0059] Furthermore, the method for determining the evaluation value corresponding to the training sample is:

[0060] weight_score=0.7*sep_score+0.3*rel_score;

[0061] Among them, weight_score is the evaluation value, sep_score is the target separation score, and rel_score is the target relevance score.

[0062] Example 1, 1. Prepare training data and train a separable classification model

[0063] Through manual annotation, we prepared the original document segments of 100 papers according to their semantic contents.

[0064] Training data format: [text block 1, text block 2 label 0 / 1]

[0065] The text block 1 / 2 can be a complete original block or a slice of the original block. The label 0 represents separation and 1 represents non-separation.

[0066] For example, a document is manually divided into text block A, text block B, text block C, and text block D according to content coherence, which means that there are semantic differences between any two adjacent text blocks A, B, C, and D.

[0067] During training, data processing is performed on these blocks:

[0068]

Text block A, text block B 0

[0069]

Text block A, text block C 0

[0070]

Text block A, text block D 0

[0071]

Text block B, text block C 0

[0072]

Text block B, text block D 0

[0073]

Text block C, text block D 0

[0074] [Part 1 of text block A, Part 2 1 of text block A]

[0075] [Part 2 of text block A, Part 3 1 of text block A]

[0076] [Part 1 of text block A, Part 3 1 of text block A]

[0077] [Part 1 of text block A, Part 3 0 of text block B]

[0078] [Part 2 of text block A, Part 2 0 of text block C]

[0079] [Part 3 of text block A, Part 1 0 of text block D]

[0080] In addition to the training samples directly constructed from the complete original blocks, each original block will be randomly divided into three slices that can have some overlapping content but not be exactly the same (for example, "Artificial intelligence is really amazing" can be sliced ​​into three slices "Artificial, intelligent, and intelligent are really amazing"). Different slices are combined in pairs, including combinations of slices belonging to the same original block and combinations of slices belonging to different original blocks.

[0081] If slice A and slice B belong to the same original cut, the label is 1, and if they belong to different original cuts, the label is 0.

[0082] In particular, if they belong to different original blocks, the slices in each original block can only randomly match one of the slices in the other original blocks to control the category balance.

[0083] Based on this, a text block related judgment model is trained by adding a fully connected layer with an output dimension of 2 on the basis of the general BERT.

[0084] By adding a softmax layer after the fully connected layer, the separability score sep_score between the two input blocks can be obtained, and the score range is 0 to 1.

[0085] 2. NSP calculation adjacency of general semantic model BERT

[0086] Based on the untrained general BERT, calculate the relevance score (nsp_score) of the current text block and the previous text block.

[0087] Since the model is pre-trained for context judgment tasks, it can directly determine whether there is context between two sentences.

[0088] In this task, two adjacent text blocks are separated by [SEP]. The first text block is preceded by [CLS], and the second text block is followed by [SEP].

[0089] For example: [CLS] Artificial intelligence is amazing [SEP] It accelerates the speed at which humans understand the world [SEP]

[0090] rel_score = bert_nsp_pre(C,S)

[0091] Where C represents the current text block and S represents the previous text block of the current line.

[0092] Bert_nsp_pre is the same as above, which is the output of Bert's final fully connected layer and then the softmax layer. The score range is also 0 to 1.

[0093] 3. Score Weighting

[0094] Considering the different weights of the two, separability has a greater impact in practice. After the experiment, the weight of the separability score in the combined evaluation function was increased, and the weight of the adjacency was reduced. The final evaluation function obtained is:

[0095] weight_score=0.7*sep_score+0.3*rel_score

[0096] 4. Usage

[0097] When a document is segmented into chunks, the chunks are fed into the trained separability prediction model and the general BERT model in pairs, obtaining scores that are recorded until the last two chunks are fed. The final score is obtained by summing all the scores and dividing them by the total number of chunks in the document. The formula is:

[0098]

[0099] Based on this, we can obtain a solution that can use scores to intuitively evaluate the effect of document segmentation. This score can be used to objectively evaluate the quality of document segmentation, which is convenient for experimental comparison.

[0100] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the scope of protection of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0101] The present invention also provides a system for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation. The specific technical solution is as follows:

[0102] The evaluation module is used to: obtain segmentation pairs obtained after segmentation processing of the document to be evaluated, input the segmentation pairs into the general semantic model in order, obtain the target evaluation value corresponding to each segmentation pair, and determine the target effect level corresponding to all target evaluation values ​​based on the corresponding relationship between the evaluation value and the effect level;

[0103] The training process of the general semantic model is specifically as follows:

[0104] Obtaining a training document, and segmenting the training document into at least two original segments;

[0105] Randomly split any original block to obtain a preset number of slices;

[0106] According to whether there is semantic correlation, any two original blocks, any two slices, and any combination of an original block and any slice are labeled to obtain n groups of training samples;

[0107] Calculate the target relevance score and target separability score corresponding to any set of training samples, and determine the evaluation value corresponding to the training sample based on the relevance score weight and the separability score weight.

[0108] It should be noted that the beneficial effects of the system for evaluating the effect of document segmentation in the large-scale retrieval enhancement generation provided by the above embodiment are the same as the beneficial effects of the method for evaluating the effect of document segmentation in the large-scale retrieval enhancement generation provided by the above embodiment, and will not be repeated here. In addition, when the system provided by the above embodiment realizes its functions, it only uses the division of the above-mentioned functional modules as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided by the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0109] like Figure 2 As shown, an electronic device 300 according to an embodiment of the present invention includes a processor 320, which is coupled to a memory 310. The memory 310 stores at least one computer program 330. The at least one computer program 330 is loaded and executed by the processor 320 to enable the electronic device 300 to implement any of the above methods. Specifically:

[0110] The electronic device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements the method for evaluating the document segmentation effect in the large model retrieval enhancement generation provided in the above embodiment. Of course, the electronic device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The electronic device 300 may also include other components for realizing the functions of the device, which will not be described in detail here.

[0111] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor to enable a computer to implement any of the above methods.

[0112] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0113] In an exemplary embodiment, a computer program product or computer program is also provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform any of the above methods.

[0114] It should be noted that the terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects and to define a specific order or precedence. Where appropriate, the order used for similar objects may be interchanged, such that the embodiments of the present application described herein can be implemented in an order other than the order shown or described.

[0115] Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention may be implemented in the form of a computer program product embodied in one or more computer-readable media containing computer-readable program code.

[0116] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0117] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation, characterized in that: include: Obtain segmentation pairs obtained after segmentation processing of the document to be evaluated, input the segmentation pairs into the general semantic model in order, obtain the target evaluation value corresponding to each segmentation pair, and determine the target effect level corresponding to all target evaluation values ​​based on the correspondence between the evaluation value and the effect level; The training process of the general semantic model is specifically as follows: Obtaining a training document, and segmenting the training document into at least two original segments; Randomly split any original block to obtain a preset number of slices; According to whether there is semantic correlation, any two original blocks, any two slices, and any combination of an original block and any slice are labeled to obtain n groups of training samples; Calculate the target relevance score and target separability score corresponding to any set of training samples, and determine the evaluation value corresponding to the training sample based on the relevance score weight and the separability score weight; The specific process of segmenting the document to be evaluated is as follows: The first method is to segment the document to be evaluated into paragraphs according to paragraphs. For any paragraph obtained after segmentation, the paragraph is segmented into sentences according to periods or semicolons. The sentence length of the sentences obtained by segmentation is determined in real time, and the sentences are associated with the serial number identifiers corresponding to the sentences to generate a sentence length record table. The length segmentation ranges corresponding to documents of different categories are determined. Based on the category of the current document to be evaluated, the target length segmentation range is determined. According to the target length segmentation range, the sentence lengths in the sentence length record table are graded, and the serial number identifier corresponding to each grade is determined. According to the number of divisions corresponding to the grade, the sentences corresponding to each serial number identifier under the grade are sliced ​​according to the number of divisions, and the slicing results corresponding to each serial number identifier are obtained; The process of determining the length segmentation range corresponding to different categories of documents is as follows: By retrieving historical documents of different categories from the historical database, determining the slicing results corresponding to each category of historical documents, integrating the slicing results, determining the length of historical sentences corresponding to two slicing results, and the length of historical sentences corresponding to three slicing results; and based on the statistical results, determining the corresponding generation segmentation length corresponding table for each category of documents; Regarding the use of the segmentation length correspondence table, by determining the category of the document to be evaluated, the segmentation length correspondence table of the corresponding category is called. In the segmentation length correspondence table, the length range corresponding to different numbers of slices can be obtained, that is, the target length segmentation range mentioned above. The target length segmentation range corresponding to the sentence length is compared in real time, and the sentence corresponding to the sentence length is segmented according to the corresponding number of segments. The second method is to segment the document to be evaluated into paragraphs, determine a fixed segmentation length, and segment any paragraph according to the fixed segmentation length to obtain at least two segments. It should be further explained that during the segmentation according to the fixed segmentation length, if there is a comma or semicolon in the segmented segment, it is necessary to re-segment, that is, segment according to the comma or semicolon, and after the segmentation, re-segment according to the fixed segmentation length starting from the first text content after the comma or semicolon; for each segmentation result, the slicing method in the first method is used; Based on the correspondence between evaluation values ​​and effect levels, the process of determining the target effect levels corresponding to all target evaluation values ​​is as follows: Determine the sum of all target evaluation values, i.e., the total evaluation value, based on all target evaluation values ​​and the target relevance score corresponding to each segmentation pair, and determine the target effect level corresponding to the total evaluation value in the corresponding relationship between the evaluation value and the effect level; The process of determining the sum of all target evaluation values ​​is as follows: Among them, score is the total evaluation value, rel_score(i) is the target relevance score corresponding to the i-th group of training samples, and weight_score(i) is the evaluation value corresponding to the i-th group of training samples.

2. The method for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation according to claim 1 is characterized in that: The method for determining the evaluation value corresponding to the training sample is: weight_score=0.7*sep_score+0.3*rel_score; Among them, weight_score is the evaluation value, sep_score is the target separation score, and rel_score is the target relevance score.

3. A system for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation, using the method for evaluating the effect of document segmentation in large-scale model retrieval enhancement generation as claimed in claim 1, characterized in that: The system comprises: The evaluation module is used to: obtain segmentation pairs obtained after segmentation processing of the document to be evaluated, input the segmentation pairs into the general semantic model in order, obtain the target evaluation value corresponding to each segmentation pair, and determine the target effect level corresponding to all target evaluation values ​​based on the corresponding relationship between the evaluation value and the effect level; The training process of the general semantic model is specifically as follows: Obtaining a training document, and segmenting the training document into at least two original segments; Randomly split any original block to obtain a preset number of slices; According to whether there is semantic correlation, any two original blocks, any two slices, and any combination of an original block and any slice are labeled to obtain n groups of training samples; Calculate the target relevance score and target separability score corresponding to any set of training samples, and determine the evaluation value corresponding to the training sample based on the relevance score weight and the separability score weight.

4. The system for evaluating document segmentation effects in large-scale model retrieval enhancement generation according to claim 3, characterized in that: The method for determining the evaluation value corresponding to the training sample is: weight_score=0.7*sep_score+0.3*rel_score; Among them, weight_score is the evaluation value, sep_score is the target separation score, and rel_score is the target relevance score.

5. An electronic device, characterized in that: The electronic device includes a processor coupled to a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the electronic device implements the method according to claim 1 or 2.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable a computer to implement the method according to claim 1 or 2.

Images