Document generation method and device, equipment, medium and product

Abstract

The invention provides a document generation method and device, equipment, a medium and a product, and the method comprises the steps: encoding retrieval fields in a retrieval field set of a target document, and obtaining an encoding vector of the target document; performing document retrieval according to the coding vector to obtain a reference document matched with the target document; and generating the target document according to the reference document, the element information of the target document and at least one retrieval field in the retrieval field set. According to the method, intelligent coding, retrieval and document generation are realized, the manpower cost is effectively reduced, the efficiency, adaptability and normalization of document generation are improved, the change of different business requirements is better met, and the quality and efficiency of document generation are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a document generation method, device, equipment, medium and product. Background Art

[0002] As an important carrier for information transmission, rule interpretation and decision-making records, the accuracy and efficiency of documents are directly related to the standardization and transparency of the operations of various institutions. Therefore, how to generate documents accurately and efficiently is an important topic that needs to be studied urgently.

[0003] In related technologies, document generation is usually achieved by using technologies based on knowledge graphs and factor extraction. However, such document generation technologies usually require manual organization of related document knowledge bases, sample data or templates, which not only increases labor costs, but also makes it difficult to adapt to changes in different business needs, and can easily lead to errors in document generation.

[0004] Therefore, there is an urgent need to provide a document generation method, device, equipment, medium and product to solve the above problems. Summary of the Invention

[0005] The present invention provides a document generation method, device, equipment, medium and product to solve the defects of the existing technology that relies on manual organization of related type document knowledge base or templates to generate documents, resulting in high labor costs and poor document generation accuracy, and realizes automated and highly accurate document generation.

[0006] The present invention provides a document generation method, comprising: Encoding a search field in a search field set of a target document to obtain an encoding vector of the target document; Performing document retrieval based on the coding vector to obtain a reference document that matches the target document; The target document is generated based on the reference document, the element information of the target document, and at least one search field in the search field set.

[0007] According to a document generation method provided by the present invention, encoding a search field in a search field set of a target document to obtain an encoding vector of the target document includes: Encoding the search fields in the search field set based on a vector encoding model to obtain an encoding vector of the target document; Among them, the vector encoding model is trained based on the first positive sample pair and the first negative sample pair in the text processing task of the general domain, the second positive sample pair and the second negative sample pair in the element extraction task of the target domain corresponding to the target document, and the sample document and the element information of the sample document in the document generation task of the target domain.

[0008] According to a document generation method provided by the present invention, the training steps of the vector encoding model include: Determine the contrast loss function of the initial encoding model according to the encoding vectors of the first positive sample pair, the encoding vectors of the first negative sample pair, the encoding vectors of the second positive sample pair, and the encoding vectors of the second negative sample pair output by the initial encoding model; Perform document retrieval according to the encoding vector of the sample document encoded by applying the element information of the sample document by the initial encoding model to obtain candidate documents matching the sample document; Determine the distillation loss function of the initial encoding model according to the candidate documents, the sample documents, and the pre-trained language model; Iteratively train the initial encoding model according to the contrast loss function and the distillation loss function to obtain the vector encoding model.

[0009] According to a document generation method provided by the present invention, the step of obtaining the retrieval field set includes: Input the candidate documents and the sample documents into the pre-trained language model to obtain the reward function of the initial encoding model; Determine the similarity distribution function of the initial encoding model according to the encoding vectors of the first positive sample pair, the encoding vectors of the first negative sample pair, the encoding vectors of the second positive sample pair, and the encoding vectors of the second negative sample pair; Determine the distillation loss function according to the distribution difference between the probability distribution function corresponding to the reward function and the similarity distribution function.

[0010] According to a document generation method provided by the present invention, the step of obtaining the retrieval field set includes: Extract the first retrieval field of the target document from the event description information of the element information of the target document; Extract the second retrieval field of the target document from the event content information of the element information of the target document; Infer and generate the third retrieval field of the target document according to the first retrieval field and the second retrieval field; Obtain the set of retrieval fields according to the first retrieval field, the second retrieval field, and the third retrieval field.

[0011] According to a document generation method provided by the present invention, the document retrieval according to the encoding vector to obtain a reference document matching the target document includes: Use the sub-encoding vector corresponding to the first retrieval field in the encoding vector as an index to perform document retrieval in the document knowledge base in the target field corresponding to the target document, and obtain a first document set; Use the multi-dimensional vector formed by fusing the sub-encoding vector corresponding to the second retrieval field and the sub-encoding vector corresponding to the third retrieval field in the encoding vector as an index to perform document retrieval in the first document set to obtain the reference document.

[0012] According to a document generation method provided by the present invention, the document retrieval according to the encoding vector to obtain a reference document matching the target document includes: According to the encoding vector, in the document knowledge base in the target field corresponding to the target document, find at least one matching document that matches each retrieval field in the set of retrieval fields; Score each of the matching documents according to the weight coefficients of the retrieval fields matched by each of the matching documents; Determine the reference document from among at least one of the matching documents according to the scoring results of each of the matching documents.

[0013] According to a document generation method provided by the present invention, the generation of the target document according to the reference document, the element information of the target document, and at least one retrieval field in the set of retrieval fields includes: Generate target prompt information according to the text content of the reference document; Input the element information of the target document, at least one retrieval field in the set of retrieval fields, and the target prompt information into a large language model to obtain the target document generated by the large language model.

[0014] The present invention also provides a document generation device, including: An encoding unit for encoding the retrieval fields in the set of retrieval fields of the target document to obtain the encoding vector of the target document; A retrieval unit for performing document retrieval according to the encoding vector to obtain a reference document matching the target document; A generation unit for generating the target document according to the reference document, the element information of the target document, and at least one retrieval field in the set of retrieval fields.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the document generation method as described in any one of the above is implemented.

[0016] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the document generation method as described in any one of the above is implemented.

[0017] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the document generation method as described in any one of the above is implemented.

[0018] The document generation method, device, equipment, medium, and product provided by the present invention perform automated encoding and retrieval by combining the retrieval fields in the retrieval field set of the target document, so as to adaptively obtain a document similar to or highly relevant to the target document as a reference document, and jointly perform automated generation of the document by combining multi-modal information such as the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set. While effectively reducing the labor cost, it improves the efficiency, adaptability, and standardization of document generation, better meets the changes in different business requirements, and ensures the quality and efficiency of document generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a flowchart of the document generation method provided by the present invention.

[0021] Figure 2 is a flowchart of the training of the vector encoding model provided by the present invention.

[0022] Figure 3 is a structural diagram of the document generation device provided by the present invention.

[0023] Figure 4 is a structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0025] As an important carrier for information transmission, rule interpretation, and decision-making records, the accuracy and efficiency of documents are directly related to the standardization and transparency of the operations of various institutions. Therefore, how to generate documents accurately and efficiently is an important topic that urgently needs to be studied at present.

[0026] The traditional method of manually writing documents is difficult to meet the needs of quickly processing a large number of affairs. Therefore, it is necessary to introduce an automatic document generation mechanism to assist staff in improving the transaction processing cycle and processing efficiency.

[0027] In related technologies, some technical personnel proposed to implement document generation based on knowledge graph technology. This technology constructs a knowledge graph by manually sorting out professional knowledge in related fields (such as law), combines natural language processing technology to match case affairs with domain rules, and uses large model prompting engineering technology to generate documents in related fields. However, this technology requires domain experts to manually sort out a large amount of relevant document professional knowledge for different affairs, and is highly dependent on the professionalism of domain experts, as well as the scope and quality of the sorting. This usually results in insufficient speed and accuracy in graph construction, and a very limited coverage range. When business requirements or domain rules are updated, the system cannot automatically identify and make corresponding adjustments, and requires manual intervention and adjustment again. It is difficult to keep up with the updates in a timely manner, and it is very easy to cause errors in document generation.

[0028] In addition, some technical personnel implement document generation through deep learning technology. This technology allows machines to learn data distributions and document features through deep learning methods, and then generates corresponding documents in combination with transaction content. However, this method requires manual collation of a large amount of sample data and label data of related documents to enable the machine to correctly identify features. The learning quality is greatly affected by human factors, the technical difficulty is high, and the implementation cost is high. Moreover, for different business requirements, such as the preferences of different regions and different users, there is diversity. The model can only learn the document generation logic of most affairs, and is prone to misjudgment for specific actual affairs, making it difficult to ensure the unified application and standardization of documents, and it is very easy to cause errors in document generation.

[0029] In addition, some technical personnel implement document generation through element extraction technology. This technology extracts specific elements for specific affairs and generates corresponding documents in combination with the document generation templates sorted out manually according to the requirements of domain standardization. However, this method requires manual sorting of domain expertise for different affairs, formulating element lists and document generation templates for different affairs. The manual sorting cost is relatively high, and it is difficult to adapt to changes in different business requirements. There are certain parsing errors in the generation of structured data, resulting in errors in document generation.

[0030] In summary, related technologies usually require manual collation of associated document knowledge bases, sample data, templates, etc., which not only increases the labor cost, but also is difficult to adapt to changes in different business requirements, and easily leads to errors in document generation.

[0031] In view of this, the present application provides a document generation method to solve the above technical problems. Figure 1 FIG. is a schematic flowchart of the document generation method provided by the present invention. It should be noted that the document generation method provided by the present application is applicable to various document generation scenarios, including but not limited to judgment document generation scenarios, contract document generation scenarios, notice document generation scenarios, etc. This embodiment does not make specific limitations in this regard. For the sake of simplicity of description, the following takes the judgment document generation scenario as an example to describe the method provided in this embodiment, and other scenarios can be modified with reference.

[0032] As Figure 1 shown, the method includes step 110, step 120, and step 130.

[0033] Step 110: Encode the retrieval fields in the retrieval field set of the target document to obtain the encoded vector of the target document.

[0034] The target document here is the document to be generated, including but not limited to judgment documents, contract documents, notice documents, etc. This embodiment does not make specific limitations in this regard.

[0035] Optionally, when generating the target document, the retrieval field set of the target document can be obtained first. Specifically, it can be a set formed by extracting and / or inferring the fields for document retrieval from the element information of the target document, or a set formed by the fields for document retrieval input by the user, etc. This embodiment does not make specific limitations in this regard. The so-called user input can be information input through a command line interface, a graphical interface, a touch input, a drop-down selection input, a voice input, a gesture input, a visual input, a brain-computer input, etc. This embodiment does not make specific limitations in this regard.

[0036] The element information here is the key content information that describes or defines the characteristics of the target document, including but not limited to the event description information used to describe the basic information of the event, such as the event number, the time of event occurrence, the area where the event occurred, the organization to which the event belongs, the event name, the background or reason for the event occurrence, etc., and the event content information used to describe the specific content information of the event, such as the situation of the event occurrence, the personnel information involved in the event, the statement information of the personnel involved in the event, the field rule information involved in the event, etc. This embodiment does not make specific limitations on this.

[0037] After obtaining the set of retrieval fields of the target document, it is possible to encode the retrieval fields in the set of retrieval fields to convert the text data encoding into a numerical vector of text semantics, thereby obtaining the encoding vector of the target document, so as to perform more accurate and efficient reference document retrieval subsequently.

[0038] It should be noted that during the encoding process, it is possible to encode the retrieval fields in the set of retrieval fields of the target document based on one or more neural network models with text encoding functions to output the encoding vector of the target document, or it is also possible to encode the retrieval fields in the set of retrieval fields of the target document based on the bag-of-words model according to the word frequency distribution to output the encoding vector of the target document, etc. This embodiment does not make specific limitations on this.

[0039] Step 120, perform document retrieval according to the encoding vector to obtain a reference document that matches the target document.

[0040] Optionally, after obtaining the encoding vector, the encoding vector can be used as an index to perform document retrieval in the document knowledge base of the target field corresponding to the target document, so as to obtain a reference document that matches the target document, that is, a document similar to or highly relevant to the target document, to provide support for subsequent document generation.

[0041] During the text retrieval process, it is possible to use the encoding vector as an overall index to perform multi-level document retrieval in the document knowledge base of the target field corresponding to the target document to obtain a reference document that matches the target document; it is also possible to divide the encoding vector into multiple sub-encoding vectors according to different retrieval fields, and use each sub-encoding vector as an index to perform hierarchical document retrieval in the document knowledge base of the target field corresponding to the target document, so as to finally obtain a reference document that matches the target document, etc. This embodiment does not make specific limitations on this.

[0042] Step 130, generate the target document according to the reference document, the element information of the target document, and at least one retrieval field in the set of retrieval fields.

[0043] Optionally, after obtaining the reference document that matches the target document, the target document can be generated based on the reference document, the element information of the target document, and at least one retrieval field associated with the document generation task in the retrieval field set.

[0044] It should be noted that during the document generation process, the data structure of the reference document can be used as the generation template for the target document, and the content information generated based on the element information of the target document and at least one retrieval field associated with the document generation task in the retrieval field set can be filled into the generation template to obtain the target document; or, the target prompt information can be generated based on the text content of the reference document to guide the large language model to generate the target document based on the content information generated according to the element information of the target document and at least one retrieval field associated with the document generation task in the retrieval field set. This embodiment does not make specific limitations on this.

[0045] The method provided in this embodiment performs automated coding and retrieval by combining the retrieval fields in the retrieval field set of the target document, adaptively obtains a document similar to or highly relevant to the target document as the reference document, and jointly performs automated generation of the document by combining multi-modal information such as the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set. While effectively reducing the labor cost, it improves the efficiency, adaptability, and standardization of document generation, better meets the changes in different business requirements, and ensures the quality and efficiency of document generation.

[0046] In some embodiments, step 110 specifically includes: Encoding the retrieval fields in the retrieval field set based on a vector encoding model to obtain the encoding vector of the target document; Among them, the vector encoding model is trained based on the first positive sample pair and the first negative sample pair under the text processing task in the general domain, the second positive sample pair and the second negative sample pair under the element extraction task in the target domain corresponding to the target document, and the sample document and the element information of the sample document under the document generation task in the target domain.

[0047] Optionally, before performing the retrieval field encoding step, a vector encoding model can be pre-trained. Specifically, an initial encoding model can be constructed first. Here, the initial encoding model can be constructed based on components such as the self-attention mechanism and the feed-forward neural network on the basis of the Transformer model architecture to effectively capture long-range dependencies in sequence data. For example, the initial encoding model can be formed by increasing the text input length to 2048, increasing the hidden layer dimension to 1024, increasing the number of model Transformer blocks to 20, and having an overall number of parameters of approximately 2.27B on the basis of a similar architecture of the Bidirectional Encoder Representations from Transformers (BERT) model.

[0048] Among them, the initial encoding model can be a model prepared for text semantic encoding after parameter initialization, or a pre-trained model with text semantic encoding function. This embodiment does not make specific limitations on this.

[0049] In addition, a large number of sample data in different fields can be collected, specifically including the first positive sample pair and the first negative sample pair in the text processing task in the general field, the second positive sample pair and the second negative sample pair in the element extraction task in the target field corresponding to the target document, and the sample document and the element information of the sample document in the document generation task in the target field.

[0050] Here, the general field is a field that does not limit a specific industry, profession, or specific application scenario, including but not limited to fields such as medical, finance, manufacturing, and education. Here, the text processing task refers to the task of processing text, including but not limited to tasks such as question answering, text retrieval, text rearrangement, and text similarity calculation. The target field is the specific field to which the target document belongs. For example, if the target document is a judgment document, the corresponding target field is the legal field. The element extraction task refers to the task of extracting and enhancing elements from text; the document generation task is the task of generating a document based on the element information.

[0051] For example, in the process of collecting the first positive sample pair and the first negative sample pair, question samples in the question answering processing task can be collected from the open source dataset in the general field as query samples, and the correct answer samples corresponding to the question samples can be collected as positive samples to form the first positive sample pair. The question samples are used as query samples and samples other than the correct answer samples corresponding to the question samples are used as negative samples to form the first negative sample pair. Similarly, for other text processing tasks, the first positive sample pair and the first negative sample pair can be obtained by referring to this method.

[0052] During the collection process of the second positive sample pairs and the second negative sample pairs, it is possible to combine large language models, such as Spark Model, and prompt engineering techniques to extract key elements from the original document data in the target domain, and then perform data augmentation on each key element without changing the semantic information, so as to use the document data pairs with unchanged semantic information of the key elements before and after augmentation as the second positive samples, and use the document data pairs with changed semantic information of the key elements before and after augmentation as the second negative samples.

[0053] During the collection process of the sample documents and the element information of the sample documents in the document generation task in the target domain, it is possible to collect the element information of the sample documents in the target domain to generate documents based on the element information to form sample documents, or to collect the sample documents in the target domain to obtain the element information of the sample documents by extracting the element information from the sample documents, etc. This embodiment does not make specific restrictions on this.

[0054] Subsequently, use the first positive sample pairs and the first negative sample pairs, the second positive sample pairs and the second negative sample pairs, as well as the sample documents and the element information of the sample documents to train the initial encoding model to obtain a vector encoding model that can accurately and efficiently perform text semantic encoding. During the training process, it is possible to use the training data set formed by the first positive sample pairs and the first negative sample pairs, the training data set formed by the second positive sample pairs and the second negative sample pairs, and the training data set formed by the sample documents and the element information of the sample documents to train the initial encoding model sequentially to obtain the vector encoding model; it is also possible to jointly construct a training data set with the first positive sample pairs and the first negative sample pairs, the second positive sample pairs and the second negative sample pairs, as well as the sample documents and the element information of the sample documents, and perform overall training on the initial encoding model to obtain the vector encoding model, etc. This embodiment does not make specific restrictions on this.

[0055] The training here can be realized by combining contrastive learning training and knowledge distillation training, or by joint training of multiple tasks (such as element extraction tasks, document generation tasks, etc.), etc. This embodiment does not make specific restrictions on this.

[0056] After obtaining the vector encoding model through training, the retrieval fields in the retrieval field set of the target document can be input into the vector encoding model, so that the vector encoding model performs text semantic feature encoding on the retrieval fields in the retrieval field set of the target document, thereby obtaining the encoding vector of the target document.

[0057] The method provided in this embodiment trains the vector encoding model through the sample data in the general domain and the target domain to achieve text semantic encoding. It can effectively reduce the manual collection cost of the sample data in the target domain, and at the same time, can achieve accurate and efficient text semantic encoding, so as to better adapt to the diverse needs of different domains in the process of document generation, improve the generation efficiency and accuracy, and enhance the generality and flexibility of document generation.

[0058] In some embodiments, the training steps of the vector encoding model include: Determine the contrast loss function of the initial encoding model according to the encoding vectors of the first positive sample pair, the encoding vectors of the first negative sample pair, the encoding vectors of the second positive sample pair, and the encoding vectors of the second negative sample pair output by the initial encoding model; Perform document retrieval based on the encoding vector of the sample document generated by encoding the element information of the sample document by the initial encoding model to obtain candidate documents matching the sample document; Determine the distillation loss function of the initial encoding model according to the candidate document, the sample document, and the pre-trained language model; Iteratively train the initial encoding model according to the contrast loss function and the distillation loss function to obtain the vector encoding model.

[0059] Optionally, during the model training process, it can be to calculate the similarity between the encoding vectors of the two samples in the encoding vector of the first positive sample pair, the similarity between the encoding vectors of the two samples in the encoding vector of the first negative sample pair, the similarity between the encoding vectors of the two samples in the encoding vector of the second positive sample pair, and the similarity between the encoding vectors of the two samples in the encoding vector of the second negative sample pair, so as to determine the contrast loss function of the initial encoding model according to the ratio between the similarity of the first positive sample pair and the similarity of the first negative sample pair, and the ratio between the similarity of the second positive sample pair and the similarity of the second negative sample pair, so as to perform model optimization training through this contrast loss function during the training process, and further make the encoding vectors of the positive sample pairs output by the model closer, while the vectors of the negative sample pairs are farther away. Among them, the contrast learning loss function The specific calculation formula is as follows: ; Where is the encoding vector output by the initial encoding model, represents the inner product operator, is the set of all positive and negative samples in this batch, represents the model temperature coefficient; is the query sample in this batch, is the negative sample corresponding to the query sample in this batch; It is the positive sample corresponding to the query sample of this batch.

[0060] In addition, the retrieval fields in the retrieval field set of the sample document can also be extracted according to the element information of the sample document, and the retrieval fields in the retrieval field set of the sample document can be encoded by using the initial encoding model to obtain the encoding vector of the sample document. Then, document retrieval is performed based on the encoding vector of the sample document to obtain candidate documents that match the sample document. The pre-trained language model is used as the teacher model, and the initial encoding model is used as the student model. The retrieval enhancement reward is calculated by applying the candidate documents and the sample document, thereby determining the distillation loss function of the initial encoding model. Then, the enhancement model is trained according to the distillation loss function to improve the effect of the retrieval enhancement part.

[0061] The pre-trained language model here can be a large language model pre-trained under text processing tasks in the general domain. The large language model (Large Language Model, LLM) is simply referred to as the large language model. It is a natural language processing (Natural Language Processing, NLP) model with a huge number of parameters. The number of model parameters and / or the complexity of the model structure exceed a preset threshold. This model is pre-trained with a large amount of language data and has a high semantic understanding ability and the ability to generate natural language. The large language model can be language models such as BERT, Generative Pretrained Transformer, Text-to-Text Transfer Transformer, Spark Large Language Model, etc. This embodiment does not make specific limitations on this.

[0062] Exemplarily, in some embodiments, the steps for determining the distillation loss function specifically include: Input the candidate document and the sample document into the pre-trained language model to obtain the reward function of the initial encoding model; Determine the similarity distribution function of the initial encoding model according to the encoding vectors of the first positive sample pair, the encoding vectors of the first negative sample pair, the encoding vectors of the second positive sample pair, and the encoding vectors of the second negative sample pair; Determine the distillation loss function according to the distribution difference between the probability distribution function corresponding to the reward function and the similarity distribution function.

[0063] Optionally, during the determination of the distillation loss function, the candidate document and the sample document can be input into the pre-trained language model, so that the pre-trained language model can predict the probability that the predicted token output according to the first data units (tokens) in the candidate document and the sample document belongs to the th token in the sample document, and determine the reward function of the initial encoding model. Among them, for the candidate document The specific calculation formula of the reward function is as follows: ; in, Candidate text Relative to the sample instrument (label) The reward value is used to measure the matching degree between the candidate text and the label; |O| is the label The length of the tag, that is, the number of tokens in the tag; , is given a candidate text and tags Under the condition of the first i−1 tokens in the training set, the i-th predicted token generated by the pre-trained language model belongs to the label O probability; Indicates the i-th token of the tag.

[0064] In summary, for a query , and a set of candidate texts , the corresponding reward score can be obtained through the above reward function calculation formula The reward score reflects the candidate text Therefore, in order to further express the probability distribution corresponding to the reward function, the reward function can be converted into a probability distribution to obtain the probability distribution function corresponding to the reward function. The function converts it into the form of probability distribution. The specific calculation formula is as follows: ; in, Candidate text The value of the corresponding reward function, is the temperature coefficient, the purpose is to control smoothness of the function; Candidate text corresponding to the query text output by the pre-trained language model The predicted probability of .

[0065] In addition, the similarity distribution function of the initial coding model can be determined based on the coding vector of the first positive sample pair, the coding vector of the first negative sample pair, the coding vector of the second positive sample pair, and the coding vector of the second negative sample pair. The specific calculation formula is as follows: ; Therefore, by calculating the distribution difference between the probability distribution function corresponding to the reward function and the similarity distribution function, the distillation loss function can be obtained. The specific calculation formula is: ; in, is the distillation loss function.

[0066] After obtaining the contrast loss function and the distillation loss function, the contrast loss function and the distillation loss function can be fused to determine the total loss function, so as to use training frameworks such as pytorch to iteratively train the initial encoding model based on the total loss function, and a vector encoding model that can perform text semantic encoding with high performance can be obtained.

[0067] The fusion here can be weighted addition, direct addition, etc., and this embodiment does not make specific limitations on this. For example, the specific calculation formula of the total loss function can be: ; where is the total loss function.

[0068] The method provided in this embodiment, through the joint training of the contrast loss and the distillation loss on the initial encoding model, combines the sample pairs in the general domain and the document generation task data in the target domain, and at the same time uses contrast learning and knowledge distillation of the pre-trained language model to further enhance the model's understanding and generation ability of complex semantics, so that the model trained accordingly can learn more accurate text semantic encoding ability, so as to more efficiently capture the semantic information of the text during the document generation process, improve the quality and semantic consistency of the generated documents, and finally achieve a high-quality document generation effect.

[0069] In some embodiments, the step of obtaining the retrieval field set in step 110 includes: In the event description information of the element information of the target document, extract the first retrieval field of the target document; In the event content information of the element information of the target document, extract the second retrieval field of the target document; According to the first retrieval field and the second retrieval field, infer and generate the third retrieval field of the target document; According to the first retrieval field, the second retrieval field and the third retrieval field, obtain the retrieval field set.

[0070] Optionally, during the process of obtaining the retrieval field set of the target document, it can be in the event description information of the element information of the target document, and through the event description prompt information, extract the field that matches the event description prompt information as the first retrieval field of the target document. This first retrieval field includes at least one field for identifying the key information of the event description of the target document, and can be adaptively determined according to actual needs. For example, for a judgment document, the first retrieval field can be at least one of the case number, year, region, court, cause of action, case name, etc.

[0071] In addition, in the event content information of the element information of the target document, through the event content prompt information, a field matching the event content prompt information can be extracted as the second retrieval field of the target document. The second retrieval field includes at least one field for identifying the key information of the event content of the target document, and can be adaptively determined according to actual needs. For example, for a judgment document, the second retrieval field can be at least one of basic case details, party information, trial process, plaintiff's claim, facts and reasons, defendant's defense, third party's defense, plaintiff's evidence in the claim, defendant's evidence in the defense, findings of the court, focus of the dispute, reasons for the judgment, relevant laws, judgment result, etc.

[0072] In addition, based on the first retrieval field and the second retrieval field, further reasoning can be performed on the element information of the target document, and through the reasoning content prompt information, a field matching the reasoning content prompt information can be extracted from the reasoning result as the third retrieval field of the target document. The second retrieval field includes at least one field for identifying the event reasoning information of the target document, and can be adaptively determined according to actual needs. For example, for a judgment document, the third retrieval field can be at least one of case type, case theme, case keywords, key points of the judgment, reasons for the analysis, etc.

[0073] It should be noted that in the process of field extraction, it can be a word matching technique, or by using relevant prompt information to guide a large language model fine-tuned in the target field of the target document to extract matching fields. This embodiment does not make specific limitations on this, so as to reduce manual intervention, enhance the intelligence level and adaptability of retrieval, and thus better meet the document retrieval needs in different fields and scenarios.

[0074] In addition, in the reasoning process, it can also be to use a large language model fine-tuned in the target field of the target document and prompt engineering technology to further reason about the element information of the target document based on the first retrieval field and the second retrieval field, so as to reduce manual intervention, enhance the intelligence level and adaptability of reasoning, and thus better meet the document retrieval needs in different fields and scenarios. For example, based on the case cause: motor vehicle traffic accident liability dispute, the specific plot information of the case: the part of traffic accident liability determination, and the specific situation classification information: 1. Single accident determination situation; 2. Accident determination + responsibility review; 3. Accident determination + review rejection of the first determination + second accident determination are inferred and output through the large language model and prompt engineering technology.

[0075] After obtaining the first retrieval field, the second retrieval field, and the third retrieval field, the retrieval field set of the target document can be integrally generated by combining the first retrieval field, the second retrieval field, and the third retrieval field.

[0076] The method provided in this embodiment extracts fields from the event description information and event content information of the target document, further infers and generates new retrieval fields based on the extracted fields, and finally integrates and generates a comprehensive set of retrieval fields, which can more accurately capture the key and distinctive information of the target document, thereby improving the accuracy and efficiency of document retrieval.

[0077] In some embodiments, step 120 specifically includes: Using the sub-encoding vector corresponding to the first retrieval field in the encoding vector as an index, perform document retrieval in the document knowledge base under the target domain corresponding to the target document to obtain a first set of documents; Using the multi-dimensional vector formed by fusing the sub-encoding vector corresponding to the second retrieval field and the sub-encoding vector corresponding to the third retrieval field in the encoding vector as an index, perform document retrieval in the first set of documents to obtain the reference document.

[0078] The document knowledge base here is a document knowledge base formed by the original documents under the target domain corresponding to the target document. It uses a pre-trained vector encoding model to convert text data into numerical vectors and store them in a database, so that in the formal reasoning process, relevant texts can be retrieved from this document knowledge base to provide reference for generating the target document.

[0079] It should be noted that in the process of constructing the document knowledge base, it is possible to first perform data preprocessing on the original documents, such as structuring the original documents according to the document data structure under the target domain, and / or cleaning the data of the original documents to remove duplicate data, missing data, and abnormal data. Subsequently, obtain the set of retrieval fields for the preprocessed original documents. The specific obtaining steps can refer to the obtaining steps of the set of retrieval fields of the target document, which will not be elaborated here. Then, use the trained vector encoding model to encode the second retrieval field and the third retrieval field in the set of retrieval fields of the preprocessed original documents into multi-dimensional vectors and store them in a database, such as a milvus vector database, etc. And store the original documents of the first retrieval field in the database to construct a document knowledge base under the target domain corresponding to the target document, so as to realize the storage of different types of elements in different ways, so as to improve the retrieval efficiency subsequently, and directly automate the construction of a vector retrieval library containing multiple sets of refined retrieval elements from the domain document library, so as to ensure that the retrieval results are highly consistent with the events of the target document in terms of plot, thereby providing strong support for subsequent document generation. And compared with the prior art that relies on manual knowledge base sorting, it can increase the domain knowledge coverage rate at a lower cost and can quickly update the vector retrieval library to adapt to the real-time update of domain knowledge, thereby improving the adaptability of document generation.

[0080] The original document here can be a historical document obtained by loading from an open-source dataset in the target field, or a historical document generated under the document generation task in the target field, etc. This embodiment does not make specific limitations on this.

[0081] Optionally, during the document retrieval process, the sub-code vector corresponding to the first retrieval field in the encoding vector of the target document can be used as an index to retrieve documents in the document knowledge base, and the documents that match the first retrieval field of the target document form the first document set. Then, the multi-dimensional vector formed by fusing the sub-code vector corresponding to the second retrieval field and the sub-code vector corresponding to the third retrieval field in the encoding vector of the target document is used as an index to retrieve documents again in the first document set, and the documents that match the second retrieval field and the third retrieval field of the target document are used as reference documents.

[0082] The matching here can be achieved by measuring the matching degree between the target document and each document in the document knowledge base through similarity, such as cosine similarity.

[0083] The method provided by this embodiment, through a hierarchical retrieval strategy, first uses the sub-code vector of the first retrieval field to perform a preliminary retrieval in the document knowledge base in the target field to obtain the first document set, and then combines the fusion vector of the second retrieval field and the third retrieval field to perform a refined retrieval in the first document set, and finally accurately locates the reference document highly relevant to the target document, so as to achieve an efficient, parallel and accurate document retrieval effect and improve the retrieval efficiency and accuracy.

[0084] In some embodiments, step 120 further includes: According to the encoding vector, in the document knowledge base in the target field corresponding to the target document, find at least one matching document that matches each retrieval field in the retrieval field set; Score each of the matching documents according to the weight coefficient of the retrieval field that each matching document matches; Determine the reference document from at least one of the matching documents according to the scoring results of each of the matching documents.

[0085] Optionally, during the document retrieval process, the encoded vector can be used as an index to search for at least one matching document in the document knowledge base in the target domain that matches each retrieval field in the retrieval field set. If the same matching document is matched by different retrieval fields, it indicates that the document is highly relevant to the target document in multiple key information dimensions. Therefore, in order to more accurately screen out the reference documents that are highly relevant to the target document, a data fusion strategy can be adopted, such as the Reciprocal Rank Fusion (RRF) strategy or the Normalized Weight Fusion strategy, to perform weighted ranking on the same matching documents that match multiple retrieval fields. Specifically, the weight coefficients of the retrieval fields corresponding to each matching document can be obtained first, and then the weight coefficients of the retrieval fields corresponding to each matching document are normalized. Based on the normalized weight coefficients, each matching document is scored according to the score fusion strategy, and each matching document is re-ranked according to the scoring results to select the most relevant document as the reference document in the ranking results.

[0086] The method provided in this embodiment can effectively improve the accuracy and relevance of document retrieval by using the encoded vector to search for documents that match the retrieval fields in the document knowledge base, scoring and weighted ranking the matching documents based on the weight coefficients of the matching fields, and accurately screening the documents that match multiple retrieval fields, ensuring that the finally determined reference documents are highly relevant to the target document in multiple key information dimensions, thereby improving the overall quality and efficiency of document retrieval.

[0087] In some embodiments, step 130 specifically includes: Generating target prompt information according to the text content of the reference document; Inputting the element information of the target document, at least one retrieval field in the retrieval field set, and the target prompt information into a large language model to obtain the target document generated by the large language model.

[0088] Optionally, in order to reduce the task volume and difficulty of knowledge collation and enable more intelligent, accurate, and standardized document generation, during the document generation process, prompt engineering and a large language model can be used to implement the generation of the target document through the one-shot technique. Specifically, target prompt information is generated based on the text content of the reference document to guide the large language model to generate the target document by applying the element information of the target document and at least one retrieval field in the retrieval field set according to the target prompt information.

[0089] For example, in the scenario of judgment document generation, through the similar case retrieval technology, the most similar case judgment document to the target document in terms of case facts, plaintiff's claims, defendant's defense, and relevant evidence can be retrieved from the legal knowledge document library as a reference document. The reasoning content and judgment content in the reference document are used as prompt information to guide the large language model to generate the reasoning content and judgment content in the target document based on the reasoning content and judgment content in the reference document as templates or examples, according to the plaintiff's claims, defendant's defense, trial information, etc. included in at least one retrieval field of the element information of the target document and the retrieval field set, thereby obtaining the target document.

[0090] The method provided in this embodiment performs intelligent generation of documents by combining similar case retrieval technology with prompt engineering and large language model generation, greatly reducing the task volume and difficulty of knowledge sorting, and ensuring the standardization and precision of document generation for similar matters, thereby ensuring the convenience, precision, and efficiency of document generation.

[0091] The document generation device provided by the present invention will be described below. The document generation device described below can be mutually corresponding and referred to with the document generation method described above.

[0092] Figure 3 is a schematic structural diagram of the document generation device provided by the present invention; as Figure 3 shown, the device includes: The encoding unit 310 is used to encode the retrieval fields in the retrieval field set of the target document to obtain the encoding vector of the target document; The retrieval unit 320 is used to perform document retrieval according to the encoding vector to obtain a reference document matching the target document; The generation unit 330 is used to generate the target document according to the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set.

[0093] The device provided in this embodiment performs automatic encoding and retrieval in combination with the retrieval fields in the retrieval field set of the target document to adaptively obtain a document similar to or highly relevant to the target document as a reference document, and jointly performs automatic generation of documents by combining multi-modal information such as the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set. While effectively reducing the labor cost, it improves the efficiency, adaptability, and standardization of document generation to better meet the changes in different business requirements and ensure the quality and efficiency of document generation.

[0094] In some embodiments, the encoding unit is specifically used for: Based on the vector encoding model, encode the retrieval fields in the set of retrieval fields to obtain the encoding vector of the target document; Among them, the vector encoding model is trained based on the first positive sample pair and the first negative sample pair in the text processing task in the general domain, the second positive sample pair and the second negative sample pair in the element extraction task in the target domain corresponding to the target document, and the sample document and the element information of the sample document in the document generation task in the target domain.

[0095] In some embodiments, the device further includes a training unit specifically configured to: Determine the contrast loss function of the initial encoding model according to the encoding vectors of the first positive sample pair, the first negative sample pair, the second positive sample pair, and the second negative sample pair output by the initial encoding model; Perform document retrieval according to the encoding vector of the sample document generated by encoding the element information of the sample document by the initial encoding model to obtain candidate documents matching the sample document; Determine the distillation loss function of the initial encoding model according to the candidate document, the sample document, and the pre-trained language model; Iteratively train the initial encoding model according to the contrast loss function and the distillation loss function to obtain the vector encoding model.

[0096] In some embodiments, the training unit is further configured to: Input the candidate document and the sample document into the pre-trained language model to obtain the reward function of the initial encoding model; Determine the similarity distribution function of the initial encoding model according to the encoding vectors of the first positive sample pair, the first negative sample pair, the second positive sample pair, and the second negative sample pair; Determine the distillation loss function according to the distribution difference between the probability distribution function corresponding to the reward function and the similarity distribution function.

[0097] In some embodiments, the device further includes a data acquisition unit specifically configured to: Extract the first retrieval field of the target document from the event description information of the element information of the target document; Extract the second retrieval field of the target document from the event content information of the element information of the target document; Infer and generate the third retrieval field of the target document according to the first retrieval field and the second retrieval field; Obtain the set of retrieval fields according to the first retrieval field, the second retrieval field, and the third retrieval field.

[0098] In some embodiments, the retrieval unit is specifically configured to: Use the sub-encoding vector corresponding to the first retrieval field in the encoding vector as an index to perform document retrieval in the document knowledge base in the target field corresponding to the target document, and obtain a first set of documents; Use the multi-dimensional vector formed by fusing the sub-encoding vector corresponding to the second retrieval field and the sub-encoding vector corresponding to the third retrieval field in the encoding vector as an index to perform document retrieval in the first set of documents to obtain the reference document.

[0099] In some embodiments, the retrieval unit is further configured to: According to the encoding vector, in the document knowledge base in the target field corresponding to the target document, find at least one matching document that matches each retrieval field in the set of retrieval fields; Score each of the matching documents according to the weight coefficient of the retrieval field matched by each matching document; Determine the reference document from at least one of the matching documents according to the scoring results of each matching document.

[0100] In some embodiments, the generation unit is specifically configured to: Generate target prompt information according to the text content of the reference document; Input the element information of the target document, at least one retrieval field in the set of retrieval fields, and the target prompt information into a large language model to obtain the target document generated by the large language model.

[0101] The device provided by the present invention is used to execute the above method embodiments. For the specific process and detailed content, please refer to the above embodiments and will not be repeated here.

[0102] Figure 4 An example of the physical structure diagram of an electronic device is shown in Figure 4As shown in the figure, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communications interface 420, and the memory 430 complete communication with each other through the communication bus 440. The processor 410 may call the logical instructions in the memory 430 to execute a document generation method, which includes: encoding the retrieval fields in the retrieval field set of the target document to obtain the encoded vector of the target document; performing document retrieval according to the encoded vector to obtain a reference document matching the target document; and generating the target document according to the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set.

[0103] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0104] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the document generation method provided by the above-mentioned various methods. The method includes: encoding the retrieval fields in the retrieval field set of the target document to obtain the encoded vector of the target document; performing document retrieval according to the encoded vector to obtain a reference document matching the target document; and generating the target document according to the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set.

[0105] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a document generation method provided by the above-mentioned various methods. The method includes: encoding the retrieval fields in the retrieval field set of the target document to obtain an encoded vector of the target document; performing document retrieval according to the encoded vector to obtain a reference document that matches the target document; and generating the target document according to the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set.

[0106] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0107] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A document generation method, characterized in that, Including: Encoding the search fields in the search field set of the target document to obtain the encoded vector of the target document; Performing document retrieval based on the encoded vector to obtain reference documents that match the target document; Generating the target document based on the reference document, the element information of the target document, and at least one search field in the search field set.

2. The document generation method according to claim 1, wherein The encoding of the search fields in the search field set of the target document to obtain the encoded vector of the target document includes: Based on the vector encoding model, encoding the search fields in the search field set to obtain the encoded vector of the target document; Among them, the vector encoding model is trained based on the first positive sample pair and the first negative sample pair in the text processing tasks in the general domain, the second positive sample pair and the second negative sample pair in the element extraction tasks in the target domain corresponding to the target document, and the sample document and the element information of the sample document in the document generation tasks in the target domain.

3. The document generation method according to claim 2, wherein The training steps of the vector encoding model include: Determining the contrast loss function of the initial encoding model according to the encoded vectors of the first positive sample pair, the first negative sample pair, the second positive sample pair, and the second negative sample pair output by the initial encoding model; Performing document retrieval based on the encoded vector of the sample document generated by applying the element information of the sample document by the initial encoding model to obtain candidate documents that match the sample document; Determining the distillation loss function of the initial encoding model according to the candidate document, the sample document, and the pre-trained language model; Iteratively training the initial encoding model according to the contrast loss function and the distillation loss function to obtain the vector encoding model.

4. The document generation method according to claim 3, wherein The determining of the distillation loss function of the initial encoding model according to the candidate document, the sample document, and the pre-trained language model includes: Inputting the candidate document and the sample document into the pre-trained language model to obtain the reward function of the initial encoding model; Determining the similarity distribution function of the initial encoding model according to the encoded vectors of the first positive sample pair, the first negative sample pair, the second positive sample pair, and the second negative sample pair; Determining the distillation loss function according to the distribution difference between the probability distribution function corresponding to the reward function and the similarity distribution function.

5. The document generation method according to any one of claims 1-4, characterized in that, The obtaining steps of the search field set include: Extracting the first search field of the target document from the event description information of the element information of the target document; Extracting the second search field of the target document from the event content information of the element information of the target document; Inferring and generating the third search field of the target document according to the first search field and the second search field; Obtaining the search field set according to the first search field, the second search field, and the third search field.

6. The document generation method according to claim 5, wherein Performing document retrieval based on the encoded vector to obtain a reference document that matches the target document includes: Using the sub-encoded vector corresponding to the first retrieval field in the encoded vector as an index to perform document retrieval in the document knowledge base under the target field corresponding to the target document, and obtaining a first document set; Using the multi-dimensional vector formed by fusing the sub-encoded vector corresponding to the second retrieval field and the sub-encoded vector corresponding to the third retrieval field in the encoded vector as an index to perform document retrieval in the first document set, and obtaining the reference document.

7. The document generation method according to any one of claims 1-4, characterized in that Performing document retrieval based on the encoded vector to obtain a reference document that matches the target document includes: According to the encoded vector, in the document knowledge base under the target field corresponding to the target document, searching for at least one matching document that matches each retrieval field in the retrieval field set; Scoring each of the matching documents according to the weight coefficients of the retrieval fields matched by each of the matching documents; Determining the reference document from at least one of the matching documents according to the scoring results of each of the matching documents.

8. The document generation method according to any one of claims 1-4, characterized in that, Generating the target document according to the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set includes: Generating target prompt information according to the text content of the reference document; Inputting the element information of the target document, at least one retrieval field in the retrieval field set, and the target prompt information into a large language model to obtain the target document generated by the large language model.

9. A document generation device, characterized in that, Including: An encoding unit configured to encode the retrieval fields in the retrieval field set of the target document to obtain an encoded vector of the target document; A retrieval unit configured to perform document retrieval according to the encoded vector to obtain a reference document that matches the target document; A generation unit configured to generate the target document according to the reference document, the element information of the target document, and at least one retrieval field in the retrieval field set.

10. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the document generation method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the document generation method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the document generation method according to any one of claims 1 to 8.

Images