A patent text abstract extraction method, system, device and storage medium
By using a BERT model and an expansion model trained in a loop, the accuracy problem of patent text summary extraction was solved, achieving efficient patent text summary extraction and reconstruction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHONGZHISHUTONG (BEIJING) INFORMATION TECH CO LTD
- Filing Date
- 2023-04-19
- Publication Date
- 2026-05-12
AI Technical Summary
Existing text summarization methods perform poorly in extracting patent texts, and struggle to accurately handle the structural and lexical features of patent texts.
A BERT-based summary extraction model and an expansion model are trained iteratively. By combining text embedding and summary analysis with expansion analysis, the accuracy of patent text summarization is improved.
It improves the accuracy and stability of patent text abstracts, effectively extracts and restores patent abstracts, and supports the processes of patent writing, analysis, and retrieval.
Smart Images
Figure CN116541513B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text semantic processing technology, and in particular to a patented text summarization extraction method, system, device, and storage medium. Background Technology
[0002] With the development of the internet and multimedia technologies, the amount of text data is growing exponentially. This text is a valuable source of information and knowledge that requires effective summarization. Text summarization is a method of simplifying source text into compact variants while retaining its knowledge and practical significance.
[0003] Text summarization aims to transform text or a collection of texts into concise texts containing key information. There are two main types of methods: extractive algorithms and generative algorithms. Correspondingly, common problems for these two types of algorithms are: extractive summarization suffers from incorrect content selection, poor sentence coherence, and limited flexibility; generative summarization is affected by out-of-vocabulary words and word repetition.
[0004] Text summarization can be categorized in many ways, such as single-document and multi-document summarization, multilingual summarization, paper generation (generating each chapter with abstracts, introductions, key points, etc.), medical report generation, sentiment summarization (summarizing viewpoints, feelings, evaluations, etc.), and dialogue summarization. Mainstream solutions are primarily based on deep learning, reinforcement learning, and transfer learning, and there are numerous related papers in this field.
[0005] Extractive summarization methods include TextRank and BertSum, which extract words from the original text and combine them to form a new summary. TextRank is modeled after PageRank, using sentences as nodes and constructing undirected weighted edges with weights representing sentence similarity. Generative summarization methods include PGN, GPT, BART, BRIO, GSum, SimCLS, and CIT+SE.
[0006] Patent texts have some unique characteristics: (1) Strong structure. Compared with other texts, patent texts have strong structural features, and the form of a single type of patent is uniform. Patent texts usually adopt a fixed form and structure, and the corresponding text structure mainly consists of five parts: abstract of specification, specification, specification drawings, claims, and abstract drawings. Each part provides a detailed explanation of the relevant content and requirements in the patent text. (2) Rigorous use of language in patents. Patents have legal effect, so the vocabulary in patent texts often uses relatively concise words and phrases. In order to avoid ambiguity, words such as "target," "first," "second," etc. are often used to distinguish between words. This kind of expression rarely appears in ordinary texts, which will have a certain impact on the extraction of abstracts. Therefore, traditional automatic text summarization methods are relatively poor in extracting patent text abstracts. Therefore, how to accurately extract patent text abstracts is an urgent problem to be solved. Summary of the Invention
[0007] In view of this, embodiments of the present invention provide a method, system, apparatus and storage medium for extracting patent text summaries, which can accurately extract patent text summaries.
[0008] On one hand, embodiments of the present invention provide a patent text digest extraction method, including:
[0009] Obtain the patent text of the patent to be extracted;
[0010] Embedded text is obtained by embedding text into the patent text;
[0011] The embedded text is analyzed using a summary extraction model to obtain the summary text. The summary extraction model is obtained by iteratively training in combination with an expansion model. Both the summary model and the expansion model are generated based on the BERT model.
[0012] Among them, the expansion model is used to perform expansion analysis on the abstract text to obtain the expanded text.
[0013] Optionally, the patent text is embedded to obtain embedded text, including:
[0014] Separate and label the chapters of the patent text;
[0015] The patent text after separation is input into the embedding layer, and the text is converted into a tensor to obtain the embedded text.
[0016] Optionally, the method further includes:
[0017] The two BERT models were trained iteratively using embedded text with pre-labeled summary tags as training data. Based on the training results, the model parameters of the two BERT models were adjusted to obtain a summary extraction model and an expansion model. The two BERT models are the first BERT model and the second BERT model, respectively.
[0018] Optionally, the two BERT models are trained iteratively using embedded text with labeled summary tags as training data, including:
[0019] The training data is used as input data and fed into the first BERT model to extract the summary and obtain the output of the first model.
[0020] The output of the first model is input into the second BERT model to perform text expansion, resulting in the output of the second model.
[0021] The training data, summary labels, first model output, and second model output are evaluated to obtain the target evaluation index.
[0022] The process involves using the output of the second model as input data, then returning it to the first BERT model for summary extraction, and obtaining the output of the first model. This process continues until the preset number of training rounds and / or the target evaluation metric exceeds the preset threshold.
[0023] Optionally, the training data, summary labels, first model output, and second model output are evaluated to obtain evaluation metrics, including:
[0024] The abstract tags and the output of the first model are subjected to a first evaluation process to obtain the first evaluation index;
[0025] The training data and the output of the second model are subjected to a second evaluation process to obtain a second evaluation index.
[0026] Based on the first and second evaluation indicators, a combined evaluation is conducted to obtain the target evaluation indicators.
[0027] Optionally, the abstract labels and the output of the first model are subjected to a first evaluation process to obtain a first evaluation metric, including:
[0028] The cosine similarity between the summary label and the output of the first model is used as the first evaluation metric.
[0029] The expression for the first evaluation indicator is as follows:
[0030]
[0031] Where e1 represents the first evaluation indicator, abstract i This indicates that the first model outputs the i-th vector representation, gt. iThis represents the i-th vector representation of the summary label.
[0032] Optionally, a second evaluation process is performed on the training data and the output of the second model to obtain a second evaluation metric, including:
[0033] The cosine similarity between the original embedded text in the training data and the output of the second model is calculated as a second evaluation metric.
[0034] The expression for the second evaluation indicator is as follows:
[0035]
[0036] Where e2 represents the first evaluation indicator, text i Origin represents the i-th vector representation output by the second model. i This represents the i-th vector representation of the original embedded text in the training data.
[0037] On the other hand, embodiments of the present invention provide a patent text summarization extraction system, including:
[0038] The first module is used to obtain the patent text of the patent to be extracted;
[0039] The second module is used to embed the patent text to obtain the embedded text.
[0040] The third module is used to perform summary analysis on the embedded text using a summary extraction model to obtain the summary text.
[0041] The fourth module is used to perform summary analysis on the embedded text using a summary extraction model to obtain the summary text; the summary extraction model is obtained by iterative training in combination with the expansion model, and both the summary model and the expansion model are generated based on the BERT model.
[0042] Among them, the expansion model is used to perform expansion analysis on the abstract text to obtain the expanded text.
[0043] On the other hand, embodiments of the present invention provide a patent text summarization extraction device, including a processor and a memory;
[0044] Memory is used to store programs;
[0045] The processor executes the program as described above.
[0046] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.
[0047] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.
[0048] This invention first obtains the patent text of the patent to be extracted; then embeds the patent text to obtain embedded text; converting the text data into embedded text that conforms to the input format of the subsequent model, thereby improving the efficiency of model analysis; using a summary extraction model, the embedded text is analyzed to obtain summary text; wherein, the summary extraction model is obtained by iterative training in combination with an expansion model, and both the summary model and the expansion model are generated based on the BERT model; wherein, the expansion model is used to perform expansion analysis on the summary text to obtain expanded text; by combining and training the summary extraction model and the expansion model in a coordinated manner, the purpose of model training is strengthened, and the accuracy of patent summary extraction is effectively improved. Attached Figure Description
[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0050] Figure 1 A schematic flowchart of a patent text abstract extraction method provided in an embodiment of the present invention;
[0051] Figure 2 A schematic diagram of the overall framework of a patent text abstract extraction method provided in an embodiment of the present invention;
[0052] Figure 3 This is a schematic diagram of the structure of the BERT model provided in an embodiment of the present invention. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0054] On the one hand, such as Figure 1 As shown, an embodiment of the present invention provides a patent text digest extraction method, including:
[0055] S100. Obtain the patent text of the patent to be extracted;
[0056] Specifically, the patent text is obtained by organizing the patent documents that require text abstract extraction. For example, the text parts such as the specification abstract, specification, and claims are organized, while non-text parts such as the specification drawings and abstract drawings are removed, ultimately forming the patent text.
[0057] S200. Embed the patent text to obtain the embedded text;
[0058] It should be noted that in some embodiments, the process includes: separating and marking each chapter of the patent text; inputting the separated and marked patent text into the embedding layer, performing text-to-tensor conversion, and obtaining the embedded text.
[0059] Specifically, in some embodiments, a word embedding layer, named the embed layer, is first established for text embedding. The word embedding layer of a BERT pre-trained model can be used to embed words into the text. The specific steps are: converting the text data into tensor data that can be input into the network model. In particular, when inputting data, separators [SEP] need to be inserted between different parts of the patent text to distinguish between different chapters, thus adapting to the strong structural characteristics of patents with the same chapter type and a single format. For example, patent texts typically adopt a fixed form and structure, corresponding to a text structure mainly composed of five parts: abstract, specification, drawings, claims, and abstract drawings.
[0060] S300. Using a summary extraction model, perform summary analysis on the embedded text to obtain the summary text;
[0061] The abstract extraction model is obtained by iteratively training in combination with the expansion model. Both the abstract model and the expansion model are generated based on the BERT model. The expansion model is used to perform expansion analysis on the abstract text to obtain the expanded text.
[0062] It should be noted that some embodiments further include: using embedded text with labeled summary tags as training data to iteratively train two BERT models, and adjusting the model parameters of the two BERT models based on the training results to obtain a summary extraction model and an expansion model; the two BERT models are the first BERT model and the second BERT model, respectively. The summary tags can be confirmed by labeling the patent text itself or by manual labeling.
[0063] In some embodiments, embedded text with pre-labeled summary tags is used as training data to iteratively train two BERT models. This includes: inputting the training data into a first BERT model to extract a summary and obtain the output of the first model; inputting the output of the first model into a second BERT model to expand the text and obtain the output of the second model; evaluating the training data, summary tags, the output of the first model, and the output of the second model to obtain a target evaluation metric; and then inputting the output of the second model back into the first BERT model to extract a summary and obtain the output of the first model, repeating this process until a preset number of training rounds is reached and / or the target evaluation metric exceeds a preset threshold.
[0064] In some embodiments, the training data, summary labels, first model output, and second model output are evaluated to obtain evaluation metrics, including: performing a first evaluation process on the summary labels and first model output to obtain a first evaluation metric; performing a second evaluation process on the training data and second model output to obtain a second evaluation metric; and performing a combined evaluation based on the first evaluation metric and the second evaluation metric to obtain a target evaluation metric.
[0065] In some embodiments, a first evaluation process is performed on the summary tags and the output of the first model to obtain a first evaluation index, including: calculating the cosine similarity between the summary tags and the output of the first model as the first evaluation index; wherein, the expression for the first evaluation index is:
[0066]
[0067] Where e1 represents the first evaluation indicator, abstract i This indicates that the first model outputs the i-th vector representation, gt. i This represents the i-th vector representation of the summary label.
[0068] In some embodiments, a second evaluation process is performed on the training data and the output of the second model to obtain a second evaluation metric, including: calculating the cosine similarity between the original embedded text in the training data and the output of the second model as the second evaluation metric; wherein, the expression for the second evaluation metric is:
[0069]
[0070] Where e2 represents the first evaluation indicator, text i Origin represents the i-th vector representation output by the second model. i This represents the i-th vector representation of the original embedded text in the training data.
[0071] Specifically, in some specific embodiments, such as Figure 2As shown, the training process of the summary extraction model and the expansion model specifically includes the following steps:
[0072] Step 1: Create a word embedding layer, named the embed layer. Here, we use the word embedding layer of the BERT pre-trained model to embed words into the text.
[0073] Step 2: Input the training data containing the embedded text into the BERT model, name this layer B1, and extract the text summary. Compare the summary extracted by the model with the ground truth (the summary of the patent text itself or the result of manual annotation), and calculate the cosine similarity between the two as an evaluation metric.
[0074]
[0075] Here, `abstract` represents the vector corresponding to the model's output summary, and `gt` represents the vector corresponding to the ground truth. This evaluation metric is used to assess the effectiveness of the summary extraction. Furthermore, the output of this layer is the final output of the entire model (i.e., the summary text).
[0076] Step 3: Input the results from the previous layer into another BERT model, named B2, and expand the summary text extracted in the previous step. Compare the expanded summary text with the original text and calculate the cosine similarity between the two as an evaluation metric.
[0077]
[0078] Here, `text` represents the expanded text output by the model, and `origin` represents the original text before summary extraction. This evaluation result reflects the final text reconstruction after computation by the two models, demonstrating a cyclical learning process.
[0079] Step 4: Repeat steps 2 and 3 to train the model until the preset number of training rounds is reached. During model training, the evaluation metrics from steps 2 and 3 are combined as the final evaluation.
[0080] e = α·e1 + β·e2
[0081] Here, α and β are the weight parameters of the evaluation indicators in steps two and three, respectively, and can be manually adjusted using empirical methods.
[0082] Step 5: Finally, a high-performance patent text generation model is obtained, along with an abstract expansion model.
[0083] The BERT model structure used in the embodiments of the present invention is as follows: Figure 3As shown, the full name of the BERT model is Bidirectional Encoder Representations from Transformer. As the name suggests, the goal of the BERT model is to train on large-scale unlabeled corpora to obtain a representation of text containing rich semantic information, i.e., a semantic representation of the text. This semantic representation is then fine-tuned for a specific NLP task and finally applied to that task. To illustrate, the process of training the BERT model to represent the text's semantics is like learning fundamental subjects such as Chinese, mathematics, English, physics, chemistry, and biology in high school, solidifying basic knowledge; while fine-tuning the model's parameters in a specific NLP task is like further strengthening one's professional skills in university based on existing knowledge, thereby acquiring specialized skills applicable to real-world scenarios. The focus of this embodiment is the process of connecting and iteratively training two BERT models; the structure of the BERT model will not be elaborated further.
[0084] In summary, compared with existing technologies, the embodiments of this invention have the following beneficial effects: First, patent texts, compared with other texts, have strong structural characteristics and unique wording, which can affect the extraction of automatic summaries. This model performs better than other models in patent summary extraction. Second, the embodiments of this invention use two models: a summary extraction model and a summarization model, implementing a cyclical learning process that continuously extracts and rewrites summaries. If the text remains stable after multiple calculations, the model's performance is considered superior. This training process and evaluation criteria are more effective than existing technologies. Third, while obtaining an excellent automatic patent text summarization model, the embodiments of this invention also obtain a patent summary rewriting model, which can restore the summary to a complete patent. These two models can effectively assist in the writing, analysis, and querying of patents, and have considerable application value.
[0085] On the other hand, embodiments of the present invention provide a patent text abstracting system, comprising: a first module for acquiring the patent text of the patent to be extracted; a second module for embedding the patent text to obtain embedded text; a third module for performing abstract analysis on the embedded text using an abstract extraction model to obtain abstract text; and a fourth module for performing abstract analysis on the embedded text using an abstract extraction model to obtain abstract text. The abstract extraction model is obtained through iterative training in combination with an expansion model, both of which are generated based on the BERT model. The expansion model is used to perform expansion analysis on the abstract text to obtain expanded text.
[0086] It should be noted that in some embodiments, a fifth module is also included, which is used to train two BERT models cyclically using embedded text with labeled summary tags as training data, and to adjust the model parameters of the two BERT models based on the training results to obtain a summary extraction model and an expansion model; the two BERT models are the first BERT model and the second BERT model, respectively.
[0087] The content of the method embodiments of the present invention is applicable to the system embodiments. The specific functions implemented by each module of the system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.
[0088] Another aspect of the present invention provides a patent text summarization extraction device, including a processor and a memory;
[0089] Memory is used to store programs;
[0090] The processor executes the program as described above.
[0091] The content of the method embodiments of the present invention is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.
[0092] Another aspect of this invention provides a computer-readable storage medium storing a program that is executed by a processor to implement the method described above.
[0093] The content of the method embodiments of the present invention is applicable to the computer-readable storage medium embodiments. The specific functions implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above methods.
[0094] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.
[0095] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.
[0096] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.
[0097] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0098] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution means, apparatus, or device (such as a computer-based device, a processor-including device, or other means that can fetch and execute instructions from, or in conjunction with, an instruction execution means, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution means, apparatus, or device.
[0099] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.
[0100] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0101] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0102] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
[0103] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.
Claims
1. A method for extracting patent text summaries, characterized in that, include: Obtain the patent text of the patent to be extracted; The patent text is embedded to obtain the embedded text; The embedded text is analyzed using a summary extraction model to obtain a summary text; wherein the summary extraction model is obtained by iterative training in combination with an expansion model, and both the summary extraction model and the expansion model are generated based on the BERT model; The expansion model is used to perform expansion analysis on the summary text to obtain expanded text; The method further includes: The two BERT models are trained iteratively using embedded text with pre-labeled summary tags as training data. Based on the training results, the model parameters of the two BERT models are adjusted to obtain a summary extraction model and an expansion model. The two BERT models are the first BERT model and the second BERT model, respectively. The step of using embedded text with labeled summary tags as training data to iteratively train two BERT models includes: Using the training data as input data, input the first BERT model to perform summary extraction and obtain the output of the first model; The output of the first model is input into the second BERT model to perform text expansion, resulting in the output of the second model. The training data, the summary labels, the first model output, and the second model output are evaluated to obtain the target evaluation index. The process involves using the output of the second model as input data, then returning the input to the first BERT model to extract the summary and obtain the output of the first model, until a preset number of training rounds is reached and / or the target evaluation index exceeds a preset threshold.
2. The patent text digest extraction method according to claim 1, characterized in that, The process of embedding the patent text to obtain embedded text includes: The chapters of the patent text are separated and labeled; The patent text after the separation mark is input into the embedding layer, and the text is converted into a tensor to obtain the embedded text.
3. The patent text digest extraction method according to claim 1, characterized in that, The evaluation process, which evaluates the training data, the summary labels, the first model output, and the second model output to obtain evaluation metrics, includes: The first evaluation process is performed on the summary tags and the output of the first model to obtain the first evaluation index; The training data and the output of the second model are subjected to a second evaluation process to obtain a second evaluation index; Based on the first evaluation index and the second evaluation index, a combined evaluation is performed to obtain the target evaluation index.
4. The patent text abstract extraction method according to claim 3, characterized in that, The first evaluation process, which evaluates the summary tags and the output of the first model to obtain a first evaluation metric, includes: The cosine similarity between the summary label and the output of the first model is calculated as the first evaluation metric. The expression for the first evaluation indicator is as follows: in, This indicates the primary evaluation indicator. This indicates that the first model outputs the... A vector representation. The first of the abstract tags Each vector represents a vector.
5. A patent text abstract extraction method according to claim 3, characterized in that, The second evaluation process, which involves processing the training data and the output of the second model to obtain a second evaluation metric, includes: The second evaluation metric is the cosine similarity between the original embedded text in the training data and the output of the second model. The expression for the second evaluation indicator is as follows: in, This indicates the primary evaluation indicator. This indicates that the second model outputs the first... A vector representation. This represents the first embedded text in the original training data. Each vector represents a vector.
6. A patent text abstracting system, characterized in that, include: The first module is used to obtain the patent text of the patent to be extracted; The second module is used to embed the patent text to obtain embedded text. The third module is used to perform summary analysis on the embedded text using a summary extraction model to obtain summary text. The fourth module is used to perform summary analysis on the embedded text using a summary extraction model to obtain summary text; wherein, the summary extraction model is obtained by iterative training in combination with an expansion model, and both the summary extraction model and the expansion model are generated based on the BERT model; The expansion model is used to perform expansion analysis on the summary text to obtain expanded text; The system further includes a fifth module, which is used to train two BERT models cyclically using embedded text labeled with summary tags as training data, and adjust the model parameters of the two BERT models based on the training results to obtain a summary extraction model and an expansion model; the two BERT models are the first BERT model and the second BERT model, respectively. The step of using embedded text with labeled summary tags as training data to iteratively train two BERT models includes: Using the training data as input data, input the first BERT model to perform summary extraction and obtain the output of the first model; The output of the first model is input into the second BERT model to perform text expansion, resulting in the output of the second model. The training data, the summary labels, the first model output, and the second model output are evaluated to obtain the target evaluation index. The process involves using the output of the second model as input data, then returning the input to the first BERT model to extract the summary and obtain the output of the first model, until a preset number of training rounds is reached and / or the target evaluation index exceeds a preset threshold.
7. A patent text summarization extraction device, comprising a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the method as described in any one of claims 1 to 5.