Intelligence abstract method and system based on big language model distillation

Abstract

The invention discloses an information abstract method and system based on large language model distillation. The information abstract method based on large language model distillation comprises the following steps: acquiring information text data; performing knowledge annotation on the information text data by using a large language model to form an information pre-abstract, prompting keywords of the information pre-abstract by using a lightweight language model, and performing distillation by using a preset template to form an information abstract, the lightweight language model is subjected to gradient updating through the difference between the information abstract and the information pre-abstract; and outputting the information abstract. According to the method, the processing speed and efficiency of extracting the intelligence abstract are well improved, the high-quality intelligence abstract can be obtained under the condition of low cost, and the requirements of various application scenes for efficient intelligence analysis are met.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligence summarization method and system based on large language model distillation. Background Art

[0002] In the existing technology, the processing of open source intelligence usually adopts manually written rules or template matching methods based on traditional machine learning technology. These methods have limited effect when processing complex and changeable open source intelligence, and are difficult to adapt to rapidly changing intelligence needs. In addition, open source intelligence may come from different languages ​​​​around the world, and it is very difficult to rely entirely on data collectors for manual labeling, which greatly increases time and labor costs.

[0003] For example, manually written rules rely on expert experience for rule design, but the rules have poor adaptability and generalization capabilities. As the amount and complexity of open source intelligence increase, manually writing rules becomes increasingly difficult and it is difficult to cope with the ever-changing intelligence scenarios.

[0004] For example, traditional machine learning template matching methods generate summaries by matching open source intelligence with templates, but the matching accuracy and summary quality are limited by template design and matching algorithms. Such methods are highly dependent on templates and are not easy to adapt to different scenarios and needs. In addition, summary annotation is completely dependent on manual work, which is time-consuming and labor-intensive. At the same time, the quality of the summary is difficult to guarantee, which further affects the training results of the automatic summary model.

[0005] In addition, it is not realistic to directly deploy a large language model with superior performance locally to generate open source intelligence summaries. First, the large language model with the best performance is expensive to deploy locally and requires a lot of computing power support, making it difficult to deploy. Second, these large language models with the best performance are not currently open source, and the data sets and training details used to train the large language models are not public, making them impossible to deploy and train locally. Third, currently only relatively small-scale lightweight large language models are open source, and their "emergence" in language understanding is not outstanding, and their performance is far inferior to that of the best large language model online.

[0006] Therefore, how to improve the analysis and processing efficiency of open source intelligence is the technical problem to be solved by the present invention. Summary of the invention

[0007] The purpose of the present invention is to provide an intelligence summarization method and system based on large language model distillation. The intelligence summarization method based on large language model distillation is a method suitable for automatic summarization of open source intelligence. It aims to utilize the powerful language understanding and generation capabilities of online large language models, combine knowledge distillation to train local lightweight language models to improve intelligence processing speed and utilization, and thus improve the analysis and processing efficiency of open source intelligence.

[0008] According to one aspect of the present invention, at least one embodiment provides an intelligence summarization method based on large language model distillation, comprising: obtaining intelligence text data; using a large language model to perform knowledge annotation on the intelligence text data to form an intelligence pre-summary, prompting keywords of the intelligence pre-summary through a lightweight language model and using a preset template distillation to form an intelligence summary, wherein the lightweight language model performs gradient update through the difference between the intelligence summary and the intelligence pre-summary; and outputting the intelligence summary.

[0009] According to another aspect of the present invention, at least one embodiment also provides a method for training a local lightweight open source large language model, including: obtaining intelligence text data, using a large language model to perform knowledge annotation on the intelligence text data to form an intelligence pre-summary; manually correcting the intelligence pre-summary to form an intelligence summary data set, dividing the intelligence summary data set into a training set and a test set, prompting keywords of the training set through a lightweight language model and using a preset template for distillation to form an intelligence summary, wherein the lightweight language model performs gradient update based on the difference between the intelligence summary and the training set; using BLEU and ROUGE-k to evaluate the intelligence summary, and iterating the lightweight language model based on the evaluation results.

[0010] According to another aspect of the present invention, at least one embodiment also provides an intelligence summarization device based on large language model distillation, comprising: a processor, suitable for implementing various instructions; and a memory, suitable for storing multiple instructions, wherein the instructions are suitable for being loaded and executed by the processor: the intelligence summarization method based on large language model distillation developed by the present invention and / or the method for training a local lightweight open source large language model developed by the present invention.

[0011] According to another aspect of the present invention, at least one embodiment further provides an intelligence summarization system based on large language model distillation, including: an intelligence summarization device based on large language model distillation developed by the present invention.

[0012] According to another aspect of the present invention, at least one embodiment further provides a computer-readable non-volatile storage medium storing computer program instructions, which, when the computer executes the program instructions, executes: the intelligence summarization method based on large language model distillation developed by the present invention and / or the method for training a local lightweight open source large language model developed by the present invention.

[0013] The present invention automatically generates summaries of open source intelligence in the above manner. Through the knowledge distillation strategy and prompt mechanism data based on the large language model, the local lightweight language model can quickly generate summaries that meet the requirements as required, reduce the reliance on manually written rules and template matching, and accurately and completely reflect the core content of open source intelligence in a smaller space, thereby greatly improving the intelligence processing speed and utilization rate, and the analysis and processing efficiency of open source intelligence is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 is a schematic diagram of an intelligence summarization system based on large language model distillation according to an embodiment of the present invention;

[0016] Figure 2 is a schematic diagram of an intelligence summarization device based on large language model distillation according to an embodiment of the present invention;

[0017] Figure 3 is a flow chart of an intelligence summarization method based on large language model distillation according to an embodiment of the present invention;

[0018] Figure 4 It is a flow chart of a method for training a local lightweight open source large language model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] In the fields of national security, commercial competition, scientific research, etc., the importance of open source intelligence is becoming increasingly prominent. However, traditional open source intelligence processing methods and technologies are limited in speed and efficiency. In order to solve this problem, this paper proposes an intelligence summary system based on large language model distillation, aiming to improve the analysis and processing efficiency of open source intelligence. Figure 1 As shown, the environment of the intelligence summarization system based on large language model distillation may include: an intelligence summarization device 100 based on large language model distillation and a server 200.

[0021] The intelligence summary device 100 based on large language model distillation can operate the server 200 through corresponding instructions, so as to read, change, add data, etc. The intelligence summary device 100 based on large language model distillation can be one or more, or can include multiple processing nodes, and the multiple processing nodes can be used as a whole externally. Figure 2 As shown, the intelligence summarization device 100 based on large language model distillation includes: a processor 202, suitable for implementing various instructions; and a memory 204, suitable for storing multiple instructions, and the instructions are suitable for being loaded and executed by the processor 202: the intelligence summarization method based on large language model distillation developed by the present invention and / or the method for training a local lightweight open source large language model developed by the present invention.

[0022] The processor 202 may be any suitable processor, for example, implemented in the form of a central processing unit, a microprocessor, an embedded processor, etc., and may adopt an architecture such as X86, ARM, etc. The memory 204 may be any suitable storage device, for example, a non-volatile storage device, including but not limited to a magnetic storage device, a semiconductor storage device, an optical storage device, etc., and may be arranged as a single storage device, a storage device array, or a distributed storage device, which is not limited in the embodiments of the present invention.

[0023] Those skilled in the art will appreciate that the structure of the intelligence summarization device 100 based on large language model distillation is only for illustration and does not limit the structure of the device. Figure 2 The transmission device is used to receive or send data via a network. In one embodiment, the transmission device is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0024] Corresponding to the intelligence summary device 100 based on large language model distillation, at least one embodiment of the present invention further provides a computer-readable non-volatile storage medium, which stores computer program instructions. When the computer executes the program instructions, it executes: the intelligence summary method based on large language model distillation developed by the present invention and / or the method for training a local lightweight open source large language model developed by the present invention. The computer-readable non-volatile storage medium can be built into the intelligence summary device 100 based on large language model distillation.

[0025] Optionally, the intelligence summary device 100 based on large language model distillation may also send the acquired intelligence text data to the server 200, so that the server 200 uses the large language model deployed thereon to perform knowledge annotation on the intelligence text data to form an intelligence pre-summary and returns it to the intelligence summary device 100 based on large language model distillation. The intelligence summary device 100 based on large language model distillation can be connected to the server 200 through a network. The above-mentioned network includes a wired network and a wireless network. The wireless network includes but is not limited to: a wide area network, a metropolitan area network, a local area network or a mobile data network. Typically, the mobile data network includes but is not limited to: a global mobile communication (GSM) network, a code division multiple access (CDMA) network, a wideband code division multiple access (WCDMA) network, a long-term evolution (LTE) communication network, a WIFI network, a ZigBee network, a network based on Bluetooth technology, etc. Different types of communication networks may be operated by different operators. The type of communication network does not constitute a limitation on the embodiments of the present invention.

[0026] The present invention uses the above-mentioned intelligence summary system based on large language model distillation, utilizes the powerful language understanding and generation capabilities of the large language model, combines knowledge distillation to train a local lightweight large language model with a smaller scale, and designs a method suitable for automatic summary of open source intelligence to improve the quality of open source intelligence summary. Compared with the existing technology, the intelligence summary system based on large language model distillation has the following advantages: it does not need to rely on manually written rules, and can generate summaries that meet the requirements by fine-tuning the local lightweight large language model; it does not need to perform full manual data labeling, and the knowledge distillation between large language models can improve the efficiency and quality of data labeling and reduce the difficulty of model training; the model has good generalization ability and can adapt to the intelligence analysis needs in different scenarios; it uses prompt mechanisms and data enhancement methods to further improve the quality of summary generation.

[0027] Under the operation of the above-mentioned software and hardware environment, at least one embodiment of the present invention provides an intelligence summary method based on large language model distillation, which can also be called an open source intelligence summary method based on large language model knowledge distillation or an intelligence automatic summary technology based on large language model knowledge distillation, which can solve the problems of the existing technology in terms of open source intelligence processing speed and quality, better improve the processing speed and utilization rate of open source intelligence, and meet the needs of various application scenarios for efficient intelligence analysis. Figure 3 The flowchart of the open source intelligence summary method based on the knowledge distillation of a large language model is shown. It should be noted that the steps shown in the flowchart of the accompanying figure can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from that shown here. The method may include the following steps:

[0028] Step S301, obtaining intelligence text data;

[0029] Step S303, using a large language model to perform knowledge annotation on intelligence text data to form an intelligence pre-summary, using a lightweight language model to prompt keywords of the intelligence pre-summary and using a preset template for distillation to form an intelligence summary, wherein the lightweight language model performs gradient update based on the difference between the intelligence summary and the intelligence pre-summary;

[0030] Step S305, output the intelligence summary.

[0031] The present invention uses the powerful language understanding and generation capabilities of the large language model in the above manner to automatically generate summaries of open source intelligence. Through the knowledge distillation strategy and prompt mechanism data based on the large model, the present invention enables the model to quickly generate summaries that meet the requirements as required, reducing the reliance on manually written rules and template matching, accurately and completely reflecting the core content of open source intelligence in a smaller space, and improving intelligence processing speed and efficiency. In short, the present invention has technical advantages in terms of summary quality, processing speed and efficiency, generalization ability, deployment cost, etc., and has higher practical value and application prospects than the existing technology.

[0032] In step S301, intelligence text data is obtained. For example, intelligence text data at the document level on the Internet is collected using a data crawler tool. i represents the i-th intelligence text (N in total), wherein the data crawler tool includes a general web crawler tool, a focused web crawler tool, an incremental web crawler tool and / or a deep web crawler tool.

[0033] General web crawlers have a large crawling range and a large number of crawls, and have high requirements for crawling speed and storage space. They are mainly used for data collection by portal search engines and large Web service providers. General web crawlers have a wide coverage and generally work in parallel. Their structure is roughly divided into the following modules: page crawling, page analysis, link filtering, page database, URL queue, and initial URL set.

[0034] Focused web crawlers are crawlers that can perform content screening. Compared with general web crawlers, they have added link evaluation modules and content evaluation modules to evaluate the importance of the content and links of the crawled pages, and sort the URL access sequence according to different importance. Focused web crawlers crawl pages related to the required topics, saving hardware and network resources.

[0035] Incremental web crawlers crawl only newly generated or changed web pages, and crawl only when needed, which reduces the amount of data downloaded by the crawler, but the crawling algorithm is more complicated. The structure of the incremental web crawler includes a crawling module, a sorting module, an update module, a local page set, a set of URLs to be crawled, and a set of local page URLs.

[0036] The deep web crawler crawls content by filling out forms. It is mainly used to crawl web pages hidden behind search forms and cannot be directly crawled through static links. The deep web crawler contains six basic modules: crawl control, parser, form analyzer, form processor, response analyzer, LVS controller, and two crawler internal data structures.

[0037] In step S303, the intelligence text data is annotated with knowledge using a large language model to form an intelligence pre-summary, the keywords of the intelligence pre-summary are prompted by a lightweight language model, and the intelligence summary is formed by distillation using a preset template, wherein the lightweight language model performs gradient update based on the difference between the intelligence summary and the intelligence pre-summary. The large language model is deployed on the server 200, including but not limited to ChatGPT, GPT-4, PaLM, mT5, Gopher, Chinchilla, Sparrow, Claude, OPT-IML, BlenderBot-3, Llama, Llama2, Jurassic, Megatron-Turing NLG, Ernie3.0Titan, Ernie Bot, GLM, GLM-130B, ChatGLM-6B, PanGu-Alpha, M6, Qwen-VL, RAPHAEL. The lightweight language model is deployed locally, including but not limited to ChatGLM-6B and LLaMa-7B.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] First, using a large language model to perform knowledge annotation on intelligence text data to form an intelligence pre-summary may include: obtaining first prompt information of the large language model, wherein the first prompt information includes at least a task prompt, an intelligence prompt, and a template prompt; inputting the first task into the large language model according to the task prompt, the intelligence text data into the large language model according to the intelligence prompt, and the first template into the large language model according to the template prompt; and obtaining an intelligence pre-summary annotated by the large language model knowledge. Optionally, the task prompt may include prompts for multiple tasks, and the multiple tasks may include the first task. Optionally, the intelligence prompt may include prompts for intelligence text data. Optionally, the template prompt may include prompts for multiple templates, and the multiple templates may include the first template.

[0040] Optionally, select the Internet online large language model base M with excellent performance t , such as ChatGPT for intelligence summary annotation. This type of model has strong language understanding and generation capabilities, providing a good foundation for open source intelligence summaries. Specifically, obtain the first prompt information of ChatGPT and convert the collected intelligence text data And given the first template T1 in the form of a prompt mechanism P t (E i ,T1) Input M t , the M t Output the corresponding intelligence summary according to the preset first task (optionally, the first task can also be input according to the prompt mechanism) The first task is "Please generate a summary of the following intelligence text based on XX research and judgment target, and the number of related core intelligence items must not exceed XX."

[0041] Second, the keywords of the intelligence pre-summary are prompted by a lightweight language model and the intelligence summary is formed by distillation using a preset template. The lightweight language model can also be a lightweight large language model. For example, the second prompt information of the lightweight language model is obtained, wherein the second prompt information at least includes a task prompt and an intelligence prompt; the second task is input into the lightweight language model according to the task prompt, and the intelligence pre-summary is input into the lightweight language model according to the intelligence prompt, and the keywords output by the lightweight language model are obtained, wherein the second task is to extract the keywords of the intelligence pre-summary; the third prompt information of the lightweight language model is obtained, wherein the third prompt information at least includes a task prompt, an intelligence prompt, a keyword prompt, and a template prompt; the third task is input into the lightweight language model according to the task prompt, the intelligence pre-summary is input according to the intelligence prompt, the keywords are input according to the keyword prompt, and the second template is input into the lightweight language model according to the template prompt, and the intelligence summary distilled by the lightweight language model is obtained.

[0042] Optionally, select a relatively small, lightweight, open-source large language model M that can be deployed offline. s, such as ChatGLM-6B or LLaMa-7B, etc., according to the two-step thinking chain prompt mechanism As M s Input, locally generated summary as output G i , T2 is a second template that is preset or input. For example, obtain the second prompt information of the lightweight language model, and according to the second prompt information, The second task inputs the lightweight language model. If the second task is "Please focus on XX to judge the target and extract keywords from the following intelligence text. The number of entries is required to be no more than XX", then the keyword K output by the lightweight language model can be obtained. i .

[0043] The present invention selects the lightweight large language model (such as ChatGLM and LLaMa, etc.), and uses the powerful language understanding and generation capabilities of the lightweight large language model to generate a more accurate and complete open source intelligence summary, which not only improves the quality of the open source intelligence summary, but also helps users quickly understand the core content of the open source intelligence.

[0044] Optionally, after getting the keyword K i After that, the lightweight language model will give a third prompt message; according to the third prompt message, T2, K i And the lightweight language model is input into the third task. If the third task is "Please analyze the target around XX, combine the intelligence text content and keyword prompts, generate an intelligence summary according to the given template, and require that the number of associated core intelligence items is no more than XX", then the intelligence summary distilled by the lightweight language model will be obtained.

[0045] In step S305, the intelligence summary is output. For example, BLEU and ROUGE-k are used to evaluate the intelligence summary. lr is the real summary length, lc is the length of the generated summary G i The length of, k represents k keywords, n represents n keywords, W k Indicates true summary and generate summary G i The weight of the precision of {1-gram, 2-gram, ..., n-gram}, P k Indicates true summary and generate summary G i The corresponding precision of {1-gram, 2-gram, ..., n-gram}, k-gram is a combination of k consecutive keywords, Count(A i ) is the true summary A i The number of k-grams in Countmatch(Gi ) is the generated summary G i and the true summary The number of k-grams matched in the result is obtained by cross-validation, and an intelligence summary that meets the requirements is output.

[0046] That is to say, the present invention uses BLEU and ROUGE evaluation indicators to evaluate the generated summary. BLUE is used to evaluate the accuracy of the generated summary, and ROUGE-k is used to evaluate the information completeness and coverage of the generated summary. First, calculate the real summary and generate summary G i The precision of {1-gram, 2-gram, ..., n-gram} (generally n is set to 4), the corresponding precision is recorded as Pk, and the weight is Wk. In order to prevent the summary from being too short, resulting in the correct content of the generated short summary, but the content is too short to be completed, Pk is artificially high, and the short text penalty BP is increased. Secondly, ROUGE can obtain multiple ROUGE values ​​according to the division method of {1-gram, 2-gram, ..., n-gram}, namely {ROUGE-1, ROUGE-2, ..., ROUGE-n}, so that the ROUGE-k indicator can be obtained for evaluating the information completeness and coverage of the generated summary, so that the present invention can automatically generate an intelligence summary that meets the requirements. The present invention optimizes the evaluation indicators and methods to comprehensively evaluate the performance of the model on the open source intelligence summary task, and continuously improves the quality and practicality of the model.

[0047] Through the above-mentioned method, the present invention reduces the difficulty of summarizing open source intelligence, improves the quality and processing speed of summary generation, and better meets the needs of national security, commercial competition, scientific research and other fields for efficient and high-quality intelligence analysis.

[0048] Under the operation of the above-mentioned software and hardware environment, at least one embodiment of the present invention provides a method for training a local lightweight open source large language model, such as Figure 4 As shown, including:

[0049] Step S402, obtaining intelligence text data, and using a large language model to perform knowledge annotation on the intelligence text data to form an intelligence pre-summary;

[0050] Step S404, manually correcting the intelligence pre-summary to form an intelligence summary data set, dividing the intelligence summary data set into a training set and a test set, using a lightweight language model to prompt keywords of the training set and using a preset template for distillation to form an intelligence summary, wherein the lightweight language model performs gradient update according to the difference between the intelligence summary and the training set;

[0051] Step S406, using BLEU and ROUGE-k to evaluate the intelligence summary, and iterating the lightweight language model based on the evaluation results.

[0052] The present invention uses the above method to fine-tune the existing open source lightweight large language model with a relatively small scale, thereby reducing the deployment cost of the local lightweight open source large language model and making the invention easy to promote and apply.

[0053] In step S402, intelligence text data is obtained, and the intelligence text data is annotated with knowledge using a large language model to form an intelligence pre-summary. This step S402 is similar to the method of obtaining intelligence text data in step S301, and the method of using a large language model to annotate intelligence text data with knowledge to form an intelligence pre-summary in step S402 is also similar to the method of step S303, which is referred to for execution and will not be repeated here.

[0054] In step S404, the intelligence pre-summary is manually corrected to form an intelligence summary data set, and the intelligence summary data set is divided into a training set and a test set. The keywords of the training set are prompted by a lightweight language model and the intelligence summary is formed by distillation using a preset template. In order not to introduce too many symbol variables, the present invention continues to borrow the same symbols of the application embodiment "Intelligence summary method based on large language model distillation" in the training embodiment "Method for training a local lightweight open source large language model" to describe the training process of the present invention. For example, It also represents the intelligence text data in the training example, The same is true for pre-summary intelligence in training examples, and so on.

[0055] First, in the training embodiment, Manual corrections are performed to reduce the burden of manually writing intelligence summaries from scratch, accelerate the construction of intelligence summary datasets, and ultimately obtain intelligence summary datasets for training local lightweight language models Therefore, the present invention can Perform random split to form a training set and test set

[0056] That is to say, the present invention performs summary generation in the first stage based on a large language model deployed online, and then significantly reduces the cost and difficulty of fully manual summary annotation through manual correction and improvement.

[0057] Second, a lightweight language model (such as ChatGLM-6B or LLaMa-7B, etc.) is used to prompt the keywords K of the training set. i The intelligence summary is formed by distillation using the preset template. The lightweight language model performs gradient updates based on the difference between the intelligence summary and the intelligence pre-summary of the training set. Assuming the loss function is L(·), the model parameter fine-tuning training is carried out as follows: α is a constant, Ms The amount of change in the model.

[0058] That is to say, the present invention designs fine-tuning strategies and prompt mechanisms to enable the model to quickly generate a summary that meets the requirements based on a given template, reduce the reliance on manually written rules and template matching, and improve the speed and efficiency of intelligence processing.

[0059] In step S406, BLEU and ROUGE-k are used to evaluate the intelligence summary, and the lightweight language model is iterated according to the evaluation results. This step S406 is similar to the method of using BLEU and ROUGE-k to evaluate the intelligence summary in step S305, and is executed with reference, and no further description is given.

[0060] Therefore, the present invention can use cross-validation to obtain evaluation results for the existing open source local lightweight open source large language model with a relatively small scale, and continuously iteratively train the local lightweight open source large language model according to the evaluation results, which not only improves the quality of automatic summarization, but also reduces the deployment cost, making the invention easy to promote and apply.

[0061] Through the above-mentioned method, the present invention combines the online large model to generate summaries with manual correction, which greatly speeds up the construction of training data. It has high summary quality, processing speed and versatility, and is easy to deploy and optimize, meeting the needs of various application scenarios for efficient intelligence analysis.

[0062] In summary, the intelligence automatic summarization technology based on large language model knowledge distillation provided by the present invention mainly realizes efficient summary annotation based on large language model knowledge distillation technology, constructs a multi-step thinking chain prompt mechanism for automatic summarization, and fine-tunes the local lightweight open source large language model to realize accurate and fast intelligence automatic summarization technology.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligence summarization method based on large language model distillation, characterized in that: include: Obtain intelligence text data; Using a large language model to perform knowledge annotation on the intelligence text data to form an intelligence pre-summary, using a lightweight language model to prompt keywords of the intelligence pre-summary and using a preset template for distillation to form an intelligence summary, wherein the lightweight language model performs gradient update according to the difference between the intelligence summary and the intelligence pre-summary; Output a summary of said intelligence.

2. The intelligence summarization method based on large language model distillation according to claim 1, characterized in that: The intelligence text data obtained includes: Intelligence text data is collected based on a data crawler tool, wherein the data crawler tool includes a general web crawler tool, a focused web crawler tool, an incremental web crawler tool and / or a deep web crawler tool.

3. The intelligence summarization method based on large language model distillation according to claim 1, characterized in that: Using a large language model to perform knowledge annotation on the intelligence text data to form an intelligence pre-summary includes: Acquire first prompt information of the large language model, wherein the first prompt information at least includes a task prompt, an intelligence prompt, and a template prompt; Inputting the first task into the large language model according to the task prompt, the intelligence text data into the intelligence prompt, and the first template into the large language model according to the template prompt; A preliminary summary of the intelligence annotated by the large language model knowledge is obtained.

4. The intelligence summarization method based on large language model distillation according to claim 3 is characterized in that: Prompting keywords of the intelligence pre-summary through a lightweight language model and using a preset template to distill and form an intelligence summary includes: Obtaining second prompt information of the lightweight language model, wherein the second prompt information at least includes a task prompt and an intelligence prompt; inputting the second task according to the task prompt and the intelligence pre-summary according to the intelligence prompt into the lightweight language model to obtain keywords output by the lightweight language model, wherein the second task is to extract keywords of the intelligence pre-summary; Obtain third prompt information of the lightweight language model, wherein the third prompt information includes at least a task prompt, an intelligence prompt, a keyword prompt and a template prompt; input the third task according to the task prompt, the intelligence pre-summary according to the intelligence prompt, the keyword according to the keyword prompt, and the second template according to the template prompt into the lightweight language model to obtain the intelligence summary distilled by the lightweight language model.

5. The intelligence summarization method based on large language model distillation according to claim 1, characterized in that: Output of said intelligence summary includes: BLEU and ROUGE-k are used to evaluate the intelligence summary. Said Said lr is the real summary A i length, lc is the length of the generated summary G i length, k represents k keywords, n represents the number of keywords, W k Indicates the true summary A i and generate summary G i The weight of the precision of {1-gram, 2-gram, ..., n-gram}, P k Indicates the true summary A i and generate summary G i The corresponding precision of {1-gram, 2-gram, ..., n-gram}, k-gram is a combination of k consecutive keywords, Count(A i ) is the true summary A i The number of k-grams in Count match (G i ) is the generated summary G i and the true summary A i The number of k-grams matched in ; The evaluation results are obtained through cross-validation and an intelligence summary that meets the requirements is output.

6. The intelligence summarization method based on large language model distillation according to claim 1, characterized in that: The large language model is ChatGPT, and the lightweight language model is ChatGLM-6B or LLaMa-7B.

7. A method for training a local lightweight open source large language model, characterized in that: include: Acquire intelligence text data, and use a large language model to perform knowledge annotation on the intelligence text data to form an intelligence pre-summary; Manually correcting the intelligence pre-summary to form an intelligence summary data set, dividing the intelligence summary data set into a training set and a test set, prompting keywords of the training set through a lightweight language model and using a preset template for distillation to form an intelligence summary; The intelligence summary is evaluated using BLEU and ROUGE-k, and the lightweight language model is iterated based on the evaluation results.

8. An intelligence summarization device based on large language model distillation, comprising: a processor adapted to implement the instructions; And a memory suitable for storing a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor: the intelligence summarization method based on large language model distillation as described in any one of claims 1-6 and / or the method for training a local lightweight open source large language model as described in claim 7.

9. Intelligence summary system based on large language model distillation, including: An intelligence summarization device based on large language model distillation as described in claim 8.

10. A computer-readable non-volatile storage medium storing computer program instructions, which, when executed by a computer, executes: the intelligence summarization method based on large language model distillation as described in any one of claims 1 to 6 and / or the method for training a local lightweight open source large language model as described in claim 7.

Images