Method and device for discovering hot topics in electric power field, medium and equipment

Abstract

The invention discloses a hot topic discovery method and device in the electric power field, a medium and equipment, and the method comprises the steps: firstly distinguishing sub-fields of the electric power field through a label marking text, then generating a question instruction through self-guidance, and constructing an instruction database; thirdly, performing incremental training on the initial large language model based on a corpus in the power field, and performing fine tuning by using an instruction database, thereby obtaining a target large language model adaptive to the power field; and finally, retrieving the latest label text by utilizing the knowledge graph, and inputting the latest label text into the model to generate the hot topic. It can be understood that according to the method, through incremental training and fine adjustment, the large language model has higher understanding and generating capacity in the electric power field, and the illusion problem is solved. Meanwhile, through the data, the labels and the mapping knowledge domain which are updated in real time, the system can provide hot topics which are more accurate and high in timeliness, and the professionality and accuracy of generated content are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device, medium and equipment for discovering hot topics in the electric power field. Background Art

[0002] The power sector contains a wide range of knowledge, including energy structure, institutional information, power theory, etc., and also involves a large amount of complex text data, such as power policies and regulations, technology development trends, etc. Traditional hot spot discovery methods are difficult to effectively process and extract this information. The Large Language Model (LLM) has shown excellent performance in tasks such as text generation, summary extraction, topic classification, and dialogue. Applying it to the hot spot discovery task in the power sector can generate hot topic descriptions more accurately and vividly.

[0003] However, general large language models have the problem of "hallucination", that is, the content generated by the model may be biased in logical consistency or factual accuracy. When the model is faced with the latest knowledge in a specific field, it is prone to unconstrained creative generation, resulting in information errors and "hallucinations". Therefore, it is crucial to solve the "hallucination" problem when applying large language models to the power field. Summary of the invention

[0004] Based on this, it is necessary to provide methods, devices, media and equipment for discovering hot topics in the power field to solve the problem that the content generated by general large language models may have deviations in logical consistency or factual accuracy.

[0005] A method for discovering hot topics in the electric power field, wherein the electric power field includes a plurality of preset sub-fields, and the method includes:

[0006] Acquire a plurality of sample label texts; wherein the label texts are marked with labels, and the labels are used to indicate the subfield to which the label texts belong;

[0007] Each sample label text is spliced ​​with a preset question instruction, and the splicing result is input into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions;

[0008] Acquire a corpus in the electric power field, and perform incremental training on an initial large language model based on the corpus in the electric power field to obtain an incremental large language model;

[0009] Fine-tuning the incremental large language model based on the question instructions in the instruction database to obtain a target large language model;

[0010] The current label text updated in real time is obtained to construct a knowledge graph, and the label text within the latest preset time period in the knowledge graph is used as the text to be retrieved. The text to be retrieved is spliced ​​with the preset question template and input into the target large language model to obtain the hot topics in the text to be retrieved.

[0011] In one embodiment, the step of obtaining a plurality of sample label texts includes:

[0012] Acquire multiple initial texts, and use a preset semantic vector model to calculate a vector of each initial text to obtain multiple text vectors; wherein each initial text is not marked with a label;

[0013] Obtain multiple labels, and use a preset semantic vector model to calculate the vector of each label to obtain multiple label vectors;

[0014] Calculate the cosine similarity between the target text vector and the multiple label vectors, and among the multiple labels, add the label corresponding to the label vector with the largest cosine similarity to the initial text corresponding to the target text vector for labeling, so as to obtain multiple sample label texts; wherein the target text vector is any one of the multiple text vectors.

[0015] In one embodiment, the step of calculating the vector of each tag using a preset semantic vector model to obtain multiple tag vectors includes:

[0016] Using T extension templates to expand the labels of the target sub-field to obtain T extension texts; wherein T ≥ 2, the target sub-field is any one of the multiple sub-fields;

[0017] Calculating vectors of the T extended texts using a preset semantic vector model to obtain T extended vectors;

[0018] The mean of the T extended vectors is used as the label vector of the target sub-domain to obtain the multiple label vectors.

[0019] In one embodiment, each sample label text is spliced ​​with a preset question instruction, and the splicing result is input into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions, including:

[0020] Acquire multiple preset question instructions, and respectively concatenate each sample label text with each preset question instruction to form a corresponding instruction seed, so as to obtain an instruction seed pool including multiple instruction seeds;

[0021] In the instruction seed pool, a plurality of preset instruction seeds are randomly sampled and input into a preset generation model to self-guide and generate a plurality of newly added question instructions;

[0022] Calculating the similarity between the target instruction and each question instruction in the instruction seed pool to obtain multiple target similarities; wherein the target instruction is any one of the multiple newly added question instructions;

[0023] If all target similarities are less than a preset similarity threshold, the target instruction is concatenated with each sample label text to form a corresponding instruction seed and added to the instruction seed pool;

[0024] If the current question instruction in the instruction seed pool is less than the preset quantity threshold, return to execute the step and subsequent steps of randomly sampling a preset plurality of instruction seeds in the instruction seed pool and inputting a preset generation model to self-guide the generation of a plurality of new question instructions, until the current question instruction in the instruction seed pool is greater than or equal to the preset quantity threshold, and add all the current question instructions in the instruction seed pool to the instruction database.

[0025] In one embodiment, the calculating the similarity between the target instruction and each question instruction in the instruction seed pool to obtain a plurality of target similarities includes:

[0026] Determine the longest common subsequence between the target instruction and the reference instruction; wherein the reference instruction is any question instruction in the instruction seed pool;

[0027] Calculating the precision rate; wherein the precision rate is the ratio between the length of the longest common subsequence and the total length of the target instruction;

[0028] Calculating a recall rate; wherein the recall rate is the ratio between the length of the longest common subsequence and the total length of the reference instruction;

[0029] The harmonic mean of the precision and the recall is calculated to obtain multiple target similarities.

[0030] In one embodiment, fine-tuning the incremental large language model based on the question instruction in the instruction database to obtain a target large language model includes:

[0031] Determine a target adjustment layer in the incremental large prediction model, introduce a bypass structure in the target adjustment layer, and perform low-rank decomposition on the bypass structure to obtain a first low-rank matrix and a second low-rank matrix;

[0032] The target adjustment layer is frozen, and the first low-rank matrix and the second low-rank matrix are fine-tuned based on the question instruction in the instruction database to obtain the target large language model.

[0033] In one embodiment, the formula for low-rank decomposition and fine-tuning the first low-rank and second low-rank matrices is expressed as:

[0034]

[0035] h=W 0 x+ΔWx=W 0 x+BAx

[0036] In the above formula, W 0 is the initial matrix; ΔW is the bypass structure; A is the first low-rank matrix; B is the second low-rank matrix; k is the input dimension; d is the output dimension; r is the predetermined hyperparameter; h is the forward propagation process.

[0037] A device for discovering hot topics in the electric power field, wherein the electric power field includes a plurality of preset sub-fields, and the device for discovering hot topics in the electric power field includes:

[0038] A text acquisition module, used to acquire a plurality of sample label texts; wherein the label texts are marked with labels, and the labels are used to indicate the sub-field to which the label texts belong;

[0039] A question instruction generation module is used to splice each sample label text with a preset question instruction, and input the splicing result into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions;

[0040] A model adjustment module is used to obtain a corpus in the electric power field, and to perform incremental training on an initial large language model based on the corpus in the electric power field to obtain an incremental large language model; and to fine-tune the incremental large language model based on the question instruction in the instruction database to obtain a target large language model;

[0041] The hot topic discovery module is used to obtain the current label text updated in real time to construct a knowledge graph, use the label text within the latest preset time period in the knowledge graph as the text to be retrieved, splice the text to be retrieved with the preset question template and input it into the target large language model to obtain the hot topics in the text to be retrieved.

[0042] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the above-mentioned method for discovering hot topics in the power field.

[0043] A device for discovering hot topics in the electric power field comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned method for discovering hot topics in the electric power field.

[0044] The present invention provides a method, device, medium and equipment for discovering hot topics in the electric power field. First, the sub-fields of the electric power field are distinguished by labeling text, and then question instructions are generated through self-guidance to build an instruction database. Next, the initial large language model is incrementally trained based on the corpus of the electric power field, and fine-tuned using the instruction database to obtain a target large language model adapted to the electric power field. Finally, the latest label text is retrieved using the knowledge graph and input into the model to generate hot topics. It can be understood that this method, through incremental training and fine-tuning, enables the large language model to have stronger understanding and generation capabilities in the electric power field, thus solving the "hallucination" problem. At the same time, through real-time updated data, labels and knowledge graphs, the system can provide more accurate and timely hot topics to ensure the professionalism and accuracy of the generated content. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] in:

[0047] Figure 1 A flowchart of the method for discovering hot topics in the power sector;

[0048] Figure 2 A flowchart for obtaining multiple sample label documents;

[0049] Figure 3 A schematic diagram of a process for constructing an instruction database including a preset number of question instructions;

[0050] Figure 4 Schematic diagram of the structure of the device for discovering hot topics in the power field;

[0051] Figure 5 Discover the block diagram of the equipment for hot topics in the power sector. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0054] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0055] like Figure 1 As shown, Figure 1 The flowchart of a method for discovering hot topics in the power sector in one embodiment is shown below. The power sector includes multiple preset sub-sectors, such as energy price changes, power supply and demand, power grid planning, international energy cooperation, energy market trends, new power systems, energy policy reports, and other sub-sectors. The steps provided by the method for discovering hot topics in the power sector in this embodiment include:

[0056] S101, obtain multiple sample label texts.

[0057] The tag text is marked with a tag, and the tag is used to indicate the subfield to which the tag text belongs. Of course, it is understandable that a tag text can be marked with at least one tag, and the tag marking method can be manual tagging or automatic identification of the subject of the sample text through natural language processing (NLP) technology and assignment of corresponding tags.

[0058] For example, suppose we extract the following text from an article: "With the increasing contradiction between electricity supply and demand, especially against the backdrop of growing energy consumption, unstable access to renewable energy, and challenges to traditional energy supply, there has been a shortage of supply in the market. In response to this situation, the government has quickly introduced a series of new energy policies aimed at alleviating supply and demand pressures by optimizing the energy structure, improving power generation efficiency, and strengthening power dispatching and energy storage construction. These policies also include regulatory measures for the electricity market, such as implementing differentiated electricity prices, promoting electricity market reforms, and strengthening international energy cooperation to ensure that the market remains balanced and stable under fluctuations in supply and demand. This series of measures will not only help resolve the tension in electricity supply and demand in the short term, but also lay the foundation for long-term energy structure transformation."

[0059] Then after labeling "electricity supply and demand" and "energy policy report", a sample label text is formed.

[0060] S102, concatenating each sample label text with a preset question instruction, and inputting the concatenation result into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions.

[0061] The preset question instructions are a series of questions or query instructions designed in advance to guide the generative model to understand the text and generate further question instructions. These instructions can be questions related to the subfield, such as: "What is the latest trend in this subfield?" or "What is the development forecast for the next year?".

[0062] Combine the sample label text with the preset question instructions into a new text input format, for example:

[0063] Instruction: What are the latest trends in this subfield?

[0064] Input: The content of this article is "With the increasing contradiction between electricity supply and demand, especially against the background of growing energy consumption, unstable access to renewable energy and challenges in traditional energy supply, the market has seen a situation of supply exceeding demand. In response to this situation, the government has quickly introduced a series of new energy policies aimed at alleviating supply and demand pressures by optimizing the energy structure, improving power generation efficiency, strengthening power dispatching and energy storage construction. These policies also include regulatory measures for the electricity market, such as promoting differentiated electricity prices, promoting electricity market reforms and strengthening international energy cooperation to ensure that the market remains balanced and stable under fluctuations in supply and demand. This series of measures will not only help resolve the tension in electricity supply and demand in the short term, but also lay the foundation for long-term energy structure transformation."

[0065] The splicing results are then input into the preset generative model, and new questions are generated through the generative model itself. This generation is based on existing inputs and rules and is a form of self-supervised learning. By generating new questions, the model can continuously expand the instruction database, optimize the ability to discover hot topics, and the questions are closer to actual needs.

[0066] S103, obtaining a corpus in the electric power field, and performing incremental training on the initial large language model based on the corpus in the electric power field to obtain an incremental large language model.

[0067] The power field corpus refers to all relevant text data sets collected in the power field, including energy policies, regulations, technical documents, equipment maintenance records, corporate trends, etc. It provides a source of knowledge and information in a specific field for training large language models.

[0068] Next, the initial large language model LLaMA model is used as the basis, and the model is incrementally trained with the collected power field corpus. Incremental training will update the weight of the model, but will not completely cover the original knowledge of the model. Through training with new data, the incremental large language model can adapt to the specific needs of the power industry based on the original general language capabilities without losing understanding of other fields.

[0069] S104, fine-tuning the incremental large language model based on the question instructions in the instruction database to obtain a target large language model.

[0070] During the fine-tuning process, the question instructions in the instruction database are used as input, these questions are input into the incremental large language model, and the corresponding domain text (as the answer) is used to train the model. Through repeated training, the model gradually learns how to accurately respond to these question instructions. The purpose of fine-tuning is to enable the model to not only have knowledge in the power field, but also to generate high-quality, highly task-related responses based on specific questions.

[0071] After fine-tuning, the model was further optimized into a "target large language model" that can not only understand complex texts in the power sector, but also give accurate and professional answers to specific questions.

[0072] S105, obtain the current label text updated in real time to construct a knowledge graph, use the label text within the latest preset time period in the knowledge graph as the text to be retrieved, splice the text to be retrieved with the preset question template and input it into the target large language model to obtain the hot topics in the text to be retrieved.

[0073] Specifically, we can first continuously obtain the latest data in the power sector through web crawlers, APIs, or manual input. These data can come from industry news, policy announcements, technical papers, etc. Then, we can label all the latest data with specific classification tags to obtain the current label text.

[0074] The collected data is then stored in the form of quads according to its label, text content, and release time. Each text is treated as an entity and associated with the sub-domain label to which it belongs. For example:

[0075] <“Title of Article A”, belongs to, “Energy Policy”, time: September 2024>.

[0076] <“Title of Article B”, belongs to, “Electricity supply and demand”, time: September 2024>.

[0077] Furthermore, the system can set the query time period as required, for example, all texts belonging to "grid planning" or "power supply and demand" in the past 30 days as texts to be retrieved.

[0078] Then combine the question template and the text to be searched into a standard input and input it into the target large language model. For example: "Task: There are a series of articles below, all in the format of <article title@@article body@@article category>, all of which are articles published in the power field within the past 30 days. Please analyze these articles and summarize 20 hot topics, and sort them according to their frequency of appearance in the articles.

[0079] Input: <[Title of Article 1]@@[Body of Article 1]@@[Category of Article 1]><[Title of Article 2]@@[Body of Article 2]@@[Category of Article 2]>…”

[0080] Finally, after receiving the concatenated input, the target large language model analyzes the content and generates hot topics that meet the requirements. These hot topics can be technology advances, policy changes, or market trends that are frequently discussed over a period of time.

[0081] The above-mentioned method for discovering hot topics in the power sector first distinguishes the sub-fields of the power sector by labeling the text, and then generates question instructions through self-guidance to build an instruction database. Next, the initial large language model is incrementally trained based on the corpus in the power sector, and fine-tuned using the instruction database to obtain a target large language model adapted to the power sector. Finally, the knowledge graph is used to retrieve the latest labeled text and input it into the model to generate hot topics. It can be understood that this method, through incremental training and fine-tuning, enables the large language model to have stronger understanding and generation capabilities in the power sector, thus solving the "hallucination" problem. At the same time, through real-time updated data, labels and knowledge graphs, the system can provide more accurate and timely hot topics to ensure the professionalism and accuracy of the generated content.

[0082] In a specific embodiment, Figure 2 As shown, S101 obtains multiple sample label texts and specifically includes the following steps:

[0083] S1011, obtaining a plurality of initial texts, and using a preset semantic vector model to calculate a vector of each initial text to obtain a plurality of text vectors.

[0084] The initial text refers to the unprocessed original text data obtained from the Internet and other channels. These texts include information such as policies, regulations, technical reports, market trends, etc. of the power industry. These initial texts are not marked with tags and need to be marked with tags through this specific embodiment.

[0085] For example, in order to collect data in the power sector, we first crawled 1.6 million Internet public documents from more than 500 websites and more than 1,000 URL addresses. These public documents provide rich information such as policies, regulations, technological developments, and market trends in the power industry; in addition, we collected the latest policy dynamics, technological advances, and market trends in real time through the WeChat public accounts of multiple power industry experts and institutions; finally, we collected relevant news, policy documents, and announcements from authoritative public websites to ensure the comprehensiveness and accuracy of field data. Then, we cleaned the data by removing HTML tags, extracting key details, and standardizing the date format, and finally retained the title, body, and release date of each text data. The field text data within nearly 30 days was used as the initial text for subsequent analysis and work.

[0086] Then, the vector of each initial text is calculated using a preset semantic vector model. Optionally, the semantic vector model in this specific embodiment is an Unsupervised SimCSE model. The Unsupervised SimCSE model encodes the initial text through its encoder (such as a pre-trained language model such as BERT). During the encoding process, the model extracts the semantic features of the initial text and compresses the text's vocabulary, sentence structure and other information into a vector of fixed dimension. SimCSE maps different forms of the same text (such as randomly discarding some words) to similar vector spaces through unsupervised contrastive learning. In this way, the semantic similarity of the text can be reflected by the distance between the vectors, and the generated semantic vector is used for subsequent similarity calculation or classification tasks. At the same time, through the method of contrastive learning, the model also compares different perturbation versions of the same text (such as randomly discarding some words), optimizes its semantic representation, and makes similar text vectors closer and dissimilar text vectors farther away. Ultimately, the trained model can convert any input text into a high-quality semantic vector for various downstream tasks.

[0087] S1012, obtaining multiple tags, and using a preset semantic vector model to calculate the vector of each tag to obtain multiple tag vectors.

[0088] In this step, multiple labels (such as "electricity supply and demand", "energy policy", etc.) are first obtained. These labels are used to indicate the subfield to which the text belongs. Then, each label is processed using the preset Unsupervised SimCSE model to convert each label into a semantic vector. This semantic vector is the result of encoding the semantic information of the label and can represent the meaning of the label in the vector space. In this way, multiple labels will generate a set of corresponding label vectors for comparison with the text vector to determine the most relevant label for the text.

[0089] S1013, calculating the cosine similarity between the target text vector and multiple label vectors, and adding the label corresponding to the label vector with the largest cosine similarity among multiple labels to the initial text corresponding to the target text vector for labeling, so as to obtain multiple sample label texts.

[0090] The target text vector is any one of the multiple text vectors, that is, the same operation is performed on all text vectors. Here, cosine similarity is used to calculate the similarity between the target text vector and each label vector. The value is between -1 and 1. The larger the value, the more similar the text and the label are. We take the label corresponding to the maximum value of cosine similarity as the label of the text and add it to the text to form a "sample label text".

[0091] This specific embodiment can accurately determine the matching degree between text and tags through semantic vector and cosine similarity calculation, so as to mark the text with the most relevant tags.

[0092] In a specific embodiment, calculating the vector of each tag using a preset semantic vector model in S1012 to obtain multiple tag vectors includes the following steps:

[0093] A1. Use T extension templates to expand the labels of the target sub-domain to obtain T extension texts.

[0094] Since labels are relatively short words, they are not completely suitable for directly calculating semantic vectors. Therefore, an extension template is defined to expand the set labels. Among them, T≥2, the extension template is a preset sentence pattern used to generate extended text, embedding short labels into complete sentences to help the semantic vector model better understand the semantics of labels. For example, the label "energy policy" is embedded in the sentence "This is an article in the field of energy policy" as extended text.

[0095] The power domain may include multiple sub-domains (such as "power supply and demand", "energy policy", "grid planning", etc.), and each label belongs to a specific sub-domain. The target sub-domain is any one of these sub-domains, that is, the specific sub-domain of the current task processing.

[0096] A2. Calculate vectors of T extended texts using a preset semantic vector model to obtain T extended vectors.

[0097] A3. The mean of the T extended vectors is used as the label vector of the target sub-domain to obtain multiple label vectors.

[0098] For example, let T = 2, define two label-prompts to expand the set label, and the label-prompts are:

[0099] prompt1="This is an article about {[label]} in {[field]}"

[0100] prompt2=”Category: {[label]}”

[0101] Then the semantic vectors of prompt1 and prompt2 are averaged as the label vector. The formula is as follows:

[0102]

[0103] Among them, fθ(p i (c)) indicates the label p i (c) Expand it.

[0104] In the above specific embodiment, by using multiple extension templates to extend the tag, the short tag can obtain more accurate and rich semantic expressions in different contexts, which is crucial for the subsequent semantic vector calculation and helps to improve the accuracy of tag matching.

[0105] In a specific embodiment, Figure 3 As shown, S102 concatenates each sample label text with a preset question instruction, and inputs the concatenation result into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions, which specifically includes the following steps:

[0106] S1021, obtaining a plurality of preset question instructions, respectively splicing each sample label text with each preset question instruction to form a corresponding instruction seed, so as to obtain an instruction seed pool including a plurality of instruction seeds.

[0107] The preset question instructions are manually written instructions to guide the model to generate the corresponding output. They may be different types of task instructions, including generation, classification, dialogue, etc. For example, the instructions may ask for a brief description of the article content or to select appropriate tags for the article.

[0108] We can manually write multiple question instructions, and then splice the sample label text with the preset question instructions to obtain an instruction seed pool including multiple instruction seeds, for example:

[0109] Example 1

[0110] “Below is a description of the Power Domain Task Order and Inputs. Please provide the appropriate responses to satisfy the request.

[0111] ##Instruction: Please briefly describe the main content of this article in no more than 20 words

[0112] ##Input: The content of this article is <...First text body...>

[0113] ##Output: <First text title>"

[0114] Example 2

[0115] “Below is a description of the Power Domain Task Order and Inputs. Please provide the appropriate responses to satisfy the request.

[0116] ##Instruction: Please select the most appropriate tag from the seven provided to classify the following article: Energy price changes, power supply and demand, power grid planning, international energy cooperation, energy market trends, new power systems, energy policy reports

[0117] ##Input: The content of this article is <...First text body...>

[0118] ##Output: <predicted label of the first text>"

[0119] S1022: Randomly sample a plurality of preset instruction seeds in the instruction seed pool and input them into a preset generation model to self-guide and generate a plurality of newly added question instructions.

[0120] For example, in the instruction seed pool, 8 task instructions are sampled from the instruction seed pool as samples for input into ChatGPT3.0 in each iteration, and then ChatGPT3.0's few-shot is used to judge the new instructions. If it is not a classification task, the input-first approach is used to generate instances, which means that when generating new instructions, the model will pay more attention to and retain the original input content, ensuring that the input part of the new instructions is richer and more accurate. For example:

[0121] Instructions: "Please briefly describe the main content of this article."

[0122] After inputting the priority generation example, a similar instruction may be generated: "Please summarize the core idea of ​​the article in two sentences."

[0123] If it is a classification task, the output-first approach is used to generate instances, which means that when generating new instructions, the diversity and accuracy of the results are given priority to ensure that the new instructions can effectively guide the model to select the correct category. For example:

[0124] Instructions: “Please select the most appropriate one from the following topic tags: electricity supply and demand, energy policy, market trends.”

[0125] After the output-first generation instance, a similar instruction may be generated: "Please select the most relevant tag based on the content of the article."

[0126] S1023, calculating the similarity between the target instruction and each question instruction in the instruction seed pool to obtain multiple target similarities.

[0127] The target instruction is any one of the multiple newly added question instructions, that is, the same operation is performed on all the newly added question instructions. Similarity can be calculated using common methods in natural language processing, such as cosine similarity, similarity of word embedding vectors, etc. The size of the similarity determines the degree of semantic proximity between two instructions.

[0128] S1024, determine whether all target similarities are less than a preset similarity threshold. If all target similarities are less than the preset similarity threshold, execute S1025, concatenate the target instruction with each sample label text to form a corresponding instruction seed and add it to the instruction seed pool.

[0129] This step is to screen the similarity. If the similarity of any instruction is greater than the threshold, it means that this new instruction is too similar to the existing instruction and may be regarded as a redundant instruction. If the similarity between the target instruction and all existing instructions in the seed pool is lower than the set similarity threshold, it means that the target instruction is unique enough and will not be repeated with existing instructions. It has added value. Execute S1025, and these newly generated instruction seeds will be added to the existing instruction seed pool as the instruction seed source for subsequent tasks.

[0130] S1026, determine whether the current question instruction in the instruction seed pool is less than the preset number threshold. If the current question instruction in the instruction seed pool is less than the preset number threshold, return to execute S1022 and subsequent steps until the current question instruction in the instruction seed pool is greater than or equal to the preset number threshold, execute S1027, and add all current question instructions in the instruction seed pool to the instruction database.

[0131] This step checks the current number of question instructions in the instruction seed pool. If the number is insufficient, that is, lower than the preset number threshold, it returns to continue generating new instructions. When the number of instructions in the instruction seed pool reaches the threshold, all the question instructions in these instruction seed pools are finally collected and added to the instruction database for use in subsequent tasks.

[0132] The above specific embodiment generates new instructions, evaluates their similarity with existing instructions, and screens out sufficiently diverse and non-redundant instructions, thereby enriching the instruction seed pool and ultimately generating an instruction database for training a large language model in the power field.

[0133] In a specific embodiment, S1023 calculates the similarity between the target instruction and each question instruction in the instruction seed pool to obtain multiple target similarities, and specifically includes the following steps:

[0134] B1. Determine the longest common subsequence between the target instruction and the reference instruction.

[0135] Among them, the reference instruction is any question instruction in the instruction seed pool, that is, the same operation is performed on all question instructions. The longest common subsequence refers to the longest character sequence shared by the target instruction and the reference instruction, which keeps the relative order unchanged. For example, for the instructions "Please briefly describe the content of the article" and "Please describe the content of the article", their longest common subsequence is "Please describe the content of the article". The length of the longest common subsequence can be used to measure the similarity between two instructions. The longer the longest common subsequence, the closer the two instructions are in content.

[0136] B2. Calculate the accuracy.

[0137] The precision is the ratio of the length of the longest common subsequence to the total length of the target instruction. This indicator is used to measure how much of the target instruction is similar to the reference instruction. A high precision means that most of the content of the target instruction appears in the reference instruction.

[0138] B3. Calculate the recall rate.

[0139] The recall rate is the ratio of the length of the longest common subsequence to the total length of the reference instruction. This indicator is used to measure how much of the reference instruction is similar to the target instruction. A high recall rate means that most of the content in the reference instruction appears in the target instruction.

[0140] B4. Calculate the harmonic mean of precision and recall to obtain the similarity of multiple targets.

[0141] The formula is as follows:

[0142]

[0143] This value F1 is one of multiple target similarities, which is used to comprehensively measure the similarity between the target instruction and the reference instruction. If the precision and recall of two instructions are high, their F1 values ​​will also be high, indicating that they are very similar.

[0144] In a specific embodiment, S104 fine-tunes the incremental large language model based on the question instruction in the instruction database to obtain the target large language model, which specifically includes the following steps:

[0145] S1041. Determine a target adjustment layer in the incremental large prediction model, introduce a bypass structure into the target adjustment layer, and perform low-rank decomposition on the bypass structure to obtain a first low-rank matrix and a second low-rank matrix.

[0146] Among them, target adjustment layers refer to certain layers selected in the large language model, usually specific network layers within the model, used to optimize or improve the model's capabilities. These layers are selected for more effective adjustments without retraining the entire model.

[0147] The bypass structure is an additional network structure that is independent of the main path of the original model and can process data in parallel. The introduction of this structure allows the model to perform additional training while maintaining its original capabilities, thereby enhancing the performance of the model. The weight matrix of the bypass structure is then low-rank decomposed into two low-rank matrices. In this process, the complex weight matrix is ​​decomposed into two smaller matrices, which can reduce the amount of calculation and improve the efficiency of model training.

[0148] S1042. Freeze the target adjustment layer, and fine-tune the first low-rank matrix and the second low-rank matrix based on the question instruction in the instruction database to obtain a target large language model.

[0149] The weights of the target adjustment layer are then frozen, and only the low-rank matrix in the introduced bypass structure is fine-tuned by asking the instruction database. This can maintain the original knowledge of the model while improving and optimizing the bypass structure, thereby improving the efficiency of fine-tuning.

[0150] In a specific embodiment, the formula for low-rank decomposition and fine-tuning the first low-rank matrix and the second low-rank matrix is ​​expressed as:

[0151]

[0152] h=W 0 x+ΔWx=W 0 x+BAx

[0153] In the above formula, W 0 is the initial matrix; ΔW is the bypass structure; A is the first low-rank matrix; B is the second low-rank matrix; k is the input dimension; d is the output dimension; r is the predetermined hyperparameter; h is the forward propagation process.

[0154] In one embodiment, Figure 4 As shown, a device for discovering hot topics in the power field is proposed, and the device includes:

[0155] The text acquisition module 401 is used to acquire a plurality of sample label texts; wherein the label texts are marked with labels, and the labels are used to indicate the sub-field to which the label texts belong;

[0156] The question instruction generation module 402 is used to splice each sample label text with a preset question instruction, and input the splicing result into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions;

[0157] The model adjustment module 403 is used to obtain a corpus in the electric power field, perform incremental training on the initial large language model based on the corpus in the electric power field to obtain an incremental large language model; and fine-tune the incremental large language model based on the question instruction in the instruction database to obtain a target large language model;

[0158] The hot topic discovery module 404 is used to obtain the current label text updated in real time to construct a knowledge graph, use the label text within the latest preset time period in the knowledge graph as the text to be retrieved, splice the text to be retrieved with the preset question template and input it into the target large language model to obtain the hot topics in the text to be retrieved.

[0159] Figure 5 FIG. 1 shows an internal structure diagram of a hot topic discovery device in the power field in one embodiment. Figure 5 As shown, the hot topic discovery device in the power field includes a processor, a memory, and a network interface connected via a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the hot topic discovery device in the power field stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the hot topic discovery method in the power field. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the hot topic discovery method in the power field. Those skilled in the art can understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the hot topic discovery device in the power field to which the present application scheme is applied. The specific hot topic discovery device in the power field may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0160] A computer-readable storage medium stores a computer program, which implements the following steps when executed by a processor: obtaining a plurality of sample label texts; wherein the label texts are marked with labels, and the labels are used to indicate the subfield to which the label texts belong; splicing each sample label text with a preset question instruction, and inputting the splicing result into a preset generation model to self-guide the generation of newly added question instructions, so as to obtain an instruction database including a preset number of question instructions; obtaining a power field corpus, and incrementally training an initial large language model based on the power field corpus to obtain an incremental large language model; fine-tuning the incremental large language model based on the question instructions in the instruction database to obtain a target large language model; obtaining a current label text updated in real time to construct a knowledge graph, using the label text within the latest preset time period in the knowledge graph as a text to be retrieved, splicing the text to be retrieved with a preset question template and inputting the result into the target large language model, so as to obtain hot topics in the text to be retrieved.

[0161] A device for discovering hot topics in the electric power field comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: obtaining a plurality of sample label texts; wherein the label texts are marked with labels, and the labels are used to indicate the subfield to which the label texts belong; splicing each sample label text with a preset question instruction, and inputting the splicing result into a preset generation model to self-guide the generation of newly added question instructions, so as to obtain an instruction database including a preset number of question instructions; obtaining an electric power field corpus, and incrementally training an initial large language model based on the electric power field corpus to obtain an incremental large language model; fine-tuning the incremental large language model based on the question instructions in the instruction database to obtain a target large language model; obtaining a current label text updated in real time to construct a knowledge graph, taking the label text within the latest preset time period in the knowledge graph as a text to be retrieved, splicing the text to be retrieved with a preset question template and inputting the result into the target large language model, so as to obtain hot topics in the text to be retrieved.

[0162] It should be noted that the above-mentioned method, device, equipment and computer-readable storage medium for discovering hot topics in the power field belong to a general inventive concept, and the contents of the embodiments of the method, device, equipment and computer-readable storage medium for discovering hot topics in the power field are applicable to each other.

[0163] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0164] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for discovering hot topics in the power field, characterized in that: The electric power field includes a plurality of preset sub-fields, and the method includes: Acquire a plurality of sample label texts; wherein the label texts are marked with labels, and the labels are used to indicate the subfield to which the label texts belong; Each sample label text is spliced ​​with a preset question instruction, and the splicing result is input into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions; Acquire a corpus in the electric power field, and perform incremental training on an initial large language model based on the corpus in the electric power field to obtain an incremental large language model; Fine-tuning the incremental large language model based on the question instructions in the instruction database to obtain a target large language model; The current label text updated in real time is obtained to construct a knowledge graph, and the label text within the latest preset time period in the knowledge graph is used as the text to be retrieved. The text to be retrieved is spliced ​​with the preset question template and input into the target large language model to obtain the hot topics in the text to be retrieved.

2. The method according to claim 1, characterized in that The obtaining of multiple sample label texts includes: Acquire multiple initial texts, and use a preset semantic vector model to calculate a vector of each initial text to obtain multiple text vectors; wherein each initial text is not marked with a label; Obtain multiple labels, and use a preset semantic vector model to calculate the vector of each label to obtain multiple label vectors; Calculate the cosine similarity between the target text vector and the multiple label vectors, and among the multiple labels, add the label corresponding to the label vector with the largest cosine similarity to the initial text corresponding to the target text vector for labeling, so as to obtain multiple sample label texts; wherein the target text vector is any one of the multiple text vectors.

3. The method according to claim 2, characterized in that The method of using a preset semantic vector model to calculate the vector of each label to obtain multiple label vectors includes: Using T extension templates to expand the labels of the target sub-field to obtain T extension texts; wherein T ≥ 2, the target sub-field is any one of the multiple sub-fields; Calculating vectors of the T extended texts using a preset semantic vector model to obtain T extended vectors; The mean of the T extended vectors is used as the label vector of the target sub-domain to obtain the multiple label vectors.

4. The method according to claim 1, characterized in that The method of splicing each sample label text with a preset question instruction and inputting the splicing result into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions, includes: Acquire multiple preset question instructions, and respectively concatenate each sample label text with each preset question instruction to form a corresponding instruction seed, so as to obtain an instruction seed pool including multiple instruction seeds; In the instruction seed pool, a plurality of preset instruction seeds are randomly sampled and input into a preset generation model to self-guide and generate a plurality of newly added question instructions; Calculating the similarity between the target instruction and each question instruction in the instruction seed pool to obtain multiple target similarities; wherein the target instruction is any one of the multiple newly added question instructions; If all target similarities are less than a preset similarity threshold, the target instruction is concatenated with each sample label text to form a corresponding instruction seed and added to the instruction seed pool; If the current question instruction in the instruction seed pool is less than the preset quantity threshold, return to execute the step and subsequent steps of randomly sampling a preset plurality of instruction seeds in the instruction seed pool and inputting a preset generation model to self-guide the generation of a plurality of new question instructions, until the current question instruction in the instruction seed pool is greater than or equal to the preset quantity threshold, and add all the current question instructions in the instruction seed pool to the instruction database.

5. The method according to claim 4, characterized in that The calculating the similarity between the target instruction and each question instruction in the instruction seed pool to obtain a plurality of target similarities includes: Determine the longest common subsequence between the target instruction and the reference instruction; wherein the reference instruction is any question instruction in the instruction seed pool; Calculating the precision rate; wherein the precision rate is the ratio between the length of the longest common subsequence and the total length of the target instruction; Calculating a recall rate; wherein the recall rate is the ratio between the length of the longest common subsequence and the total length of the reference instruction; The harmonic mean of the precision and the recall is calculated to obtain multiple target similarities.

6. The method according to claim 1, characterized in that The fine-tuning of the incremental large language model based on the question instruction in the instruction database to obtain a target large language model includes: Determine a target adjustment layer in the incremental large prediction model, introduce a bypass structure in the target adjustment layer, and perform low-rank decomposition on the bypass structure to obtain a first low-rank matrix and a second low-rank matrix; The target adjustment layer is frozen, and the first low-rank matrix and the second low-rank matrix are fine-tuned based on the question instruction in the instruction database to obtain the target large language model.

7. The method according to claim 1, characterized in that The formula for low-rank decomposition and fine-tuning the first low-rank matrix and the second low-rank matrix is ​​expressed as: h=W0x+ΔWx=W0x+BAx In the above formula, W 0 is the initial matrix; ΔW is the bypass structure; A is the first low-rank matrix; B is the second low-rank matrix; k is the input dimension; d is the output dimension; r is the predetermined hyperparameter; h is the forward propagation process.

8. A device for discovering hot topics in the electric power field, characterized in that: The power field includes a plurality of preset sub-fields, and the hot topic discovery device in the power field includes: A text acquisition module, used to acquire a plurality of sample label texts; wherein the label texts are marked with labels, and the labels are used to indicate the sub-field to which the label texts belong; A question instruction generation module is used to splice each sample label text with a preset question instruction, and input the splicing result into a preset generation model to self-guide the generation of new question instructions, so as to obtain an instruction database including a preset number of question instructions; A model adjustment module is used to obtain a corpus in the electric power field, and to perform incremental training on an initial large language model based on the corpus in the electric power field to obtain an incremental large language model; and to fine-tune the incremental large language model based on the question instruction in the instruction database to obtain a target large language model; The hot topic discovery module is used to obtain the current label text updated in real time to construct a knowledge graph, use the label text within the latest preset time period in the knowledge graph as the text to be retrieved, splice the text to be retrieved with the preset question template and input it into the target large language model to obtain the hot topics in the text to be retrieved.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.

10. A device for discovering hot topics in the power field, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

Images