Power grid field keyword extraction method, training method, medium and equipment

The training method enhances keyword extraction in the power grid field by using a domain-specific dictionary and neural networks, addressing accuracy issues and improving data processing efficiency.

CN119089896APending Publication Date: 2024-12-06YUNNAN POWER GRID CO LTD ELECTRIC POWER RES INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411127633.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Current keyword extraction methods struggle to accurately extract power grid field characteristics and identify professional vocabulary, leading to inefficiencies in data processing and threat identification.

Method used

A training method involving a power grid domain dictionary, pseudo data set construction, pre-trained language model adjustments, and secondary adjustments using bidirectional long short-term memory networks and convolutional neural networks to enhance keyword extraction accuracy.

Benefits of technology

Provides an efficient and accurate keyword extraction solution for the power grid field, improving data processing and threat identification capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089896A_ABST
    Figure CN119089896A_ABST
Patent Text Reader

Abstract

The invention discloses a power grid field keyword extraction method, a training method, a medium and equipment, and the method comprises the steps: firstly obtaining a power grid field dictionary and constructing a power grid field pseudo data set in a training stage, inputting the pseudo data set into a pre-training language model, and carrying out the preliminary adjustment, thereby obtaining an adjusted power grid field pre-training model. And then obtaining a first text data set of the power grid field, preprocessing the first text data set to obtain a target text data set, and finally inputting the target text data set into the adjusted pre-training model for secondary adjustment to obtain a final power grid field keyword extraction model. In the application stage, a to-be-extracted data set is input into the trained power grid field keyword extraction model, a target label sequence is obtained, power grid field keywords are extracted from the to-be-extracted data set based on the target label sequence, and a keyword list is obtained. The invention provides an efficient and accurate power grid field keyword extraction solution.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field, and in particular to a keyword extraction method, training method, medium and equipment in the field of power grid. Background Art

[0002] With the deep integration of industrialization and informatization, as well as the rapid development of information technologies such as intelligence, the Internet of Things, and big data, information and data in the power grid field are growing exponentially. Against this background, keyword extraction in the power grid field has become crucial. Keyword extraction technology can help extract key information from a large amount of power grid data for monitoring, management, and optimization of power grid operations, and can also identify data leaks and potential threats on the Internet.

[0003] However, the current common keyword extraction methods have shown poor results when facing the specific needs of the power grid field. It is difficult to accurately extract the characteristics of the power grid field and identify professional vocabulary. Summary of the invention

[0004] Based on this, it is necessary to provide keyword extraction methods, training methods, media and equipment in the power grid field to solve the problem of difficulty in accurately extracting power grid field characteristics and identifying professional vocabulary.

[0005] A training method for a keyword extraction model in the field of power grid, the method comprising:

[0006] Acquire a power grid domain dictionary, and construct a power grid domain pseudo data set based on the power grid domain dictionary;

[0007] Inputting the power grid field pseudo data set into the pre-trained language model for preliminary adjustment to obtain an adjusted power grid field pre-trained model;

[0008] Acquire a first text data set in the field of power grid, and preprocess the first text data set to obtain a target text data set;

[0009] The target text data set is input into the pre-trained model in the power grid field for secondary adjustment to obtain an adjusted power grid field keyword extraction model.

[0010] In one embodiment, constructing a pseudo data set in the power grid domain based on the power grid domain dictionary includes:

[0011] Randomly extracting k preset power grid domain terms from the power grid domain dictionary to construct pseudo sentence level text;

[0012] All power grid domain keywords in the pseudo sentence level text are marked, the first character of the power grid domain keyword is represented as the first character, and the non-first character of the power grid domain keyword is represented as the second character, so as to obtain the power grid domain pseudo data set.

[0013] In one embodiment, the step of inputting the power grid domain pseudo data set into a pre-trained language model for preliminary adjustment to obtain an adjusted power grid domain pre-trained model includes:

[0014] Inputting the power grid field pseudo data set into the hidden layer of the pre-trained language model for feature extraction to obtain a first text feature;

[0015] Mapping the first text feature to a label domain using a linear function to obtain a first text label prediction value; wherein the label domain is composed of K power grid domain terms;

[0016] Inputting the first text label prediction value into the conditional random field layer of the pre-trained language model to model the label transition probability in the text label prediction value and calculate the label score based on the label transition probability, and outputting the label sequence based on the maximum score principle;

[0017] The cross entropy loss function is used to calculate the gradient and update the model based on the gap between the output label sequence and the corresponding true label to obtain the adjusted power grid field pre-training model.

[0018] In one embodiment, the preprocessing of the first text data set to obtain a target text data set includes:

[0019] Removing noise from the first text data set and segmenting the text data set according to a preset text length to obtain a second text data set;

[0020] All power grid domain keywords in the second text data set are marked, the first character of the power grid domain keyword is represented in the form of the fourth character, the non-first character of the power grid domain keyword is represented in the form of the fifth character, and the characters in the second text data set except the power grid domain keyword are represented in the form of the sixth character to obtain the target text data set.

[0021] In one embodiment, inputting the target text data set into a pre-trained model in the power grid field for secondary adjustment to obtain an adjusted power grid field keyword extraction model includes:

[0022] Inputting the target text data set into the hidden layer of the power grid domain pre-training model for feature extraction to obtain a second text feature;

[0023] Input the second text feature into a bidirectional long short-term memory network of a pre-trained model in the power grid field to extract global features, and input the second text feature into a convolutional neural network of a pre-trained model in the power grid field to extract local features, and fuse the local features of the global feature set to obtain a fused feature;

[0024] Mapping the fused features into a label domain using a linear function to obtain a second text label prediction value;

[0025] Inputting the second text label prediction value into the conditional random field layer of the pre-trained language model to model the label transition probability in the text label prediction value and calculate the label score based on the label transition probability, and outputting the predicted label sequence based on the maximum score principle;

[0026] The cross entropy loss function is used to calculate the gradient and update the model based on the gap between the output predicted label sequence and the corresponding true label to obtain the adjusted keyword extraction model in the power grid field.

[0027] A method for extracting keywords in the field of power grid, the method comprising:

[0028] Obtain a data set to be extracted, input the data set to be extracted into a keyword extraction model in the power grid field, and obtain an output target label sequence; wherein the keyword extraction model in the power grid field is trained by the training method of the above-mentioned keyword extraction model in the power grid field.

[0029] The power grid field keywords in the to-be-extracted data set are extracted based on the target tag sequence to obtain a power grid field keyword list.

[0030] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of a keyword extraction method in the field of power grid.

[0031] A terminal device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a keyword extraction method in the field of power grid.

[0032] The present invention provides a keyword extraction method, training method, medium and equipment in the field of power grid. In the training stage, a power grid field dictionary is first obtained and a pseudo data set in the field of power grid is constructed. Then, the pseudo data set is input into a pre-trained language model, and a preliminary adjustment is performed to obtain an adjusted pre-trained model in the field of power grid. Then, a first text data set in the field of power grid is obtained, and it is pre-processed to obtain a target text data set. Finally, the target text data set is input into the adjusted pre-trained model, and a secondary adjustment is performed to obtain a final keyword extraction model in the field of power grid. In the application stage, the data set to be extracted is input into a trained keyword extraction model in the field of power grid, a target label sequence is obtained, and keywords in the field of power grid are extracted from the data set to be extracted based on the target label sequence to obtain a keyword list. The present invention overcomes the problem that the general keyword extraction method is not effective when applied in the field of power grid, and provides an efficient and accurate keyword extraction solution in the field of power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] in:

[0035] Figure 1 A flowchart of a training method for a keyword extraction model in the power grid field;

[0036] Figure 2 This is a flowchart of the keyword extraction method in the power grid field;

[0037] Figure 3 This is a structural block diagram of the terminal equipment. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0040] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0041] like Figure 1 As shown, Figure 1The figure is a flow chart of a method for training a keyword extraction model in the power grid field in one embodiment. The steps provided by the method for training a keyword extraction model in the power grid field in this embodiment include:

[0042] S101, obtaining a power grid domain dictionary, and constructing a power grid domain pseudo data set based on the power grid domain dictionary.

[0043] Among them, the power grid domain dictionary is a vocabulary set specially built for the power grid field, which contains commonly used terms, keywords and professional vocabulary in this field. These words may include power grid equipment names, technical terms, operation methods, fault types, etc. Exemplarily, the power grid domain dictionary includes transformer, distribution network, voltage fluctuation, and power quality. The constructed power grid domain pseudo data set is synthetic data generated for training models, which simulates some characteristics and patterns in real data in the power grid field.

[0044] In a specific embodiment, the step of constructing a pseudo data set in the power grid domain based on the power grid domain dictionary includes:

[0045] (1) Randomly extract k preset power grid domain terms from the power grid domain dictionary to construct pseudo-sentence level text.

[0046] The k here can be set based on demand, and k ≥ 3 is generally required.

[0047] For example, one of the pseudo-sentence-level texts is represented as: “The voltage fluctuation of the transformer will affect the power quality.”

[0048] (2) All the power grid keywords in the pseudo-sentence-level text are annotated, the first character of the power grid keyword is represented as the first character, and the non-first character of the power grid keyword is represented as the second character, so as to obtain a pseudo data set of the power grid domain.

[0049] Optionally, the first character of the power grid domain keyword is marked as “B” (Beginning) in the form of the first character, and the non-first character of the power grid domain keyword is marked as “I” (Inside) in the form of the second character.

[0050] Then the annotated pseudo-sentence in the previous example is expressed as: "The B-voltage I-fluctuation of the B-transformer I-will affect the B-electrical I-energy I-quality I-."

[0051] By repeatedly labeling multiple pseudo-sentence-level texts, the required pseudo dataset in the power grid field can be obtained.

[0052] S102, inputting a pseudo data set in the power grid field into a pre-trained language model for preliminary adjustment to obtain an adjusted pre-trained model in the power grid field.

[0053] It is understandable that the fine-tuned pre-trained model contains the preliminary learned knowledge and features of the power grid domain. The adjustment at this stage lays the foundation for further fine-tuning based on real power grid data.

[0054] In a specific embodiment, the step of inputting a pseudo data set in the power grid field into a pre-trained language model for preliminary adjustment includes:

[0055] (1) Input the pseudo data set in the power grid field into the hidden layer of the pre-trained language model for feature extraction to obtain the first text feature.

[0056] Specifically, the constructed pseudo dataset in the power grid field is input into the hidden layer of the pre-trained language model. The hidden layer processes the input data and extracts the feature vector of each word. These feature vectors represent the semantic information of the input text. The process is expressed as:

[0057] h s =BERT′(X s )

[0058] X s ={x s1 , x s2 , …, x sn}

[0059]

[0060] Among them, X s represents a pseudo dataset in the power grid field, n is the length of the text, h s represents the first text feature, and d represents the word embedding dimension.

[0061] (2) Use a linear function to map the first text feature to the label domain to obtain a first text label prediction value.

[0062] The purpose of the linear function here is to map text features to a higher-dimensional label domain, and the label domain consists of K power grid domain terms. The label prediction value obtained after the linear transformation indicates the possibility of a certain word in the text belonging to which label.

[0063] The process is expressed as:

[0064] p s =Linear s (h s )

[0065]

[0066] Among them, p s represents the first text label prediction value, p sn It represents the tag prediction value of the nth keyword, and t is the number of tag types.

[0067] (3) Inputting the first text label prediction value into the conditional random field layer of the pre-trained language model to model the label transition probability in the text label prediction value and calculate the label score based on the label transition probability, and outputting the label sequence based on the maximum score principle.

[0068] Among them, the conditional random field (CRF) is a probabilistic graphical model for sequence labeling tasks, which can consider the dependencies between labels in the sequence, thereby improving the accuracy of labeling. In the keyword extraction task, the CRF layer is used to process the label prediction value, model the transition probability between labels, and output the optimal label sequence.

[0069] The process of calculating label scores is expressed as:

[0070]

[0071] in, Represents the label transition probability between two adjacent labels.

[0072] Then, the CRF layer selects the label sequence with the highest score as the final output label sequence according to the maximum score principle. This step can use the Viterbi algorithm to find the optimal path.

[0073] (4) The cross entropy loss function is used to calculate the gradient and update the model based on the gap between the output label sequence and the corresponding true label to obtain the adjusted power grid domain pre-training model.

[0074] We use the back-propagation algorithm to calculate the gradient of the loss with respect to the model parameters. The back-propagation algorithm passes the gradient of the loss function back to each layer of the neural network through the chain rule to calculate the gradient of each parameter. Then, the gradient descent algorithm (such as stochastic gradient descent SGD, Adam, etc.) is used to update the model parameters so that the model can better fit the training data.

[0075] S103, obtaining a first text data set in the field of power grid, and preprocessing the first text data set to obtain a target text data set.

[0076] The first text data set may be derived from documents, reports, logs, etc. in various power grid fields.

[0077] In a specific embodiment, the step of preprocessing the first text data set to obtain the target text data set includes:

[0078] (1) Remove noise from the first text data set and segment the text according to a preset text length to obtain a second text data set.

[0079] The noise here includes irrelevant characters, symbols, erroneous data, etc. Removing these noises helps improve the effect of model training. Then, in order to meet the requirements of model input, the long text needs to be split into multiple text segments of appropriate length.

[0080] (2) All power grid keywords in the second text data set are marked, the first character of the power grid keyword is represented as the fourth character, the non-first character of the power grid keyword is represented as the fifth character, and the characters other than the power grid keyword in the second text data set are represented as the sixth character to obtain the target text data set.

[0081] Optionally, the first character of the power grid domain keyword is marked as "B" (Beginning) in the form of the fourth character. The non-first character of the power grid domain keyword is marked as "I" (Inside) in the form of the fifth character. The characters other than the power grid domain keyword in the second text data set are marked as "0" (Outside) in the form of the sixth character.

[0082] For example, suppose our text dataset contains the following sentence: "Smart grid improves the efficiency of the power system." In this sentence, "smart grid" and "power system" are keywords in the field of power grid. Then the annotated sentence is expressed as: "Smart grid improves the efficiency of the power system."

[0083] By repeating the labeling operation on multiple such sentences, the required target text dataset can be obtained.

[0084] S104, inputting the target text data set into the pre-trained model in the power grid field for secondary adjustment to obtain an adjusted keyword extraction model in the power grid field.

[0085] In the process of training the keyword extraction model, re-adjusting the pre-trained model is an important step to improve the performance of the model in a specific field. This process further trains the model using data from a specific field to better adapt it to the needs of a specific task.

[0086] In a specific embodiment, the target text data set is input into the pre-trained model in the power grid field for secondary adjustment, and the step of obtaining the adjusted power grid field keyword extraction model includes:

[0087] (1) Input the target text dataset into the hidden layer of the power grid pre-training model for feature extraction to obtain the second text feature.

[0088] Specifically, after the target text dataset is input into the model, it first passes through the hidden layer (e.g., Transformer or other neural network layer) to extract the feature representation of the text, which captures the contextual information of the text.

[0089] The process is expressed as:

[0090] h b =BERT(X e )

[0091]

[0092] X e ={x e1 , x e2 , …, x em}

[0093] Among them, X e represents the target text dataset, m is the length of the text, h b represents the second text feature, and d is the word embedding dimension.

[0094] (2) Input the second text feature into the bidirectional long short-term memory network of the pre-trained model in the power grid field to extract the global feature, and input it into the convolutional neural network of the pre-trained model in the power grid field to extract the local feature, and fuse the global feature set and the local feature to obtain the fused feature.

[0095] Specifically, BiLSTM is used to capture the global features in the text, taking into account the dependencies between words in the text. The process is expressed as:

[0096]

[0097] h G_L =LSTM 1 (h b ), b∈[1,m]

[0098] h G_R =LSTM 2 (h b ), b∈[m,1]

[0099] Among them, AVG represents the bitwise averaging operation on the output of the bidirectional LSTM network.

[0100] CNN is used to capture local features in text, especially patterns of phrases or local contexts. The process is expressed as:

[0101] h L =CNN(h b ), b∈[1,m]

[0102] Finally, the features extracted by BiLSTM and CNN are fused to form fused features. The process is expressed as:

[0103]

[0104] (3) Use a linear function to map the fused features to the label domain to obtain the second text label prediction value.

[0105] The process is expressed as:

[0106] p e =Linear e (h e )

[0107]

[0108] Among them, p e Represents the second text label prediction value, p em It represents the tag prediction value of the mth keyword, and t is the number of tag types.

[0109] (4) Inputting the second text label prediction value into the conditional random field layer of the pre-trained language model to model the label transition probability in the text label prediction value and calculate the label score based on the label transition probability, and outputting the predicted label sequence based on the maximum score principle.

[0110] Among them, the conditional random field (CRF) is a probabilistic graphical model for sequence labeling tasks, which can consider the dependencies between labels in the sequence, thereby improving the accuracy of labeling. In the keyword extraction task, the CRF layer is used to process the label prediction value, model the transition probability between labels, and output the optimal label sequence.

[0111] The process of calculating label scores is expressed as:

[0112]

[0113] in, Represents the label transition probability between two adjacent labels.

[0114] Then, the CRF layer selects the label sequence with the highest score as the final output label sequence according to the maximum score principle. This step can use the Viterbi algorithm to find the optimal path.

[0115] (5) The cross entropy loss function is used to calculate the gradient and update the model based on the gap between the output predicted label sequence and the corresponding true label to obtain the adjusted keyword extraction model in the power grid field.

[0116] We use the back-propagation algorithm to calculate the gradient of the loss with respect to the model parameters. The back-propagation algorithm passes the gradient of the loss function back to each layer of the neural network through the chain rule to calculate the gradient of each parameter. Then, the gradient descent algorithm (such as stochastic gradient descent SGD, Adam, etc.) is used to update the model parameters so that the model can better fit the training data.

[0117] The training method of the above-mentioned power grid field keyword extraction model first obtains the power grid field dictionary and constructs a pseudo data set in the power grid field during the training stage, and then inputs the pseudo data set into the pre-trained language model, performs preliminary adjustments, and obtains the adjusted power grid field pre-trained model. Then, the first text data set in the power grid field is obtained, pre-processed to obtain the target text data set, and finally the target text data set is input into the adjusted pre-trained model, and secondary adjustments are performed to obtain the final power grid field keyword extraction model. The present invention overcomes the problem that the general keyword extraction method is not effective when applied in the power grid field, and provides an efficient and accurate power grid field keyword extraction solution.

[0118] like Figure 2 As shown, the present invention also provides a method for extracting keywords in the power grid field. The steps provided by the method for extracting keywords in the power grid field in this embodiment include:

[0119] S201, obtaining a data set to be extracted, inputting the data set to be extracted into a keyword extraction model in the power grid field, and obtaining an output target label sequence.

[0120] We obtain a data set that needs to be extracted for keywords. This data set contains a large amount of text information in the power grid field. Then, the prepared data set to be extracted is input into a pre-trained power grid field keyword extraction model. This model has been trained using the training method of the power grid field keyword extraction model mentioned above, so that it can recognize and extract keywords in the power grid field.

[0121] The keyword extraction model in the power grid domain processes the input data set and outputs a target label sequence. This label sequence annotates the labels of each word or character in the input text, for example, which are the first characters of the keyword (B), which are non-first characters of the keyword (I), and which are non-keyword characters (O).

[0122] S202, extracting keywords in the power grid field from the data set to be extracted based on the target tag sequence to obtain a list of keywords in the power grid field.

[0123] We can first create an empty keyword list to store the extracted keywords. Then traverse each character in the target tag sequence and its corresponding label. When the label "B" is encountered, mark this as the beginning of a new keyword and record the character. When the label "I" is encountered, if the previous label is "B" or "I", continue to record the character, indicating that this is part of the current keyword. When the label "O" is encountered, it means that the current keyword ends, and the complete keyword is added to the keyword list. If there are still unfinished keywords when traversing to the end of the text, they are also added to the keyword list. Through the above steps, all the keywords extracted from the target tag sequence constitute the final power grid field keyword list.

[0124] This can effectively identify and extract keywords in the power grid field from the data set to be extracted, and generate a list containing all keywords. This automated processing method greatly improves the efficiency and accuracy of text data processing in the power grid field.

[0125] Figure 3 FIG. 2 shows an internal structure diagram of a terminal device in an embodiment. Figure 3 As shown, the terminal device includes a processor, a memory and a network interface connected via a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the terminal device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the method for extracting keywords in the power grid field. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the method for extracting keywords in the power grid field. Those skilled in the art can understand that Figure 3 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0126] A computer-readable storage medium stores a computer program, which implements the following steps when executed by a processor: obtaining a power grid domain dictionary and constructing a power grid domain pseudo data set based on the power grid domain dictionary; inputting the power grid domain pseudo data set into a pre-trained language model for preliminary adjustment to obtain an adjusted power grid domain pre-trained model; obtaining a first text data set in the power grid domain, pre-processing the first text data set to obtain a target text data set; inputting the target text data set into the power grid domain pre-trained model for secondary adjustment to obtain an adjusted power grid domain keyword extraction model.

[0127] And implement the following steps: obtain the data set to be extracted, input the data set to be extracted into the power grid field keyword extraction model, and obtain the output target label sequence; extract the power grid field keywords in the data set to be extracted based on the target label sequence to obtain a power grid field keyword list.

[0128] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: obtaining a power grid domain dictionary and constructing a power grid domain pseudo data set based on the power grid domain dictionary; inputting the power grid domain pseudo data set into a pre-trained language model for preliminary adjustment to obtain an adjusted power grid domain pre-trained model; obtaining a first text data set in the power grid domain and pre-processing the first text data set to obtain a target text data set; inputting the target text data set into the power grid domain pre-trained model for secondary adjustment to obtain an adjusted power grid domain keyword extraction model.

[0129] And implement the following steps: obtain the data set to be extracted, input the data set to be extracted into the power grid field keyword extraction model, and obtain the output target label sequence; extract the power grid field keywords in the data set to be extracted based on the target label sequence to obtain a power grid field keyword list.

[0130] It should be noted that the above-mentioned keyword extraction method, training method, device and computer-readable storage medium in the power grid field belong to a general inventive concept, and the contents in the embodiments of the keyword extraction method, training method, device and computer-readable storage medium in the power grid field are applicable to each other.

[0131] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0132] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0133] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A training method for keyword extraction model in power grid field, It is characterized in that The method comprises: Acquire a power grid domain dictionary, and construct a power grid domain pseudo data set based on the power grid domain dictionary; Inputting the power grid field pseudo data set into the pre-trained language model for preliminary adjustment to obtain an adjusted power grid field pre-trained model; Acquire a first text data set in the field of power grid, and preprocess the first text data set to obtain a target text data set; The target text data set is input into the pre-trained model in the power grid field for secondary adjustment to obtain an adjusted power grid field keyword extraction model.

2. The method according to claim 1, It is characterized in that The constructing a pseudo data set in the power grid field based on the power grid field dictionary includes: Randomly extracting k preset power grid domain terms from the power grid domain dictionary to construct pseudo sentence level text; All power grid domain keywords in the pseudo sentence level text are marked, the first character of the power grid domain keyword is represented as the first character, and the non-first character of the power grid domain keyword is represented as the second character, so as to obtain the power grid domain pseudo data set.

3. The method according to claim 2, It is characterized in that The step of inputting the pseudo data set in the power grid field into the pre-trained language model for preliminary adjustment to obtain the adjusted pre-trained model in the power grid field includes: Inputting the power grid field pseudo data set into the hidden layer of the pre-trained language model for feature extraction to obtain a first text feature; Mapping the first text feature to a label domain using a linear function to obtain a first text label prediction value; wherein the label domain is composed of K power grid domain terms; Inputting the first text label prediction value into the conditional random field layer of the pre-trained language model to model the label transition probability in the text label prediction value and calculate the label score based on the label transition probability, and outputting the label sequence based on the maximum score principle; The cross entropy loss function is used to calculate the gradient and update the model based on the gap between the output label sequence and the corresponding true label to obtain the adjusted power grid field pre-training model.

4. The method according to claim 1, It is characterized in that The preprocessing of the first text data set to obtain a target text data set includes: Removing noise from the first text data set and segmenting the text data set according to a preset text length to obtain a second text data set; All power grid domain keywords in the second text data set are marked, the first character of the power grid domain keyword is represented in the form of the fourth character, the non-first character of the power grid domain keyword is represented in the form of the fifth character, and the characters in the second text data set except the power grid domain keyword are represented in the form of the sixth character to obtain the target text data set.

5. The method according to claim 1, It is characterized in that The step of inputting the target text data set into the pre-trained model in the power grid field for secondary adjustment to obtain an adjusted keyword extraction model in the power grid field includes: Inputting the target text data set into the hidden layer of the power grid domain pre-training model for feature extraction to obtain a second text feature; Input the second text feature into a bidirectional long short-term memory network of a pre-trained model in the power grid field to extract global features, and input the second text feature into a convolutional neural network of a pre-trained model in the power grid field to extract local features, and fuse the local features of the global feature set to obtain a fused feature; Mapping the fused features into a label domain using a linear function to obtain a second text label prediction value; Inputting the second text label prediction value into the conditional random field layer of the pre-trained language model to model the label transition probability in the text label prediction value and calculate the label score based on the label transition probability, and outputting the predicted label sequence based on the maximum score principle; The cross entropy loss function is used to calculate the gradient and update the model based on the gap between the output predicted label sequence and the corresponding true label to obtain the adjusted keyword extraction model in the power grid field.

6. A keyword extraction method in the power grid field, It is characterized in that The method: Obtain a data set to be extracted, input the data set to be extracted into a keyword extraction model in the power grid field, and obtain an output target label sequence; wherein the keyword extraction model in the power grid field is trained by the method according to claim 1; The power grid field keywords in the to-be-extracted data set are extracted based on the target tag sequence to obtain a power grid field keyword list.

7. A computer-readable storage medium, It is characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor is caused to execute the steps of the method according to any one of claims 1 to 6.

8. A terminal device, It is characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 6.