Entity relation extraction method for power equipment and related device

Abstract

This application provides a method and related apparatus for entity relation extraction for power equipment, relating to the field of natural language processing technology. The method includes: acquiring text information describing the power equipment; extracting a sentence to be processed from the text information; extracting features from the characters of the sentence to be processed to obtain character features; extracting features from the radicals of the sentence to be processed to obtain radical features; fusing the character features and radical features to obtain fused features; and inputting the fused features into an entity relation extraction network to obtain entity relations extracted from the sentence to be processed. In this application, features at different levels, namely character features and radical features, can be extracted from the sentence to be processed. Since radical features can convey semantic information, fusing radical features and character features can shorten the distance between entities of different power equipment, thereby accurately extracting entity relations. Furthermore, this method has generalization ability across different power equipment.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, specifically to a method and related apparatus for extracting entity relationships for power equipment. Background Technology

[0002] With the development of the digital economy, the construction of digital and smart grids has become crucial in the power energy sector. On the one hand, entering the deeper waters of digital transformation urgently requires theoretical and technical support from independently controllable artificial intelligence technologies to facilitate information flow, gain situational awareness of public opinion and development trends related to important power equipment, and promptly issue early warnings and adjust production accordingly. On the other hand, to complete the construction of a smart grid, it is necessary to extract relevant data from the entire process into knowledge, and condense that knowledge into wisdom. Among these technologies, entity relation extraction (ERP), as an information extraction technique, is one of the key theoretical technologies for converting information into knowledge.

[0003] However, in the field of power equipment, entity names are long, span a wide range, and contain a large number of specialized terms. Therefore, extracting entity relationships for power equipment has become a challenge. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, this application aims to provide a method and related apparatus for extracting entity relationships of power equipment, which can accurately extract entity relationships related to power equipment.

[0005] In a first aspect, this application provides a method for extracting entity relationships for power equipment, the method comprising:

[0006] Obtain text information describing the power equipment;

[0007] Extract the sentence to be processed from the text information;

[0008] Character features are extracted from the characters of the sentence to be processed.

[0009] The radicals of the sentence to be processed are extracted to obtain radical features;

[0010] The character features and the radical features are fused to obtain fused features;

[0011] The fused features are input into the entity relation extraction network to obtain the entity relations extracted from the sentence to be processed.

[0012] In some embodiments, the step of extracting character features from the characters of the sentence to be processed to obtain character features includes:

[0013] The sentence to be processed is segmented according to Chinese characters to obtain a character sequence;

[0014] The character sequence is input into the first natural language feature extraction model to obtain the character features of the sentence to be processed.

[0015] In some embodiments, the step of extracting radical features from the sentence to be processed to obtain radical features includes:

[0016] Extract the Chinese radicals contained in the sentence to be processed to obtain the radical sequence;

[0017] The radical sequence is input into the second natural language feature extraction model to obtain the radical features of the sentence to be processed.

[0018] In some embodiments, before performing feature fusion on the character features and the radical features to obtain the fused features, the method further includes:

[0019] Remove noise information from the character features.

[0020] In some embodiments, removing noise information from the character features includes:

[0021] The character features are input into the text denoising model to obtain the labeling type of each character in the sentence to be processed. The available labeling types are derived from a type set, which includes the following labeling types: related to power equipment and unrelated to power equipment.

[0022] According to the positional order in the sentence to be processed, the annotation types of each character are concatenated to obtain the annotation sequence;

[0023] The annotation type in the annotation sequence is converted into a digital value based on the transformation function to obtain the attention vector after deduplication and noise removal; wherein, the transformation function is used to enhance the features related to power equipment and weaken the features unrelated to power equipment.

[0024] Multiplying the attention vector and the character feature yields the noise-removed character feature.

[0025] In some embodiments, the entity relation extraction network includes a subject location prediction module and a relation prediction module. The step of inputting the fused features into the entity relation extraction network to obtain the entity relations extracted from the sentence to be processed includes:

[0026] The fused features are input into the subject position prediction module to obtain the subject's start and end positions;

[0027] The fusion feature, the start position and end position of the subject are input into the relationship prediction module to obtain at least one prediction information output by the relationship prediction module. Each prediction information includes the start position and end position of an object, as well as the relationship between the subject and the object.

[0028] Extract the main body from the sentence to be processed based on the start and end positions of the main body;

[0029] Extract each object from the sentence to be processed based on its start and end positions;

[0030] For each object, construct an entity relation that includes the subject, the object, and the relationship between the subject and the object.

[0031] In some embodiments, the relationship prediction module includes multiple binary classification modules for predicting different relationships. The step of inputting the fusion features, the start position and end position of the subject into the relationship prediction module to obtain at least one prediction message output by the relationship prediction module includes:

[0032] The fusion features, the start position and the end position of the subject are respectively input into each binary classification module to obtain the candidate prediction information output by each binary classification module. The candidate prediction information includes the start position and the end position of an object, as well as the relationship between the subject and the object.

[0033] Filter out erroneous candidate predictions and determine at least one remaining candidate prediction as the at least one prediction output by the relationship prediction module.

[0034] In some embodiments, the feature fusion of the character features and the radical features to obtain fused features includes:

[0035] The character features and the radical features are concatenated to obtain the fused features.

[0036] Secondly, this application provides an entity relationship extraction device for power equipment, the device comprising:

[0037] The text acquisition module is used to acquire text information describing power equipment;

[0038] The sentence extraction module is used to extract the sentence to be processed from the text information;

[0039] The character feature extraction module is used to extract character features from the sentence to be processed to obtain character features;

[0040] The radical feature extraction module is used to extract radical features from the sentence to be processed to obtain radical features;

[0041] The fusion module is used to fuse the character features and the radical features to obtain fused features;

[0042] The relation extraction module is used to input the fused features into the entity relation extraction network to obtain the entity relations extracted from the sentence to be processed.

[0043] In some embodiments, the character feature extraction module is used for:

[0044] The sentence to be processed is segmented according to Chinese characters to obtain a character sequence;

[0045] The character sequence is input into the first natural language feature extraction model to obtain the character features of the sentence to be processed.

[0046] In some embodiments, the radical feature extraction module is used for:

[0047] Extract the Chinese radicals contained in the sentence to be processed to obtain the radical sequence;

[0048] The radical sequence is input into the second natural language feature extraction model to obtain the radical features of the sentence to be processed.

[0049] In some embodiments, it also includes:

[0050] The noise reduction module is used to remove noise information from the character features before performing feature fusion on the character features and the radical features to obtain the fused features.

[0051] In some embodiments, the noise reduction module is used for:

[0052] The character features are input into the text denoising model to obtain the labeling type of each character in the sentence to be processed. The available labeling types are derived from a type set, which includes the following labeling types: related to power equipment and unrelated to power equipment.

[0053] According to the positional order in the sentence to be processed, the annotation types of each character are concatenated to obtain the annotation sequence;

[0054] The annotation type in the annotation sequence is converted into a digital value based on the transformation function to obtain the attention vector after deduplication and noise removal; wherein, the transformation function is used to enhance the features related to power equipment and weaken the features unrelated to power equipment.

[0055] Multiplying the attention vector and the character feature yields the noise-removed character feature.

[0056] In some embodiments, the relation extraction module includes:

[0057] The main body position prediction unit is used to input the fused features into the main body position prediction module to obtain the start position and end position of the main body;

[0058] A relationship prediction unit is used to input the fusion feature, the start position and end position of the subject into the relationship prediction module to obtain at least one prediction information output by the relationship prediction module. Each prediction information includes the start position and end position of an object, as well as the relationship between the subject and the object.

[0059] The subject extraction unit is used to extract the subject from the sentence to be processed based on the start and end positions of the subject.

[0060] An object extraction unit is used to extract each object from the sentence to be processed based on the start and end positions of each object.

[0061] The relationship establishment unit is used to construct entity relationships for each object, including the subject, the object, and the relationship between the subject and the object.

[0062] In some embodiments, the relationship prediction module includes multiple binary classification modules for predicting different relationships, and the relationship prediction unit is used for:

[0063] The fusion features, the start position and the end position of the subject are respectively input into each binary classification module to obtain the candidate prediction information output by each binary classification module. The candidate prediction information includes the start position and the end position of an object, as well as the relationship between the subject and the object.

[0064] Filter out erroneous candidate predictions and determine at least one remaining candidate prediction as the at least one prediction output by the relationship prediction module.

[0065] In some embodiments, the fusion module is used to concatenate the character features and the radical features to obtain the fused features.

[0066] Thirdly, this application provides an electronic device, the electronic device comprising:

[0067] Memory, used to store programs; and

[0068] A processor is configured to execute any of the entity relation extraction methods for power equipment provided in this application by invoking the program stored in the memory.

[0069] Fourthly, this application provides a computer-readable storage medium storing computer instructions for causing a computer to execute any of the entity relationship extraction methods for power equipment provided in this application.

[0070] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the entity relationship extraction methods for power equipment disclosed herein.

[0071] In summary, the embodiments of this application can extract features at different levels from the sentence to be processed, namely character features and radical features. Since radical features can convey semantic information, fusing radical features and character features can shorten the distance between entities of different power devices, thereby accurately extracting entity relationships. Moreover, this method has generalization ability among different power devices. Attached Figure Description

[0072] Figure 1 This is a schematic diagram illustrating an application scenario of an entity relationship extraction method for power equipment provided in an embodiment of this application.

[0073] Figure 2 A flowchart illustrating the entity relationship extraction method for power equipment provided in this application embodiment;

[0074] Figure 3 A flowchart illustrating the extraction of character features and radical features provided in an embodiment of this application;

[0075] Figure 4 A model block diagram for extracting character features provided in an embodiment of this application;

[0076] Figure 5 This is a schematic diagram illustrating a process for text denoising based on character features, provided as an embodiment of this application.

[0077] Figure 6 A model block diagram for extracting radical features provided in an embodiment of this application;

[0078] Figure 7 Another flowchart illustrating the entity relationship extraction method for power equipment provided in this application embodiment;

[0079] Figure 8 Another flowchart illustrating the entity relationship extraction method for power equipment provided in this application embodiment;

[0080] Figure 9 A schematic diagram of the framework of the entity relationship extraction network provided in the embodiments of this application;

[0081] Figure 10 Another flowchart illustrating the entity relationship extraction method for power equipment provided in this application embodiment;

[0082] Figure 11 A network framework diagram is shown in the entity relationship extraction method for power equipment provided in the embodiments of this application.

[0083] Figure 12 This is a schematic diagram of the entity relationship extraction device for power equipment provided in an embodiment of this application;

[0084] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0085] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0086] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0087] Currently, the main model architectures for entity relation extraction include two types: the pipeline model architecture and the joint model architecture. Among them:

[0088] The cascaded model architecture treats entity-relation extraction as two separate stages: entity extraction and relation extraction. These stages are trained separately, which causes errors from the first stage to propagate to the second stage, ignoring potential association information between entities and relations.

[0089] The joint model architecture extracts entities and relationships simultaneously during a single training process, which can uncover potential semantic feature associations.

[0090] However, regardless of the model architecture mentioned above, when extracting entity relationships in the power equipment field, the two model architectures mentioned above cannot function well in the power equipment field because the entity names are long, the scope is large, and there are many professional terms.

[0091] In view of this, embodiments of this application provide a method and related apparatus for extracting entity relationships for power equipment.

[0092] This application uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. The above are only preferred embodiments of this application. It should be noted that due to the limitations of textual expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this application, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other occasions without modification, should all be considered within the scope of protection of this application.

[0093] Figure 1 This illustration shows an application scenario diagram of an entity relationship extraction method for power equipment provided in an embodiment of this application. For example... Figure 1 As shown, the development of power equipment will generate relevant news reports or research papers. Entity relationships can be extracted from these news reports and research papers. For ease of description, both news reports and papers are collectively referred to as text information in this embodiment. Figure 1 In order to automatically and effectively mine the entity relationships of power equipment, electronic device 101, which extracts entity relationships, can collect textual information on news reports and research papers distributed across the network to obtain the entity relationships to be mined. For example, from... Figure 1 News reports are extracted from the content provided by the news server 102 in the middle. Figure 1 Research papers can be extracted from papers provided by the Chinese paper server 103.

[0094] For power equipment, in order to accurately and effectively mine entity relationships, embodiments of this application propose using multi-level fusion features to shorten the spatial distance between entities of different power equipment, thereby mining accurate entity relationships. For example... Figure 2 As shown, it includes the following:

[0095] S201, Obtain text information describing the power equipment.

[0096] like Figure 1 The textual information described may come from news reports or academic papers.

[0097] In this embodiment, each news report and each paper can correspond to one text message. Since the processing methods for different text messages are the same, this embodiment uses one text message as an example to illustrate the method for extracting entity relationships.

[0098] S202, Extract the sentence to be processed from the text information.

[0099] For example, the text information includes "The company has added two 1600KVA power transformers, one of which was allocated to the provincial company." In implementation, this can be split into two sentences: one is "The company has added two 1600KVA power transformers," and the other is "One of which was allocated to the provincial company." The two sentences can be used separately for entity relation extraction. Similarly, since the processing method for each sentence is the same, this embodiment uses one sentence as an example for illustration.

[0100] S203, extract character features from the characters of the sentence to be processed to obtain character features.

[0101] Taking "The company has added two 1600KVA power transformers" as an example, each word is treated as a separate character. For example, "company" is two characters and "1600" is four characters.

[0102] S204: Extract the radical features of the sentence to be processed to obtain radical features.

[0103] It should be noted that there is no restriction on the order in which S203 and S204 are executed.

[0104] S205, perform feature fusion on character features and radical features to obtain fused features.

[0105] S206, input the fused features into the entity relation extraction network to obtain the entity relations extracted from the sentence to be processed.

[0106] In this embodiment, features at different levels, namely character features and radical features, can be extracted from the sentence to be processed. Since radical features can convey semantic information, fusing radical features and character features can shorten the distance between entities of different power equipment. Therefore, even in the field of power equipment, where entity names are long, have a wide range, and contain many specialized terms, multi-level feature fusion can minimize the differences and accurately extract entity relationships. Furthermore, this method has generalization ability across different power equipment.

[0107] Therefore, the embodiments of this application can accurately transform unstructured text data into knowledge, which helps to improve the construction of smart grids and supports regulatory work such as power equipment situational awareness and risk warning.

[0108] For ease of understanding, the implementation of the main steps will be described by way of example below. It should be understood that the following exemplary embodiments are used to illustrate the entity relationship extraction method of the present application, and do not specifically limit the method.

[0109] 1) Regarding character feature extraction

[0110] like Figure 3As shown, it can be implemented as follows:

[0111] S311, split the sentence to be processed according to Chinese characters to obtain a character sequence.

[0112] Continuing with the example of "The company newly added two 1600KVA power transformers", each character can be split using spaces to obtain the character sequence: "The company newly added two 1600KVA power transformers".

[0113] Of course, in other embodiments, a specified connector can be used to connect adjacent characters. For example, the obtained character sequence is "co - mp - an - y - ne - wl - y - ad - de - d - tw - o - 1 - 6 - 0 - 0 - K - V - A - po - we - r - tr - an - sf - or - me - rs", where "-" is the connector.

[0114] It should be noted that any method capable of splitting different characters is applicable to the embodiments of this application.

[0115] S312, input the character sequence into the first natural language feature extraction model to obtain the character features of the sentence to be processed.

[0116] During implementation, the first natural speech feature extraction model can adopt the BERT model structure. To distinguish it from other BERT models in the embodiments of this application, the first natural language feature extraction model can be simply referred to as BERTC.

[0117] BERTC can be trained in a self - supervised manner to facilitate the extraction of expressive character features. For example, two sentences can be used to construct sample pairs, including positive sample pairs and negative sample pairs. Among them, the positive sample pairs include two context - coherent sentences. For example, for a sentence, the next sentence of this sentence can be extracted from different texts containing this sentence to construct a positive sample. Similarly, for a sentence, a sentence can be randomly selected from the corpus (most likely not logically coherent with this sentence) to construct a negative sample pair. Then during self - supervised training, the BERTC model can be used to predict the next sentence of the previous sentence in the sample pair, or predict the previous sentence of the next sentence in the sample pair. During the prediction process, as Figure 4 shown, the BERT model will extract intermediate features and make predictions based on these intermediate features. Based on the prediction results and the corresponding sample pairs, the loss of the prediction can be determined to adjust the parameters of the BERTC model until the model converges.

[0118] In the case where the BERTC model is trained, the character sequence is input into the BERTC model, and the intermediate features output by the model for the character sequence are used as the character features of the character sequence.

[0119] In this embodiment of the application, based on the first natural language feature extraction model, a deep understanding of character sequences can be achieved, thereby extracting expressive high-level features to provide a data foundation for accurate entity relationship extraction.

[0120] In some embodiments, noise, i.e., information unrelated to electrical equipment, may exist in the character sequence. To obtain more expressive fused features that better describe the characteristics of the sentence to be processed, this application proposes further noise removal from the character features. That is, before fusing character features and radical features to obtain the fused features, noise information in the character features needs to be removed. Therefore, further refining the character features can guide entity relation extraction and improve the accuracy of the extracted entity relations.

[0121] In some possible embodiments, such as Figure 5 As shown, removing noise information from character features can be implemented as follows:

[0122] S501, Input the character features into the text denoising model to obtain the labeling type of each character in the sentence to be processed. The available labeling types are derived from a type set, which includes the following labeling types: related to power equipment and unrelated to power equipment.

[0123] The category "unrelated to power equipment" can be further subdivided into two types: noise type and irrelevant type. Therefore, the type set includes three categories: "related to power equipment," "noise type," and "irrelevant." Each character is labeled with an appropriate type selected from this set.

[0124] S502: Concatenate the annotation types of each character according to their position order in the sentence to be processed to obtain the annotation sequence.

[0125] S503 converts each label type in the label sequence into a digital quantity based on the transformation function to obtain the attention vector after deduplication and noise removal; wherein, the transformation function is used to enhance features related to power equipment and weaken features unrelated to power equipment.

[0126] S504 multiplies the attention vector and the character features to obtain the noise-removed character features.

[0127] For example, in one possible implementation, such as Figure 5As shown, the text denoising model consists of two parts: a bidirectional long short-term memory (BiLSTM) network and a conditional random field (CRF) layer. The character feature Hc is input into the BiLSTM to obtain the intermediate vector representation H. Bi As shown in formula (1):

[0128]

[0129] In formula (1), the character feature H c The length is M, and the character feature is H. Bi After processing, an intermediate vector representation H that can predict the label type is obtained. Bi , This represents the extraction of the intermediate vector H of length M. Bi The character feature has M feature components, each corresponding to a component of an intermediate vector, i.e.

[0130] Represent the intermediate vector H Bi The input is fed into a CRF for decoding to obtain the predicted sequence Y. s Each Chinese character in the text sequence is divided into three types: "related to power equipment", "noise", and "irrelevant", as shown in formula (2):

[0131]

[0132] In formula (2), each character obtains its corresponding type.

[0133] In order to perform operations with character features, the transformation function I in this embodiment marks the positions of tags related to power equipment as 1 and the positions of tags that are not related as 0, thereby strengthening the features related to power equipment and weakening the features that are not related to power equipment, and finally obtaining the attention vector E after removing noise, as shown in formula (3):

[0134] E = [e1, e2, ..., e M ] = I(Y s (3)

[0135] Therefore, in formula (3), the attention vector E is a series of strings consisting of 0 or 1.

[0136] In some other possible embodiments, the embodiments of the present application may also use multiple classifiers to classify and predict the types of the same character, and obtain the confidence of the prediction results. For example, it is predicted that the character A is of the type "related to circuit devices", and the confidence is 0.8. Then, during implementation, a conversion function can be used to process the predicted type and the corresponding confidence to obtain the corresponding attention vector. For example, for the predicted type of "related to power equipment", its confidence is used as the digital quantity of the labeled type, and for the type of "unrelated to power equipment", (1 - confidence) is used as its labeled type digital quantity. For example, the labeled type of character 1 is "related to power equipment", and the predicted confidence is 0.9, then its digital quantity is 0.9; the labeled type of character 2 is "related to power equipment", and the predicted confidence is 0.8, then its digital quantity is 0.8; the labeled type of character 3 is "unrelated to power equipment", and the predicted confidence is 0.9, then the corresponding digital quantity is 0.1. Thus, the attention vectors obtained for these 3 characters are (0.9, 0.8, 0.1). After multiplying this attention vector by the character features, it can also achieve the purpose of strengthening the features related to power equipment and weakening the features unrelated to power equipment.

[0137] In summary, in the embodiments of the present application, by removing duplicate noises from character features, character features with strong expressiveness can be further refined, features unrelated to power equipment can be removed, which can better guide entity relationship extraction and establish a data basis for entity relationship extraction.

[0138] 2) Regarding the extraction of radical features

[0139] As Figure 3 shown, it can be implemented as:

[0140] S321, extract the Chinese radicals contained in the sentence to be processed to obtain a radical sequence.

[0141] Continuing with the example of "The company newly added two 1600KVA power transformers", according to Chinese radicals, the radical sequence can be obtained: "丷口斤土一囗1600KVA田力又厂口".

[0142] S322, input the radical sequence into the second natural language feature extraction model to obtain the radical features of the sentence to be processed.

[0143] Taking the extraction of character features as an example, in the embodiments of the present application, a BERT model structure can also be used to construct the second natural language feature extraction model. The second natural language feature extraction model is abbreviated as BERTR

[0144] BERTR can be trained in a self-supervised manner to extract expressive radical features. For example, sample pairs can be constructed using two sentences, including positive and negative sample pairs. A positive sample pair consists of two sentences with coherent context; for instance, for a given sentence, the next sentence can be extracted from different texts containing that sentence to construct a positive sample. Similarly, for a given sentence, a sentence can be randomly selected from the corpus (likely logically disjointed) to construct a negative sample pair. Then, during self-supervised training, the BERTR model can predict the sentence following the preceding sentence in a sample pair, or predict the sentence preceding the following sentence in a sample pair. During the prediction process, such as... Figure 6 As shown, the BERTR model extracts intermediate features and makes predictions based on these features. Based on the prediction results and the corresponding sample pairs, the prediction loss can be determined, and the parameters of the BERTR model can be adjusted until the model converges.

[0145] Once the BERTR model is trained, the character sequence is input into the BERTR model, and the intermediate features output by the model for the radical sequence are used as the radical features of the radical sequence.

[0146] In this embodiment, based on the second natural language feature extraction model, the radical sequence can be deeply understood, and radical features with semantic information can be extracted, thereby extracting high-level features with expressive power, so as to provide a data foundation for accurate entity relationship extraction.

[0147] 3) Regarding feature fusion

[0148] One possible implementation is to calculate the average of radical features and character features as the fusion feature. Of course, for different types of text information, a weighted sum of character features and radical features can also be used to determine the fusion feature. In implementation, the corresponding fusion method can be determined according to actual needs.

[0149] Furthermore, to better utilize radical and character features for entity relation extraction, character and radical features can be concatenated to obtain fused features. Concatenated features can fully preserve the information from both character and radical features, describing the sentence to be processed at different levels, thus improving the accuracy of entity relation extraction.

[0150] 4) Regarding the extraction of entity relations

[0151] like Figure 7 As shown, it can be implemented as follows:

[0152] S701 inputs the fused features into the subject position prediction module to obtain the start and end positions of the subject.

[0153] S702, input the fused features, the start position and end position of the subject into the relationship prediction module to obtain at least one prediction information output by the relationship prediction module. Each prediction information includes the start position and end position of an object, as well as the relationship between the subject and the object.

[0154] S703 extracts the subject from the sentence to be processed based on the start and end positions of the subject.

[0155] S704, extract each object from the sentence to be processed based on the start and end positions of each object.

[0156] S705, for each object, construct entity relations that include the subject, the object, and the relationship between the subject and the object.

[0157] Continuing with the example of "The company adds two 1600KVA power transformers", the predicted start position of the subject is 14 and the end position is 18, so the predicted subject position is (14, 18). The subject extracted from the sentence to be processed is "power transformer", and the predicted "quantity" relation is (quantity, 5, 6), corresponding to the object "two units". This results in a triple {power transformer, quantity, two units} as an entity relation.

[0158] In this embodiment, the subject, object, and the relationship between them can be accurately determined from the sentence to be processed based on fusion features, thereby achieving automatic extraction of entity relationships. Because fusion features can reduce the differences between entities of different power devices, they are applicable to different power devices. Therefore, the entity relationship extraction method provided in this embodiment has better generalization ability.

[0159] In some embodiments, to accurately predict different relationships, the relationship prediction module includes multiple binary classification modules for predicting different relationships. Accordingly, step S702 can be implemented as follows: Figure 8 As shown, it includes the following steps:

[0160] S801, the fusion features, the start position and end position of the subject are input into each binary classification module respectively, and the candidate prediction information output by each binary classification module is obtained. The candidate prediction information includes the start position and end position of an object, as well as the relationship between the subject and the object.

[0161] S802, filter out erroneous candidate prediction information, and determine at least one remaining candidate prediction information as at least one prediction information output by the relationship prediction module.

[0162] This allows for the definition of filtering conditions to eliminate erroneous candidate predictions. For example, each binary classification module can provide candidate predictions and output their confidence scores. A low confidence score indicates low reliability of the predictions output by the binary classifier; therefore, candidate predictions with confidence scores below a prediction threshold can be filtered out. The prediction threshold can be set to 0.5, but a larger threshold can be set for higher accuracy requirements.

[0163] In another embodiment, the reliability of the object's position prediction result can be verified. For example, by comparing the object's start and end positions, if the end position is before the start position, it indicates that the prediction result is incorrect, and therefore candidate predictions where the object's end position is before the start position can be filtered out.

[0164] For example, let's continue with the example of "the company adds two 1600KVA power transformers". Assume that six types of relationships can be predicted, including "belongs to", "is / is", "follows", "equipment specifications", "quantity", and "parameters".

[0165] From each binary classification module, positions with a confidence score greater than 0.5 for the predicted relation type are selected as candidate object positions. The start position Ohead and end position Otail of the objects for six relations can be predicted. Removing results where the end position precedes the start position yields the predicted relation and object position set Uo = {(relation1, Ohead1, Otail1), (relation2, Ohead2, Otail2), ...}. Mapping the obtained object and object position results to the input sentence yields the actual text content. For example, in the sentence "The company added two 1600KVA power transformers", the predicted subject position result is (14, 18), and the corresponding subject is "power transformer". The predicted set Uo = {(quantity, 5, 6), (equipment specification, 7, 13)}, and the corresponding objects are "two" and "1600KVA". The relations are "quantity" and "equipment specification". Finally, we get the triples {power transformer, quantity, two} and {power transformer, equipment specification, 1600KVA}, which completes the entity relation prediction.

[0166] In summary, by filtering out erroneous information in this application example, the accuracy of entity relationship extraction can be improved.

[0167] For example, a fusion pointer classification network can be used as an entity relation extraction network. It consists of a subject prediction part (i.e., subject position prediction module) and an object prediction part (i.e., relation prediction module). The subject prediction part consists of two binary classifiers, predicting the start and end positions of the subject, respectively. The object prediction part consists of twelve binary classifiers, with each pair of binary classifiers corresponding to one relation type, predicting the start and end positions of six relations: "belongs to," "is / is," "follows," "equipment specifications," "quantity," and "parameters." The fusion pointer classification network uses the fusion feature Hcr as input to the subject prediction part and the labeled subject start and end positions as labels for training. The object prediction part uses the fusion vector Hcr and the predicted subject position vector as input, and the object start and end positions as labels for training.

[0168] like Figure 9 The diagram shows a schematic of a fusion pointer classification network. Figure 9 In this process, an attention vector E is extracted and applied to the character feature Hr. Then, a fusion function G is used to fuse the character feature Hr and the radical feature Hc to obtain a fused feature Hcr. The fused feature Hcr is input to two classifiers in the subject prediction part of the fusion pointer classification network to obtain the subject's position vector representation, i.e., the subject's start and end positions. Each relation includes two classifiers, which predict the start and end positions of objects under that relation, thus obtaining candidate prediction information output by different relation prediction modules.

[0169] In implementation, each sentence can be extracted from the text sample set and labeled with the entity relations it contains. This yields a training sample set and a validation sample set constructed from multiple sentences and their labels. The training sample set can then be used to train a fusion pointer classification network, resulting in an entity relation extraction network.

[0170] To facilitate a systematic understanding of the entity relationship extraction method for power equipment provided in the embodiments of this application, the following is combined with... Figure 10 Explanation of entity relationship extraction from news report data, such as... Figure 10 The following are included:

[0171] S1001, preprocess the news report data to obtain the text set D.

[0172] In implementation, a web crawler for news reports on power equipment is constructed, crawling content including standard changes, cutting-edge technological developments, fault reports, and Q&A related to power equipment. The crawled news reports on power equipment are preprocessed, such as formatting the text and removing meaningless stop words, ultimately yielding a text set D = {d1, d2, ..., dn}, where di represents a preprocessed news report.

[0173] Based on the text collection D, a domain-specific vocabulary set P is obtained through manual summarization and expert consultation. At the same time, the text collection D is annotated. Specifically, each sentence in the news report di is annotated according to the predefined entity relationship types, obtaining the power equipment news report training data set S = {s1, s2, …, sm; T}, where sj is the training sentence and T is the annotation corresponding to each sentence. The annotation content consists of several triples {Subject, s, Relation, r, Object, o}. Among them, the subject is usually a power equipment, and the object is usually the content corresponding to the power equipment under a specific relationship. For example, in the sentence "The company newly added two 1600KVA power transformers", "power transformers" is the subject, the relationship is "quantity", and the object is "two".

[0174] S1002, Use the publicly available news data in the public domain as the basic data, and combine it with the news text collection D obtained in S1001 to form the power equipment news report corpus resource library D' = {d1', d2', …, dk'}.

[0175] S1003, Based on the corpus resource library D', train BERTC and BERTR, and train the fusion pointer classification network.

[0176] Among them, for all text passages in the power equipment news report corpus resource library D', on the one hand, they are converted into input texts Dr at the Chinese radical level according to Chinese radicals, and on the other hand, they are segmented according to Chinese characters to form training texts Dc at the Chinese character level. For example, if there is a sentence "The company newly added two 1600KVA power transformers" in the passage, it is converted into "The company newly added two 1600KVA power transformers" by separating with spaces according to Chinese characters, and then according to Chinese radicals, it can be obtained as "Eight mouths, one catty, one piece of soil, one mouth, 1600KVA, one field, one force, one factory, one mouth". According to these two different levels of input, the encoder BERT of the bidirectional depth self-attention transformation network is pre-trained according to the training method of the language model, obtaining the corresponding pre-trained language models BERTC and BERTR.

[0177] According to the annotated information and the training method described above, train the fusion pointer classification network.

[0178] S1004, Extract entity relationships from the news report data of power equipment based on the trained model.

[0179] As Figure 11 shown, first, obtain the text information describing the power equipment. Then, extract the sentence to be processed from the text information. In the embodiments of this application, it is default to preprocess the sentence to be processed.

[0180] The sentence to be processed is segmented according to Chinese characters to obtain a character sequence. The character sequence is then input into BERTC to obtain the character features of the sentence to be processed. At the same time, the Chinese radicals contained in the sentence to be processed are extracted to obtain a radical sequence. The radical sequence is then input into BERTR to obtain the radical features of the sentence to be processed.

[0181] For example, Figure 11 As shown, radicals are extracted from the sentence to be processed to obtain the Chinese radical level input Sr, and Chinese characters are segmented to obtain the Chinese character level input Sc.

[0182] Using the models BERTC and BERTR, we obtain two levels of vector representations, namely character features H. r and radical features H c As shown in formulas (4) and (5), Sr and Sc have the same length and are M.

[0183]

[0184]

[0185] For character features, a text denoising model is further employed for noise reduction. For example... Figure 11 As shown, the intermediate vector representation is obtained from the input BiLSTM, and then input into the Conditional Random Field layer to obtain the predicted labeled sequence Ys. The labeled sequence is then converted into an attention vector E through a binary function.

[0186] After that, as Figure 11 As shown, the attention vector E is multiplied by the Chinese character feature Hc, and then input into the fusion function G along with the radical feature representation of Chinese characters for fusion processing, resulting in the fusion feature Hcr obtained by concatenating the two vectors. The fusion process is shown in formula (6):

[0187] Hcr=concatenates(G(Hc,Hr,E)) (6)

[0188] Finally, as Figure 11 As shown, the obtained fusion features are input into the fusion pointer network to extract entity relationships.

[0189] Based on the same technical concept, this invention also provides an entity relationship extraction device for power equipment, such as... Figure 12 As shown, the device 1200 includes:

[0190] The text acquisition module 1201 is used to acquire text information describing power equipment;

[0191] Sentence extraction module 1202 is used to extract the sentence to be processed from the text information;

[0192] The character feature extraction module 1203 is used to extract character features from the sentence to be processed to obtain character features;

[0193] The radical feature extraction module 1204 is used to extract radical features from the sentence to be processed to obtain radical features;

[0194] The fusion module 1205 is used to fuse the character features and the radical features to obtain fused features;

[0195] The relation extraction module 1206 is used to input the fused features into the entity relation extraction network to obtain the entity relations extracted from the sentence to be processed.

[0196] In some embodiments, the character feature extraction module is used for:

[0197] The sentence to be processed is segmented according to Chinese characters to obtain a character sequence;

[0198] The character sequence is input into the first natural language feature extraction model to obtain the character features of the sentence to be processed.

[0199] In some embodiments, the radical feature extraction module is used for:

[0200] Extract the Chinese radicals contained in the sentence to be processed to obtain the radical sequence;

[0201] The radical sequence is input into the second natural language feature extraction model to obtain the radical features of the sentence to be processed.

[0202] In some embodiments, it also includes:

[0203] The noise reduction module is used to remove noise information from the character features before performing feature fusion on the character features and the radical features to obtain the fused features.

[0204] In some embodiments, the noise reduction module is used for:

[0205] The character features are input into the text denoising model to obtain the labeling type of each character in the sentence to be processed. The available labeling types are derived from a type set, which includes the following labeling types: related to power equipment and unrelated to power equipment.

[0206] According to the positional order in the sentence to be processed, the annotation types of each character are concatenated to obtain the annotation sequence;

[0207] The annotation type in the annotation sequence is converted into a digital value based on the transformation function to obtain the attention vector after deduplication and noise removal; wherein, the transformation function is used to enhance the features related to power equipment and weaken the features unrelated to power equipment.

[0208] Multiplying the attention vector and the character feature yields the noise-removed character feature.

[0209] In some embodiments, the relation extraction module includes:

[0210] The main body position prediction unit is used to input the fused features into the main body position prediction module to obtain the start position and end position of the main body;

[0211] A relationship prediction unit is used to input the fusion feature, the start position and end position of the subject into the relationship prediction module to obtain at least one prediction information output by the relationship prediction module. Each prediction information includes the start position and end position of an object, as well as the relationship between the subject and the object.

[0212] The subject extraction unit is used to extract the subject from the sentence to be processed based on the start and end positions of the subject.

[0213] An object extraction unit is used to extract each object from the sentence to be processed based on the start and end positions of each object.

[0214] The relationship establishment unit is used to construct entity relationships for each object, including the subject, the object, and the relationship between the subject and the object.

[0215] In some embodiments, the relationship prediction module includes multiple binary classification modules for predicting different relationships, and the relationship prediction unit is used for:

[0216] The fusion features, the start position and the end position of the subject are respectively input into each binary classification module to obtain the candidate prediction information output by each binary classification module. The candidate prediction information includes the start position and the end position of an object, as well as the relationship between the subject and the object.

[0217] Filter out erroneous candidate predictions and determine at least one remaining candidate prediction as the at least one prediction output by the relationship prediction module.

[0218] In some embodiments, the fusion module is used to concatenate the character features and the radical features to obtain the fused features.

[0219] The specific functions and examples of each module and sub-module of the apparatus in this application embodiment can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0220] Based on the same inventive concept, this application also provides an electronic device, a readable storage medium, and a computer program product.

[0221] Figure 13 A schematic block diagram of an example electronic device 1300 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0222] like Figure 13 As shown, device 1300 includes a computing unit 1301, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1302 or a computer program loaded from storage unit 1308 into random access memory (RAM) 1303. The RAM 1303 may also store various programs and data required for the operation of device 1300. The computing unit 1301, ROM 1302, and RAM 1303 are interconnected via bus 1304. Input / output (I / O) interface 1305 is also connected to bus 1304.

[0223] Multiple components in device 1300 are connected to I / O interface 1305, including: input unit 1306, such as keyboard, mouse, etc.; output unit 1307, such as various types of monitors, speakers, etc.; storage unit 1308, such as disk, optical disk, etc.; and communication unit 1309, such as network card, modem, wireless transceiver, etc. Communication unit 1309 allows device 1300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0224] The computing unit 1301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1301 performs the various methods and processes described above, such as the entity relation extraction method for power equipment. For example, in some embodiments, the entity relation extraction method for power equipment may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1300 via ROM 1302 and / or communication unit 1309. When the computer program is loaded into RAM 1303 and executed by the computing unit 1301, one or more steps of the entity relation extraction method for power equipment described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured by any other suitable means (e.g., by means of firmware) to perform an entity relation extraction method for power devices.

[0225] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0226] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0227] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0228] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0229] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0230] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0231] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0232] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this application should be included within the scope of protection of this application.

Claims

1. A method for extracting entity relationships for power equipment, characterized in that, The method comprises: obtaining text information describing power equipment; extracting a to-be-processed sentence from the text information; character feature extraction is performed on the characters of the to-be-processed sentence to obtain character features; the character features are input into a text denoising model to obtain the label types of each character in the to-be-processed sentence, wherein the label types are selected from a type set, and the type set includes the following label types: related to power equipment and unrelated to power equipment; the label types of the characters are concatenated in the order of their positions in the to-be-processed sentence to obtain a label sequence; each label type in the label sequence is converted into a digital quantity based on a conversion function to obtain a denoised attention vector, wherein the conversion function is used to enhance the features related to power equipment and weaken the features unrelated to power equipment; for a type predicted to be related to power equipment, the conversion function uses the confidence thereof as the digital quantity of the label type, and for a type unrelated to power equipment, 1-confidence is used as the digital quantity of the label type; the attention vector and the character features are multiplied to obtain denoised character features; radical feature extraction is performed on the to-be-processed sentence to obtain radical features; the denoised character features and the radical features are fused to obtain fused features; the fused features are input into an entity relationship extraction network to obtain an entity relationship extracted from the to-be-processed sentence.

2. The method of claim 1, wherein, The character feature extraction on the to-be-processed sentence comprises: the to-be-processed sentence is segmented according to Chinese characters to obtain a character sequence; the character sequence is input into a first natural language feature extraction model to obtain the character features of the to-be-processed sentence.

3. The method of claim 1, wherein, The radical feature extraction on the to-be-processed sentence comprises: Chinese radicals contained in the to-be-processed sentence are extracted to obtain a radical sequence; the radical sequence is input into a second natural language feature extraction model to obtain the radical features of the to-be-processed sentence.

4. The method according to any one of claims 1 to 3, characterized in that, The entity relationship extraction network comprises a subject position prediction module and a relationship prediction module, and the input of the fused features into the entity relationship extraction network to obtain an entity relationship extracted from the to-be-processed sentence comprises: the fused features are input into the subject position prediction module to obtain the start position and the end position of a subject; the fused features, the start position and the end position of the subject are input into the relationship prediction module to obtain at least one piece of predicted information output by the relationship prediction module, wherein each piece of the predicted information includes the start position and the end position of an object, and the relationship between the subject and the object; a subject is extracted from the to-be-processed sentence based on the start position and the end position of the subject; each object is extracted from the to-be-processed sentence based on the start position and the end position of each object; for each object, an entity relationship including the subject, the object, and the relationship between the subject and the object is constructed.

5. The method of claim 4, wherein, The relationship prediction module comprises a plurality of binary classification modules for predicting different relationships, the fusion feature, the start position and the end position of the subject are input into the relationship prediction module, and at least one piece of prediction information output by the relationship prediction module is obtained, comprising: The fusion feature, the start position and the end position of the subject are input into each binary classification module respectively, and candidate prediction information output by each binary classification module is obtained, the candidate prediction information comprising the start position and the end position of an object, and the relationship between the subject and the object; The incorrect candidate prediction information is filtered out, and the remaining at least one piece of candidate prediction information is determined as the at least one piece of prediction information output by the relationship prediction module.

6. The method according to any one of claims 1-3, characterized in that, The character feature and the component feature are fused to obtain the fusion feature, comprising: The character feature and the component feature are spliced to obtain the fusion feature.

7. An entity relation extraction apparatus for power equipment, characterized by comprising: The device comprises: A text acquisition module is configured to acquire text information describing power equipment; A sentence extraction module is configured to extract a to-be-processed sentence from the text information; A character feature extraction module is configured to extract a character feature of the to-be-processed sentence; The character feature is input into a text denoising model to obtain a label type of each character in the to-be-processed sentence, wherein the label type is selected from a type set, and the type set comprises the following label types: related to power equipment and unrelated to power equipment; The label types of the characters are spliced in the order of positions in the to-be-processed sentence to obtain a label sequence; Each label type in the label sequence is converted into a digital quantity based on a conversion function to obtain a denoised attention vector, wherein the conversion function is used to enhance the feature related to power equipment and weaken the feature unrelated to power equipment; for a type predicted to be related to power equipment, the conversion function uses the confidence thereof as the digital quantity of the label type, and for a type unrelated to power equipment, 1-confidence is used as the digital quantity of the label type; The denoised attention vector and the character feature are multiplied to obtain a denoised character feature; A component feature extraction module is configured to extract a component feature of the to-be-processed sentence; A fusion module is configured to fuse the denoised character feature and the component feature to obtain a fusion feature; A relationship extraction module is configured to input the fusion feature into an entity relationship extraction network to obtain an entity relationship extracted from the to-be-processed sentence. The electronic device comprises:

8. An electronic device, comprising: A memory is configured to store a program; and a processor is configured to execute the power equipment-oriented entity relationship extraction method by calling the program stored in the memory. The computer readable storage medium stores computer instructions for causing the computer to execute the power equipment-oriented entity relationship extraction method.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing the computer to execute the power equipment-oriented entity relationship extraction method.

Images