Power network asset key feature extraction method based on pseudo-twinning neural network
By using a pseudo-twin neural network in the extraction of power network asset information, combining LSTM and convolutional attention network to process text and image data, and fusing the two through similar anastomosis judgment layers, it is solved that it is difficult to integrate heterogeneous data in the prior art. The key features of the power network assets are achieved more accurately and comprehensively extracted.
Patent Information
- Application Number
- CN202411270840.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art is difficult to effectively integrate heterogeneous data of power network assets, especially when processing the combination of text and image data, and it is difficult to accurately extract key features.
Using a pseudo-twin neural network method, a dual-branch neural network with LSTM and convolutional attention network is constructed, text and image data are processed respectively, and the two are fused through a similar anastomosis judgment layer to achieve the consistency judgment of text and image.
It significantly improves the accuracy and comprehensiveness of the extraction of asset information in power networks, can more accurately judge the consistency between text description and image, and improves the practicality of feature extraction.
Smart Images
Figure CN119441819A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning, and in particular, to a method for extracting key features of power network assets based on a pseudo-twin neural network. Background Art
[0002] The information of power network assets is usually distributed on various web pages, including text descriptions (such as equipment models, technical parameters, instructions, etc.) and image data (such as equipment photos, schematic diagrams, etc.). Traditional methods are difficult to effectively integrate these heterogeneous data and extract accurate key features from them.
[0003] Existing methods usually rely on manual labeling and classification of power asset information, which is not only time-consuming and labor-intensive, but also prone to errors. Especially when faced with large-scale data, the efficiency and accuracy of manual methods are difficult to meet the needs; many key information is often a combination of text and images, such as a text description of a device and a corresponding image diagram. Existing machine learning models can usually only process a single type of data (such as plain text or plain images), and it is difficult to effectively process and fuse these two types of data, making it impossible to accurately determine whether a text description matches the corresponding image.
[0004] Based on the above background, the present invention provides a method for extracting key features of power network assets based on pseudo twin neural networks. The present invention constructs a pseudo twin neural network to process text and image data separately, so that the model can simultaneously understand and integrate the information of these two modalities; by combining LSTM and convolutional attention network, LSTM is used to process text data to capture important information and contextual relationships in text sequences; convolutional neural network (CNN) is combined with attention mechanism to process image data to highlight the key parts related to power assets in the image; the consistency between text description and image content is judged by similarity matching judgment layer. In this way, it is possible to judge not only whether the text and image each contain information about power assets, but also whether the two describe the same asset, thereby improving the comprehensiveness and accuracy of feature extraction. Summary of the invention
[0005] The purpose of the embodiment of the present application is to provide a method for extracting key features of power network assets based on a pseudo-twin neural network. The present invention proposes a method for extracting key features of power network assets based on a pseudo-twin neural network. By integrating LSTM (Long Short-Term Memory Network) and convolutional attention mechanism (CNN and Attention), the method can process text and image data simultaneously, and effectively fuse the two types of data through a similarity matching judgment layer, significantly improving the accuracy and comprehensiveness of power network asset information extraction.
[0006] To achieve the above objectives, this application provides the following technical solutions:
[0007] In the first aspect, the present invention discloses a method for extracting key features of power network assets based on a pseudo-twin neural network, comprising the following steps: S1. Using a crawler tool or an API interface to capture web pages containing power system asset information, annotate and preprocess the data, and construct a training data set; S2. Constructing a pseudo-twin neural network based on an LSTM (long short-term memory network) branch and a convolutional attention network branch; S3. Using the constructed data set to train the pseudo-twin neural network model until the model converges; S4. Using the trained pseudo-twin neural network model to extract key features of power network assets in new web pages.
[0008] More specifically, the above-mentioned method for extracting key features of power network assets based on pseudo-twin neural network is characterized in that the preprocessing described in step S1 includes: 1. Text data preprocessing, extracting text data from web pages, cleaning text, removing irrelevant characters and stop words, and converting text into vector representation; 2. Image data preprocessing, extracting image data from web pages, adjusting image size, ensuring that all images are of the same size, performing data enhancement, and normalizing pixel values.
[0009] More specifically, the above-mentioned method for extracting key features of power network assets based on pseudo-twin neural network is characterized in that, in step S1, the training data set includes a pure text data set, a pure image data set, and a text plus image data set; the sample labels of the pure text data set are text labels; the sample labels of the pure image data set are image labels, and the labels of the text plus image data set include text labels, image labels, and text-image matching labels.
[0010] More specifically, the above-mentioned method for extracting key features of power network assets based on pseudo-twin neural network is characterized in that, in step S2, a pseudo-twin neural network based on LSTM (long short-term memory network) branch and convolutional attention network branch is constructed; the pseudo-twin neural network includes: a first branch neural network, a second branch neural network, and a similarity matching judgment layer.
[0011] The first branch of the neural network is used to process text data in web pages, including the LSTM feature extraction layer and the text feature determination layer, which aims to determine the power network assets based on the text.
[0012] The second branch of the neural network is used to process image data in web pages, including a feature extraction layer that combines CNN (convolutional neural network) and Attention (attention mechanism) and an image feature determination layer, aiming to determine the power network assets of the image.
[0013] Similarity matching judgment layer: The text feature zt extracted by the LSTM feature extraction layer of the first branch neural network and the image feature zp extracted by the feature extraction layer of the second branch neural network combined with CNN and Attention are judged to match and evaluate the matching between the text and the image.
[0014] The pseudo-twin neural network supports text-only input, image-only input, and text and image combined input.
[0015] More specifically, the above-mentioned method for extracting key features of power network assets based on pseudo-twin neural network is characterized in that the loss of the pseudo-twin neural network consists of three parts, including the first branch text feature loss Ltext, which is used to judge the power network assets on the text; the second branch image feature loss Limage, which is used to judge the power network assets on the image; the twin similarity matching loss Lsimilarity, which is used to evaluate the correlation between text and image, that is, whether the power assets described in the text are consistent with the corresponding image.
[0016] More specifically, the above-mentioned method for extracting key features of power network assets based on pseudo twin neural network is characterized in that the calculation of the first branch text feature loss Ltext is as follows:
[0017]
[0018] N represents the number of samples; yi represents the text label of sample i, 1 represents that the text information contains power network asset information, 0 represents that the text information does not contain power network asset information, and y'i represents the predicted probability of the text information of sample i.
[0019] More specifically, the above-mentioned method for extracting key features of power network assets based on pseudo twin neural network is characterized in that the calculation of the second branch image feature loss Limage is as follows:
[0020]
[0021] N represents the number of samples; pi represents the image label of sample i, 1 represents that the image contains power network asset information, 0 represents that the image does not contain power network asset information, and p'i represents the image prediction probability of sample i.
[0022] More specifically, the above-mentioned method for extracting key features of power network assets based on pseudo twin neural network is characterized in that the twin similarity loss Lsimilarity is calculated as follows:
[0023]
[0024] N represents the number of samples; si represents the matching label of sample i, that is, whether the text information and the image match, 1 represents matching, and 0 represents non-matching; Represents the text features of sample i and image features The distance between them; m is a threshold used to control the minimum distance between negative samples (text information and images do not match).
[0025] More specifically, the above-mentioned method for extracting key features of power network assets based on pseudo twin neural network is characterized in that the total loss Ltotal of the pseudo twin neural network is calculated as follows:
[0026] Ltotal=αLtext+βLimage+γLsimilarity,
[0027] Among them, α, β, and γ are weight coefficients used to balance the importance of different loss functions.
[0028] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention can more comprehensively extract the key features of power network assets by simultaneously processing text and image data through the dual-branch neural network of the pseudo-twin neural network; by jointly optimizing multiple loss functions (text branch loss, image branch loss, twin similarity matching loss), the overall performance of the model can be improved; the model can perform feature extraction while also identifying the key features of power network assets, thereby improving the practicality of the model. The present invention can be applied to different types of data sources, including power system asset information from multiple sources such as official websites, social media, and industry reports. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0030] Figure 1 A flowchart of a method for extracting key features of power network assets based on pseudo-twin neural network provided for the application.
[0031] Figure 2 A structural diagram of a pseudo-twin neural network model provided for the application. DETAILED DESCRIPTION
[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0033] The terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0034] The terms "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and should not be understood as indicating or implying relative importance, nor should they be understood as requiring or implying any such actual relationship or order between these entities or operations.
[0035] The basic idea of the present invention is to use crawler tools or API interfaces to capture web pages containing power system asset information, annotate and preprocess the data, and construct a training data set; construct a pseudo-twin neural network based on LSTM (long short-term memory network) branches and convolutional attention network branches; use the constructed data set to train the pseudo-twin neural network model until the model converges; use the trained pseudo-twin neural network model to extract key features of power network assets in new web pages. By constructing a pseudo-twin neural network, text and image data are processed separately, so that the model can simultaneously understand and integrate information from these two modalities; by combining LSTM and convolutional attention networks, LSTM is used to process text data and capture important information and contextual relationships in text sequences; convolutional neural networks are combined with attention mechanisms to process image data and highlight key parts of images related to power assets; in this way, the model can more accurately extract features related to power network assets in text and images.
[0036] like Figure 1 As shown, the present invention provides a method for extracting key features of power network assets based on a pseudo twin neural network, comprising the following steps:
[0037] S1. Use crawler tools or API interfaces to crawl web pages containing power system asset information, annotate and preprocess the data, and build a training data set;
[0038] 1. Text data preprocessing: extract text data from web pages, clean the text, remove irrelevant characters and stop words, and convert the text into a vector representation;
[0039] 2. Image data preprocessing: extract image data from web pages, adjust image size, ensure that all images are of the same size, perform data enhancement, and normalize pixel values;
[0040] 3. Construct training data sets including pure text data sets, pure image data sets and text plus image data sets; the sample labels of the pure text data sets are text labels, the sample labels of the pure image data sets are image labels, and the labels of the text plus image data sets include text labels, image labels and text-image matching labels.
[0041] sample Data Types Data Labels Sample 1 text Text Labels Sample 2 image Image Tags Sample 3 Text plus image Text labels, image labels, matching labels ...... ...... ......
[0042] S2. Construct a pseudo-twin neural network based on LSTM (long short-term memory network) branch and convolutional attention network branch;
[0043] like Figure 2 As shown, the pseudo-twin neural network includes: a first branch neural network, a second branch neural network, and a similarity matching judgment layer.
[0044] The first branch of the neural network is used to process text data in web pages, including the LSTM feature extraction layer and the text feature determination layer, which is designed to determine the power network assets based on the text;
[0045] The calculation of the first branch text feature loss Ltext is as follows:
[0046]
[0047] N represents the number of samples; yi represents the text label of sample i, 1 represents that the text information contains power network asset information, 0 represents that the text information does not contain power network asset information, and y'i represents the predicted probability of the text information of sample i.
[0048] The second branch of the neural network is used to process image data in web pages. It includes a feature extraction layer that combines CNN (convolutional neural network) and Attention (attention mechanism) and an image feature determination layer, which aims to determine the power network assets of the image.
[0049] The calculation of the second branch image feature loss Limage is as follows:
[0050]
[0051] N represents the number of samples; pi represents the image label of sample i, 1 represents that the image contains power network asset information, 0 represents that the image does not contain power network asset information, and p'i represents the image prediction probability of sample i.
[0052] Similarity matching judgment layer: The text feature zt extracted by the LSTM feature extraction layer of the first branch neural network and the image feature zp extracted by the feature extraction layer of the second branch neural network combined with CNN and Attention are judged to match and evaluate the matching between the text and the image.
[0053] The twin similarity loss Lsimilarity is calculated as follows:
[0054]
[0055] N represents the number of samples; si represents the matching label of sample i, that is, whether the text information and the image match, 1 represents matching, and 0 represents mismatch; d(zit,zip) represents the distance between the text feature zit and the image feature zip of sample i; m is a threshold used to control the minimum distance between negative samples (text information and images do not match).
[0056] The total loss Ltotal of the pseudo-twin neural network is calculated as follows:
[0057] Ltotal=αLtext+βLimage+γLsimilarity,
[0058] Among them, α, β, and γ are weight coefficients used to balance the importance of different loss functions; the pseudo-twin neural network supports text-only input, image-only input, and text and image combined input.
[0059] S3. Use the constructed data set to train the pseudo-twin neural network model until the model converges;
[0060] S4. Use the trained pseudo-twin neural network model to extract key features of power network assets in new web pages.
[0061] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for extracting key features of power network assets based on pseudo-twin neural network, characterized in that: The following steps are involved: S1. Use crawler tools or API interfaces to crawl web pages containing power system asset information, annotate and preprocess the data, and build a training data set; S2. Construct a pseudo-twin neural network based on LSTM (long short-term memory network) branch and convolutional attention network branch; S3. Use the constructed data set to train the pseudo-twin neural network model until the model converges; S4. Use the trained pseudo-twin neural network model to extract key features of power network assets in new web pages.
2. According to claim 1, a method for extracting key features of power network assets based on pseudo-twin neural network is characterized in that: In the step S1, a crawler tool or an API interface is used to crawl web pages containing power system asset information, the data is annotated and preprocessed, and a training data set is constructed; the preprocessing includes:
1. text data preprocessing, extracting text data from web pages, cleaning text, removing irrelevant characters and stop words, and converting text into vector representation; 2. image data preprocessing, extracting image data from web pages, adjusting image size, ensuring that all images are of the same size, performing data enhancement, and normalizing pixel values.
3. According to claim 1, a method for extracting key features of power network assets based on pseudo twin neural network is characterized in that: The training data set includes a pure text data set, a pure image data set and a text plus image data set; the sample labels of the pure text data set are text labels; the sample labels of the pure image data set are image labels, and the labels of the text plus image data set include text labels, image labels and text-image matching labels.
4. According to claim 1, a method for extracting key features of power network assets based on pseudo-twin neural network is characterized in that: In the step S2, a pseudo twin neural network based on an LSTM (long short-term memory network) branch and a convolutional attention network branch is constructed; The pseudo twin neural network includes: a first branch neural network, a second branch neural network, and a similarity matching judgment layer. The first branch of the neural network is used to process text data in web pages, including the LSTM feature extraction layer and the text feature determination layer, which aims to determine the power network assets based on the text. The second branch of the neural network is used to process image data in web pages, including a feature extraction layer that combines CNN (convolutional neural network) and Attention (attention mechanism) and an image feature determination layer, aiming to determine the power network assets of the image. Similarity matching judgment layer: The text feature zt extracted by the LSTM feature extraction layer of the first branch neural network and the image feature zp extracted by the feature extraction layer of the second branch neural network combined with CNN and Attention are judged to match and evaluate the matching between the text and the image. The pseudo-twin neural network supports text-only input, image-only input, and text and image combined input.
5. The method for extracting key features of power network assets based on pseudo twin neural network according to claim 1 is characterized in that: The loss of the pseudo-twin neural network consists of three parts, including the first branch text feature loss Ltext, which is used to judge the power network assets of the text; the second branch image feature loss Limage, which is used to judge the power network assets of the image; the twin similarity matching loss Lsimilarity, which is used to evaluate the correlation between the text and the image, that is, whether the power assets described in the text are consistent with the corresponding image.
6. The method for extracting key features of power network assets based on pseudo twin neural network according to claim 1 is characterized in that: The calculation of the first branch text feature loss Ltext is as follows: N represents the number of samples; yi represents the text label of sample i, 1 represents that the text information contains power network asset information, 0 represents that the text information does not contain power network asset information, and y'i represents the predicted probability of the text information of sample i.
7. The method for extracting key features of power network assets based on pseudo twin neural network according to claim 3 is characterized in that: The calculation of the second branch image feature loss Limage is as follows: N represents the number of samples; pi represents the image label of sample i, 1 represents that the image contains power network asset information, 0 represents that the image does not contain power network asset information, and p'i represents the image prediction probability of sample i.
8. The method for extracting key features of power network assets based on pseudo twin neural network according to claim 3 is characterized in that: The twin similarity loss Lsimilarity is calculated as follows: N represents the number of samples; si represents the matching label of sample i, that is, whether the text information and the image match, 1 represents matching, and 0 represents non-matching; Represents the text features of sample i and image features The distance between m is a threshold used to control the minimum distance between negative samples (text information and images do not match).
9. The method for extracting key features of power network assets based on pseudo-twin neural network according to claim 1 is characterized in that: The total loss Ltotal of the pseudo-twin neural network is calculated as follows: Ltotal=αLtext+βLimage+γLsimilarity, Among them, α, β, and γ are weight coefficients used to balance the importance of different loss functions.