Power network asset discrimination method and system based on multi-task and hierarchical multi-modal decision
Patent Information
- Application Number
- CN202411270841.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-12-03
AI Technical Summary
The existing power network asset classification technology has problems such as single-modal limitations, single decision-making strategies and insufficient processing efficiency, making it difficult to effectively process complex multimodal data and meet real-time requirements.
Using a parallel processing mechanism, text and image samples are input into independent multi-task learning models, and the classification results of multiple modalities are combined through a hierarchical multi-modal decision-making strategy, considering the correlation between text and image samples and asset level. Weight, realize the precise judgment of power network assets.
The classification accuracy of power network assets and the real-time response capability of the system are improved, and the balance between real-time and accuracy of power network assets is achieved.
Smart Images

Figure CN119066474A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning, and in particular, to a method and system for identifying power network assets based on multi-task and hierarchical multimodal decision-making. Background Art
[0002] As power systems become increasingly complex and networked, asset management and security monitoring of power networks have become particularly important. However, the content of web pages related to power systems is complex and diverse, containing a large amount of text and image data. Traditional classification methods are difficult to effectively process these multimodal data. In addition, since power network assets involve assets of different levels such as core, key, and peripheral, their classification and monitoring place higher requirements on the real-time and accuracy of the system.
[0003] Existing power network asset classification technologies often have the following problems: 1. Single-mode limitation: Many methods only focus on one mode in text or images, ignoring the diversity of power network-related web page content, resulting in incomplete classification results. 2. Single decision strategy: Traditional classification methods mostly use simple decision rules, which are difficult to adapt to complex asset level identification requirements. 3. Insufficient processing efficiency: When processing a large number of complex web pages, the processing efficiency of traditional methods is often difficult to meet the real-time requirements, affecting the response speed of the system.
[0004] Based on the above background, the present invention provides a method and system for distinguishing power network assets based on multi-task and hierarchical multimodal decision-making. The present invention adopts a parallel processing mechanism to process multiple text and image samples at the same time, which greatly improves the processing speed and real-time response capability of the system; by inputting text and image samples into independent multi-task learning models respectively, the category and authenticity of web page content are captured, and each model focuses on processing data of a specific modality, thereby improving the classification accuracy of power network assets; by introducing a hierarchical multimodal decision-making strategy, the correlation and importance between text and image samples, as well as the weights of different asset levels are fully considered. This strategy can integrate the classification results of multiple modalities to make more accurate asset level judgments. The present invention achieves a balance between real-time and accuracy in distinguishing power network assets through parallel processing, multi-task learning models and hierarchical multimodal decision-making mechanisms. Summary of the invention
[0005] The purpose of the embodiment of the present application is to provide a method and system for distinguishing power network assets based on multi-task and hierarchical multimodal decision-making. The present invention inputs text and image samples into independent multi-task learning models respectively, taking into account both authenticity identification and asset grade classification. Each model focuses on processing data of a specific modality, thereby improving the classification accuracy of power network assets; by introducing a hierarchical multimodal decision-making strategy, the relevance and importance between text and image samples, as well as the weights of different asset grades, are fully considered. This strategy integrates the classification results of multiple modalities to achieve more accurate asset grade discrimination.
[0006] To achieve the above objectives, this application provides the following technical solutions:
[0007] In a first aspect, the present invention discloses a method for distinguishing electric power network assets based on multi-task and hierarchical multimodal decision making, comprising the following steps: S1, using a crawler tool to crawl the HTML content of a target electric power system-related web page, extracting text paragraphs and images, and constructing a text data set and an image data set; S2, using the constructed text and image data sets to respectively train a text multi-task learning model and an image multi-task learning model; S3, optimizing a hierarchical multimodal decision strategy based on the classification results of a single sample based on the multi-task model; S4, dividing a new electric power system-related web page into multiple text samples and image samples;
[0008] S5. Input the samples in parallel into the trained multi-task learning model of text and image to obtain the preliminary classification results of each sample; S6. Use the optimized hierarchical multimodal decision strategy to perform a fusion evaluation on the classification results of all text and image samples; S7. Output the classification results of the power assets to which the entire webpage belongs.
[0009] More specifically, the above-mentioned method for identifying power network assets based on multi-task and hierarchical multimodal decision-making is characterized in that the extraction of text paragraphs in step S1 includes: cleaning HTML tags and removing noise data, such as scripts, advertisements, etc.; performing pre-processing operations such as word segmentation, part-of-speech tagging, and removal of stop words on the text; segmenting the text, extracting key paragraphs, and forming independent text samples. The image extraction includes pre-processing such as cropping and scaling to unify the size of all images. The text data set and image data set are constructed, and authenticity labels and category labels are added to each text and image sample. The authenticity label is whether it is a power system-related network asset, 1 indicates that it is a power system-related network asset, and 0 indicates that it is not a power system-related network asset. The category label is marked as core assets, key assets, peripheral assets, or counterfeit assets according to the content.
[0010] More specifically, the above-mentioned power network asset identification method based on multi-task and hierarchical multimodal decision-making is characterized in that the text multi-task learning model is based on the BERT pre-trained language model, the image multi-task learning model is a convolutional neural network based on adaptive attention, and the multi-task training goal is asset authenticity identification and asset classification.
[0011] More specifically, the above-mentioned method for identifying power network assets based on multi-task and hierarchical multi-modal decision-making is characterized in that the multi-modal decision-making strategy includes the following steps:
[0012] 1. For each asset level k, calculate the preliminary scores of the text samples and image samples in the same web page for that level respectively and
[0013] 2. Combine the scores of text samples and image samples to obtain the preliminary score Sk of asset level k;
[0014] 3. According to the importance of asset level k in the web page and the authenticity ratio of the sample, the preliminary score is weighted and adjusted to obtain the final comprehensive score of asset level k
[0015] 4. The comprehensive score of each asset level k Perform normalization processing;
[0016] 5. The highest scoring asset class Total score Compared with the decision threshold θf, when , the webpage is classified as this asset level, otherwise the webpage is marked as “undetermined”.
[0017] More specifically, the above-mentioned method for identifying power network assets based on multi-task and hierarchical multimodal decision-making is characterized in that the asset level k score of the text sample in the web page is calculated as follows:
[0018]
[0019] N T represents the number of text samples in a web page, T represents the set of text samples in a web page, and αk represents the weight of asset level k in the text sample; the asset level k score of the image sample in the web page is calculated as follows:
[0020]
[0021] N Irepresents the number of image samples in the web page, I represents the set of image samples in the web page, and βk represents the weight of asset level k in the image samples; the preliminary score Sk of the asset level k is calculated as follows:
[0022]
[0023] More specifically, the above-mentioned method for identifying power network assets based on multi-task and hierarchical multimodal decision-making is characterized in that the preliminary score is weighted and adjusted, and the calculation method is as follows:
[0024]
[0025] ωk is the relative importance of asset level k, γk is the sample proportion of real power network assets in the samples judged as level k on the web page, and Pk is the proportion of samples judged as level k in the web page to the total samples. The comprehensive score After normalization, the calculation method is as follows:
[0026]
[0027] There are 4 asset classes in total.
[0028] More specifically, the above-mentioned method for identifying power network assets based on multi-task and hierarchical multimodal decision-making is characterized in that the core assets indicate that the power asset information contained in the web page has high sensitivity and importance; the key assets indicate that the power asset content contained in the web page is important but not as sensitive as the core assets; the peripheral assets indicate that the power assets contained in the web page are relatively minor; and the counterfeit assets indicate that the power asset content contained in the web page is unreliable and contains false information.
[0029] More specifically, the above-mentioned method for identifying power network assets based on multi-task and hierarchical multimodal decision-making is characterized in that if the decision strategy determines that the result is "undetermined", the web page and all the text samples and image samples it contains are recorded and backed up and submitted for manual review.
[0030] In a second aspect, the present invention discloses a power network asset identification system based on multi-task and hierarchical multi-modal decision-making, including a data acquisition module, a parallel processing module, a multi-task classification module, a multi-level decision module and a identification module;
[0031] Data acquisition module: used to grab data from the target power system-related web pages, parse the HTML structure of the web pages, extract text paragraphs and images related to the power system, and form independent samples; Parallel processing module: used to manage the parallel processing capabilities of the system system, receive independent samples uploaded by the data acquisition module, and use multi-core GPUs to execute multi-task classification modules in parallel to process each text paragraph and each image sample;
[0032] Multi-task classification module: used to extract features from text and image data in web pages, determine whether they are related to power assets and classify power assets, and send the output results to the multi-level decision module; Multi-level decision module: used to perform preliminary fusion of the classification results of samples (text or images) in web pages. Use weighted decision fusion strategy to generate the overall classification results of web pages;
[0033] Discrimination and log management module: used to output power system web asset classification results for use in security monitoring, risk assessment and other business processes. At the same time, the classification results and related data are logged for subsequent analysis, tuning and model improvement.
[0034] Compared with the prior art, the present invention has the following beneficial effects: by inputting text and image samples into independent multi-task learning models, the present invention takes into account both authenticity identification and asset grade classification, and each model focuses on processing data of a specific modality, thereby improving the classification accuracy of power network assets; by introducing a hierarchical multimodal decision strategy, the relevance and importance between text and image samples, as well as the weights of different asset grades, are fully considered. This strategy integrates the classification results of multiple modalities to achieve more accurate asset grade discrimination. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0036] Figure 1 A flowchart of a power network asset identification method based on multi-task and hierarchical multimodal decision-making is provided for the application.
[0037] Figure 2 A module structure diagram of a power network asset identification system based on multi-task and hierarchical multimodal decision-making provided for the application. DETAILED DESCRIPTION
[0038] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0039] The terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0040] The terms "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and should not be understood as indicating or implying relative importance, nor should they be understood as requiring or implying any such actual relationship or order between these entities or operations.
[0041] The basic idea of the present invention is to use a crawler tool to crawl the HTML content of the target power system-related web page, extract text paragraphs and images, and construct a text data set and an image data set; use the constructed text and image data sets to train the text multi-task learning model and the image multi-task learning model respectively; optimize the hierarchical multimodal decision strategy based on the classification results of a single sample of the multi-task model; divide the new power system-related web page into multiple text samples and image samples; input the samples in parallel into the trained text and image multi-task learning model to obtain the preliminary classification results of each sample; use the optimized hierarchical multimodal decision strategy to perform a fusion evaluation on the classification results of all text and image samples; and output the classification results of the power assets to which the entire web page belongs. Construct an independent multi-task learning model for text and image samples, taking into account both authenticity identification and asset grade classification. Each model focuses on processing data of a specific modality, thereby improving the classification accuracy of power network assets; by introducing a hierarchical multimodal decision strategy, fully consider the correlation and importance between text and image samples, as well as the weights of different asset grades. This strategy integrates the classification results of multiple modalities to achieve more accurate asset grade discrimination.
[0042] like Figure 1 As shown, the present invention provides a method for identifying power network assets based on multi-task and hierarchical multi-modal decision-making, comprising the following steps:
[0043] S1. Use crawler tools to crawl the HTML content of target power system related web pages, extract text paragraphs and images, and construct text data sets and image data sets;
[0044] 8. Extracting text paragraphs includes: cleaning HTML tags, removing noise data, such as scripts, advertisements, etc.; performing preprocessing operations such as word segmentation, part-of-speech tagging, and removing stop words on the text; segmenting the text, extracting key paragraphs, and forming independent text samples. Extracting images includes preprocessing such as cropping and scaling to unify the size of all images. Construct text data sets and image data sets, and label each text and image sample with authenticity and category labels. The authenticity label is whether it is a power system-related network asset. 1 indicates that it is a power system-related network asset, and 0 indicates that it is not a power system-related network asset. Label the category label according to the content: core assets, key assets, peripheral assets, or counterfeit assets. Core assets indicate that the power asset information contained in the web page has high sensitivity and importance; key assets indicate that the power asset content contained in the web page is important, but not as sensitive as core assets; peripheral assets indicate that the power assets contained in the web page are relatively minor; counterfeit assets indicate that the power asset content contained in the web page is unreliable and contains false information.
[0045] S2. Use the constructed text and image datasets to train the text multi-task learning model and the image multi-task learning model respectively;
[0046] The text multi-task learning model uses a BERT pre-trained language model, and the image multi-task learning model uses a convolutional neural network based on adaptive attention. The multi-task training objectives are asset authenticity identification and asset classification.
[0047] S3. Optimizing the hierarchical multimodal decision strategy for the classification results of a single sample based on the multi-task model; the multimodal decision strategy includes the following steps:
[0048] 1. For each asset level k, calculate the preliminary scores of the text samples and image samples in the same web page for that level respectively and
[0049] The asset level k-score of a text sample is calculated as follows:
[0050]
[0051] N T represents the number of text samples in a web page, T represents the set of text samples in a web page, and αk represents the weight of asset level k in the text sample;
[0052] The asset level k-score of an image sample is calculated as follows:
[0053]
[0054] N I represents the number of image samples in the web page, I represents the set of image samples in the web page, and βk represents the weight of asset level k in the image sample.
[0055] 2. Combine the scores of text samples and image samples to obtain the preliminary score Sk of asset level k;
[0056] The preliminary score Sk for asset class k is calculated as follows:
[0057] 3. According to the importance of asset level k in the web page and the authenticity ratio of the sample, the preliminary score is weighted and adjusted to obtain the final comprehensive score of asset level k
[0058] A weighted adjustment is made to the preliminary score, calculated as follows:
[0059]
[0060] ωk is the relative importance of asset level k, γk is the sample proportion of real power network assets among the samples judged as level k on the web page, and Pk is the proportion of samples judged as level k on the web page to the total samples.
[0061] 4. The comprehensive score of each asset level k Perform normalization processing;
[0062] The calculation method is as follows: There are 4 asset classes in total.
[0063] 5. The highest scoring asset class Total score Compared with the decision threshold θf, when , the webpage is classified as this asset level, otherwise the webpage is marked as “undetermined”.
[0064] If the decision strategy determines that the result is "undetermined", the web page and all the text samples and image samples it contains will be backed up and submitted for manual review.
[0065] S4, dividing the new power system related web page into multiple text samples and image samples;
[0066] S5. Input the samples into the trained multi-task learning model of text and image in parallel to obtain the preliminary classification results of each sample;
[0067] S6. Use the optimized hierarchical multimodal decision strategy to perform fusion evaluation on the classification results of all text and image samples;
[0068] S7. Output the classification result of the electric power assets to which the entire webpage belongs.
[0069] like Figure 2 As shown, the present invention provides a power network asset identification system based on multi-task and hierarchical multi-modal decision-making, including a data acquisition module, a parallel processing module, a multi-task classification module, a multi-level decision module and a identification module;
[0070] Data acquisition module: used to capture data from target power system-related web pages, parse the HTML structure of the web pages, extract text paragraphs and images related to the power system, and form independent samples;
[0071] Parallel processing module: used to manage the parallel processing capability of the system, receive independent samples uploaded by the data acquisition module, use multi-core GPU to execute multi-task classification modules in parallel, and process each text and each image sample;
[0072] Multi-task classification module: used to extract features from text and image data in web pages, determine whether they are related to power assets and classify power assets, and send the output results to the multi-level decision-making module;
[0073] Multi-level decision module: used to initially integrate the classification results of samples (text or images) in web pages. Use weighted decision fusion strategy to generate the overall classification results of web pages;
[0074] Discrimination and log management module: used to output power system web asset classification results for use in security monitoring, risk assessment and other business processes. At the same time, the classification results and related data are logged for subsequent analysis, tuning and model improvement.
[0075] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for identifying power network assets based on multi-task and hierarchical multi-modal decision making, characterized in that: The following steps are involved: S1. Use crawler tools to crawl the HTML content of target power system related web pages, extract text paragraphs and images, and construct text data sets and image data sets; S2. Use the constructed text and image datasets to train the text multi-task learning model and image multi-task learning model respectively; S3, optimize the hierarchical multimodal decision strategy for the classification results of single samples based on the multi-task model; S4, dividing the new power system related web page into multiple text samples and image samples; S5. Input the samples into the trained multi-task learning model of text and image in parallel to obtain the preliminary classification results of each sample; S6. Use the optimized hierarchical multimodal decision strategy to perform fusion evaluation on the classification results of all text and image samples; S7. Output the classification result of the electric power assets to which the entire webpage belongs.
2. The method for identifying power network assets based on multi-task and hierarchical multi-modal decision-making according to claim 1 is characterized in that: In step S1, a crawler tool is used to crawl the HTML content of the target power system-related web page, extract text paragraphs and images, and construct a text data set and an image data set; the extraction of text paragraphs includes: cleaning HTML tags, removing noise data, such as scripts, advertisements, etc.; performing pre-processing operations such as word segmentation, part-of-speech tagging, and removing stop words on the text; segmenting the text, extracting key paragraphs, and forming independent text samples. The image extraction includes pre-processing such as cropping and scaling to unify the size of all images. The text data set and image data set are constructed, and each text and image sample is marked with an authenticity label and a category label. The authenticity label is whether it is a power system-related network asset, 1 indicates that it is a power system-related network asset, and 0 indicates that it is not a power system-related network asset. The category label is marked as a core asset, a key asset, a peripheral asset, or a counterfeit asset according to the content.
3. The method for identifying power network assets based on multi-task and hierarchical multi-modal decision-making according to claim 1 is characterized in that: In step S2, the constructed text and image datasets are used to train a text multi-task learning model and an image multi-task learning model respectively; the text multi-task learning model is based on the BERT pre-trained language model, the image multi-task learning model is a convolutional neural network based on adaptive attention, and the multi-task training objectives are asset authenticity identification and asset classification.
4. The method for identifying power network assets based on multi-task and hierarchical multi-modal decision-making according to claim 1 is characterized in that: In the step S3, a hierarchical multimodal decision strategy is optimized based on the classification result of a single sample based on a multi-task model; The multimodal decision-making strategy includes the following steps:
1. For each asset level k, calculate the preliminary scores of the text samples and image samples in the same web page for that level respectively and 2. Combine the scores of text samples and image samples to obtain the preliminary score Sk of asset level k; 3. According to the importance of asset level k in the web page and the authenticity ratio of the sample, the preliminary score is weighted and adjusted to obtain the final comprehensive score of asset level k 4. The comprehensive score of each asset level k Perform normalization processing; 5. The highest scoring asset class Total score Compared with the decision threshold θf, when When the webpage is classified as the asset level, otherwise the webpage is marked as "undetermined".
5. A method and system for identifying power network assets based on multi-task and hierarchical multi-modal decision-making according to claim 4, characterized in that: The asset level k score of the text sample in the web page is calculated as follows: N T represents the number of text samples in a web page, T represents the set of text samples in a web page, and αk represents the weight of asset level k in the text sample; the asset level k score of the image sample in the web page is calculated as follows: N I represents the number of image samples in the web page, I represents the set of image samples in the web page, and βk represents the weight of asset level k in the image samples; the preliminary score Sk of the asset level k is calculated as follows:
6. The method for identifying power network assets based on multi-task and hierarchical multi-modal decision-making according to claim 4 is characterized in that: The preliminary score is weighted and adjusted as follows: ωk is the relative importance of asset level k, γk is the sample proportion of real power network assets in the samples judged as level k on the web page, and Pk is the proportion of samples judged as level k in the web page to the total samples. The comprehensive score After normalization, the calculation method is as follows: There are 4 asset classes in total.
7. A method and system for identifying power network assets based on multi-task and hierarchical multi-modal decision-making according to claim 2, characterized in that: The core assets indicate that the power asset information contained in the web page is highly sensitive and important; the key assets indicate that the power asset content contained in the web page is important but not as sensitive as the core assets; the peripheral assets indicate that the power assets contained in the web page are relatively minor; the counterfeit assets indicate that the power asset content contained in the web page is unreliable and contains false information.
8. The method and system for identifying power network assets based on multi-task and hierarchical multi-modal decision-making according to claim 4, characterized in that: If the decision strategy determines that the result is "undetermined", the webpage and all the text samples and image samples contained therein are recorded and backed up and submitted for manual review.
9. A power network asset identification system based on multi-task and hierarchical multi-modal decision-making, characterized in that: It includes data acquisition module, parallel processing module, multi-task classification module, multi-level decision module and discrimination module; Data acquisition module: used to grab data from the target power system-related web pages, parse the HTML structure of the web pages, extract text paragraphs and images related to the power system, and form independent samples; Parallel processing module: used to manage the parallel processing capabilities of the system system, receive independent samples uploaded by the data acquisition module, and use multi-core GPUs to execute multi-task classification modules in parallel to process each text paragraph and each image sample; Multi-task classification module: used to extract features from text and image data in web pages, determine whether they are related to power assets and classify power assets, and send the output results to the multi-level decision-making module; Multi-level decision module: used to initially integrate the classification results of samples (text or images) in web pages. Use weighted decision fusion strategy to generate the overall classification results of web pages; Discrimination and log management module: used to output power system web asset classification results for use in security monitoring, risk assessment and other business processes. At the same time, the classification results and related data are logged for subsequent analysis, tuning and model improvement.
Citation Information
Patent Citations
Training and identification method and system for identification model, equipment and medium
CN112651410A
Feature fusion and decision fusion mixed multi-modal emotion recognition method
CN112800875A
Data analysis method and system based on data classification
CN117077071A
Power distribution network power equipment identification method based on multi-modal data fusion
CN118245740A
Asset information classification and display method and device based on artificial intelligence, and medium
CN118245903A
Cited By
Mobile game complex scene self-adaptive operation method based on multi-task learning
CN119868963A
