Power grid compliance area risk prediction method and system based on transfer learning

Abstract

The invention discloses a power grid compliance area risk prediction method and system based on transfer learning. The method comprises the steps of obtaining marketing inspection business flow charts of a power grid source domain and a target domain and corresponding text record data in the flow charts; performing data processing on the text record data of the source domain and the target domain, and generating a source domain text graph and a target domain text graph in combination with the flow chart; based on transfer learning, using a GCN and Transform combined network to learn the source domain text graph, and using the target domain text graph to perform network fine tuning and domain adaptation to obtain a compliance region risk prediction model; and performing compliance area risk prediction on the latest marketing inspection business flow chart of the target domain and the corresponding text record data in the flow chart. The method can rapidly adapt to a new complex and changeable power grid environment, and improves the risk prediction accuracy, efficiency and applicability when a power grid enterprise carries out inspection services in different regions.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence risk detection and relates to a method and system for predicting compliance area risks of power grid marketing audit business flows based on transfer learning. Background Art

[0002] Grid marketing audit, which is the extended business process of grid audit, is a key task in grid management and control. It supervises and inspects all aspects of grid operation and maintenance to ensure the safe, reliable and efficient operation of the grid and prevent possible risks. Through audit, potential hidden dangers and faults in grid operation can be discovered and handled in a timely manner to prevent major accidents. At the same time, systematic audit helps to find weak links in grid operation, optimize the configuration of equipment and lines, and improve the overall reliability and stability of the grid. In addition, grid marketing audit can ensure that grid operation complies with relevant regulations and industry standards and avoid risks caused by illegal operations. By analyzing and improving the business processes of the grid, grid marketing audit can reduce resource waste and operational errors, and improve the operational efficiency and control level of the grid. In addition, grid marketing audit helps to establish and improve the emergency plan of the grid, improve the ability to respond to emergencies, and ensure that power supply can be quickly restored in an emergency.

[0003] Although manual audits play an important role in power grid management and control, there are many shortcomings and problems in risk prediction of compliance areas. Specifically, manual audits usually require a lot of time and human resources. Especially when faced with a large and complex power grid system, the efficiency is obviously insufficient, and it is difficult to detect and deal with problems in a timely manner. Manual audits rely on the experience and judgment of auditors and are easily affected by subjective factors, resulting in inconsistent and inaccurate results. In addition, manual audits are difficult to make full use of large-scale historical data and real-time data, and cannot conduct comprehensive and in-depth risk analysis and prediction. Due to human resource limitations, the coverage of manual audits is usually limited, making it difficult to conduct a comprehensive inspection of all areas and equipment of the power grid, and it is easy to miss potential risk points. At the same time, manual audits require a lot of manpower and financial resources, especially when frequent and in-depth inspections are required, the cost increases significantly.

[0004] In order to solve the above problems, the current rapidly developing artificial intelligence methods, especially deep learning, have been introduced into the compliance area risk prediction of power grid audit. However, deep learning methods usually require a large amount of labeled data to train the model, but power grid audit data is often limited and expensive. In addition, power grid data is time-varying and regionally different, and the power grid conditions in different regions may vary greatly, causing deep learning models to need to be retrained at specific times in specific areas, which is time-consuming and labor-intensive. Although other learning methods perform well in specific fields, they may have limitations in power grid audit risk prediction. For example, reinforcement learning requires a large amount of interactive data and exploration processes, and there may be practical difficulties in applying it to high-risk fields such as power grid audit. Although ensemble learning can improve the stability and predictive ability of the model, its training and tuning process is complicated, and its effect is limited when data is scarce. Summary of the invention

[0005] In order to address the deficiencies in the prior art, the present invention provides a power grid compliance area risk prediction method and system based on transfer learning. Through transfer learning technology and utilizing a large amount of existing historical data and experience, it can effectively complete the compliance area inspection and risk prediction of the power grid audit leaf flow chart, quickly adapt to the new complex and changeable power grid environment, and improve the risk prediction accuracy, efficiency and applicability of power grid enterprises when conducting audit business in different areas.

[0006] The present invention adopts the following technical solution.

[0007] A first aspect of the present invention provides a compliance area risk prediction method based on transfer learning, comprising:

[0008] Obtain the marketing audit business flow chart of the power grid source domain and the target domain and the corresponding text record data in the flow chart;

[0009] The text record data of the source domain and the target domain are cleaned, entity feature recognized, and converted into JSON format in turn, and then combined with their respective marketing audit business process diagrams to generate source domain text graphs and target domain text graphs;

[0010] The source domain text graph is learned using the GCN and Transformer joint network to obtain a general feature extraction model. The general feature extraction model is transferred to the target domain text graph using transfer learning technology for network fine-tuning training and domain adaptation. During fine-tuning training, cross-validation is used to increase the performance of the model and adversarial training is performed using a domain discriminator to obtain a compliance area risk prediction model.

[0011] The compliance area risk prediction model is used to predict the compliance area risk of the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram to obtain the compliance area risk prediction results.

[0012] Preferably, the data in the source domain includes marketing audit flow charts in different fields of non-predicted target areas and corresponding audit reports, fault records, and equipment maintenance log text record data in the flow charts;

[0013] The data in the target domain includes the marketing audit business process flow chart of the predicted target area and the corresponding equipment status, load conditions, environmental conditions real-time monitoring data in the process chart, power grid audit reports and records, maintenance logs of the target area, and text record data of feedback and complaint records from power grid users.

[0014] Preferably, the text record data of the source domain and the target domain are respectively cleaned, entity feature recognized and converted into JSON format, and then combined with their respective marketing audit business process diagrams to generate a source domain text graph and a target domain text graph, specifically including:

[0015] (1) Perform data cleaning on text record data, including deduplication, missing value processing, invalid value or outlier processing, noise processing, and deletion of unreasonable data;

[0016] (2) Input the cleaned text record data into the Bert-CRF model for entity feature recognition to obtain key text information, including date, time, text, and value;

[0017] (3) Convert the key text information into JSON format to obtain structured text information in JSON format;

[0018] (4) The structured text information in JSON format is combined with the corresponding marketing audit flowchart to perform Networkx modeling, with the steps in the flowchart as nodes, the transfer relationships between the steps as edges, and the structured text information in JSON format corresponding to the steps as node features to generate a text graph.

[0019] Preferably, the (2) is as follows:

[0020] (2.1) BERT represents: Given a cleaned text record data X = [x1,…,x n ], BERT converts it into a word embedding sequence H = [h1,…,h n ]

[0021] H = BERT(X) = [h1,…,h n ]

[0022] (2.2) CRF probability: Given BERT’s output H = [h1,…,h n ], the CRF layer calculates the label sequence Y = [y1,…,y n], predict the label of each word based on the conditional probability, and extract key features according to the label of the word and the corresponding position in the original sentence. The conditional probability calculation formula is:

[0023]

[0024] in Transfer score for true label; For label y i The score weight of is the predicted possible label transfer score; U y′i is the possible label y′ predicted i The score weight of are all possible tag sequences; Y′ is The elements in Y′ include several y′ i .

[0025] Preferably, in the joint network, GCN aggregates the node topology information of the text graph to obtain the aggregated node representation, and serializes the aggregated text graph through singular value decomposition to obtain a token sequence; the token sequence is sent to the Transformer layer, and the node features in the token sequence are encoded using multi-head attention MHA to obtain the encoded node representation; the encoded node representation and the aggregated node representation are concatenated to update the node representation; the above process is iterated multiple times to form a final node representation, which is sent to the classifier for compliance area risk prediction.

[0026] Preferably, the singular value decomposition serialization process is:

[0027]

[0028] in is the smoothed adjacency matrix; LN represents the normalization operation; D represents the degree matrix of matrix A; e v express The token sequence obtained by conversion; Represents a smoothed adjacency matrix A row of ; A represents the matrix representation of the text graph; For topology-aware projection function: Map the high-order features of each node to a low-dimensional feature space; represents the dimension of the original feature space, where the feature vector of each node has |V| elements, |V| is the number of nodes in the graph; represents the dimension of the target feature space, where d represents the representation dimension of each node in the low-dimensional space; U represents the left singular vector matrix, Λ represents the singular value diagonal matrix, and Λ represents the right singular vector matrix; U, Λ, and V are obtained by pairing the matrices Perform singular value decomposition to obtain .

[0029] Preferably, the classifier defines compliance area risk prediction as a node classification task, and defines risk levels as four risk levels: normal, minor violation, moderate violation, and major violation.

[0030] Preferably, the compliance area risk prediction model is used to predict the compliance area risk of the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram to obtain the compliance area risk prediction result, including:

[0031] First, the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram are collected in real time, and data cleaning, entity feature recognition, JSON format conversion and text graph generation are performed on them to obtain the latest target domain text graph, which is input into the compliance area risk prediction model for compliance area risk prediction, and the corresponding compliance area risk prediction results are output, including risk level, location, and compliance status;

[0032] The compliance area risk prediction results are then presented in the form of a report. After taking corresponding measures, the latest marketing audit business process diagram and the corresponding text record data in the process diagram are input into the compliance area risk prediction model for prediction. The latest marketing audit business process diagram and the corresponding text record data in the process diagram are added to the target domain data to update the data set for model fine-tuning, form a feedback mechanism, and continuously improve model performance.

[0033] Preferably, the loss function used in the transfer learning process is:

[0034] Source domain loss function

[0035]

[0036] Target domain loss function

[0037]

[0038] in, is the target domain loss; is the domain adversarial loss; D(·) is the domain discriminator; f(·) is the feature extractor; P s , P t are the data distribution of the source domain and the target domain respectively; N is the number of samples in the source domain; M is the number of classification categories; w cis the category weight, used to balance the category imbalance; ic is the true label of domain sample i belonging to category c; p ic is the probability that source domain sample i is predicted to be category c; P(c) is the prior probability of category c in the source domain; Indicates the expected calculation on the source domain samples; Indicates the expected calculation on the target domain samples.

[0039] A second aspect of the present invention provides a power grid compliance regional risk prediction system based on transfer learning, comprising:

[0040] The marketing audit data collection and acquisition module is used to respectively acquire the marketing audit business process diagram of the power grid source domain and the target domain and the corresponding text record data in the process diagram;

[0041] The data processing and conversion module is used to perform data cleaning, entity feature recognition and JSON format conversion on the text record data of the source domain and the target domain respectively, and then generate the source domain text graph and the target domain text graph in combination with their respective marketing audit business process diagrams;

[0042] The model training module is used to use the GCN and Transformer joint network to learn the source domain text graph to obtain a general feature extraction model; the general feature extraction model is transferred to the target domain text graph using transfer learning technology for network fine-tuning training and domain adaptation. During fine-tuning training, cross-validation is used to increase the performance of the model and adversarial training is performed using the domain discriminator to obtain a compliance area risk prediction model;

[0043] The compliance area risk prediction module is used to use the compliance area risk prediction model to perform compliance area risk prediction on the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram to obtain the compliance area risk prediction results.

[0044] A third aspect of the present invention provides a terminal, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0045] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method when executed by a processor.

[0046] Compared with the prior art, the beneficial effects of the present invention include at least:

[0047] The present invention widely collects mature marketing audit flowcharts in other fields, converts the power grid audit business flow into a text graph, pre-trains it to obtain a general risk feature extraction model, and then generalizes it to the data set of the current task for fine-tuning and adaptation, and finally trains to obtain a power grid marketing audit process compliance area risk prediction model with high classification accuracy.

[0048] The present invention uses the graph structure of a text graph to represent each link and their relationships in the business flow, providing a more intuitive and detailed risk analysis, so that subsequent models can more accurately capture and analyze the complex dependencies and potential risks in the audit business flow, and improve the accuracy and reliability of compliance risk prediction.

[0049] The present invention combines GCN and Transformer technologies in the processing of text graphs, and designs a loss function. In the process of model training and fine-tuning, domain adaptation is considered, a domain discriminator is added, and the problem of category imbalance is considered. The domain discriminator is used for adversarial training to reduce the distribution difference between the source domain and the target domain data, thereby improving the generalization ability and classification accuracy of the model, and can effectively model the complex relationship in the power grid audit business flow, capture the dependency and topological structure between nodes, and enable the model to simultaneously use local structural information and global dependency, thereby enhancing the ability to predict the compliance risk of the business flow and improving the processing ability of complex business flows. The feature extractor obtained by pre-training of the present invention not only extracts features, but also adjusts these features so that the domain discriminator cannot easily distinguish whether they are from the source domain or the target domain. Through this adversarial training, the feature extractor is finally prompted to generate more general features, which helps the model to quickly adapt to new tasks. In addition to considering the classification target, the total loss of the target domain of the present invention also adds a domain alignment target, namely, domain adversarial loss, so that the model can accurately classify in the target domain and reduce the difference between domain distributions.

[0050] The present invention can achieve efficient training based on a small amount of labeled data through transfer learning, reduce the demand for a large amount of high-quality labeled data, and reduce the cost and time of data labeling. At the same time, the model can automatically predict compliance risks, reduce the workload of manual audits, improve the automation level of audit work, and provide intelligent decision support for auditors. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the overall framework of the present invention;

[0052] Figure 2 It is a schematic flow chart of the method of the present invention;

[0053] Figure 3 It is a schematic diagram of data processing conversion in the present invention;

[0054] Figure 4 It is the joint network structure diagram of GCN and Transformer in the present invention;

[0055] Figure 5 It is a schematic diagram of the serialization of the figure in the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only embodiments of a part of the present invention, rather than all embodiments. Based on the spirit of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the protection scope of the present invention.

[0057] Embodiment 1 of the present invention provides a compliance area risk prediction method based on transfer learning, by collecting a wide range of mature and good audit leaf expansion process flow charts, cleaning their text records and performing BERT-CRF key information mining preprocessing operations to convert them into structured text data, and then using Networkx to convert them into text graphs, and jointly using the graph neural network model GCN and the language model Transformer to simultaneously perform text encoding and graph information aggregation on the obtained text graph to more accurately capture the node semantics. After training, a general risk feature extraction model can be obtained, which can automatically extract key features from compliance area data. Next, using transfer learning technology, the general risk feature extraction model is migrated to the target area of ​​the audit leaf expansion business flow of the current task, and the latest data of the target area is used to fine-tune and optimize the model to adapt to the specific risk characteristics of the area. At the same time, the prediction performance and reliability of the model are evaluated by the cross-validation method, such as Figure 1-Figure 5 As shown, the method specifically includes:

[0058] S1, obtaining the power grid source domain data and target domain data required for transfer learning, including the marketing audit business process diagram and the corresponding text record data in the process diagram;

[0059] Further preferably, two types of data sets, source domain and target domain, are collected. The source domain data include other power grid areas, marketing audit flowcharts in other fields, and corresponding text record files. These data usually contain features and patterns similar to the target task, but not necessarily identical, and will be used to pre-train a general feature extraction model. The target domain data refers to the data of a specific power grid area that currently needs to perform compliance area risk prediction. Its text record files include real-time monitoring data of equipment status, load conditions, environmental conditions, etc., power grid audit reports and records, maintenance logs of the target area, and feedback and complaint records of power grid users. These data are used to fine-tune the pre-trained model to improve its performance in the target domain.

[0060] In specific implementation, the source domain data includes marketing audit flowcharts in different fields of non-predicted target areas and corresponding audit reports, fault records, and equipment maintenance log text record data in the flowcharts; examples of marketing audit flowcharts in different fields of non-predicted target areas are as follows:

[0061] The user capacity expansion process in the power industry involves all aspects from application, survey, design, construction to acceptance, and ensures the accuracy and compliance of each step through audit reports, fault records and equipment maintenance logs;

[0062] The establishment of branches in the financial industry requires approval, security review, system testing and other steps, and also relies on detailed reports and records to ensure the security of business systems and the effectiveness of employee training;

[0063] The construction and operation preparation of new facilities in the healthcare industry must also strictly comply with safety standards and quality control, and ensure the quality of medical services and patient safety through comprehensive document management.

[0064] The target domain data includes the marketing audit business process flow chart of the predicted target area and the corresponding equipment status, load conditions, environmental conditions real-time monitoring data in the process chart, power grid audit reports and records, maintenance logs of the target area, and feedback and complaint records of power grid users.

[0065] This step collects mature and compliant marketing audit process data in different fields, including flowcharts and corresponding audit reports, fault records, equipment maintenance logs and other text record files as source domain data, and collects the existing audit reports and business expansion business flow records of the audit leaf flowcharts that need to be analyzed as target domain data; the collected and obtained power grid marketing audit business flow diagrams and the corresponding text record data in the flowcharts provide original data for subsequent data processing and model training to ensure the comprehensiveness and reliability of the data.

[0066] It can be understood that in the business / operation flow chart, the part that is executed in accordance with the requirements specified in the document is the compliance area, otherwise it is the non-compliance area, indicating that there are non-compliance issues or possible risk points.

[0067] S2, respectively, performs data cleaning, entity feature recognition, and JSON format conversion on the text record data of the source domain and the target domain, and then generates the source domain text graph and the target domain text graph by combining their respective marketing audit business process diagrams;

[0068] Further preferably, after collecting the data, the collected text data needs to be processed and converted, including data cleaning, such as removing missing values, duplicate data, outliers, noise data, etc., to ensure the quality of the data, and then using the natural language processing (NLP) technology named entity recognition (NER) based on the BERT-CRF model to identify key entities and features in the text data, such as equipment name, fault type, maintenance time, etc., to mine key information, and then convert it into a unified structured format JSON based on the mining results; combined with the marketing audit flowchart, the network modeling tool NetworkX is used to model the audit leaf expansion flowchart into a text graph, where the nodes are the steps in the flowchart, the edges are the dependencies between the steps, and the node attributes are the corresponding structured text records, whose function is to extract key features, construct a structured text graph, and provide input data for model training, thereby completing the training data preparation.

[0069] The details are as follows:

[0070] (1) Data cleaning of text record data, including deduplication (i.e., removing duplicate records to ensure data uniqueness), missing value processing, invalid value or outlier processing, noise processing, and filling or deleting unreasonable record data; the main processing object of the data cleaning process is dirty data. Dirty data itself has the characteristics of inconsistency and inaccuracy, which directly affects the explicit and implicit value of the data, that is, directly affects the quality of the data.

[0071] 1.1), Deduplication: Remove duplicate records from the data set to avoid deviations caused by calculating the same data multiple times.

[0072] 1.2) Missing value processing: Missing values ​​refer to clustering, grouping, deletion or truncation of data due to lack of information in rough data. It means that the value of one or some attributes in the existing data set is incomplete. For power grid data, data missing may be caused by failure of data collection or data preservation at the metering point. The methods for missing value processing include: deletion, interpolation, and not processing missing values.

[0073] 1.3) Invalid or outlier processing: Invalid or outlier values ​​refer to those unreasonable values ​​in the data set. Outliers in the data set may be caused by sensor failure, manual input errors or abnormal events. These outliers will lead to incorrect conclusions in some scenarios (such as linear regression models, K-means clustering, etc.). Detection of outliers includes simple statistical analysis, 3σ principle, box plots, etc.; outlier processing methods include deletion, treating as missing values, mean correction, capping method, binning method, regression interpolation, multiple interpolation and no processing.

[0074] 1.4), Noise processing: Noise processing is to reduce the noise data in the data set and improve the training effect and prediction accuracy of the model.

[0075] 1.5) Delete unreasonable data: Remove records in the data set that are obviously inconsistent with logic or business rules to ensure the rationality and consistency of the data.

[0076] (2) Identify key entities and features in text data. BERT-CRF in natural language processing technology will be used to mine key information. In the process, some of the information mined will be standardized, such as unifying the formats of date, time, text and numerical values: the text record data after data cleaning is input into the named entity recognition technology Bert-CRF model for entity feature recognition, to achieve key information mining, and obtain key text information, including date, time, text and numerical values. Among them, date refers to a specific day, usually in the format of year-month-day or year-month-day; time refers to a specific moment, usually in the format of hour:minute, which can be 24-hour system or 12-hour system, and can include AM / PM; text refers to descriptive or narrative content, usually including detailed description of the event, user feedback or other relevant information; numerical value refers to specific numbers or quantitative information, which can be integers, decimals, negative numbers or percentages.

[0077] BERT-CRF is a sequence labeling algorithm that combines BERT (Bidirectional Encoder Representations from Transformers) and CRF (Conditional Random Fields). BERT provides powerful context representation capabilities and uses CRF to globally optimize sequence labeling, thereby achieving high-precision labeling in tasks such as named entity recognition (NER). BERT-CRF optimizes parameters through end-to-end training, allowing the model to capture both local and global features. The details are as follows:

[0078] (2.1) BERT represents: Given an input sequence (i.e., text record data after data cleaning) X = [x1,…,x n ], BERT converts it into a word embedding sequence H = [h1,…,h n ]

[0079] H = BERT(X) = [h1,…,h n ]

[0080] (2.2) CRF probability: Given BERT’s output H = [h1,…,h n ], the CRF layer calculates the label sequence Y = [y1,…,y n ]:

[0081]

[0082] in is the true label transfer score, is the label y i The score weight of Score the predicted possible label transfers; is the possible label y′ predicted i The score weight of are all possible tag sequences; Y′ is The elements in i ;

[0083] The above Y is the label sequence, y i is the i-th token (h i )’s corresponding label; The values ​​of are all learnable parameters of CRF.

[0084] For the entity feature recognition, for example, if the target recognition is a date such as October 20, 2024 / October 20, 2024 / 2024.10.24, the recognized label should be the date, and the time label should not be recognized.

[0085] Each tag in the tag sequence belongs to a tag set, such as a set containing the following:

[0086] B-DATE (indicates the beginning of the date), I-DATE (indicates the continuation of the date), B-TIME (indicates the beginning of time), B-VALUE (indicates the beginning of a numerical value), and O (indicates a common word that is not an entity).

[0087] CRF predicts the label for each word. This process predicts a probability for each label in the label set and selects the largest probability as the label of the word. A specific example is as follows:

[0088] Input text: "The meeting will be held on December 15, 2023 at 10:30 AM with 120 participants.

[0089] "

[0090] BERT word segmentation: ["meeting","will be held","2023","year","12","month","15","day","morning","10",":","30","held",",","a total of","120","people","participated","."]

[0091] CRF prediction: ["O","O","B-DATE","I-DATE","I-DATE","I-DATE","I-DATE","I-DATE","B- TIME","I-TIME","I-TIME","I-TIME","O","O","O","B-VALUE","I-VALUE","O","O"]

[0092] Information integration: extract key features based on word labels and their corresponding positions in the original sentences.

[0093] {

[0094] "Date":"December 15, 2023",

[0095] "time":"10:30 AM",

[0096] "Value":"120"

[0097] }

[0098] (3) Convert the key text information into JSON format to obtain structured text information in JSON format;

[0099] (4) The JSON format structured text information is combined with the corresponding marketing audit flowchart to perform Networkx modeling, with the steps in the flowchart as nodes, the transfer relationships between the steps as edges, and the text records corresponding to the steps (i.e., JSON format structured text information) as node features to generate a text graph. NetworkX is a powerful Python library for creating, manipulating, and analyzing complex network structures. It supports undirected graphs, directed graphs, and multigraphs, and provides rich functions for adding nodes and edges, calculating network characteristics, performing advanced analysis (such as shortest paths and centrality measurements), and drawing graphs. NetworkX is easy to use and suitable for research and development of various network analysis applications.

[0100] S3, based on transfer learning, uses the GCN and Transformer joint network to learn the source domain text graph, and uses the target domain text graph to perform network fine-tuning and domain adaptation to obtain a compliance area risk prediction model;

[0101] Further preferably, transfer learning has significant advantages in dealing with data scarcity, domain differences, and dynamic changes, and is therefore more practical and efficient in predicting compliance regional risks in power grid marketing audits. Because transfer learning can make full use of large-scale historical data and real-time data, and conduct in-depth risk analysis and prediction by learning complex patterns and relationships in the data, it can reduce dependence on human resources and reduce audit costs. At the same time, transfer learning has good adaptability and can be applied in different power grid environments and conditions. It can adapt to new data and needs by fine-tuning the model to maintain efficient risk prediction capabilities. In addition, by continuously introducing new data and knowledge, transfer learning can continuously improve and optimize risk prediction models, maintain continuous improvement in prediction capabilities, and significantly improve the effectiveness and efficiency of power grid audits. Therefore, the present invention uses a method based on transfer learning to predict compliance regional risks in the power grid marketing audit process.

[0102] Transformer alone performs well in processing pure text tasks such as machine translation and text generation, and can capture long-distance dependencies and complex semantic relationships. However, when processing tasks that need to make full use of graph structure information, Transformer may not be able to effectively capture the local relationships between nodes and the topological structure of the graph. Therefore, considering the characteristics of the text graph, the present invention chooses to combine GCN and Transformer to make up for this deficiency. By comprehensively utilizing local and global information, the processing effect of text graph tasks is improved. GCN will be nested in the Transformer block of each layer. By combining GCN and Transformer to capture the neighbor topological information and text attribute information of the nodes in the text graph to generate high-quality low-dimensional representations of nodes for risk prediction, and use transfer learning to let the model first train on mature and good audit leaf expansion flowcharts in different fields to obtain a general risk feature extraction model with high generalization ability, and then fine-tune it on the dataset of the current task to adapt to the current task. This step combines the graph neural network GCN and the language model Transformer to train the processed text graph, learn to process graph structure data and capture complex relationships and patterns, and thus establish a compliance area risk prediction model. During the training process, transfer learning technology can be used to fine-tune the pre-trained model and a small amount of new data to improve the performance and generalization ability of the model. The details are as follows:

[0103] The graph data obtained in the training data preparation phase is input into GCN to aggregate the neighbor information of the nodes to capture the topological relationship of the nodes to obtain the node representation update at the graph level. As the first enhancement, the graph serialization is then sent to the Transformer to further enhance the node representation by jointly encoding the text attributes and topological attributes of the nodes. After multiple rounds of iterations, the enhanced topological information and text information of the nodes are spliced ​​as their final representation, and finally sent to the classifier for node classification to achieve compliance area risk prediction. The model will be trained on data sets in different fields, and finally a general risk feature extraction model will be obtained, which can automatically extract key features from compliance area data; the trained general model will be transferred to the target area data set for fine-tuning using transfer learning technology, and cross-validation will be used to further increase the performance of the model.

[0104] In specific implementation, the model learning process will adopt transfer learning to build a compliance area risk prediction model through two stages: pre-training and fine-tuning.

[0105] First, in the pre-training stage, the joint model is trained using source domain data to learn basic features and patterns. The source domain text graph is learned using the GCN and Transformer joint network to obtain a general feature extraction model.

[0106] Then, in the fine-tuning stage, the general feature extraction model is moved to the target domain text graph using transfer learning technology for network fine-tuning. Cross-validation is used during fine-tuning training to further increase the performance of the model and obtain a compliant area risk prediction model. The general feature extraction model is fine-tuned using the target domain data to make it better suited to the target task. Specifically, K-fold cross-validation is used: first, the data set is divided into K subsets, and the model is trained with K-1 subsets each time. The remaining 1 subset is used for verification. This process is repeated for K rounds, and each subset takes turns as a verification set. In each round, the model hyperparameters can be adjusted according to the results, and the model performance indicators are recorded. After all rounds are completed, the average of the performance indicators of each round is calculated to obtain the overall performance evaluation of the model, so that the model with the best performance in the K rounds is selected as the final model.

[0107] In the fine-tuning stage, the domain discriminator module is introduced, and adversarial training is used to narrow the distribution difference between the source domain and the target domain, so as to improve the generalization ability and classification accuracy of the model. Specifically, the pre-trained feature extractor not only extracts features, but also adjusts these features so that the domain discriminator cannot easily distinguish whether they are from the source domain or the target domain. Through this adversarial training, the feature extractor will eventually generate more general features, which will help the model quickly adapt to new tasks.

[0108] Through this learning method, the model can make full use of existing data knowledge and accurately predict compliance area risks in the power grid marketing audit business flow, thereby improving audit efficiency and accuracy.

[0109] Further preferably, in the joint network, GCN is nested in the Transformer layer, which captures the topological information and text information of the nodes in the text graph, generates the final node representation, and sends it to the classifier for compliance area risk prediction. Specifically, the following steps are the flow process of model data. This model needs to be placed on the data set and the parameters adjusted after multiple training to obtain a general feature extraction model:

[0110] (1) GCN aggregates the node topology information of the text graph set A to obtain the node representation

[0111] in represents the set of all node features of each graph in the text graph set A at round l, is the node feature set after l-1 rounds of updates, Represents the features of a node on a graph g in the text graph set A at round l;

[0112] (2) In order to use Transformer to capture node text information, the graph needs to be converted into a unified token sequence. Specifically, the text graph set A after the node feature update in the previous step is serialized by singular value decomposition as follows to obtain the token sequence e v :

[0113]

[0114] in is a smoothed adjacency matrix; L represents the Lth power of the matrix;

[0115] LN represents the normalization operation; D represents the degree matrix of matrix A; e v express The token sequence obtained by conversion; Represents a smoothed adjacency matrix A row of ; A represents the matrix representation of the text graph;

[0116] For topology-aware projection function: Map the high-level features of each node to a low-dimensional feature space; represents the dimension of the original feature space, where the feature vector of each node has |V| elements, |V| is the number of nodes in the graph; Represents the dimension of the target feature space, where d is a small number representing the representation dimension of each node in the low-dimensional space;

[0117] U, Λ, V are Perform singular value decomposition to obtain, where U represents the left singular vector matrix, Λ represents the singular value diagonal matrix, Λ represents the right singular vector matrix,

[0118] (3) The token sequence e v Send it to the Transformer layer, use multi-head attention MHA to encode the node text features in the token sequence, and obtain the encoded node representation The details are as follows:

[0119]

[0120] MHA(e v )=Concat(head1,…,head h )

[0121]

[0122] Among them, head j is the jth attention head;

[0123] Concat represents a concatenation operation, which is used to merge multiple vectors or matrices into one; softmax is a function that converts a vector into a probability distribution; Q represents a query matrix; K represents a key matrix; V represents a value matrix; T represents the transpose of a matrix; B represents a learnable bias term; is the node feature representation of the lth layer; W j represents the weight matrix;

[0124] It is understandable that transformer is a multi-head attention mechanism. h For one of the intentions, when the model is actually trained, all nodes in all graphs will be operated at once. Operation, that is, the input data is represented as a tensor / multidimensional array. v Represents a node feature sequence, which is simply a vector. Each element of the vector is a node feature on a graph. The node feature will be updated after multiple rounds of iterations. When l=0, it means the initial node feature, that is, the feature before being sent to the transformer.

[0125] (4) Node representation and node representation Indicates splicing and updates node representation The details are as follows;

[0126]

[0127] (5) According to the number of layers of the set model, iterate (1)-(4) multiple times to form the final node representation, which is then sent to the classifier for compliance area risk prediction.

[0128] In summary, the design is as follows Figure 4 The model structure of the combined GCN and Transformer shown in the figure embeds GCN in the Transformer layer. First, GCN aggregates the node topology information of the text graph to obtain the updated node representation. Then serialize the graph, the serialization method is as follows Figure 5 As shown in the figure, it mainly performs singular value decomposition, and then sends the serialized token to Transformer to use multi-head attention MHA to encode the text features of the node features. Finally, the encoded node representation and the node representation after message aggregation are concatenated as the node representation. Multiple iterations finally form the node representation to perform the node risk prediction task.

[0129] The classifier defines compliance area risk prediction as a node classification task, and defines the risk levels as normal, minor violation, medium violation and severe violation. That is, the risk levels of node compliance are normal, minor violation, medium violation and severe violation.

[0130] Further preferably, in the process of transfer learning, since the data distribution of the source domain and the target domain may be different, domain adaptation needs to be considered, and the distribution difference between the two domains needs to be reduced when designing the loss function. The designed loss function is as follows:

[0131] Source domain loss function

[0132]

[0133] The loss function is the cross entropy loss, which takes into account the problem of class imbalance in the dataset because there is an imbalance in risk level categories.

[0134] Target domain loss function

[0135]

[0136] is the target domain loss, which measures the accuracy of model classification just like the source domain loss, while The domain adversarial loss aims to make the feature distribution of the model consistent between the source domain and the target domain;

[0137] D is the domain discriminator, f is the feature extractor, P s , P tare the data distribution of the source domain and the target domain respectively; N is the number of samples in the source domain; M is the number of classification categories; w c is the category weight, used to balance the category imbalance; ic is the true label of domain sample i belonging to category c; p ic is the probability that source domain sample i is predicted to be category c; P(c) is the prior probability of category c in the source domain; Indicates the expected calculation on the source domain samples; Indicates the expected calculation on the target domain samples.

[0138] S4, using the compliance area risk prediction model to predict the compliance area risk of the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram, and obtain the compliance area risk prediction result.

[0139] Further preferably, this step uses the trained model to predict the compliance area risk of the new data, obtains the real-time power grid marketing audit business flow data of the current task and inputs it into the model. According to the input power grid marketing audit business flow data, the model can predict the corresponding risk level or compliance status. Its role is to realize automated risk prediction, improve the efficiency and accuracy of the audit, and help power grid managers to discover and deal with potential risks in a timely manner, as follows:

[0140] First, the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram (the latest audit data, expansion business record data, etc.) are collected in real time, and the data is preprocessed, including data cleaning, entity feature recognition, JSON format conversion and text graph generation, to obtain the latest target domain text graph, which is input into the compliance area risk prediction model fine-tuned in the previous stage for compliance area risk prediction, and the corresponding compliance area risk prediction results are output, including risk level, location, compliance status, etc.; it can be understood that each node in the text graph corresponds to each process operation on the marketing audit process diagram, the input of the model is the text graph, and the output is the risk level category of each node, that is, the category. If it is normal, it means that the corresponding process operation is compliant, otherwise the corresponding operation is not compliant, that is, the location. At the same time, the risk level can also be converted into a numerical value, that is, a score, and the score mainly shows the degree of compliance in the form of numbers.

[0141] Then, an early warning report is generated based on the risk prediction results, and the risk prediction results of the compliant area are presented in the form of a report to the grid decision makers and maintenance personnel to facilitate decision-making and taking corresponding measures. Decision makers perform response actions based on the report and update the relevant report content. After taking corresponding measures, new real-time data of the target area will be formed, which can be input into the compliant area risk prediction model for prediction, and at the same time form a feedback mechanism to continuously improve the model performance.

[0142] The goal of high-energy-consuming industry classification and abnormal inspection (corresponding to the task of marketing audit, it is the risk inspection and classification of each operation on the audit operation flow chart) is to improve the efficiency and accuracy of audit work through automation and intelligent technology, accurately identify potential risk areas in the power grid audit business flow, optimize resource allocation, enhance the decision-making support capabilities of auditors, reduce human errors, and enhance the generalization ability of the model in different audit business flows and compliance scenarios to ensure the safe operation of the power grid.

[0143] Embodiment 2 of the present invention provides a power grid compliance area risk prediction system based on transfer learning, including:

[0144] The marketing audit data collection and acquisition module is used to obtain the power grid source domain data and target domain data, including the marketing audit business process diagram and the corresponding text record data in the process diagram;

[0145] The data processing and conversion module is used to perform data cleaning, entity feature recognition and JSON format conversion on the text record data of the source domain and the target domain respectively, and then generate the source domain text graph and the target domain text graph in combination with their respective marketing audit business process diagrams;

[0146] The model training module is used to use the GCN and Transformer joint network to learn the source domain text graph to obtain a general feature extraction model; the general feature extraction model is transferred to the target domain text graph using transfer learning technology for network fine-tuning training and domain adaptation. During fine-tuning training, cross-validation is used to increase the performance of the model and adversarial training is performed using the domain discriminator to obtain a compliance area risk prediction model;

[0147] The compliance area risk prediction module is used to use the compliance area risk prediction model to perform compliance area risk prediction on the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram to obtain the compliance area risk prediction results.

[0148] Embodiment 3 of the present invention provides a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0149] Embodiment 4 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method when executed by a processor.

[0150] Compared with the prior art, the beneficial effects of the present invention include at least:

[0151] The present invention widely collects mature marketing audit flowcharts in other fields, converts the power grid audit business flow into a text graph, pre-trains it to obtain a general risk feature extraction model, and then generalizes it to the data set of the current task for fine-tuning and adaptation, and finally trains to obtain a power grid marketing audit process compliance area risk prediction model with high classification accuracy.

[0152] The present invention uses the graph structure of a text graph to represent each link and their relationships in the business flow, providing a more intuitive and detailed risk analysis, so that subsequent models can more accurately capture and analyze the complex dependencies and potential risks in the audit business flow, and improve the accuracy and reliability of compliance risk prediction.

[0153] The present invention combines GNN and Transformer technology to effectively model the complex relationships in power grid audit business flows, capture the dependencies and topological structures between nodes, and enable the model to simultaneously utilize local structural information and global dependencies, thereby enhancing the ability to predict business flow compliance risks and improving the ability to handle complex business flows.

[0154] The present invention can achieve efficient training based on a small amount of labeled data through transfer learning, reduce the demand for a large amount of high-quality labeled data, and reduce the cost and time of data labeling. At the same time, the model can automatically predict compliance risks, reduce the workload of manual audits, improve the automation level of audit work, and provide intelligent decision support for auditors.

[0155] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0156] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0157] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0158] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for predicting grid compliance regional risk based on transfer learning, characterized in that the method include: Obtain the marketing audit business flow chart of the power grid source domain and the target domain and the corresponding text record data in the flow chart; The text record data of the source domain and the target domain are cleaned, entity feature recognized and format converted respectively, and then combined with their respective marketing audit business process diagrams to generate source domain text graphs and target domain text graphs; The source domain text graph is learned using the GCN and Transformer joint network to obtain a general feature extraction model. The general feature extraction model is transferred to the target domain text graph using transfer learning technology for network fine-tuning training and domain adaptation. During fine-tuning training, cross-validation is used to increase the performance of the model and adversarial training is performed using a domain discriminator to obtain a compliance area risk prediction model. The compliance area risk prediction model is used to predict the compliance area risk of the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram to obtain the compliance area risk prediction results.

2. According to claim 1, a method for predicting grid compliance regional risk based on transfer learning is characterized in that: The data in the source domain includes marketing audit flow charts in different fields of non-predicted target areas and corresponding audit reports, fault records, and equipment maintenance log text record data in the flow charts; The data in the target domain includes the marketing audit business process flow chart of the predicted target area and the corresponding equipment status, load conditions, environmental conditions real-time monitoring data in the process chart, power grid audit reports and records, maintenance logs of the target area, and text record data of feedback and complaint records from power grid users.

3. The method for predicting grid compliance regional risk based on transfer learning according to claim 1 is characterized in that: The text record data of the source domain and the target domain are respectively cleaned, entity feature recognized and format converted, and then combined with their respective marketing audit business process diagrams to generate a source domain text graph and a target domain text graph, specifically including: (1) Perform data cleaning on text record data, including deduplication, missing value processing, invalid value or outlier processing, noise processing, and deletion of unreasonable data; (2) Input the cleaned text record data into the Bert-CRF model for entity feature recognition to obtain key text information, including date, time, text, and value; (3) Convert the key text information into JSON format to obtain structured text information in JSON format; (4) The structured text information in JSON format is combined with the corresponding marketing audit flowchart to perform Networkx modeling, with the steps in the flowchart as nodes, the transfer relationships between the steps as edges, and the structured text information in JSON format corresponding to the steps as node features to generate a text graph.

4. The method for predicting grid compliance regional risk based on transfer learning according to claim 3 is characterized by: The details of (2) are as follows: (2.1) BERT represents: Given a cleaned text record data X = [x1,…,x n ], BERT converts it into a word embedding sequence H = [h1,…,h n ] H=BERT(X)=[h1,…,h n ] (2.2) CRF probability: Given BERT’s output H = [h1,…,h n ], the CRF layer calculates the label sequence Y = [y1,…,y n ], predict the label of each word based on the conditional probability, and extract key features according to the label of the word and the corresponding position in the original sentence. The conditional probability calculation formula is: in Transfer score for true label; For label y i The score weight of Score the predicted possible label transfers; is the possible label y′ predicted i The score weight of are all possible tag sequences; Y′ is The elements in Y′ include several y′ i .

5. The method for predicting grid compliance regional risk based on transfer learning according to claim 1 is characterized in that: In the joint network, GCN aggregates the node topology information of the text graph to obtain the aggregated node representation, and serializes the aggregated text graph through singular value decomposition to obtain a token sequence; the token sequence is sent to the Transformer layer, and the node features in the token sequence are encoded using multi-head attention MHA to obtain the encoded node representation; the encoded node representation and the aggregated node representation are concatenated to update the node representation; the above process is iterated multiple times to form the final node representation, which is sent to the classifier for compliance area risk prediction.

6. The method for predicting grid compliance regional risk based on transfer learning according to claim 5 is characterized by: The process of singular value decomposition serialization is: in is the smoothed adjacency matrix; LN represents the normalization operation; D represents the degree matrix of matrix A; e v express The token sequence obtained by conversion; Represents a smoothed adjacency matrix A row of ; A represents the matrix representation of the text graph; For topology-aware projection function: Map the high-order features of each node to a low-dimensional feature space; represents the dimension of the original feature space, where the feature vector of each node has |V| elements, |V| is the number of nodes in the graph; represents the dimension of the target feature space, where d represents the representation dimension of each node in the low-dimensional space; U represents the left singular vector matrix, Λ represents the singular value diagonal matrix, and Λ represents the right singular vector matrix; U, Λ, and V are obtained by pairing the matrices Perform singular value decomposition to obtain .

7. The method for predicting grid compliance regional risk based on transfer learning according to claim 5 is characterized by: The classifier defines compliance area risk prediction as a node classification task and defines four risk levels: normal, minor violation, medium violation, and severe violation.

8. The method for predicting grid compliance regional risk based on transfer learning according to claim 1 is characterized by: The compliance area risk prediction model is used to predict the compliance area risk of the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram to obtain the compliance area risk prediction results, including: First, the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram are collected in real time, and data cleaning, entity feature recognition, JSON format conversion and text graph generation are performed on them to obtain the latest target domain text graph, which is input into the compliance area risk prediction model for compliance area risk prediction, and the corresponding compliance area risk prediction results are output, including risk level, location, and compliance status; The compliance area risk prediction results are then presented in the form of a report. After taking corresponding measures, the latest marketing audit business process diagram and the corresponding text record data in the process diagram are input into the compliance area risk prediction model for prediction. The latest marketing audit business process diagram and the corresponding text record data in the process diagram are added to the target domain data to update the data set for model fine-tuning, form a feedback mechanism, and continuously improve model performance.

9. The method for predicting grid compliance regional risk based on transfer learning according to claim 1 is characterized by: The loss function used in the transfer learning process is: Source domain loss function Target domain loss function in, is the target domain loss; is the domain adversarial loss; D(·) is the domain discriminator; f(·) is the feature extractor; P s , P t are the data distribution of the source domain and the target domain respectively; N is the number of samples in the source domain; M is the number of classification categories; w c is the category weight, used to balance the category imbalance; ic is the true label of domain sample i belonging to category c; p ic is the probability that source domain sample i is predicted to be category c; P(c) is the prior probability of category c in the source domain; Indicates the expected calculation on the source domain samples; Indicates the expected calculation on the target domain samples.

10. A power grid compliance regional risk prediction system based on transfer learning, using the method according to any one of claims 1 to 9, characterized in that: The system comprises: The marketing audit data collection and acquisition module is used to respectively acquire the marketing audit business process diagram of the power grid source domain and the target domain and the corresponding text record data in the process diagram; The data processing and conversion module is used to perform data cleaning, entity feature recognition and format conversion on the text record data of the source domain and the target domain respectively, and then generate the source domain text graph and the target domain text graph in combination with their respective marketing audit business process diagrams; The model training module is used to use the GCN and Transformer joint network to learn the source domain text graph to obtain a general feature extraction model; the general feature extraction model is transferred to the target domain text graph using transfer learning technology for network fine-tuning training and domain adaptation. During fine-tuning training, cross-validation is used to increase the performance of the model and adversarial training is performed using the domain discriminator to obtain a compliance area risk prediction model; The compliance area risk prediction module is used to use the compliance area risk prediction model to perform compliance area risk prediction on the latest marketing audit business process diagram of the target domain and the corresponding text record data in the process diagram to obtain the compliance area risk prediction results.

11. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images