Semantic-level fuzzy matching method based on graph structure for power business data

Abstract

The present invention relates to a semantic-level fuzzy matching method based on graph structure for power business data, comprising the following steps: step 1, preprocessing the original channel routing data in the power grid data; step 2, structured representation of the routing topology information of the preprocessed data to form structured routing topology data. The present invention can effectively match data from two perspectives, namely, the substring level and the network topology level.

Description

Technical Field

[0001] The present invention belongs to the technical field of data matching, and relates to a semantic-level fuzzy matching method for electric power communication service data, and in particular to a semantic-level fuzzy matching method based on a graph structure for electric power service data. Background Art

[0002] The current data sources of power communication services have the following problems: (1) The data format is not unified and the naming rules of network element link services are not unified; (2) The data structure is complex and large in scale, requiring a lot of manpower for cleaning and verification; (3) Random factors such as human errors introduce topological logic errors. In order to solve the above problems, research on cleaning and verification technology for power communication management ledgers and network management data is carried out to improve the intelligent level of operation and maintenance management, reduce the pressure on operation and maintenance personnel, and reduce costs.

[0003] Traditional power communication management ledger and network management data automation processing technology mainly performs matching verification based on the edit distance between power communication equipment names from a character level perspective. This method can only consider the matching of each power communication equipment name separately, and cannot effectively use the power communication network topology information to infer the missing power communication equipment information, resulting in poor matching effect.

[0004] In the prior art, the traditional power communication management ledger automation processing technology is mainly based on character-level matching, and matches different power communication equipment name data by calculating the edit distance between the power communication equipment names.

[0005] The main operation process is as follows.

[0006] 1. Obtain a list of power communication equipment as benchmark data;

[0007] 2. Cut the device names in the business channel data to form individual device names;

[0008] 3. Calculate the edit distance between the device name formed in step 2 and each device name in the benchmark data in step 1 respectively;

[0009] 4. Sort the edit distances obtained in step 3, and take the first K with the smallest edit distances as recommended matching data;

[0010] The calculation formula for the edit distance is as follows:

[0011]

[0012] Formula 1-1 Edit distance calculation formula

[0013] Where i and j represent the subscripts of string a and string b, both starting from 1 and increasing.

[0014] The above traditional methods have the following two important problems:

[0015] 1. Calculating the edit distance between two device name strings at the character level will process the device name strings with the same weight, which cannot grasp the important words in the device name well. For example, if the word to be matched is "Ta Gou Bian", there are two device names "Ta Wan Bian" and "Ta Gou" in the benchmark data. Obviously, the matching result of "Ta Gou Bian" should be "Ta Gou", but the edit distance between the word to be matched "Ta Gou Bian" and "Ta Wan Bian" and "Ta Gou" is 1, which cannot be accurately matched.

[0016] 2. The traditional method only uses the edit distance between two device names as a matching indicator, and does not effectively utilize the communication topology information in the power communication management ledger. The topology information can help determine the missing related device name information.

[0017] In summary, there are many non-standard data in the power communication business data that are manually maintained and reported. The traditional character-level string matching technology has the following problems: 1. It considers the weight of each character equally and ignores the important word features in the string; 2. It does not effectively utilize the topological information in the power communication network, resulting in poor matching effect.

[0018] After searching, no public documents of the prior art identical or similar to the present invention were found. Summary of the invention

[0019] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a semantic-level fuzzy matching method based on graph structure for power communication management ledgers and network management data, which can effectively match data from two perspectives: substring level and network topology level.

[0020] The present invention solves the practical problem by adopting the following technical solutions:

[0021] A semantic-level fuzzy matching method based on graph structure for power business data includes the following steps:

[0022] Step 1: preprocessing the original channel routing data in the power grid data;

[0023] Step 2: Structurally represent the routing topology information of the preprocessed data to form structured routing topology data.

[0024] Moreover, the specific steps of preprocessing the original channel routing data in the power grid data in step 1 include:

[0025] (1) First, the original channel routing data in the power grid data is semantically segmented using methods such as Jieba word segmentation to form structured linked list data;

[0026] (2) Secondly, the structured linked list data is preliminarily de-redundant using the Python regular expression method to remove punctuation marks and other characters that have no practical meaning;

[0027] (3) Then, all structured linked list data are divided into two parts: accurate historical data and business data to be cleaned and verified according to whether the structure and description method are complete;

[0028] (4) Finally, the accurate historical data is organized into a basic data table containing a site list, a device list, and a fiber list. The site list and the device list contain accurate site and device name information, and the fiber list contains various accurate channel routing data. This basic data table will be used as the matching benchmark data source in subsequent steps.

[0029] Moreover, the specific steps of step 2 include:

[0030] ⑴. First, perform semantic segmentation on the routing channel data to be cleaned and verified to form: ①. A sequence of sites and devices to be matched ② A sequence of connectors;

[0031] ⑵. Use the matching algorithm to match and disambiguate each field in the sequence of sites and devices to be matched;

[0032] ⑶. Re-encode each connector in the connector sequence using the One-hot encoding rule to form a One-hot encoded connector sequence;

[0033] (4) Cluster the connector sequence after One-hot encoding using the Kmeans algorithm;

[0034] ⑸. For each category of connector sequence after clustering, restore the one-hot encoding sequence of the center point to the corresponding connector sequence to generate a regular expression for the topological routing of the category;

[0035] ⑹. Use the corresponding regular expression to reorganize the site and device information after matching and disambiguation for each routing channel data to form structured routing topology data.

[0036] Moreover, the specific steps of step 2 (3) include:

[0037] ⑴. Convert the site and equipment lists in the basic data table into site and equipment word bags and construct a Trie tree;

[0038] ⑵. Construct a corresponding topological network diagram based on the channel routing data in the fiber table in the basic data table as a query diagram;

[0039] ⑶. Input the site and device name data to be matched as the query field into the Trie tree for search and matching. If the site and device name in the word bag are completely matched, skip the subsequent steps and directly use the site and device name data as the output result. If there is no complete match in the Trie tree, execute steps (4), (5), and (6);

[0040] (4) For query fields that are not fully matched, the semantic distance between each site name and the field in the bag of words is calculated using the maximum common substring length algorithm, the results are sorted, and the nearest sites and device names are selected as candidate sets for semantic matching;

[0041] ⑸. Construct a small subgraph of the routing links to which the query fields that are not fully matched belong, input the constructed small subgraph into the query graph for subgraph matching, and take the node names corresponding to the nearest matching results with the closest graph similarity as the candidate set for structural matching;

[0042] ⑹. Calculate the intersection of the semantically matched candidate set and the structurally matched candidate set, and take the site and device names with the closest semantic distance in the intersection as the output result to further improve the recommendation accuracy of the candidate set.

[0043] Advantages and beneficial effects of the present invention:

[0044] 1. The present invention aims to solve the problem that the traditional character-level string matching technology fails to take into account the characteristics of important words in the string. Starting from the substring level, the maximum common substring length between the power communication equipment string to be matched and the reference data is calculated as the matching degree;

[0045] 2. In view of the defect that the traditional character-level string matching technology fails to utilize the topological information in the power communication network, the present invention generates a topological network diagram of the power communication equipment according to the service channel data, and uses the subgraph matching method to further refine the matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a process flow chart of the present invention;

[0047] Figure 2 This is a flow chart of the site and device information matching and disambiguation method of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings:

[0049] The power communication management ledger and network management data contain a large amount of power business routing topology information with inconsistent description methods, and each routing topology information contains a large amount of site and equipment information with inconsistent naming rules and ambiguity. Therefore, the cleaning and verification of the actual power communication management ledger and network management data can be divided into two parts: 1. Structuring the routing topology with unified rules; 2: Matching and disambiguating the site and equipment names.

[0050] A semantic-level fuzzy matching method based on graph structure for power business data, such as Figure 1 and Figure 2 As shown, the following steps are included:

[0051] Step 1: preprocessing the original channel routing data in the power grid data;

[0052] The specific steps of preprocessing the original channel routing data in the power grid data in step 1 include:

[0053] (1) First, the original channel routing data in the power grid data is semantically segmented using methods such as Jieba word segmentation to form structured linked list data;

[0054] (2) Secondly, the structured linked list data is preliminarily de-redundant using the Python regular expression method to remove punctuation marks and other characters that have no practical meaning;

[0055] (3) Then, all structured linked list data are divided into two parts: accurate historical data and business data to be cleaned and verified according to whether the structure and description method are complete;

[0056] (4) Finally, the accurate historical data is organized into a basic data table containing a site list, a device list, and a fiber list. The site list and the device list contain accurate site and device name information, and the fiber list contains various accurate channel routing data. This basic data table will be used as the matching benchmark data source in subsequent steps.

[0057] Step 2: Structurally represent the routing topology information of the preprocessed data to form structured routing topology data;

[0058] The specific steps of step 2 include:

[0059] (1) First, perform semantic segmentation on the routing channel data to be cleaned and verified to form: ① a sequence of sites and devices to be matched ② a sequence of connectors;

[0060] (2) Use the matching algorithm to match and disambiguate each field in the sequence of sites and devices to be matched;

[0061] (3) Re-encode each connector in the connector sequence using the one-hot encoding rule to form a one-hot encoded connector sequence;

[0062] (4) Cluster the connector sequence after one-hot encoding using the Kmeans algorithm;

[0063] (5) For each category of connector sequence after clustering, restore the one-hot encoding sequence of the center point to the corresponding connector sequence to generate a regular expression for the topological routing of the category;

[0064] (6) Use the corresponding regular expression to reorganize the site and device information after matching and disambiguation for each routing channel data to form structured routing topology data.

[0065] The specific steps of step (3) of step 2 include:

[0066] In the process of structurally representing the routing topology, it is necessary to match and disambiguate the site and device information in the routing topology. The specific steps of step (3) of step 2 include:

[0067] (1) Convert the site and device lists in the basic data table into site and device word bags and construct a Trie tree;

[0068] (2) Construct a corresponding topological network diagram based on the channel routing data in the fiber table in the basic data table as a query diagram;

[0069] (3) Input the site and device name data to be matched as the query field into the Trie tree for search and matching. If the site and device name in the word bag are completely matched, skip the subsequent steps and directly use the site and device name data as the output result. If there is no complete match in the Trie tree, execute steps (4), (5), and (6);

[0070] (4) For query fields that are not fully matched, the semantic distance between each site name and the field in the bag of words is calculated using the maximum common substring length algorithm, the results are sorted, and the nearest site and device names are selected as the candidate set for semantic matching;

[0071] (5) Construct a small subgraph of the routing links to which the query fields that are not fully matched belong, input the constructed small subgraph into the query graph for subgraph matching, and take the node names corresponding to the several matching results with the closest graph similarity as the candidate set for structural matching;

[0072] (6) Calculate the intersection of the semantically matched candidate set and the structurally matched candidate set, and take the site and device names with the closest semantic distance in the intersection as the output result to further improve the recommendation accuracy of the candidate set.

[0073] Two examples of matching recommendations are given below to verify the advantages and effects of the present invention:

[0074] 110kv Yuquan Substation ['110kv Yinquan Substation', '110kv Shengquan Substation', '110kv Luquan Substation', '110kv Wenquan Substation', '110kV Yuquan Road Substation', '110kv Jinquan Substation', '110kV Baiquan Substation', '110kv Huiquan Substation']

[0075] 110kv Yuquan Substation ['110kv Yuquan Road Substation', '110kv Yinquan Substation', '110kv Shengquan Substation', '110kv Luquan Substation', '110kv Wenquan Substation', '110kv Jinquan Substation', '110kv Baiquan Substation', '110kv Huiquan Substation']

[0076] Two methods match recommendation example 1

[0077] Hengxin Heshuo Photovoltaic [‘Efisun Photovoltaic’, ‘35kV Hengxin Heshuo Photovoltaic’, ‘Ruoqiang Haiwei Photovoltaic’]

[0078] Hengxin Heshuo Photovoltaic ['35kV Hengxin Heshuo Photovoltaic', 'Efisun Photovoltaic', 'Ruoqiang Haiwei Photovoltaic']

[0079] Two methods match recommendation example 2

[0080] The above two matching recommendation methods show the comparison of the recommendation results of the traditional matching method based only on the character level and the semantic matching + graph structure matching method proposed in the present invention. In each example, the first line is the recommendation result of the traditional matching method, and the second line is the result of the semantic matching + graph structure matching method. The leftmost string in each row is the site to be matched, and the list is the candidate set of matching sites. The sites in the candidate set are arranged from left to right according to the confidence level.

[0081] From the above results, it can be seen that since the traditional character-level matching method only considers the edit distance of the string, the recommendation results are very unreliable. For example, the correct matching result of "110kv Yuquan Substation" is obviously "110kV Yuquan Road Substation", and since the edit distance between "110kV Yinquan Road Substation" and it is only 1, it is ranked first in the recommended candidate set; while the matching method proposed in this patent comprehensively considers the semantic distance between sites and the graph structure information, the semantic distance between "110kV Yuquan Road Substation" and "110kv Yuquan Substation" is the smallest, and the similarity of the subgraph with "110kV Yuquan Road Substation" as the central node is the largest with the similarity of the entire routing link, so the matching method proposed in this patent ranks the most similar site name first in the recommended candidate set.

[0082] The semantic-level fuzzy matching method based on graph structure of the present invention is proposed based on the power communication business data. It has good data generalization ability in the power communication business data and can be conveniently applied to various data matching tasks in the context of power communication business and obtain good results. When solving practical engineering problems, the idea of ​​integrated semantic fuzzy matching can have strong robustness.

[0083] Starting from the substring level, the present invention calculates the maximum common substring length between the character string of the power communication equipment to be matched and the reference data as the matching degree; and generates a topological network diagram of the power communication equipment according to the service channel data, and uses the graph matching method to further refine the matching.

[0084] It should be emphasized that the embodiments of the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation modes. Any other implementation modes derived by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.

Claims

1. A semantic-level fuzzy matching method based on graph structure for power business data, characterized by: The following steps are involved: Step 1: preprocessing the original channel routing data in the power grid data; Step 2: Structurally represent the routing topology information of the preprocessed data to form structured routing topology data; The specific steps of step 2 include: ⑴. First, perform semantic segmentation on the routing channel data to be cleaned and verified to form: ①. A sequence of sites and devices to be matched ② A sequence of connectors; ⑵. Use the matching algorithm to match and disambiguate each field in the sequence of sites and devices to be matched; ⑶. Re-encode each connector in the connector sequence using the One-hot encoding rule to form a One-hot encoded connector sequence; (4) Cluster the connector sequence after One-hot encoding using the Kmeans algorithm; ⑸. For each category of connector sequence after clustering, restore the one-hot encoding sequence of the center point to the corresponding connector sequence to generate a regular expression for the topological routing of the category; (6) Reorganize the site and device information after matching and disambiguation of each routing channel data using the corresponding regular expression to form structured routing topology data; The specific steps of step (2) of step 2 include: ⑴. Convert the site and equipment lists in the basic data table into site and equipment word bags and construct a Trie tree; ⑵. Construct a corresponding topological network diagram based on the channel routing data in the fiber table in the basic data table as a query diagram; ⑶. Input the site and device name data to be matched as the query field into the Trie tree for search and matching. If the site and device name in the word bag are completely matched, the subsequent steps are skipped and the site and device name data are directly used as the output result; if there is no complete match in the Trie tree, execute steps (4), (5), and (6); (4) For query fields that are not fully matched, the semantic distance between each site name and the field in the bag of words is calculated using the maximum common substring length algorithm, the results are sorted, and the nearest sites and device names are selected as candidate sets for semantic matching; ⑸. Construct a small subgraph of the routing links to which the query fields that are not fully matched belong, input the constructed small subgraph into the query graph for subgraph matching, and take the node names corresponding to the nearest matching results with the closest graph similarity as the candidate set for structural matching; ⑹. Calculate the intersection of the semantically matched candidate set and the structurally matched candidate set, and take the site and device names with the closest semantic distance in the intersection as the output result to further improve the recommendation accuracy of the candidate set.

2. According to claim 1, a graph-structured semantic-level fuzzy matching method for power business data is characterized by: The specific steps of preprocessing the original channel routing data in the power grid data in step 1 include: (1) First, the original channel routing data in the power grid data is semantically segmented using the Jieba word segmentation method to form structured linked list data; (2) Secondly, the structured linked list data is preliminarily de-redundant using the Python regular expression method to remove punctuation characters that have no practical meaning; (3) Then, all structured linked list data are divided into two parts: accurate historical data and business data to be cleaned and verified according to whether the structure and description method are complete; (4) Finally, the accurate historical data is organized into a basic data table containing a site list, a device list, and a fiber list. The site list and the device list contain accurate site and device name information, and the fiber list contains various accurate channel routing data. This basic data table will be used as the matching benchmark data source for subsequent steps.

Images