Distribution network line risk prediction method based on natural language processing and rule matching

By cleaning and feature extraction of road construction information on Rust language and BERT natural language learning models, and using latitude and longitude coordinates to match the distribution network circuit diagram, the accuracy of information extraction in multi-source heterogeneous data is solved, and the accuracy of information extraction in multi-source heterogeneous data is improved Accuracy and safety assurance of road construction risks.

CN119168139APending Publication Date: 2024-12-20MAINTENANCE BRANCH OF STATE GRID FUJIAN ELECTRIC POWER +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411229923.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-12-20

AI Technical Summary

Technical Problem

The prior art has problems with data format diversity and error information when extracting road construction information from multi-source heterogeneous data, making it difficult to achieve accurate information extraction and integration.

Method used

The Rust language and BERT natural language learning model is adopted, combined with a bidirectional encoder and self-attention mechanism, text cleaning and feature extraction of road construction information is carried out, and the latitude and longitude coordinates are used to match the distribution network circuit diagram, and the vertical line length is calculated as a risk judgment. Conditions, set the warning level.

Benefits of technology

The integration of multi-source heterogeneous data and accurate extraction of key information are achieved, and the accuracy of judgment of road construction risks is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119168139A_ABST
    Figure CN119168139A_ABST
Patent Text Reader

Abstract

The invention relates to a distribution network line risk prediction method based on natural language processing and rule matching. The method comprises the specific steps that road construction information data in a target webpage are collected based on a Rust language and by means of a plurality of databases; the method comprises the following steps: constructing a BERT natural language learning model based on a Transformers architecture, and carrying out pre-training, wherein the BERT natural language learning model adopts a bidirectional encoder and a self-attention mechanism; performing text cleaning on the acquired road construction information data, performing word segmentation on the road construction information data by utilizing a BERT natural language learning model, and extracting event feature data in the road construction information data; extracting time and place information of each event from the feature data of the road construction information data by using a pre-trained BERT natural language learning model; collecting latitude and longitude coordinates of the power distribution network circuit diagram, and drawing the circuit diagram on the map according to the latitude and longitude coordinates; and converting each event site in the road construction information data into latitude and longitude coordinates.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data mining and natural language processing, and in particular to a distribution network line risk prediction method based on natural language processing and rule matching. Background Art

[0002] With the continuous advancement of smart city construction, the collection and processing of road construction data has become increasingly important. Traditionally, the collection of road construction information relies on manual reports and government announcements, which is not only time-consuming and labor-intensive, but also difficult to update in real time. Therefore, being able to extract road construction information from multi-source heterogeneous data in real time and accurately is of great significance to urban traffic management, distribution network maintenance and other fields.

[0003] Currently, large-scale automatic collection of road construction information on the Internet can be achieved through Internet crawler technology, but the data formats on the Internet are diverse and contain a large amount of unstructured text, which brings challenges to accurate information extraction. In addition, due to the wide range of data sources, different data sources may contain different degrees of erroneous or redundant information, which requires an effective data cleaning and integration method.

[0004] The development of NLP can be traced back to the 1950s. In the early days, researchers mainly focused on rule-based methods, that is, artificially defining some grammatical rules to recognize and parse text. However, this method has great limitations, because natural language is very flexible and changeable, and it is difficult to fully cover it with a set of fixed rules. With the advancement of computer technology and the accumulation of a large amount of data, deep learning technology has achieved remarkable results in the field of NLP in the past few decades. Among them, pre-trained models represented by BERT have attracted widespread attention in the field of NLP. This method can be pre-trained on a large amount of text data, and then trained for specific tasks by fine-tuning, which can greatly improve the performance of NLP models.

[0005] In the prior art CN106504746A, a method for extracting structured traffic road condition information from voice data is disclosed, which includes obtaining voice containing required road condition information and performing voice preprocessing; transcribing the preprocessed voice data into text to obtain voice recognition text; segmenting the voice recognition text and tagging the segmentation results with parts of speech; screening the segmented text to filter out useless information content; extracting key information; aggregating road section information; aggregating road condition information; and post-processing road condition information.

[0006] In the prior art, the data obtained is from voice, and then the voice data is converted into text data. There are errors in the accuracy of translation during the conversion process. At the same time, the source of the voice data obtained is not as wide as the source of the text version. At the same time, the key information is extracted by support vector machine, and then the extracted key information is combined by location. The method of extracting key information adopts classification and makes judgments based on probability, which also has errors in the results. Summary of the invention

[0007] In order to solve the above problems existing in the prior art, the present invention proposes a distribution network line risk prediction method based on natural language processing and rule matching.

[0008] The technical solution of the present invention is as follows:

[0009] On the one hand, the present invention proposes a distribution network line risk prediction method based on natural language processing and rule matching, and the specific steps include:

[0010] Based on Rust language and using several databases to collect road construction information data from the target web page;

[0011] Build and pre-train a BERT natural language learning model based on the Transformers architecture, which uses a bidirectional encoder and self-attention mechanism;

[0012] The collected road construction information data is cleaned up, and then the road construction information data is segmented using the BERT natural language learning model, and then the feature data of each event in the road construction information data is extracted;

[0013] The pre-trained BERT natural language learning model is used to extract the time and location information of each event from the feature data of the road construction information data;

[0014] Collect the longitude and latitude coordinates of the distribution network route map, and draw the route map on the map based on the longitude and latitude coordinates; convert the locations of each event in the road construction information data into longitude and latitude coordinates, and mark them on the map to complete the matching of each event with the distribution network route map;

[0015] The distance between each event point and all points on the route map is calculated by traversal, and the length of the perpendicular median line connecting each event point to the two nearest points adjacent to the event point on the route map is set as the risk judgment condition, and the warning level is set according to the length of the perpendicular median line.

[0016] As a preferred implementation, the steps of performing text cleaning on the collected road construction information data, then segmenting the road construction information data using the BERT natural language learning model, and then extracting feature data of each event in the road construction information data are specifically as follows:

[0017] Remove unnecessary characters and punctuation marks from the road construction information data text, and delete special symbols and blank characters;

[0018] The word segmenter corresponding to the pre-trained BERT natural language learning model is loaded using the Transformers database to segment the road construction information data with redundant characters removed and organize it into vocabulary fragments;

[0019] Input the vocabulary fragment into the BERT natural language learning model to obtain the feature vector.

[0020] As a preferred implementation, the step of extracting the time and location information of each event from the feature data of the road construction information data using the pre-trained BERT natural language learning model is specifically as follows:

[0021] In the BERT natural language learning model, the relationship between temporal information and long-distance dependencies in the feature vector is captured based on position encoding and multi-head attention mechanism;

[0022] Fine-tune the parameters of the BERT natural language learning model based on different task training sets to enhance the ability to understand the data set;

[0023] Use the fine-tuned BERT natural language learning model to extract time and place information from the input vocabulary fragment;

[0024] Convert the extracted time and location information into the specified output format for output.

[0025] As a preferred implementation, the steps of collecting the longitude and latitude coordinates of the distribution network route map and drawing the route map on the map according to the longitude and latitude coordinates; converting each event location in the road construction information data into longitude and latitude coordinates and marking them on the map, and completing the matching steps of each event with the distribution network route map are as follows:

[0026] Get the longitude and latitude coordinates of the distribution network map and store them as a json file;

[0027] Use openlayers in webgis to draw a route map on the map based on the json file;

[0028] Call the Amap API: geocoder.getAddress to convert the event location into longitude and latitude coordinates;

[0029] Print the event's latitude and longitude coordinates on the distribution network map to match the distribution network map with the event.

[0030] On the other hand, the present invention proposes a distribution network line risk prediction system based on natural language processing and rule matching, comprising:

[0031] The information collection module is based on the Rust language and uses several databases to collect road construction information data from the target web page;

[0032] BERT model building module, which builds and pre-trains the BERT natural language learning model based on the Transformers architecture. The BERT natural language learning model uses a bidirectional encoder and self-attention mechanism;

[0033] The feature data extraction module performs text cleaning on the collected road construction information data, then uses the BERT natural language learning model to segment the road construction information data, and then extracts the feature data of each event in the road construction information data;

[0034] The event information extraction module uses the pre-trained BERT natural language learning model to extract the time and location information of each event from the feature data of the road construction information data;

[0035] The distribution network information matching module collects the longitude and latitude coordinates of the distribution network route map and draws the route map on the map based on the longitude and latitude coordinates; converts the locations of each event in the road construction information data into longitude and latitude coordinates and marks them on the map to complete the matching of each event with the distribution network route map;

[0036] The risk level warning module traverses and calculates the distance between each event point and all points on the route map, sets the length of the median perpendicular line connecting each event point to the two nearest points adjacent to the event point on the route map as the risk judgment condition, and sets the warning level according to the length of the median perpendicular line.

[0037] As a preferred implementation, the steps of performing text cleaning on the collected road construction information data, then segmenting the road construction information data using the BERT natural language learning model, and then extracting feature data of each event in the road construction information data are specifically as follows:

[0038] Remove unnecessary characters and punctuation marks from the road construction information data text, and delete special symbols and blank characters;

[0039] The word segmenter corresponding to the pre-trained BERT natural language learning model is loaded using the Transformers database to segment the road construction information data with redundant characters removed and organize it into vocabulary fragments;

[0040] Input the vocabulary fragment into the BERT natural language learning model to obtain the feature vector.

[0041] As a preferred implementation, the step of extracting the time and location information of each event from the feature data of the road construction information data using the pre-trained BERT natural language learning model is specifically as follows:

[0042] In the BERT natural language learning model, the relationship between temporal information and long-distance dependencies in the feature vector is captured based on position encoding and multi-head attention mechanism;

[0043] Fine-tune the parameters of the BERT natural language learning model based on different task training sets to enhance the ability to understand the data set;

[0044] Use the fine-tuned BERT natural language learning model to extract time and place information from the input vocabulary fragment;

[0045] Convert the extracted time and location information into the specified output format for output.

[0046] As a preferred implementation, the steps of collecting the longitude and latitude coordinates of the distribution network route map and drawing the route map on the map according to the longitude and latitude coordinates; converting each event location in the road construction information data into longitude and latitude coordinates and marking them on the map, and completing the matching steps of each event with the distribution network route map are as follows:

[0047] Get the longitude and latitude coordinates of the distribution network map and store them as a json file;

[0048] Use openlayers in webgis to draw a route map on the map based on the json file;

[0049] Call the Amap API: geocoder.getAddress to convert the event location into longitude and latitude coordinates;

[0050] Print the event's latitude and longitude coordinates on the distribution network map to match the distribution network map with the event.

[0051] On the other hand, the present invention proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements a distribution network line risk prediction method based on natural language processing and rule matching as described in any embodiment of the present invention.

[0052] On the other hand, the present invention proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a distribution network line risk prediction method based on natural language processing and rule matching as described in any embodiment of the present invention.

[0053] The present invention has the following beneficial effects:

[0054] 1. The present invention is based on the Rust language combined with the functions of several databases to capture road construction information data fragments with different structures in the target web page, so as to integrate multi-source heterogeneous information in different web pages.

[0055] 2. This paper uses the bidirectional encoder and self-attention mechanism in the BERT model to extract key information from a large amount of captured data, greatly enhancing the model's ability to capture language structure.

[0056] 3. The BERT model of the present invention fine-tunes parameters for different task training sets to generate a BERT model that meets the information comprehension capabilities of different fields, thereby improving the accuracy of feature extraction.

[0057] 4. The present invention analyzes the actual longitude and latitude coordinates of the distribution network line map and the longitude and latitude coordinates of each extracted event on the same map, takes them as the actual judgment conditions of road construction risks, and serves as a warning for the possible degree of danger, thereby improving the level of safety protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of the steps of the present invention;

[0059] Figure 2 A flowchart for web crawlers and data analysis;

[0060] Figure 3 Flowchart of the time extraction steps. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0065] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0066] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0067] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0068] Embodiment 1:

[0069] See also Figure 1 , a distribution network line risk prediction method based on natural language processing and rule matching, the specific steps include:

[0070] Based on Rust language and using several databases to collect road construction information data from the target web page;

[0071] In this embodiment, the databases used include reqwest and spider databases. The reqwest library is used to make network requests, and the spider library is used to perform efficient page content parsing. When designing the crawler, the focus is on the structural features of the target web page and the information fragments related to road construction are captured. The specific crawler code design is as follows:

[0072]

[0073] Step 2: Implement the Scraper trait to parse HTML

[0074]

[0075] Then use the Scraper trait to parse HTML:

[0076]

[0077] Clean the captured road construction related information fragments and delete the ` <script>`标签内容。

[0078]

[0079] 进行信息抓取的爬虫主体部分设计:

[0080] 为每个查询创建一个异步任务来抓取数据。

[0081]

[0082]

[0083] 等待所有抓取任务完成,并将结果通过nlp后端解析

[0084]

[0085] 该步骤的流程图如图2所示。

[0086] 构建基于Transformers架构的BERT自然语言学习模型并进行预训练,所述BERT自然语言学习模型采用双向编码器和自注意力机制;

[0087] 在本实施例中,双向编码器:BERT的双向编码器设计使其在预测每个单词的表示时,能够同时考虑到前后文信息。这种全面的上下文理解力是通过复杂的自注意力机制实现的,极大地增强了模型对语言结构的捕捉能力,尤其是在处理从网页、社交媒体和公共记录等来源收集的非结构化或半结构化文本数据时。

[0088] 自注意力机制:自注意力机制允许模型在处理输入时自动评估各个单词之间的相关性,并据此调整其表达。这一机制尤其适用于识别和理解道路施工报道中的关键词汇和句子结构,从而精确提取出施工地点、时间以及相关的安全风险等信息。

[0089] 对采集到的道路施工信息数据进行文本清理,接着利用BERT自然语言学习模型将道路施工信息数据进行分词,再提取道路施工信息数据中的各事件特征数据;

[0090] 在本实施例中,各操作步骤的具体代码如下:

[0091] Step1:文本清理。

[0092] 去除不必要的字符和标点,处理特殊符号和空白字符。

[0093] cleaned_text=re.sub(r'[^\w\s()\(\)\-]',”,raw_text)

[0094] Step2:使用预训练模型进行特征提取。

[0095] 利用预训练的BERT模型提取文本中的特征

[0096] 1、使用`transformers`库加载预训练的BERT模型和对应的分词器

[0097] from transformers import BertTokenizer,BertModel)

[0098] tokenizer=BertTokenizer.from_pretrained('bert-base-chinese')

[0099] model=BertModel.from_pretrained('bert-base-chinese')

[0100] 2、将清理后的文本分成词汇片段(tokens)

[0101] inputs=tokenizer(cleaned_text,return_tensors='pt')

[0102] 获取特征向量

[0103] 3、将分词结果输入BERT模型,获取文本的特征向量

[0104] outputs=model(**inputs)

[0105] last_hidden_states=outputs.last_hidden_state

[0106] 4、考虑到地址间可能用箭头等连接以表示路段,所以我们对地址间不超过三个字符进行连接。

[0107] address_text=text[start_index:end_index].strip()

[0108] filtered_addresses_last.append((address_text,start_index))

[0109] 利用预训练的BERT自然语言学习模型从道路施工信息数据的特征数据中提取出各事件的时间和地点信息;

[0110] 在本实施例中,各操作步骤的具体代码如下:

[0111] Step3:事件及所在文本位置提取。

[0112] 从特征向量中提取出时间和地点信息。

[0113] 1、位置编码和多头注意力机制

[0114] BERT模型使用位置编码(Positional Encoding)和多头注意力机制(Multi-headAttention)来捕捉文本中的时序信息和长距离依赖关系。

[0115] Attention(Q,K,V)=softmax((QK^T) / sqrt(d_k))*V

[0116] 2、微调模型

[0117] 在特定任务数据集上对模型进行微调,以增强其对特定领域(如道路施工报告)的理解能力。

[0118] from transformers import BertForTokenClassification,Trainer,TrainingArguments

[0119] model=BertForTokenClassification.from_pretrained('bert-base-chinese',num_labels=3)

[0120] training_args=TrainingArguments(output_dir='. / results',num_train_epochs=3)

[0121] trainer=Trainer(model=model,args=training_args,train_dataset=train_dataset,eval_dataset=eval_dataset)

[0122] trainer.train()

[0123] 3、事件抽取

[0124] 利用微调后的模型,对输入文本进行预测,抽取出时间和地点信息。

[0125] inputs=tokenizer(test_text,return_tensors='pt')

[0126] outputs=model(**inputs)

[0127] predictions=outputs.logits.argmax(dim=-1)

[0128] Step4:解析预测结果。

[0129] 将预测结果转换为人类可读的格式,提取出具体的时间和地点信息。

[0130] tokens=tokenizer.convert_ids_to_tokens(inputs['input_id s'][0])

[0131] time_entities=[tokens[i]for iinrange(len(predictions[0]))ifpredictions[0][i]==1]

[0132] location_entities=[tokens[i]for iinrange(len(predictions[0]))ifpredictions[0][i]==2]

[0133] 该步骤的流程图如图3所示。

[0134] 采集配电网线路图的经纬度坐标,并根据经纬度坐标在地图上绘制出线路图;将道路施工信息数据中的各事件地点转换为经纬度坐标,并标记于地图上,完成各事件与配电网线路图的匹配;

[0135] 遍历计算各事件点与线路图上所有点的距离,设置各事件点到线路图上与事件点相邻最近两个点连成线段的中垂线长度作为风险判断条件,根据中垂线长度分等级设置预警等级。

[0136] 作为本实施例的优选实施方式,所述对采集到的道路施工信息数据进行文本清理,接着利用BERT自然语言学习模型将道路施工信息数据进行分词,再提取道路施工信息数据中的各事件特征数据步骤具体为:

[0137] 去除道路施工信息数据文本中的非必要字符与标点,删除特殊符号和空白字符;

[0138] 利用Transformers数据库加载预训练的BERT自然语言学习模型对应的分词器对去除多余字符的道路施工信息数据进行分词,并整理成词汇片段;

[0139] 将词汇片段输入至BERT自然语言学习模型中获取特征向量。

[0140] 作为本实施例的优选实施方式,所述利用预训练的BERT自然语言学习模型从道路施工信息数据的特征数据中提取出各事件的时间和地点信息步骤具体为:

[0141] 在BERT自然语言学习模型中基于位置编码和多头注意力机制来捕捉特征向量中的时序信息与长距离依赖的关系;

[0142] 基于不同的任务训练集对BERT自然语言学习模型的参数进行微调整,增强对数据集的理解能力;

[0143] 利用微调后的BERT自然语言学习模型对输入的词汇片段抽取时间和地点信息;

[0144] 将抽取出的时间和地点信息转换成指定的输出格式进行输出。

[0145] 在本实施例中,使用JioNLP工具包,使用NLP任务预处理提取事件事件;使用正则匹配与jieba分词器组合的方式提取事件地点;相邻两段时间间隔视为一个事件,对事件通过文本摘要模型进行摘要提取,再将所有的事件、时间、地点进行组合,实现一个个完整事件提取。

[0146] 作为本实施例的优选实施方式,所述采集配电网线路图的经纬度坐标,并根据经纬度坐标在地图上绘制出线路图;将道路施工信息数据中的各事件地点转换为经纬度坐标,并标记于地图上,完成各事件与配电网线路图的匹配步骤具体为:

[0147] 获取获取配电网线路图的经纬度坐标,存储为json文件;

[0148] 使用webgis中openlayers在地图上根据json文件绘制出线路图;

[0149] 调用高德地图API:geocoder.getAddress,将事件地点转化为经纬度坐标;

[0150] 将事件经纬度坐标打在配电网线路图地图上,完成配电网线路图与事件的匹配。

[0151] 在本实施例中,除了调用高德地图的API进行事件地点的经纬度坐标转化,还可以使用Google Maps Geocoding API、百度地图Geocoding API等地理编码API接口;或者各种地理编码软件。

[0152] 实施例二:

[0153] 基于自然语言处理与规则匹配的配网线路风险预测系统,包括:

[0154] 信息采集模块,基于Rust语言并利用若干数据库来采集目标网页中的道路施工信息数据;

[0155] BERT模型构建模块,构建基于Transformers架构的BERT自然语言学习模型并进行预训练,所述BERT自然语言学习模型采用双向编码器和自注意力机制;

[0156] 特征数据提取模块,对采集到的道路施工信息数据进行文本清理,接着利用BERT自然语言学习模型将道路施工信息数据进行分词,再提取道路施工信息数据中的各事件特征数据;

[0157] 事件信息提取模块,利用预训练的BERT自然语言学习模型从道路施工信息数据的特征数据中提取出各事件的时间和地点信息;

[0158] 配电网信息匹配模块,采集配电网线路图的经纬度坐标,并根据经纬度坐标在地图上绘制出线路图;将道路施工信息数据中的各事件地点转换为经纬度坐标,并标记于地图上,完成各事件与配电网线路图的匹配;

[0159] 风险等级预警模块,遍历计算各事件点与线路图上所有点的距离,设置各事件点到线路图上与事件点相邻最近两个点连成线段的中垂线长度作为风险判断条件,根据中垂线长度分等级设置预警等级。

[0160] 作为本实施例的优选实施方式,所述对采集到的道路施工信息数据进行文本清理,接着利用BERT自然语言学习模型将道路施工信息数据进行分词,再提取道路施工信息数据中的各事件特征数据步骤具体为:

[0161] 去除道路施工信息数据文本中的非必要字符与标点,删除特殊符号和空白字符;

[0162] 利用Transformers数据库加载预训练的BERT自然语言学习模型对应的分词器对去除多余字符的道路施工信息数据进行分词,并整理成词汇片段;

[0163] 将词汇片段输入至BERT自然语言学习模型中获取特征向量。

[0164] 作为本实施例的优选实施方式,所述利用预训练的BERT自然语言学习模型从道路施工信息数据的特征数据中提取出各事件的时间和地点信息步骤具体为:

[0165] 在BERT自然语言学习模型中基于位置编码和多头注意力机制来捕捉特征向量中的时序信息与长距离依赖的关系;

[0166] 基于不同的任务训练集对BERT自然语言学习模型的参数进行微调整,增强对数据集的理解能力;

[0167] 利用微调后的BERT自然语言学习模型对输入的词汇片段抽取时间和地点信息;

[0168] 将抽取出的时间和地点信息转换成指定的输出格式进行输出。

[0169] 作为本实施例的优选实施方式,所述采集配电网线路图的经纬度坐标,并根据经纬度坐标在地图上绘制出线路图;将道路施工信息数据中的各事件地点转换为经纬度坐标,并标记于地图上,完成各事件与配电网线路图的匹配步骤具体为:

[0170] 获取获取配电网线路图的经纬度坐标,存储为json文件;

[0171] 使用webgis中openlayers在地图上根据json文件绘制出线路图;

[0172] 调用高德地图API:geocoder.getAddress,将事件地点转化为经纬度坐标;

[0173] 将事件经纬度坐标打在配电网线路图地图上,完成配电网线路图与事件的匹配。

[0174] 实施例三:

[0175] 本实施例一种电子设备,包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序,所述处理器执行所述程序时实现本发明任一实施例所述的一种基于自然语言处理与规则匹配的配网线路风险预测方法。

[0176] 实施例四:

[0177] 一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现本发明任一实施例所述的一种基于自然语言处理与规则匹配的配网线路风险预测方法。

[0178] 以上所述仅为本发明的实施例,并非因此限制本发明的专利范围,凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本发明的专利保护范围内。< / script>

Claims

1. A distribution network line risk prediction method based on natural language processing and rule matching, characterized in that: The specific steps include: Based on Rust language and using several databases to collect road construction information data from the target web page; Build and pre-train a BERT natural language learning model based on the Transformers architecture, which uses a bidirectional encoder and self-attention mechanism; The collected road construction information data is cleaned up, and then the road construction information data is segmented using the BERT natural language learning model, and then the feature data of each event in the road construction information data is extracted; The pre-trained BERT natural language learning model is used to extract the time and location information of each event from the feature data of the road construction information data; Collect the longitude and latitude coordinates of the distribution network route map, and draw the route map on the map based on the longitude and latitude coordinates; convert the locations of each event in the road construction information data into longitude and latitude coordinates, and mark them on the map to complete the matching of each event with the distribution network route map; The distance between each event point and all points on the route map is calculated by traversal, and the length of the perpendicular median line connecting each event point to the two nearest points adjacent to the event point on the route map is set as the risk judgment condition, and the warning level is set according to the length of the perpendicular median line.

2. The distribution network line risk prediction method based on natural language processing and rule matching according to claim 1 is characterized in that: The steps of performing text cleaning on the collected road construction information data, then segmenting the road construction information data using the BERT natural language learning model, and then extracting feature data of each event in the road construction information data are specifically as follows: Remove unnecessary characters and punctuation marks from the road construction information data text, and delete special symbols and blank characters; The word segmenter corresponding to the pre-trained BERT natural language learning model is loaded using the Transformers database to segment the road construction information data with redundant characters removed and organize it into vocabulary fragments; Input the vocabulary fragment into the BERT natural language learning model to obtain the feature vector.

3. The distribution network line risk prediction method based on natural language processing and rule matching according to claim 2 is characterized in that: The steps of extracting the time and location information of each event from the feature data of the road construction information data using the pre-trained BERT natural language learning model are specifically as follows: In the BERT natural language learning model, the relationship between temporal information and long-distance dependencies in the feature vector is captured based on position encoding and multi-head attention mechanism; Fine-tune the parameters of the BERT natural language learning model based on different task training sets to enhance the ability to understand the data set; Use the fine-tuned BERT natural language learning model to extract time and place information from the input vocabulary fragment; Convert the extracted time and location information into the specified output format for output.

4. The distribution network line risk prediction method based on natural language processing and rule matching according to claim 1 is characterized in that: The steps of collecting the longitude and latitude coordinates of the distribution network route map and drawing the route map on the map according to the longitude and latitude coordinates; converting each event location in the road construction information data into longitude and latitude coordinates and marking them on the map, and completing the matching of each event with the distribution network route map are specifically as follows: Get the longitude and latitude coordinates of the distribution network map and store them as a json file; Use openlayers in webgis to draw a route map on the map based on the json file; Call the Amap API: geocoder.getAddress to convert the event location into longitude and latitude coordinates; Print the event's latitude and longitude coordinates on the distribution network map to match the distribution network map with the event.

5. The distribution network line risk prediction system based on natural language processing and rule matching is characterized by: include: The information collection module is based on the Rust language and uses several databases to collect road construction information data from the target web page; BERT model building module, which builds and pre-trains the BERT natural language learning model based on the Transformers architecture. The BERT natural language learning model uses a bidirectional encoder and self-attention mechanism; The feature data extraction module performs text cleaning on the collected road construction information data, then uses the BERT natural language learning model to segment the road construction information data, and then extracts the feature data of each event in the road construction information data; The event information extraction module uses the pre-trained BERT natural language learning model to extract the time and location information of each event from the feature data of the road construction information data; The distribution network information matching module collects the longitude and latitude coordinates of the distribution network route map and draws the route map on the map according to the longitude and latitude coordinates; Convert the locations of each event in the road construction information data into longitude and latitude coordinates and mark them on the map to match each event with the distribution network line map; The risk level warning module traverses and calculates the distance between each event point and all points on the route map, sets the length of the median perpendicular line connecting each event point to the two nearest points adjacent to the event point on the route map as the risk judgment condition, and sets the warning level according to the length of the median perpendicular line.

6. The distribution network line risk prediction system based on natural language processing and rule matching according to claim 5 is characterized in that: The steps of performing text cleaning on the collected road construction information data, then segmenting the road construction information data using the BERT natural language learning model, and then extracting feature data of each event in the road construction information data are specifically as follows: Remove unnecessary characters and punctuation marks from the road construction information data text, and delete special symbols and blank characters; The word segmenter corresponding to the pre-trained BERT natural language learning model is loaded using the Transformers database to segment the road construction information data with redundant characters removed and organize it into vocabulary fragments; Input the vocabulary fragment into the BERT natural language learning model to obtain the feature vector.

7. The distribution network line risk prediction system based on natural language processing and rule matching according to claim 6 is characterized in that: The steps of extracting the time and location information of each event from the feature data of the road construction information data using the pre-trained BERT natural language learning model are specifically as follows: In the BERT natural language learning model, the relationship between temporal information and long-distance dependencies in the feature vector is captured based on position encoding and multi-head attention mechanism; Fine-tune the parameters of the BERT natural language learning model based on different task training sets to enhance the ability to understand the data set; Use the fine-tuned BERT natural language learning model to extract time and place information from the input vocabulary fragment; Convert the extracted time and location information into the specified output format for output.

8. The distribution network line risk prediction system based on natural language processing and rule matching according to claim 5 is characterized in that: The steps of collecting the longitude and latitude coordinates of the distribution network route map and drawing the route map on the map according to the longitude and latitude coordinates; converting each event location in the road construction information data into longitude and latitude coordinates and marking them on the map, and completing the matching of each event with the distribution network route map are specifically as follows: Get the longitude and latitude coordinates of the distribution network map and store them as a json file; Use openlayers in webgis to draw a route map on the map based on the json file; Call the Amap API: geocoder.getAddress to convert the event location into longitude and latitude coordinates; Print the event's latitude and longitude coordinates on the distribution network map to match the distribution network map with the event.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements a distribution network line risk prediction method based on natural language processing and rule matching as described in any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a distribution network line risk prediction method based on natural language processing and rule matching as described in any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • Method for extracting structured traffic road condition information from voice data

    CN106504746A