Power distribution network project evaluation method and system based on distributed file system

Abstract

The present invention belongs to the field of engineering risk assessment. The present invention provides a method and system for evaluating power distribution network engineering based on a distributed file system, which obtains relevant data; pre-processes the acquired relevant data, uses a distributed file system to store the differentially removed relevant data in a distributed manner, and performs secondary processing of the corresponding data in a distributed manner to form a data set; extracts feature information from the data set, and uses a pre-trained decision tree model to predict the transaction risk level under different feature information; constructs a curve chart or benefit map based on the feature information and prediction results of the decision tree model. The present invention can accurately evaluate the investment benefits of power distribution network engineering by conducting comprehensive data collection and analysis on investment projects, combined with economic indicators and risk assessment, and provide scientific basis for decision makers, and conduct real-time monitoring and adjustment to improve the efficiency and results of the project.

Description

Technical Field

[0001] The present invention belongs to the field of engineering risk assessment, and in particular relates to a method and system for evaluating an electric power distribution network engineering based on a distributed file system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of the power industry, the scale and complexity of power distribution network projects are increasing. In order to ensure the feasibility and benefits of investment projects, a comprehensive investment benefit evaluation is required. Investment benefit evaluation refers to the analysis of the economic and social benefits of investment projects, and on this basis, the technical feasibility, economic benefits and risks faced by investment projects are concluded accordingly, which serves as the basis for project decision-making.

[0004] Traditional evaluation methods face difficulties in data collection and processing. Traditional review methods rely on limited data samples and empirical judgments, making it difficult to comprehensively and accurately evaluate the benefits of project investment. In addition, existing review tools cannot effectively process and analyze large amounts of complex data, resulting in possible errors and inaccuracies in the evaluation results. The ability to conduct risk assessment and cost optimization is also limited, and potential risk factors cannot be accurately identified and predicted. In addition, the analysis and optimization of cost structures are not comprehensive and detailed enough.

[0005] These defects are mainly due to the inadequacy of traditional post-evaluation methods in data processing and analysis. Traditional methods can usually only process small-scale data sets, and it is difficult for traditional methods to efficiently process and analyze large amounts of complex data. In addition, traditional methods are also limited in their technical applications in data mining and other areas, and are unable to fully utilize the potential information in the data. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a method and system for evaluating power distribution network projects based on a distributed file system. The present invention can accurately evaluate the investment benefits of power distribution network projects by comprehensively collecting and analyzing investment projects, combining economic indicators and risk assessment, and provide scientific basis for decision makers, and conduct real-time monitoring and adjustment to improve the efficiency and results of the project. In addition, the present invention can identify and predict potential risk factors by analyzing historical data and other relevant data, and take corresponding measures for risk control and management.

[0007] According to some embodiments, the present invention adopts the following technical solutions:

[0008] A method for evaluating a power distribution network project based on a distributed file system comprises the following steps:

[0009] Obtain data on the project scale, investment cost, operation and maintenance costs, winning bid information, and power load demand of power distribution network projects;

[0010] Preprocess the acquired relevant data and convert them into time series data respectively;

[0011] Group the converted time series data and remove data differences according to the groups;

[0012] Using a distributed file system, the relevant data with differences removed is stored in a distributed manner, and the corresponding data is processed in a distributed manner to form a data set;

[0013] Extract feature information from the data set and use the pre-trained decision tree model. Each node in the decision tree model represents a feature information, each branch represents the corresponding feature value, and the final leaf node represents the classification result to predict the transaction risk level under different feature information.

[0014] Construct a curve chart or benefit map based on the feature information and prediction results of the decision tree model.

[0015] As an optional implementation method, the specific process of obtaining data related to the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand of the power distribution network project includes:

[0016] The project scale-related data include line length, number of substations and equipment capacity;

[0017] The investment cost-related data include construction costs, equipment purchase costs and labor costs;

[0018] The operating and maintenance cost-related data include equipment maintenance costs, staff wages and energy consumption costs;

[0019] The power load demand related data include power consumption, load curve and peak-to-valley difference;

[0020] The data related to the winning bid information include the investment amount, payment amount, whether the winning bidder is on the list of subcontractors, whether there is a commissioning time and the winning bidder's historical engineering reputation.

[0021] As an optional implementation, the specific process of preprocessing the acquired relevant data includes: standardizing each data according to the time series format, removing missing values ​​and outliers in the standardized data, and then smoothing the data to eliminate noise and fluctuations.

[0022] As an optional implementation, the specific process of converting into time series data includes: determining a time interval, arranging each item of preprocessed data according to the determined time interval and a preset time sequence, and forming the time series data.

[0023] As an optional implementation, the specific process of grouping the converted time series data includes: storing the time series data in the data center, grouping the time series data according to the preset collection and storage cycle of the data center, and obtaining multiple data groups of corresponding data types.

[0024] As an optional implementation, the specific process of performing differential data removal according to groups includes:

[0025] Calculate the difference coefficient between the current data group and the previous data group under the corresponding data types in sequence;

[0026] Determine whether the value of the differentiation coefficient is within a preset range. If the value of the differentiation coefficient is not within the preset range, remove the corresponding data group;

[0027] If the value of the differentiation coefficient is within the preset range, the corresponding data group is retained until all data groups are traversed to obtain the time series data with differentiation removed.

[0028] As a further step, the calculation process of the difference coefficient between the current data group and the previous data group includes: for:

[0029] ;

[0030] in, is the coefficient of variation, represents the covariance between the current data set X and the previous data set Y, represents the mean of the current data set X, Represents the mean of the previous data set Y.

[0031] As an optional implementation, the specific process of performing distributed secondary processing of corresponding data includes: storing different data groups on different nodes respectively, aggregating data using query statements, and performing data calculation using Map function and Reduce function.

[0032] As an optional implementation, the training of the decision tree model is performed based on the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand related data of historical power distribution network projects.

[0033] As an optional implementation, the characteristic information is selected based on the evaluation requirements of the power distribution network project, and multiple factors such as the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand related data of the power distribution network project are used as the characteristic information.

[0034] As an optional implementation, the decision tree model is constructed according to the improved ID3 algorithm, and the specific construction process includes:

[0035] (1) Initialize data;

[0036] (2) Determine whether the attribute of the feature information is a discrete attribute. If it is a discrete attribute, jump to step (3). Otherwise, determine the number of values ​​after discretization, apply the K-means++ algorithm for discretization, and replace the original continuous value with the discrete value.

[0037] (3) Use the SAG (a, P, Q) algorithm to calculate the importance of each feature information;

[0038] (4) Select the most important feature information as the split node to split the sample set;

[0039] (5) Repeat steps (3) and (4) to select the remaining feature information segmentation sample sets until all feature information is used as split nodes;

[0040] (6) Generate a decision tree model and perform pruning operations.

[0041] A power distribution network engineering evaluation system based on a distributed file system, comprising:

[0042] A data acquisition module is configured to acquire data related to the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand of the power distribution network project;

[0043] A data preprocessing module is configured to preprocess the acquired relevant data and convert them into time series data respectively;

[0044] A data grouping module is configured to group each of the converted time series data and remove data differences according to the groups;

[0045] The distributed processing module is configured to use a distributed file system to store the relevant data with the difference removed in a distributed manner, and to perform secondary processing of the corresponding data in a distributed manner to form a data set;

[0046] The model classification module is configured to extract feature information from the data set, and use the pre-trained decision tree model. Each node in the decision tree model represents a feature information, each branch represents the corresponding feature value, and the final leaf node represents the classification result to predict the transaction risk level under different feature information;

[0047] The visualization module is configured to construct a curve chart or a benefit map according to the feature information and prediction results of the decision tree model.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention can improve the accuracy and comprehensiveness of the assessment, and support more refined risk assessment and cost optimization. By using the parallel processing of the distributed file system, it can process and analyze large-scale and complex data sets and deeply mine the valuable information hidden in the data. By comprehensively considering various factors such as investment cost, operating income, risk and market demand, more accurate investment benefit indicators can be obtained, and a scientific basis can be provided for decision makers.

[0050] The present invention makes full use of the high reliability, high scalability, data locality and high throughput characteristics of the distributed file system HDFS to achieve storage, processing and evaluation of large-scale data. By using the distributed file system, various data of the power distribution network project can be stored in a distributed manner on multiple nodes in the cluster. This distributed storage method of data ensures high reliability of the data, because each data block will be copied to multiple nodes, even if a node fails, the data still exists reliably on other nodes. This data redundancy mechanism provides fault tolerance and ensures the reliability and availability of data.

[0051] As the amount of data in the power distribution network project increases, the present invention can expand the storage capacity of the HDFS cluster by simply adding more nodes. This horizontal expansion capability allows the system to easily cope with the growing data demand. HDFS also has the characteristics of data locality and high throughput, which is crucial for the implementation of post-investment benefit review methods and tools for power distribution network projects. Data locality means that data is stored close to the computing nodes, which can minimize data transmission and network overhead and improve data access speed and efficiency; it can process data in a parallel and high-throughput manner, providing fast and efficient data access capabilities.

[0052] The present invention has a wide range of applications and can also strengthen the closed-loop control of the entire investment process, focus on efficiency and benefits, improve the level of investment decision-making, and improve the investment decision-making mechanism. It can also conduct a comprehensive, systematic, and objective analysis and evaluation of the entire process of project planning, preparatory work, construction, and operation after the project is completed and accepted and put into use or has been in operation for a certain period of time.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0055] Figure 1 The present invention is a schematic diagram of a post-assessment process of investment benefits of a power distribution network project according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0057] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0058] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0059] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0060] Embodiment 1

[0061] A post-evaluation method for investment benefits of power distribution network projects based on HDFS technology, such as Figure 1 As shown, the following steps are included:

[0062] Step 1: Data collection and storage

[0063] Based on the power grid data center, all data of power distribution network projects are comprehensively collected and stored, including data such as project scale, investment cost, operation and maintenance cost, and power load demand. The collected data is transferred to HDFS for storage. The high scalability and fault tolerance of HDFS enable it to easily handle the storage needs of large-scale data and ensure the reliability and availability of data.

[0064] Step 2: Convert the collected data into time series data. The specific technical means are as follows:

[0065] Step 2.1: Data Collection:

[0066] Collect scale data of power distribution network projects from engineering design documents, field survey reports, construction drawings and other materials, including line length, number of substations, equipment capacity, etc.

[0067] Collect investment cost data of power distribution network projects from financial statements, project budgets, procurement records and other materials, including construction costs, equipment procurement costs, labor costs, etc.

[0068] Collect operation and maintenance cost data of power distribution network projects from maintenance records, energy consumption statistics, staff salary lists and other materials, including equipment maintenance costs, staff salaries, energy consumption costs, etc.

[0069] Collect power load demand data of power distribution network projects from sources such as power user information, meter readings, load forecasting models, etc., including power consumption, load curve, peak-to-valley difference, etc.

[0070] The collected data is preprocessed as raw data and finally used as feature information for model construction and training.

[0071] Step 2.2: Establish a data middle platform: Establish a power grid data middle platform as a platform for centralized data storage and management; the power grid data middle platform includes components such as data warehouse and data lake.

[0072] Step 2.3: Data format standardization: Standardize all data according to the time series format to ensure data consistency and comparability.

[0073] Step 2.4: Time series data conversion steps:

[0074] Step 2.4.1: Data cleaning and preprocessing: Remove missing values ​​and outliers from the standardized data to ensure the integrity and accuracy of the data. Smooth the data to eliminate noise and fluctuations.

[0075] Step 2.4.2: Time series processing

[0076] Arrange the items and preprocessed data in a preset time sequence to form time series data. Determine the time interval, such as hourly, daily, monthly, etc. The preset time sequence extracts the existing time series data from the database, data warehouse or historical records. This method is particularly suitable for analyzing data with historical trends and patterns, such as investment costs, operation and maintenance costs of power distribution network projects, etc.

[0077] Step 2.4.3: Data storage

[0078] Store time series data in a database or data warehouse to ensure that the data can be queried and visualized.

[0079] Through the above steps, the data of the project scale, investment cost, operation and maintenance cost, power load demand, etc. of this method can be converted into time series data and stored in the power grid data center, providing a basis for subsequent analysis and mining.

[0080] Step 3: Remove data differences from each converted time series data. The specific technical means are as follows:

[0081] Step 3.1: Group the time series data according to the collection and storage cycle of the preset data center to obtain multiple data groups corresponding to the data types;

[0082] Step 3.2: Calculate the differentiation coefficient between the current data group and the previous data group under the corresponding data type; because the data center will be affected by its own parameters, the data collected and stored at a certain moment will have a large deviation. Therefore, this technology can screen out abnormal monitoring values ​​through the differentiation coefficient, thereby ensuring the accuracy of historical data and real-time data. The formula for the differentiation coefficient is ;in, is the coefficient of variation, represents the covariance between the current data set X and the previous data set Y, represents the mean of the current data set X, Represents the mean of the previous data set Y.

[0083] Step 3.3: Determine whether the value of the differentiation coefficient is within a preset range;

[0084] Step 3.4: If the value of the differentiation coefficient is not within the preset range, the corresponding data group is removed;

[0085] Step 3.5: If the value of the differentiation coefficient is within the preset range, the corresponding data group is retained until all data groups are traversed to obtain the processed time series data.

[0086] Step 4: Data cleaning and transformation

[0087] As a data transfer station, the data center is connected to the HDFS distributed file system wirelessly, and the data in the data transfer station is transferred to the HDFS distributed file system. The data processing tools in the Hadoop ecosystem, such as MapReduce and Hive, are used to clean, convert and aggregate the data stored in the HDFS. This step can remove invalid or redundant data and convert the data into a format that can be used for analysis.

[0088] In this embodiment, the secondary data processing process is applied, that is, in the differentiated processing process of the data center, the data is processed initially, and for the data stored in HDFS, the distributed processing capability is used to perform a more efficient secondary processing. The specific steps are as follows:

[0089] Leverage distributed processing capabilities: Data processing tools in the HDFS and Hadoop ecosystem (such as MapReduce and Hive) have powerful distributed processing capabilities, which can process large-scale data in parallel and improve the efficiency and speed of data processing.

[0090] Data integration and unified standards: Storing data in HDFS can achieve data integration and unified standards, which is convenient for subsequent data analysis and mining. Data from different data sources may suffer from data processing loss and other problems during the initial standardization process, which can be uniformly processed and organized in HDFS.

[0091] Data processing and transformation: The Hadoop ecosystem provides a variety of data processing and transformation tools that can perform complex transformations, aggregations, and calculations on data. Through tools such as Hive and MapReduce, data can be processed and processed again to obtain more valuable data. The steps for data aggregation using Hive are as follows:

[0092] Use Hive's SQL-like language (HiveQL) to write query statements for data aggregation. Example:

[0093] SELECT column1, SUM(column2) AS total_sum

[0094] FROM table_name

[0095] GROUP BY column1;

[0096] Write Map function and Reduce function to calculate and process data. Example in this embodiment:

[0097] Map function: public class MyMapper extends Mapper<LongWritable, Text,Text, IntWritable> {

[0098] public void map(LongWritable key, Text value, Context context) {

[0099] / / Process input value and emit intermediate key-value pairs

[0100] }

[0101] }

[0102] Reduce function:

[0103] public class MyReducer extends Reducer<Text, IntWritable, Text,IntWritable>{

[0104] public void reduce(Text key, Iterable <intwritable>values, Contextcontext) {

[0105] / / Aggregate values ​​based on the key

[0106] }

[0107] }

[0108] The specific calculation formulas include sum, average, count, maximum and minimum. The corresponding formulas are as follows:

[0109] ; ;Count=n;Max=max(x1,x2,...,x n ); Min = min(x1,x2,...,x n )wait.

[0110] Among them, x i represents the i-th data, and n is the total number of data.

[0111] Through the Hive and MapReduce tools in the Hadoop ecosystem, data can be flexibly aggregated and calculated to meet complex data processing requirements, and the above formulas can be used to perform various statistics and analyses on the data.

[0112] Fully utilize cluster resources: HDFS and the Hadoop ecosystem can fully utilize cluster resources to achieve parallel processing and distributed computing of data, and improve the speed and efficiency of data processing. Through reasonable data partitioning and task scheduling, the performance advantages of the cluster can be fully utilized.

[0113] Data quality control: Re-preprocessing and cleaning in HDFS can further improve the quality and accuracy of the data. This embodiment can ensure that the quality of the data meets the needs of analysis and modeling by identifying and processing invalid or redundant data.

[0114] Step 5: Data processing and segmentation

[0115] According to the results of data cleaning and conversion, the data is processed and segmented. Data processing includes processing missing data into non-missing values ​​according to business logic and converting enumeration variables into numerical variables. Data segmentation requires the data to be split into two parts: training data set (80%) and test data set (20%).

[0116] Among them, the training data set is used to train the evaluation model, and the test data set is used to test whether the evaluation model is accurate.

[0117] Of course, when it is officially applied, the trained and tested evaluation model can be used to perform predictive classification on the newly acquired project information.

[0118] The cleaned data appear as feature information and are input into the model as preset data on the winning bidder’s list of subcontractors, whether there is a time for commissioning, and the credibility of the winning bidder.

[0119] Step 6: Build the evaluation model

[0120] The training data set is used to build a decision tree model using the improved ID3 algorithm.

[0121] In this embodiment, a decision tree model is constructed based on characteristic information such as investment amount, payment amount, the list of subcontractors of the winning bidder, whether it has been put into operation, and the credibility of the winning bidder. Each node represents a characteristic information, each branch represents a characteristic value, and the final leaf node represents a classification result, which predicts the transaction risk level under different operations.

[0122] Of course, in other embodiments, other characteristic information may be selected to construct a decision tree model, for example, several factors may be selected from the scale data, investment cost data, operation and maintenance cost data, and power load demand data of the power distribution network project as characteristic information.

[0123] Among them, the improved ID3 algorithm is as follows:

[0124] Data initialization: Count the number of samples in the training set. Suppose the training set is D, with a total of K categories. Count the number of samples in each category D1, ..., Dk.

[0125] Attribute discretization: Determine whether the attribute (i.e., feature information) is a discrete attribute. If it is a discrete attribute, jump to step 3. Otherwise, determine the number of values ​​after discretization, apply the K-means++ algorithm for discretization, and replace the original continuous value with a discrete value. Regarding the K-means++ algorithm, it is an algorithm for clustering. By selecting the initialized cluster center and iteratively updating the cluster center, k clusters and corresponding cluster centers are finally obtained. The key to the algorithm is to select the initial cluster center as far as possible and iteratively update until convergence.

[0126] Calculate the importance of attributes: Calculate the importance of attributes (i.e. feature information) using the SAG(a, P, Q) algorithm. The SAG algorithm is used to calculate the importance of attributes. The importance of attribute a is evaluated by calculating the ratio of the number of equivalence classes when attribute a is added to conditional attribute P and when conditional attribute P is not added. SAG(a, P, Q) represents the importance of attribute a to conditional attribute P and equivalence relation Q, and can be used to select important conditional attributes for segmentation.

[0127] Split the sample set: Select the conditional attribute with the greatest importance as the splitting node to split the sample set.

[0128] Continue to split the sample set: Repeat steps 3 and 4, and select the remaining conditional attributes to split the sample set until all conditional attributes are used as splitting nodes.

[0129] Pruning: Generate a decision tree and perform pruning operations to improve the generalization ability of the model.

[0130] Table 1 Decision tree model feature information table

[0131]

[0132] Step 7: Visualize the results

[0133] Based on the model output results, an asset efficiency and benefit map was constructed, and an asset two-dimensional matrix analysis model was drawn according to the map colors to assist in the investment analysis of power grid construction. Using the two-dimensional matrix analysis chart, a reserve project construction ranking model was built to help improve subjective problems in project construction and innovate the application of results.

[0134] This embodiment can accurately evaluate the investment benefits of power distribution network projects by collecting and analyzing comprehensive data on investment projects, combining economic indicators and risk assessment, and provide scientific basis for decision makers. It can also monitor and adjust in real time through analytical tools to improve the efficiency and results of projects. By analyzing historical data and other relevant data, potential risk factors can be identified and predicted, and corresponding measures can be taken for risk control and management. In addition, it can also help project investors analyze and optimize the cost structure of the project, thereby improving the benefits of project investment.

[0135] Embodiment 2

[0136] A power distribution network engineering evaluation system based on a distributed file system, comprising:

[0137] A data acquisition module is configured to acquire data related to the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand of the power distribution network project;

[0138] A data preprocessing module is configured to preprocess the acquired relevant data and convert them into time series data respectively;

[0139] A data grouping module is configured to group each of the converted time series data and remove data differences according to the groups;

[0140] The distributed processing module is configured to use a distributed file system to store the relevant data with the difference removed in a distributed manner, and to perform secondary processing of the corresponding data in a distributed manner to form a data set;

[0141] The model classification module is configured to extract feature information from the data set, and use the pre-trained decision tree model. Each node in the decision tree model represents a feature information, each branch represents the corresponding feature value, and the final leaf node represents the classification result to predict the transaction risk level under different feature information;

[0142] The visualization module is configured to construct a curve chart or a benefit map according to the feature information and prediction results of the decision tree model.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.< / intwritable>

Claims

1. A method for evaluating a power distribution network project based on a distributed file system, characterized in that: The following steps are involved: Obtain data on the project scale, investment cost, operation and maintenance costs, winning bid information, and power load demand of power distribution network projects; Preprocess the acquired relevant data and convert them into time series data respectively; Group the converted time series data and remove data differences according to the groups; Using a distributed file system, the relevant data with differences removed is stored in a distributed manner, and the corresponding data is processed in a distributed manner to form a data set; Extract feature information from the data set and use the pre-trained decision tree model. Each node in the decision tree model represents a feature information, each branch represents the corresponding feature value, and the final leaf node represents the classification result to predict the transaction risk level under different feature information. Construct a curve chart or benefit map based on the feature information and prediction results of the decision tree model; The specific process of obtaining data on the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand of power distribution network projects includes: Establish a power grid data center as a platform for data set storage and management, standardize various data in the format of time series, remove missing values ​​and outliers in the standardized data, smooth the data, and eliminate noise and fluctuations; Arrange the pre-processed data in a preset time sequence to form time series data, and extract the existing time series data from the database, data warehouse or historical records in a preset time sequence at a determined time interval; The project scale-related data include line length, number of substations and equipment capacity; The investment cost-related data include construction costs, equipment purchase costs and labor costs; The operating and maintenance cost-related data include equipment maintenance costs, staff wages and energy consumption costs; The power load demand related data include power consumption, load curve and peak-to-valley difference; The data related to the winning bid information include the investment amount, payment amount, whether the winning bidder is on the list of subcontractors, whether there is a commissioning time, and the winning bidder's historical project reputation; The specific process of performing differential data removal according to groups includes: Calculate the difference coefficient between the current data group and the previous data group under the corresponding data types in sequence; Determine whether the value of the differentiation coefficient is within a preset range. If the value of the differentiation coefficient is not within the preset range, remove the corresponding data group; If the value of the differentiation coefficient is within the preset range, the corresponding data group is retained until all data groups are traversed to obtain the time series data with differentiation removed; Clean, transform and aggregate data stored in distributed file systems; The training of the decision tree model is based on the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand related data of historical power distribution network projects; The characteristic information is selected based on the evaluation requirements of the power distribution network project, and multiple factors from the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand related data of the power distribution network project are used as the characteristic information.

2. The method for evaluating a power distribution network project based on a distributed file system according to claim 1, characterized in that: The specific process of preprocessing the acquired relevant data includes: standardizing each data according to the time series format, removing missing values ​​and outliers in the standardized data, and then smoothing the data to eliminate noise and fluctuations.

3. The method for evaluating a power distribution network project based on a distributed file system according to claim 1, characterized in that: The specific process of converting to time series data includes: determining the time interval, arranging the pre-processed data according to the determined time interval and the preset time sequence to form the time series data.

4. The method for evaluating a power distribution network project based on a distributed file system according to claim 1, characterized in that: The specific process of grouping the converted time series data includes: storing the time series data in the data center, grouping the time series data according to the preset collection and storage cycle of the data center, and obtaining multiple data groups of corresponding data types.

5. The method for evaluating a power distribution network project based on a distributed file system according to claim 1, characterized in that: Differentiation coefficient for: ； in, is the coefficient of variation, represents the covariance between the current data set X and the previous data set Y, represents the mean of the current data set X, Represents the mean of the previous data set Y.

6. The method for evaluating a power distribution network project based on a distributed file system according to claim 1, characterized in that: The specific process of distributed secondary processing of corresponding data includes: storing different data groups on different nodes, aggregating data using query statements, and calculating data using Map functions and Reduce functions.

7. The method for evaluating a power distribution network project based on a distributed file system according to claim 1, characterized in that: The decision tree model is constructed according to the improved ID3 algorithm, and the specific construction process includes: (1) Initialize data; (2) Determine whether the attribute of the feature information is a discrete attribute. If it is a discrete attribute, jump to step (3). Otherwise, determine the number of values ​​after discretization, apply the K-means++ algorithm for discretization, and replace the original continuous value with the discrete value. (3) Use the SAG (a, P, Q) algorithm to calculate the importance of each feature information; (4) Select the most important feature information as the split node to split the sample set; (5) Repeat steps (3) and (4) to select the remaining feature information segmentation sample sets until all feature information is used as split nodes; (6) Generate a decision tree model and perform pruning operations.

8. A power distribution network engineering evaluation system based on a distributed file system, characterized in that: include: A data acquisition module is configured to acquire data related to the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand of the power distribution network project; A data preprocessing module is configured to preprocess the acquired relevant data and convert them into time series data respectively; A data grouping module is configured to group each of the converted time series data and remove data differences according to the groups; The distributed processing module is configured to use a distributed file system to store the relevant data with the difference removed in a distributed manner, and to perform secondary processing of the corresponding data in a distributed manner to form a data set; The model classification module is configured to extract feature information from the data set, and use the pre-trained decision tree model. Each node in the decision tree model represents a feature information, each branch represents the corresponding feature value, and the final leaf node represents the classification result to predict the transaction risk level under different feature information; A visualization module is configured to construct a curve chart or a benefit map according to the feature information and prediction results of the decision tree model; The specific process of obtaining data on the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand of power distribution network projects includes: The project scale-related data include line length, number of substations and equipment capacity; The investment cost-related data include construction costs, equipment purchase costs and labor costs; The operating and maintenance cost-related data include equipment maintenance costs, staff wages and energy consumption costs; The power load demand related data include power consumption, load curve and peak-to-valley difference; The data related to the winning bid information include the investment amount, payment amount, whether the winning bidder is on the list of subcontractors, whether there is a commissioning time, and the winning bidder's historical project reputation; The specific process of performing differential data removal according to groups includes: Calculate the difference coefficient between the current data group and the previous data group under the corresponding data types in sequence; Determine whether the value of the differentiation coefficient is within a preset range. If the value of the differentiation coefficient is not within the preset range, remove the corresponding data group; If the value of the differentiation coefficient is within the preset range, the corresponding data group is retained until all data groups are traversed to obtain the time series data with differentiation removed; The training of the decision tree model is based on the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand related data of historical power distribution network projects; The characteristic information is selected based on the evaluation requirements of the power distribution network project, and multiple factors from the project scale, investment cost, operation and maintenance cost, winning bid information and power load demand related data of the power distribution network project are used as the characteristic information.

9. The power distribution network engineering evaluation system based on a distributed file system as claimed in claim 8, characterized in that: The specific process of distributed secondary processing of corresponding data includes: storing different data groups on different nodes, aggregating data using query statements, and calculating data using Map functions and Reduce functions.

10. The power distribution network engineering evaluation system based on a distributed file system as claimed in claim 8, characterized in that: The decision tree model is constructed according to the improved ID3 algorithm, and the specific construction process includes: (1) Initialize data; (2) Determine whether the attribute of the feature information is a discrete attribute. If it is a discrete attribute, jump to step (3). Otherwise, determine the number of values ​​after discretization, apply the K-means++ algorithm for discretization, and replace the original continuous value with the discrete value. (3) Use the SAG (a, P, Q) algorithm to calculate the importance of each feature information; (4) Select the most important feature information as the split node to split the sample set; (5) Repeat steps (3) and (4) to select the remaining feature information segmentation sample sets until all feature information is used as split nodes; (6) Generate a decision tree model and perform pruning operations.

Images