Multi-source heterogeneous data processing method and device in digital substation industrial control system

Abstract

The invention discloses a multi-source heterogeneous data processing method and device in an industrial control system of a digital substation, relates to the technical field of power data processing, and mainly aims to improve the cleaning accuracy of data. The method comprises the following steps: acquiring to-be-processed multi-source heterogeneous data in a digital substation industrial control system; performing classification processing on the to-be-processed multi-source heterogeneous data to obtain to-be-processed data under different classification categories; and determining a data processing mode corresponding to each classification category, and processing the to-be-processed data under the corresponding classification category by using each data processing mode to obtain processed data corresponding to each classification category. The multi-source heterogeneous data processing method is suitable for processing the multi-source heterogeneous data in the digital substation industrial control system.

Description

Technical Field

[0001] The present invention relates to the technical field of power data processing, and in particular to a method and device for processing multi-source heterogeneous data in a digital substation industrial control system. Background Art

[0002] As the nerve center of power industry production, the industrial control system of digital substation undertakes important functions such as monitoring, control, protection and data collection, and is an important part of the key information infrastructure of the power system. With the rapid development of information technology, the informatization level of the industrial control system of digital substation has gradually improved, and it has developed from traditional hardware control to intelligent and automated direction, realizing real-time monitoring, intelligent dispatching and automated control of the operation status of the power system. With the increase in the number of access services and the continuous improvement of data collection technology, the data form in the industrial control system of digital substation has become more and more complex and the data scale has become larger and larger. The data of the industrial control system of digital substation may come from various sensors, equipment, systems and intelligent instruments. In order to ensure the availability of data, multi-source data needs to be cleaned.

[0003] Currently, different data are usually cleaned through a unified data cleaning process. However, this unified data cleaning process may not be applicable to all types of data, resulting in low accuracy of data cleaning. Summary of the invention

[0004] The present invention provides a method and device for processing multi-source heterogeneous data in a digital substation industrial control system, which mainly aims to improve the accuracy of data cleaning.

[0005] According to a first aspect of the present invention, a method for processing multi-source heterogeneous data in a digital substation industrial control system is provided, comprising:

[0006] Obtain the multi-source heterogeneous data to be processed in the digital substation industrial control system;

[0007] Classify and process the multi-source heterogeneous data to be processed to obtain data to be processed under different classification categories;

[0008] Determine the data processing method corresponding to each of the classification categories, and use each of the data processing methods to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each of the classification categories.

[0009] According to a second aspect of the present invention, there is provided a multi-source heterogeneous data processing device in a digital substation industrial control system, comprising:

[0010] An acquisition unit, used for acquiring multi-source heterogeneous data to be processed in the digital substation industrial control system;

[0011] A classification unit, used for classifying the multi-source heterogeneous data to be processed to obtain data to be processed under different classification categories;

[0012] The processing unit is used to determine the data processing method corresponding to each of the classification categories, and use each of the data processing methods to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each of the classification categories.

[0013] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-source heterogeneous data processing method in the digital substation industrial control system.

[0014] According to a fourth aspect of the present invention, there is provided a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned multi-source heterogeneous data processing method in a digital substation industrial control system when executing the program.

[0015] According to a method and device for processing multi-source heterogeneous data in a digital substation industrial control system provided by the present invention, compared with the current method of cleaning different data through a unified data cleaning process, the present invention obtains the multi-source heterogeneous data to be processed in the digital substation industrial control system; then classifies the multi-source heterogeneous data to be processed to obtain the data to be processed under different classification categories; finally determines the data processing method corresponding to each classification category, and uses each data processing method to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each classification category. In this way, by pre-classifying the multi-source heterogeneous data in the digital substation industrial control system, the data to be processed under different classification categories are obtained, and then the data processing method applicable to different classification categories is determined, and finally the data processing method is used to process the data under the corresponding classification category, and the applicable data processing method is determined by the data category, thereby improving the accuracy of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0017] Figure 1 A flow chart of a multi-source heterogeneous data processing method in a digital substation industrial control system provided by an embodiment of the present invention is shown;

[0018] Figure 2 A flowchart of another method for processing multi-source heterogeneous data in a digital substation industrial control system provided by an embodiment of the present invention is shown;

[0019] Figure 3 A schematic diagram of the structure of a multi-source heterogeneous data processing device in a digital substation industrial control system provided by an embodiment of the present invention is shown;

[0020] Figure 4 It shows a schematic diagram of the structure of another multi-source heterogeneous data processing device in a digital substation industrial control system provided by an embodiment of the present invention;

[0021] Figure 5 A schematic diagram of the physical structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0023] At present, different data are cleaned through a unified data cleaning process. This unified data cleaning process may not be applicable to all types of data, resulting in low accuracy of data cleaning.

[0024] In order to solve the above problems, an embodiment of the present invention provides a method for processing multi-source heterogeneous data in a digital substation industrial control system. Figure 1 As shown, the method includes:

[0025] 101. Obtain the multi-source heterogeneous data to be processed in the digital substation industrial control system.

[0026] Among them, the multi-source heterogeneous data to be processed include equipment current, equipment voltage, electric energy metering data, energy consumption statistics, equipment circuit breaker status, equipment switch status, alarm information, equipment fault status, ambient temperature, ambient humidity and other data in the digital substation industrial control system.

[0027] Specifically, the multi-source heterogeneous data to be processed are collected from the equipment current and equipment voltage collected by the internal sensors of the digital substation, the electric energy metering data and energy consumption statistics from the smart meter, the equipment circuit breaker status, equipment switch status, alarm information, equipment fault status from the equipment monitoring system, and the ambient temperature and ambient humidity from the environmental monitoring equipment. Then, the multi-source heterogeneous data to be processed are classified, and suitable data processing methods are determined for different classification categories. Finally, different data processing methods are used to process the data under the corresponding classification categories. This can avoid the situation where a unified data processing flow is not applicable to all data when processing all multi-source heterogeneous data. Therefore, the embodiment of the present invention can improve the data processing accuracy by determining suitable processing methods for different categories of data and processing them using suitable processing methods.

[0028] 102. Classify and process the multi-source heterogeneous data to be processed to obtain the data to be processed under different classification categories.

[0029] Among them, different classification categories include: equipment parameter category, environmental parameter category, safety parameter category, etc.

[0030] Specifically, the collected multi-source heterogeneous data are classified according to device parameters S, environmental parameters H, and safety parameters A. For example, the final classification result is: device parameters S include: device current I, device voltage U, device circuit breaker state ECB, device switch state TF; environmental parameters H include: electric energy metering data ED, energy consumption statistics ECS, ambient temperature T, ambient humidity RH; safety parameters A include: alarm information SAI, device fault state DAI; device parameters S, environmental parameters H, and safety parameters A can be expressed as:

[0031]

[0032] Since multi-source heterogeneous data comes from different data sources and different sensors and other collection devices, in order to determine the appropriate data processing method for different multi-source heterogeneous data, it is first necessary to accurately classify the multi-source heterogeneous data and determine the processing method for the multi-source heterogeneous data to be processed according to different classification categories. This can make the determined processing method more suitable for data in different categories. Through classification processing, the processing problems in various types of data can be solved more specifically, thereby improving the processing accuracy of multi-source heterogeneous data and thus improving the overall data quality.

[0033] 103. Determine the data processing method corresponding to each classification category, and use each data processing method to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each classification category.

[0034] For the embodiment of the present invention, after the multi-source heterogeneous data to be processed are classified and the data to be processed under different classification categories are obtained, the applicable processing methods are determined for the different classification categories, and finally the data to be processed under the corresponding classification categories are cleaned using different processing methods. The classification processing in the embodiment of the present invention makes the data processing process more orderly and efficient. The use of appropriate processing methods and tools for different categories of data can avoid unnecessary waste of resources and reduce processing costs. At the same time, classification processing helps to process different categories of data in parallel, thereby improving processing efficiency. At the same time, optimizing processing algorithms and processes for specific categories of data can also further improve processing speed.

[0035] According to a method for processing multi-source heterogeneous data in a digital substation industrial control system provided by the present invention, compared with the current method of cleaning different data through a unified data cleaning process, the present invention obtains the multi-source heterogeneous data to be processed in the digital substation industrial control system; then classifies the multi-source heterogeneous data to be processed to obtain the data to be processed under different classification categories; finally determines the data processing method corresponding to each classification category, and uses each data processing method to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each classification category. Thus, by pre-classifying the multi-source heterogeneous data in the digital substation industrial control system, the data to be processed under different classification categories are obtained, and then the data processing method applicable to different classification categories is determined, and finally the data processing method is used to process the data under the corresponding classification category, and the applicable data processing method is determined by the data category, thereby improving the accuracy of data processing.

[0036] Further, in order to better illustrate the process of processing multi-source heterogeneous data in the digital substation industrial control system, as a refinement and extension of the above embodiment, the embodiment of the present invention provides another method for processing multi-source heterogeneous data in the digital substation industrial control system, such as Figure 2 As shown, the method includes:

[0037] 201. Obtain the multi-source heterogeneous data to be processed in the digital substation industrial control system.

[0038] Specifically, the multi-source heterogeneous data to be processed are obtained from databases corresponding to different acquisition systems, monitoring systems and other devices in the digital substation industrial control system.

[0039] 202. Classify and process the multi-source heterogeneous data to be processed to obtain data to be processed under different classification categories.

[0040] For the embodiment of the present invention, in order to determine the data processing method applicable to each multi-source heterogeneous data to be processed, it is first necessary to classify the multi-source heterogeneous data to be processed. Based on this, step 202 specifically includes: determining the feature vector corresponding to each of the multi-source heterogeneous data to be processed; determining the centroid vector corresponding to different classification categories; calculating the distance between the feature vector and the centroid vector corresponding to the different classification categories, and based on the distance, dividing each of the multi-source heterogeneous data to be processed into the different classification categories; based on the multi-source heterogeneous data in the different classification categories after division, calculating the updated centroid vector corresponding to the different classification categories; based on the updated centroid vector, re-dividing each of the multi-source heterogeneous data to be processed into the different classification categories until the updated centroid vector does not change, and the multi-source heterogeneous data to be processed that is finally divided into the different classification categories is determined as the data to be processed under the different classification categories.

[0041] Specifically, firstly, the feature vectors corresponding to each of the multi-source heterogeneous data to be processed are determined by means of word embedding or the like, and different classification categories are defined as K clusters. Firstly, the centroid vectors corresponding to the initial centroids corresponding to the K clusters are initialized. For the feature vectors corresponding to each of the multi-source heterogeneous data to be processed, the distances from each of the multi-source heterogeneous data to be processed to the K centroid vectors are calculated, and each of the multi-source heterogeneous data to be processed is assigned to the cluster corresponding to the centroid vector closest to the distance. Then, for each cluster, the centroid of each cluster and its corresponding centroid vector are recalculated, and each of the multi-source heterogeneous data to be processed is re-divided into different clusters, so as to continuously divide each of the multi-source heterogeneous data to be processed until the position of the centroid does not change, that is, the centroid vector does not change, and finally the multi-source heterogeneous data to be processed divided into different clusters are determined as data to be processed under different classification categories.

[0042] Furthermore, if the data processing method corresponding to a certain classification category in each classification category includes: data redundancy processing method, data format processing method, data standardization processing method, data exception processing method, etc., then it is necessary to process the corresponding classification category, that is, the data to be processed under the certain classification category, according to the data redundancy processing method, data format processing method, data standardization processing method, and data exception processing method. The specific processing process is shown in the following steps.

[0043] 203. Process the data to be processed under the corresponding classification category using a data redundancy processing method to obtain redundantly processed data.

[0044] Specifically, in the collected multi-source heterogeneous data to be processed, different data acquisition systems and monitoring systems may collect and process the same parameters, resulting in data redundancy. Therefore, it is necessary to perform redundancy removal on the data. The processing method is as follows:

[0045] Equipment current I, equipment voltage U, electric energy metering data ED, energy consumption statistics ECS are uniformly retained to two decimal places, and ambient temperature T, ambient humidity RH are uniformly retained to one decimal place; if the data of the same parameter collected by different systems are consistent, any of them will be retained and the rest will be directly discarded; if the data of the same parameter collected by different systems are inconsistent, the average value of the same parameter collected by different systems will be used. Represents this parameter, and the calculation method is:

[0046]

[0047] Among them, M i Different systems collect different values ​​of the same parameter, M iIt can represent the equipment current I, equipment voltage U, electric energy metering data ED, energy consumption statistics ECS, ambient temperature T, ambient humidity RH, and n is the number of different values ​​of the same parameter collected by different systems. If the equipment circuit breaker state ECB, equipment switch state TF, alarm information SAI, and equipment fault state DAI are inconsistent, the final state is determined according to the principle of minority obeys majority. If the equipment state is completely contradictory, manual on-site confirmation is performed and the result is retained. Therefore, the redundantly processed data can be obtained through the above-mentioned redundant processing method. Redundant data will take up additional storage space, especially in large databases, where this waste is particularly obvious. Removing redundant data can significantly reduce the use of storage space, reduce storage costs, and improve storage efficiency. When data is redundant, update and maintenance operations need to be modified simultaneously in multiple locations, which increases the complexity of the operation and the risk of errors. After removing redundancy, data updates and maintenance only need to be performed in one location, which simplifies the operation process and reduces the possibility of errors.

[0048] 204. Use the data format processing method to unify the format of the redundantly processed data under the corresponding classification category to obtain data with a unified format.

[0049] Specifically, for example, the unit of the device current I in the device parameter S is set to ampere, the unit of the device voltage U is set to volt, when the device circuit breaker state is open, the device circuit breaker state ECB=1, when the device circuit breaker state is closed, the device circuit breaker state ECB=0, when the device switch is open, the device switch state TF=1, when the device switch is closed, the device switch state TF=0; in the environmental parameter H, the unit of the electric energy metering data ED and the energy consumption statistics ECS is set to kilowatt-hour, the unit of the ambient temperature T is set to degrees Celsius, and the unit of the ambient humidity RH is set to percentage; the safety parameter A includes the alarm information SAI and the device fault state DAI. Because the data of the alarm information contains multiple characters, when there is no alarm information, SAI=0, when there is an alarm information, SAI=1, when the device is not faulty, DAI=0, when the device is faulty, DAI=1, and the alarm information is transmitted in JSON format, and the format is as follows:

[0050]

[0051] Among them, timestamp is the time when the alarm occurs. For example, the format can be: year-month-day-hour-minute-second, represented by 4 digits, 2 digits, 2 digits, 2 digits, 2 digits, 2 digits, and 2 digits respectively. Device_id is the unique identifier of the device, represented by 6 digits. Priority is the alarm priority. 1 / 2 / 3 / 4 represent low alarm priority / medium alarm priority / high alarm priority / dangerous alarm priority respectively. Location is the location where the alarm occurs. Description is the alarm description information. Status records the alarm status. Unresolved / Resolved represents that the alarm is not resolved / the alarm is resolved. The fault status is transmitted in JSON format. The format is as follows:

[0052]

[0053] Thus, according to the above format unification method, the data after format unification can be obtained. It should be noted that the above format unification method is only exemplary and does not specifically limit the present invention. The embodiment of the present invention can reduce data errors and anomalies caused by inconsistent formats by unifying the format of data. When all data follow the same format standard, it is easier to identify and process abnormal values, thereby improving the overall quality and accuracy of the data.

[0054] 205. Use a data standardization processing method to standardize the data in a unified format under a corresponding classification category to obtain standardized data.

[0055] For the embodiment of the present invention, after the data is formatted in a unified manner, it is also necessary to perform standardized processing on the data after the formatting is unified. Based on this, step 205 specifically includes: determining a data collection period of the data to be processed corresponding to the data after the formatting is unified; if the data collection period is less than a preset collection period, determining non-periodic data collected before the preset collection period is reached in the data after the formatting is unified, and deleting the non-periodic data in the data after the formatting is unified to obtain first periodic data, and determining the first periodic data as the data after the standardized processing; if the data collection period is greater than the preset collection period, performing interpolation processing on the data after the formatting is unified to obtain second periodic data, and determining the second periodic data as the data after the standardized processing, wherein the method for performing interpolation processing on the data after the formatting is unified includes: determining the data to be inserted x at the current time j; j The corresponding data x in the previous moment after the unified format j-1 , the data x after the format is unified at the next moment j+1 , the total number of data that needs to be inserted at adjacent moments m, and the data x after the format is unified based on the previous moment j-1, the data x after the format is unified at the next moment j+1 , the total number of data to be inserted at adjacent moments m, determine the number of data to be inserted at the current moment x j ,

[0056] The data collection cycle refers to the period of data collection in seconds, minutes, hours, etc. The preset collection cycle is set according to actual needs.

[0057] Specifically, for the equipment current I, equipment voltage U, electric energy metering data ED, energy consumption statistics ECS, ambient temperature T, and ambient humidity RH of different time granularities, if the unified data collection time interval is seconds (preset collection period), if the time granularity of the data collection period of the data to be processed is milliseconds or microseconds, then only the data at the whole second is retained in the data of the unified format, and the rest of the data (non-periodic data) can be deleted. If the time granularity is minutes or hours, that is, the data collection period is greater than the preset collection period, the linear interpolation method is used to interpolate the data, as shown in the following formula:

[0058]

[0059] Among them, x j is the jth data inserted, j = 1, 2, ..., m, m is the total number of data inserted at adjacent moments, x j+1 is the data of the moment after the current moment, x j-1 is the data of the moment before the current moment, x j , x j-1 , x j+1 It can represent any data such as equipment current I, equipment voltage U, electric energy metering data ED, energy consumption statistics ECS, ambient temperature T, ambient humidity RH, etc. For the equipment circuit breaker status ECB, equipment switch status TF, alarm information SAI, and equipment fault status DAI, for devices that only record abnormal states, the data in the normal state is padded with 0, so that the data can be supplemented, that is, expanded, by interpolating the data. Therefore, through the above method, it is possible to achieve standardized processing of the data after the format is unified to obtain standardized data. In the data set, data may be missing due to various reasons (such as equipment failure, data entry error, etc.). The homology interpolation processing of the present invention can use known data points to estimate these missing values, thereby improving the data set and improving the integrity of the data.

[0060] 206. Perform exception processing on the standardized data under the corresponding classification category using a data exception processing method to obtain exception processed data.

[0061] For the embodiment of the present invention, after determining the data after the standardization processing, it is also necessary to perform exception processing on the data after the standardization processing. Based on this, step 206 specifically includes: determining the numerical type data in the data after the standardization processing, sorting the numerical type data in the order of data collection time from small to large, and obtaining the sorted numerical type data; determining all the data from the second data to the last data in the sorted numerical type data as a target data, and determining p forward data sorted before the target data in the sorted numerical type data; based on each of the forward data, determining the initial abnormal evaluation value corresponding to the target data, and determining the evaluation weight corresponding to the initial abnormal evaluation value, multiplying the initial abnormal evaluation value by the evaluation weight to obtain the abnormal evaluation value corresponding to the target data; based on the initial abnormal evaluation value and the abnormal evaluation value, determining the abnormal evaluation upper limit value and the abnormal evaluation lower limit value corresponding to the target data, and the abnormal evaluation upper limit value and the abnormal evaluation lower limit value constitute the abnormal evaluation interval corresponding to the target data; judging whether the target data is within the abnormal evaluation interval, if the target data is within the abnormal evaluation interval, If the target data is within the normal evaluation interval, the target data is judged to be normal data, otherwise, the target data is judged to be abnormal data; if the target data is abnormal data, p backward data sorted after the target data are determined in the sorted numerical type data; based on each of the backward data, an initial repair evaluation value corresponding to the abnormal data is determined, and a repair evaluation weight corresponding to the initial repair evaluation value is determined, and the initial repair evaluation value is multiplied by the repair evaluation weight to obtain the abnormal repair evaluation value corresponding to the abnormal data; based on the initial repair evaluation value and the abnormal repair evaluation weight, the abnormal repair evaluation value corresponding to the abnormal data is obtained. value, determine the repair evaluation upper limit value and the repair evaluation lower limit value corresponding to the abnormal data, and the repair evaluation upper limit value and the repair evaluation lower limit value constitute the repair evaluation interval corresponding to the abnormal data; judge whether the abnormal data is within the repair evaluation interval, if the abnormal data is within the repair evaluation interval, determine that the abnormal data is unrepairable data, then remove the abnormal data from the numerical type data to obtain the data after the abnormality is processed; if the abnormal data is not within the repair evaluation interval, repair the abnormal data to obtain the data after the abnormality is processed.

[0062] Among them, numerical type data: such as equipment current, equipment voltage, electric energy metering data, energy consumption statistics, ambient temperature, ambient humidity and other data; evaluation weight and re-evaluation weight are values ​​set according to actual needs.

[0063] Specifically, for the numerical data in the standardized data, the numerical data are sorted in the order of data collection time from small to large to obtain sorted numerical data. After the sorted numerical data, the average value of the first p data is calculated, and the average value is determined as the initial abnormal evaluation value ave1. The specific calculation formula is as follows:

[0064]

[0065] Among them, x t-k is the k data before time t, k = 1, 2, ..., p, and the abnormal evaluation value thr1 of the p data before time t is defined as shown in the following formula:

[0066] thr1=λ1×ave1

[0067] Among them, λ1 is the evaluation weight corresponding to the initial abnormal evaluation value. The numerical data anomaly detection is shown in the following formula:

[0068]

[0069] Among them, s z Indicates the data status at time t, including abnormal data and normal data, x t Indicates the numerical data at time t, x t-k , x t It can represent device current I, device voltage U, electric energy metering data ED, energy consumption statistics ECS, ambient temperature T, ambient humidity RH, etc. For abnormal data, calculate the average value of p backward data sorted after the target data, and determine the average value as the initial repair evaluation value ave2, where the calculation formula of the initial repair evaluation value is as follows:

[0070]

[0071] Among them, x t ' -k is the k data after time t, k = 1, 2, ..., p, x t ' -k It can represent the equipment current I, equipment voltage U, electric energy metering data ED, energy consumption statistics ECS, ambient temperature T, ambient humidity RH, etc. The abnormal repair evaluation value thr2 of p data after time t is defined as:

[0072] thr2=λ2×ave2

[0073] Among them, λ2 is the repair evaluation weight corresponding to the initial repair evaluation value, x t Indicates numeric data type, if ave2-thr2≤x t≤ ave2 + thr2, then it is determined that the abnormal data is irreparable data, and the abnormal data is excluded from the numerical type data to obtain the data after abnormal processing; if x t > ave2 + thr2 or x t < ave2 - thr2, then repair the abnormal data, and the specific repair method is as follows:

[0074] x t = ave2

[0075] Thus, the abnormal data of the data is processed in the above manner. The abnormality of power data is often related to factors such as sudden changes in power load and imbalance between energy supply and demand. By identifying abnormal data, the changing trend of power demand can be understood more accurately, providing data support for the rational allocation of power resources. At the same time, by repairing abnormal data, the accuracy and integrity of the data set can be significantly improved, providing a more reliable basis for subsequent data analysis and decision-making.

[0076] In another embodiment of the present invention, the abnormal data of the status type data in the data after normalization processing can also be processed in the following manner: Determine the status type data in the data after normalization processing, sort the status type data in ascending order according to the status acquisition time to obtain the sorted status type data; Determine any data in the sorted status type data as a target status data, determine the forward status data at the previous acquisition time corresponding to the target status data, and the backward status data at the next acquisition time; Judge whether the target status data is the same as the forward status data, and judge whether the target status data is the same as the backward status data; If the target status data is the same as at least one of the forward status data and the backward status data, it is determined that the target status data is normal data, and no abnormal processing is required for the target status data; If the target status data is not the same as both the forward status data and the backward status data, it is determined that the target status data is a mutation data; Determine the total number of mutation data and the total number of status type data in the status type data, and determine the preset mutation data threshold corresponding to the total number of status type data; If the total number of mutation data is less than or equal to the preset mutation data threshold, the mutation data is determined as abnormal data, and the abnormal data is replaced with the forward status data at the previous acquisition time corresponding to the abnormal data to obtain the data after abnormal processing.

[0077] Among them, the status type data includes data such as the device circuit breaker status, the device switch status, the device fault status, etc. Specifically, firstly, the status type data is sorted in the order of data collection time from small to large to obtain the sorted status type data, and then the sorting starts from the second status data in the sorted status type data to the second to last status data. For example, to determine whether the second status data is abnormal data, the forward status data that is adjacent to the second status data and sorted in front of it is determined, and the backward status data that is adjacent to the second status data and sorted behind it is determined, and then it is determined whether the second status data is the same as the forward status data and the backward status data. If the second status data is the same as at least one of the forward status data and the backward status data, then the second status data is determined to be normal data. On the contrary, if the second status data is the same as the forward status data and the backward status data, then the second status data is determined to be normal data. If the state data is different from both the forward state data and the backward state data, the second state data is determined to be mutation data. Thus, according to the above method, the mutation data can be determined in all state type data, and the total number of mutation data and the total number of state type data can be calculated. If the total number of mutation data is less than or equal to the preset mutation data threshold (a value set according to actual needs), all mutation data are determined to be abnormal data, and the mutation data is replaced with the state data that is sorted before the mutation data and adjacent to the mutation data, thereby obtaining data after abnormal processing. The embodiment of the present invention can timely discover and repair problems in the power system through abnormal processing, such as equipment failure, network fluctuations, data transmission errors, etc., thereby reducing the frequency and scope of failures. In order to better explain the process of performing abnormal processing on state type data in the embodiment of the present invention, the following formula can be used for abnormal processing:

[0078]

[0079] Among them, s t is the state value at time t, s t-1 is the state value at time t-1, s t+1 is the state value at time t+1, s t ,s t-1 ,s t+1 It can represent the detection equipment circuit breaker status ECB, equipment switch status TF, equipment fault status DAI, etc. D1(t) is used to judge s t With s t-1 Are they the same? D2(t) is used to determine whether s t With s t+1 Are they the same? This is used to determine whether t Whether a mutation occurs, if a mutation occurs, let D(t) = 1, if no mutation occurs, let D(t) = 0, which can be expressed as:

[0080]

[0081] Calculate the number of mutations D in each p data:

[0082]

[0083] Where t represents the current moment, which is set to 1, and p represents the pth moment, i.e., p data. It should be noted that the above p data can be the overall state type data, or the last p data corresponding to the state type data currently being detected for anomaly. If the preset mutation data threshold for mutations in each p data is D′, if D ≤ D′, then the mutation data is abnormal data, and the state type data s at moment t-1 is used. t-1 Instead of the state type data s at time t t :

[0084] s t =s t-1

[0085] Furthermore, after processing the data to be processed under different classification categories in the digital substation industrial control system, it is also necessary to conduct a quality assessment of the processed data, that is, a processing effect assessment. Based on this, the method includes: counting the total number of cleaned data corresponding to the processed data corresponding to each of the classification categories, and counting the total number of processed data corresponding to the multi-source heterogeneous data to be processed, and determining the total number of missing data based on the total number of processed data and the total number of cleaned data; for any classification category in each of the classification categories, determining the data density within a preset field corresponding to each of the processed data corresponding to the arbitrary classification category; determining the processed data whose data density is greater than a preset density threshold as a cluster center; based on the cluster center, The processed data are clustered to obtain clustering results; the data that does not belong to any cluster in the clustering results are determined as abnormal data under the arbitrary classification category, and the total number of abnormal data corresponding to each classification category is determined; based on the total number of missing data and the total number of cleaned data, the integrity score corresponding to the processed data is determined, and based on the total number of abnormal data and the total number of cleaned data, the accuracy score corresponding to the processed data is determined; based on the integrity score and the accuracy score, the data processing score corresponding to the processed data is determined, and based on the data processing score, the data processing level corresponding to the processed data is determined, so as to determine the data processing effect on the multi-source heterogeneous data to be processed based on the data processing level.

[0086] Specifically, it is necessary to first determine the abnormal data in the processed data. The specific method is: first determine several definitions: core point (cluster center): processed data with more than MinPts number within radius r; boundary point (boundary processed data): the number of processed data within radius r is less than MinPts, but falls within the neighborhood of the core point; noise point (abnormal data): a point that is neither a core point nor a boundary point. The specific clustering method is: for any classification category, for each processed data, calculate the set of data within its preset neighborhood (the preset neighborhood is set according to actual needs). When the number of data in the set exceeds the preset density threshold (the preset data density threshold is set according to actual needs), the processed data is determined as the core point (cluster center), and then check whether the remaining points are in the neighborhood of the core point, and determine the remaining points in the neighborhood of the core point as boundary points, and the remaining points not in the neighborhood of the core point as noise points, and then determine the processed data corresponding to the noise point as abnormal data. After determining the abnormal data under each classification category, determine the data processing score according to the following formula:

[0087]

[0088] Among them, N a T a The total number of all cleaned data in the time period, that is, the total number of processed data corresponding to each classification category, q is the total number of missing value data (the total number of data to be processed is subtracted from the total number of cleaned data to obtain the total number of missing value data), y is the total number of abnormal data, Q is the missing data ratio, Y is the abnormal data ratio, and then the ratio interval to which the missing data ratio belongs is determined, and the integrity score P corresponding to the ratio interval is determined q At the same time, determine the proportion interval to which the abnormal data proportion belongs, and determine the accuracy score P corresponding to the proportion interval y ,

[0089] Examples of completeness scores corresponding to different ratio intervals and accuracy scores corresponding to different ratio intervals are as follows: for example, if the missing data ratio is 3%, the corresponding completeness score is determined to be 4 points by the following formula; if the abnormal data ratio is 3.5%, the corresponding accuracy score is determined to be 3 points by the following formula. It should be noted that the following examples are merely illustrative and do not specifically limit the embodiments of the present invention.

[0090]

[0091] The comprehensive score P (data processing score) of the processed data can be determined as follows:

[0092] P=P q +P y

[0093] The data quality level L (data processing level) of the processed data is defined according to P:

[0094]

[0095] It can be seen from the above formula that the scoring interval to which the data processing score belongs is first determined, and then the data processing level corresponding to the scoring interval is determined. For example, if the scoring interval to which the data processing score belongs is [9, 10], the data processing level is determined to be excellent. Finally, the data quality level L after processing and cleaning is used to reflect the implementation effect of the multi-source heterogeneous data processing method of the digital substation industrial control system. The embodiment of the present invention can further reflect the processing effect of each data processing method on the corresponding classified data by determining the implementation effect of the multi-source heterogeneous data processing method. When the processing effect is not good, the data to be processed is reprocessed, or the appropriate processing method is re-determined for different classification categories, and the data to be processed is reprocessed using the re-determined processing method, thereby improving the processing accuracy of the multi-source heterogeneous data to be processed in the digital substation industrial control system.

[0096] According to another multi-source heterogeneous data processing method in a digital substation industrial control system provided by the present invention, compared with the current method of cleaning different data through a unified data cleaning process, the present invention obtains the multi-source heterogeneous data to be processed in the digital substation industrial control system; then classifies the multi-source heterogeneous data to be processed to obtain the data to be processed under different classification categories; finally determines the data processing method corresponding to each classification category, and uses each data processing method to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each classification category. Thus, by pre-classifying the multi-source heterogeneous data in the digital substation industrial control system, the data to be processed under different classification categories are obtained, and then the data processing method applicable to different classification categories is determined, and finally the data processing method is used to process the data under the corresponding classification category, and the applicable data processing method is determined by the data category, thereby improving the accuracy of data processing.

[0097] Further, as Figure 1 The specific implementation of the present invention provides a multi-source heterogeneous data processing device in a digital substation industrial control system, such as Figure 3 As shown, the device includes: an acquisition unit 31, a classification unit 32 and a processing unit 33.

[0098] The acquisition unit 31 may be used to acquire the multi-source heterogeneous data to be processed in the digital substation industrial control system.

[0099] The classification unit 32 may be used to classify the multi-source heterogeneous data to be processed to obtain data to be processed in different classification categories.

[0100] The processing unit 33 may be used to determine the data processing method corresponding to each of the classification categories, and use each of the data processing methods to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each of the classification categories.

[0101] In a specific application scenario, in order to use the data processing method to process the data to be processed under the corresponding classification category, the processed data corresponding to each classification category is obtained, such as Figure 4 As shown, the processing unit 33 includes a redundancy processing module 331 , a format processing module 332 , a standardization module 333 , and an exception processing module 334 .

[0102] The redundancy processing module 331 can be used to process the to-be-processed data under the corresponding classification category by using the data redundancy processing method to obtain data after redundancy processing.

[0103] The format processing module 332 can be used to perform format unification processing on redundantly processed data under corresponding classification categories using the data format processing method to obtain data with unified format.

[0104] The standardization module 333 may be used to perform standardization processing on the data in a unified format under the corresponding classification category using the data standardization processing method to obtain standardized data.

[0105] The exception processing module 334 can be used to perform exception processing on the standardized data under the corresponding classification category using the data exception processing method to obtain exception-processed data.

[0106] In a specific application scenario, in order to perform standardization processing on the data after the format is unified, the standardization module 333 can be specifically used to determine the data collection period of the to-be-processed data corresponding to the data after the format is unified; if the data collection period is less than the preset collection period, then the non-periodic data collected without reaching the preset collection period is determined in the data after the format is unified, and the non-periodic data is deleted from the data after the format is unified to obtain first periodic data, and the first periodic data is determined as the data after the standardized processing; if the data collection period is greater than the preset collection period, then the data after the format is unified is interpolated to obtain second periodic data, and the second periodic data is determined as the data after the standardized processing, wherein the method for interpolating the data after the format is unified includes: determining the data x to be inserted at the current time j; jThe corresponding data x in the previous moment after the unified format j-1 , the data x after the format is unified at the next moment j+1 , the total number of data that needs to be inserted at adjacent moments m, and the data x after the format is unified based on the previous moment j-1 , the data x after the format is unified at the next moment j+1 , the total number of data to be inserted at adjacent moments m, determine the number of data to be inserted at the current moment x j ,

[0107] In a specific application scenario, in order to perform exception processing on the data after standardization, the exception processing module 334 can be specifically used to determine the numerical type data in the data after standardization, sort the numerical type data in the order of data collection time from small to large, and obtain the sorted numerical type data; determine all the data from the second data to the last data in the sorted numerical type data as a target data, and determine the p forward data sorted before the target data in the sorted numerical type data; based on each of the forward data, determine the initial abnormal evaluation value corresponding to the target data, and determine the evaluation weight corresponding to the initial abnormal evaluation value, multiply the initial abnormal evaluation value by the evaluation weight to obtain the abnormal evaluation value corresponding to the target data; based on the initial abnormal evaluation value and the abnormal evaluation value, determine the abnormal evaluation upper limit value and the abnormal evaluation lower limit value corresponding to the target data, and the abnormal evaluation upper limit value and the abnormal evaluation lower limit value constitute the abnormal evaluation interval corresponding to the target data; judge whether the target data is within the abnormal evaluation interval, if the target data is within the abnormal evaluation interval , then the target data is judged to be normal data, otherwise, the target data is judged to be abnormal data; in the case where the target data is abnormal data, p backward data sorted after the target data are determined in the sorted numerical type data; based on each of the backward data, an initial repair evaluation value corresponding to the abnormal data is determined, and a repair evaluation weight corresponding to the initial repair evaluation value is determined, and the initial repair evaluation value is multiplied by the repair evaluation weight to obtain the abnormal repair evaluation value corresponding to the abnormal data; based on the initial repair evaluation value and the abnormal repair evaluation value, the abnormal repair evaluation value corresponding to the abnormal data is determined. Determine a repair evaluation upper limit value and a repair evaluation lower limit value corresponding to the abnormal data, and the repair evaluation upper limit value and the repair evaluation lower limit value constitute a repair evaluation interval corresponding to the abnormal data; judge whether the abnormal data is within the repair evaluation interval, if the abnormal data is within the repair evaluation interval, determine that the abnormal data is unrepairable data, then remove the abnormal data from the numerical type data to obtain the data after the abnormality is processed; if the abnormal data is not within the repair evaluation interval, repair the abnormal data to obtain the data after the abnormality is processed.

[0108] In a specific application scenario, in order to perform exception processing on the data after standardization processing, the exception processing module 334 can be specifically used to determine the state type data in the data after standardization processing, sort the state type data in the order of state acquisition time from small to large, and obtain the sorted state type data; determine any data in the sorted state type data as a target state data, determine the forward state data at the previous acquisition time and the backward state data at the next acquisition time corresponding to the target state data; determine whether the target state data is the same as the forward state data, and determine whether the target state data is the same as the backward state data; if the target state data is the same as the forward state data, then the target state data is the same as the backward state data; if the target state data is the same as the forward state data, then the target state data is the same as the backward state data; if the target state data is the same as the forward state data, then the target state data is the same as the backward state data. If the target state data is the same as at least one of the state data in the backward state data, the target state data is determined to be normal data, and there is no need to perform exception processing on the target state data; if the target state data is the same as the forward state data and is different from the backward state data, the target state data is determined to be mutation data; the total number of mutation data and the total number of state type data are determined in the state type data, and a preset mutation data threshold corresponding to the total number of state type data is determined; if the total number of mutation data is less than or equal to the preset mutation data threshold, the mutation data is determined to be abnormal data, and the abnormal data is replaced by the forward state data at the previous acquisition time corresponding to the abnormal data to obtain the data after the exception processing.

[0109] In a specific application scenario, in order to evaluate the data processing result, the device further includes: a statistical unit 34 , a determination unit 35 and a clustering unit 36 ​​.

[0110] The statistical unit 34 can be used to count the total number of cleaned data corresponding to the processed data corresponding to each of the classification categories, and count the total number of data to be processed corresponding to the multi-source heterogeneous data to be processed, and determine the total number of missing data based on the total number of data to be processed and the total number of cleaned data.

[0111] The determination unit 35 may be configured to determine, for any classification category among the classification categories, a data density within a preset field corresponding to each of the processed data corresponding to the arbitrary classification category.

[0112] The determining unit 35 may be specifically configured to determine the processed data having a data density greater than a preset density threshold as a cluster center.

[0113] The clustering unit 36 ​​may be used to cluster the processed data corresponding to the arbitrary classification categories based on the cluster centers to obtain clustering results.

[0114] The determining unit 35 may be specifically configured to determine the data that does not belong to any cluster in the clustering result as abnormal data under any classification category, and determine the total number of abnormal data corresponding to each classification category.

[0115] The determination unit 35 can be specifically used to determine the completeness score corresponding to the processed data based on the total number of missing data and the total number of cleaned data, and determine the accuracy score corresponding to the processed data based on the total number of abnormal data and the total number of cleaned data.

[0116] The determination unit 35 can be specifically used to determine the data processing score corresponding to the processed data based on the integrity score and the accuracy score, and determine the data processing level corresponding to the processed data based on the data processing score, so as to determine the data processing effect on the multi-source heterogeneous data to be processed based on the data processing level.

[0117] In a specific application scenario, in order to classify the multi-source heterogeneous data to be processed, the classification unit 32 includes a determination module 321 , a division module 322 , and a calculation module 323 .

[0118] The determination module 321 may be used to determine a feature vector corresponding to each of the multi-source heterogeneous data to be processed.

[0119] The determination module 321 may also be used to determine centroid vectors corresponding to different classification categories.

[0120] The division module 322 may be used to calculate the distance between the feature vector and the centroid vectors corresponding to the different classification categories, and based on the distance, divide each of the multi-source heterogeneous data to be processed into the different classification categories.

[0121] The calculation module 323 may be used to calculate updated centroid vectors corresponding to different classification categories based on the divided multi-source heterogeneous data in different classification categories.

[0122] The division module 322 can also be used to re-divide each of the multi-source heterogeneous data to be processed into the different classification categories based on the updated centroid vector until the updated centroid vector does not change, and the multi-source heterogeneous data to be processed that are finally divided into the different classification categories are determined as data to be processed under different classification categories.

[0123] It should be noted that for other corresponding descriptions of the functional modules involved in the multi-source heterogeneous data processing device in the digital substation industrial control system provided in the embodiment of the present invention, reference can be made to Figure 1 The corresponding description of the method shown will not be repeated here.

[0124] Based on the above Figure 1 The method shown, accordingly, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the following steps are implemented: obtaining multi-source heterogeneous data to be processed in the digital substation industrial control system; classifying and processing the multi-source heterogeneous data to be processed to obtain data to be processed under different classification categories; determining the data processing method corresponding to each of the classification categories, and using each of the data processing methods to process the data to be processed under the corresponding classification category, to obtain processed data corresponding to each of the classification categories.

[0125] Based on the above Figure 1 The method shown and Figure 3 The embodiment of the device shown in the figure, the embodiment of the present invention also provides a physical structure diagram of a computer device, such as Figure 5 As shown, the computer device includes: a processor 41, a memory 42, and a computer program stored in the memory 42 and executable on the processor, wherein the memory 42 and the processor 41 are both arranged on a bus 43, and the processor 41 implements the following steps when executing the program: obtaining the multi-source heterogeneous data to be processed in the digital substation industrial control system; classifying and processing the multi-source heterogeneous data to be processed to obtain data to be processed under different classification categories; determining the data processing method corresponding to each of the classification categories, and using each of the data processing methods to process the data to be processed under the corresponding classification category, to obtain the processed data corresponding to each of the classification categories.

[0126] Through the technical solution of the present invention, compared with the method of cleaning different data through a unified data cleaning process, the present invention obtains the multi-source heterogeneous data to be processed in the digital substation industrial control system; then classifies the multi-source heterogeneous data to be processed to obtain the data to be processed under different classification categories; finally determines the data processing method corresponding to each classification category, and uses each data processing method to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each classification category. In this way, by pre-classifying the multi-source heterogeneous data in the digital substation industrial control system, the data to be processed under different classification categories are obtained, and then the data processing method applicable to different classification categories is determined, and finally the data processing method is used to process the data under the corresponding classification category, and the applicable data processing method is determined by the data category, thereby improving the accuracy of data processing.

[0127] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing multi-source heterogeneous data in a digital substation industrial control system, characterized in that: include: Obtain the multi-source heterogeneous data to be processed in the digital substation industrial control system; Classify and process the multi-source heterogeneous data to be processed to obtain data to be processed under different classification categories; Determine the data processing method corresponding to each of the classification categories, and use each of the data processing methods to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each of the classification categories.

2. The method according to claim 1, characterized in that If the data processing method corresponding to any classification category in each of the classification categories includes: data redundancy processing method, data format processing method, data standardization processing method, and data exception processing method; The method of processing the data to be processed under the corresponding classification category by using the data processing method to obtain processed data corresponding to each classification category includes: Using the data redundancy processing method to process the data to be processed under the corresponding classification category to obtain redundantly processed data; Using the data format processing method, the redundantly processed data under the corresponding classification category is processed in a unified format to obtain data in a unified format; Using the data standardization processing method, standardize the data in the unified format under the corresponding classification category to obtain standardized data; The data exception processing method is used to perform exception processing on the standardized data under the corresponding classification category to obtain exception-processed data.

3. The method according to claim 2, characterized in that The method of using the data standardization processing method to standardize the data in a unified format under the corresponding classification category to obtain the standardized data includes: Determine the data collection cycle of the to-be-processed data corresponding to the data after the unified format; If the data collection period is less than the preset collection period, then determining the non-periodic data collected before the preset collection period in the data after the format is unified, deleting the non-periodic data in the data after the format is unified, obtaining the first periodic data, and determining the first periodic data as the standardized data; If the data collection period is greater than the preset collection period, the data after the unified format is interpolated to obtain second periodic data, and the second periodic data is determined as the standardized data, wherein the method for interpolating the data after the unified format includes: determining the data to be inserted x at the current time j j The corresponding data x in the previous moment after the unified format j-1 , the data x after the format is unified at the next moment j+1 , the total number of data that needs to be inserted at adjacent moments m, and the data x after the format is unified based on the previous moment j-1 , the data x after the format is unified at the next moment j+1 , the total number of data to be inserted at adjacent moments m, determine the number of data to be inserted at the current moment x j , 4. The method according to claim 2, characterized in that: The method of using the data exception processing method to perform exception processing on the standardized data under the corresponding classification category to obtain the exception processed data includes: Determine the numerical type data in the standardized data, sort the numerical type data in the order of data collection time from small to large, and obtain sorted numerical type data; Determine all data from the second data to the last data in the sorted numerical data as target data respectively, and determine p forward data sorted before the target data in the sorted numerical data; Based on each of the forward data, determine an initial abnormal evaluation value corresponding to the target data, determine an evaluation weight corresponding to the initial abnormal evaluation value, and multiply the initial abnormal evaluation value by the evaluation weight to obtain an abnormal evaluation value corresponding to the target data; Based on the initial abnormality evaluation value and the abnormality evaluation value, determine an abnormality evaluation upper limit value and an abnormality evaluation lower limit value corresponding to the target data, and the abnormality evaluation upper limit value and the abnormality evaluation lower limit value constitute an abnormality evaluation interval corresponding to the target data; Determine whether the target data is within the abnormality evaluation interval, if the target data is within the abnormality evaluation interval, determine that the target data is normal data, otherwise, determine that the target data is abnormal data; In the case where the target data is abnormal data, determining p backward data sorted after the target data in the sorted numerical data; Based on each of the backward data, determine the initial repair evaluation value corresponding to the abnormal data, determine the repair evaluation weight corresponding to the initial repair evaluation value, and multiply the initial repair evaluation value by the repair evaluation weight to obtain the abnormal repair evaluation value corresponding to the abnormal data; Based on the initial repair evaluation value and the abnormal repair evaluation value, determine the repair evaluation upper limit value and the repair evaluation lower limit value corresponding to the abnormal data, and the repair evaluation upper limit value and the repair evaluation lower limit value constitute the repair evaluation interval corresponding to the abnormal data; Determine whether the abnormal data is within the repair evaluation interval. If the abnormal data is within the repair evaluation interval, determine that the abnormal data is unrepairable data, and then remove the abnormal data from the numerical data to obtain the data after the abnormality is processed; If the abnormal data is not within the repair evaluation interval, the abnormal data is repaired to obtain the data after the abnormality is processed.

5. The method according to claim 4, characterized in that The method of using the data exception processing method to perform exception processing on the standardized data under the corresponding classification category to obtain the exception processed data includes: Determine the state type data in the standardized data, sort the state type data in the order of state collection time from small to large, and obtain sorted state type data; Determine any data in the sorted state type data as a target state data, and determine the forward state data at a previous acquisition time and the backward state data at a subsequent acquisition time corresponding to the target state data; determining whether the target state data is the same as the forward state data, and determining whether the target state data is the same as the backward state data; If the target state data is the same as the forward state data and at least one of the backward state data, the target state data is determined to be normal data, and there is no need to perform abnormal processing on the target state data; If the target state data is the same as the forward state data and the backward state data is different, then the target state data is determined to be mutation data; Determining the total amount of mutation data and the total amount of state type data in the state type data, and determining a preset mutation data threshold corresponding to the total amount of state type data; If the total number of mutation data is less than or equal to the preset mutation data threshold, the mutation data is determined as abnormal data, and the abnormal data is replaced by the forward state data at the previous collection time corresponding to the abnormal data to obtain the data after the abnormality processing.

6. The method according to claim 1, characterized in that After the data to be processed under the corresponding classification category is processed by the data processing method to obtain the processed data corresponding to each classification category, the method further includes: Counting the total number of cleaned data corresponding to the processed data corresponding to each of the classification categories, and counting the total number of data to be processed corresponding to the multi-source heterogeneous data to be processed, and determining the total number of missing data based on the total number of data to be processed and the total number of cleaned data; For any classification category in each of the classification categories, determine the data density in a preset field corresponding to each of the processed data corresponding to the arbitrary classification category; Determining the processed data whose data density is greater than a preset density threshold as a cluster center; Based on the cluster center, clustering the processed data corresponding to the arbitrary classification category to obtain a clustering result; Determine the data that does not belong to any cluster in the clustering result as abnormal data under the arbitrary classification category, and determine the total number of abnormal data corresponding to each classification category; Determine a completeness score corresponding to the processed data based on the total number of missing data and the total number of cleaned data, and determine an accuracy score corresponding to the processed data based on the total number of abnormal data and the total number of cleaned data; Based on the completeness score and the accuracy score, a data processing score corresponding to the processed data is determined, and based on the data processing score, a data processing level corresponding to the processed data is determined, so as to determine the data processing effect on the multi-source heterogeneous data to be processed based on the data processing level.

7. The method according to claim 1, characterized in that The classifying and processing the multi-source heterogeneous data to be processed to obtain the data to be processed under different classification categories includes: Determine the feature vector corresponding to each of the multi-source heterogeneous data to be processed; Determine the centroid vectors corresponding to different classification categories; Calculating the distance between the feature vector and the centroid vectors corresponding to the different classification categories, and based on the distance, classifying each of the multi-source heterogeneous data to be processed into the different classification categories; Based on the multi-source heterogeneous data in different classification categories after division, the updated centroid vectors corresponding to different classification categories are calculated; Based on the updated centroid vector, each of the multi-source heterogeneous data to be processed is re-divided into the different classification categories until the updated centroid vector does not change, and the multi-source heterogeneous data to be processed that are finally divided into the different classification categories are determined as the data to be processed under different classification categories.

8. A multi-source heterogeneous data processing device in a digital substation industrial control system, characterized in that: include: An acquisition unit, used for acquiring multi-source heterogeneous data to be processed in the digital substation industrial control system; A classification unit, used for classifying the multi-source heterogeneous data to be processed to obtain data to be processed under different classification categories; The processing unit is used to determine the data processing method corresponding to each of the classification categories, and use each of the data processing methods to process the data to be processed under the corresponding classification category to obtain the processed data corresponding to each of the classification categories.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images