Log data processing method based on cable phase measurement

Abstract

The invention discloses a log data processing method based on cable phase measurement, and particularly relates to the field of log data processing, which comprises the following steps: constructing a feature vector set containing multi-dimensional description, and establishing a classification reference of short-term and long-term feature abnormal modes of each cable number; monitoring the latest log record of each cable in real time, and generating abnormal recognition results corresponding to short-term fluctuation and long-term drift; establishing a first-level repair strategy based on short-term fluctuation anomaly characteristics, and generating a repair value sequence by using an interpolation algorithm to reconstruct short-term fluctuation anomaly data of the phase log; establishing a second-level repair strategy based on the long-term migration abnormal characteristics, and reconstructing the long-term migration abnormal data of the phase log based on polynomial fitting and piecewise regression; and performing abnormal identification on the data after the repair execution is finished again, judging whether the phase log data is repaired or not, if so, ending the log data repair, and if not, sending out a phase abnormal alarm signal.

Description

Technical Field

[0001] The present invention relates to the technical field of log data processing, and more specifically, to a log data processing method based on cable phase measurement. Background Art

[0002] Cable phase data logs are an important data source for monitoring the operating status of power systems, and their accuracy directly affects the operation evaluation, fault diagnosis, and optimal control of the system. However, during long-term operation, cable phase data is vulnerable to various interference sources, including equipment failures, electromagnetic noise, sensor drift, and transient interference from the external environment. These problems may lead to data quality problems such as anomalies, missing data, noise, and drift in the log data, damaging the accuracy and integrity of the data. Traditional methods for detecting and repairing the quality of phase data mostly rely on static threshold determination and manual intervention, unable to meet the real-time and intelligent requirements in complex power systems. Especially in the case of faults, inaccurate data anomalies will spread rapidly, increasing the difficulty of phase data analysis.

[0003] Therefore, how to propose a log data processing method based on cable phase measurement to meet the requirements of maintaining high-quality and stable data under different interference conditions, providing reliable data support for the real-time monitoring of power systems, while significantly reducing the need for manual intervention and improving the intelligent level of data quality management is an urgent problem to be solved.

[0004] To solve the above problems, a technical solution is provided as follows. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a log data processing method based on cable phase measurement to solve the problems proposed in the above background art.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] S1: Integrate the multi-dimensional features of the cable historical phase log data, construct a feature vector set containing multi-dimensional descriptions, and establish classification benchmarks for short-term and long-term feature anomaly patterns for each cable number.

[0008] S2: Set a log monitoring sliding window, monitor the latest log records of each cable in real time, and generate anomaly recognition results corresponding to short-term fluctuations and long-term drifts respectively.

[0009] S3: Establish a first-level repair strategy based on the short-term fluctuation anomaly characteristics, and use the interpolation algorithm to generate a sequence of repair values to reconstruct the short-term fluctuation anomaly data of the phase log.

[0010] S4: Establish a second-level repair strategy based on the characteristics of long-term offset anomalies, and reconstruct the long-term offset anomaly data of the phase log by generating trend repair data pairs based on polynomial fitting and piecewise regression;

[0011] S5: Re-identify the anomalies of short-term fluctuations and long-term drifts for the data after the repair is completed, and determine whether the phase log data is repaired. If so, end the log data repair. If not, send out a phase anomaly alarm signal.

[0012] In a preferred embodiment, in S1, integrating the multi-dimensional features of the cable historical phase log data, constructing a feature vector set containing multi-dimensional descriptions, and establishing the classification benchmarks for the short-term and long-term feature anomaly patterns of each cable number specifically include:

[0013] Screen the records in the cable historical phase log data, delete the logs with missing items, incomplete formats and duplicate contents, and obtain the original record set with complete data;

[0014] Read the original record set item by item, and establish time-series phase historical log data groups corresponding to different cable numbers based on the upload timestamp and cable number of the data;

[0015] Independently analyze the time-series phase historical log data groups corresponding to each cable number, extract the static features of the data from the historical logs, including the mean, extreme values, standard deviation and fluctuation range of the phase data, and construct a static feature description set;

[0016] Extract the dynamic feature values of the historical logs through time-series analysis, including the change rate of the phase data and the moving window mean of the set window size, and construct a dynamic feature description set;

[0017] Extract the frequency domain feature values of the historical logs through frequency domain analysis, including the high-frequency amplitude and the proportion of high-frequency energy, and construct a frequency domain feature description set;

[0018] Integrate the static features, dynamic features and frequency domain features, and establish a feature vector set containing multi-dimensional descriptions;

[0019] Extract the log feature vector set corresponding to the normal operation period of the cable history, and divide the feature vectors into short-term and long-term feature categories through feature clustering, and each category corresponds to a comprehensive feature group;

[0020] Based on the comprehensive feature groups corresponding to different feature categories of the phase history logs of each cable number, establish the classification benchmarks for the short-term and long-term feature anomaly patterns of the phase history logs of each cable number. The matrix expression form corresponding to the classification benchmark is:

[0021]

[0022] Among them, D is the matrix corresponding to the classification criterion, DS and DL are the classification criteria for short-term and long-term feature anomaly patterns respectively, and X max , X min , X rate , X avg are respectively the maximum value, minimum value, change rate and mean value of the phase data, σ is the standard deviation of the phase data, and F high , E high are respectively the high-frequency amplitude and high-frequency energy ratio in the frequency domain of the phase data, and W avg is the moving window mean value.

[0023] In a preferred embodiment, in S2, a log monitoring sliding window is set to monitor the latest log records of each cable in real time. The specific steps for generating anomaly recognition results corresponding to short-term fluctuations and long-term drifts respectively include:

[0024] Set the log monitoring sliding window of the cable with the cable number as the retrieval identifier, and obtain the latest log record of each cable in real time;

[0025] For the log data in each monitoring sliding window, construct a feature vector matrix based on the feature anomaly classification criterion, map the data in the sliding window to the feature space one by one, and extract the key features representing different anomaly patterns;

[0026] Cluster the key features representing different anomaly patterns by using the multi-center iterative distribution method, calculate the Euclidean distance from the classification criterion in the feature vector matrix, set the distance offset anomaly threshold, and generate anomaly recognition results corresponding to short-term fluctuations and long-term drifts respectively through threshold comparison. Establish a repair task for each log data with an anomaly recognition result;

[0027] According to the feature deviation situation and anomaly influence size order of the anomaly recognition results, use the weight assignment algorithm to prioritize the selection of the repair path of the phase log data, and generate a priority sequence for the execution of the repair path. Specifically:

[0028] Obtain the anomaly recognition results of the log data corresponding to all cable numbers, and uniformly convert the anomaly recognition results of each type into the expression of the feature deviation ratio;

[0029] Preset the influence weight coefficients of short-term fluctuations and long-term drift anomalies, weight and sum the anomaly feature deviation ratios of short-term fluctuations and long-term drifts, calculate the comprehensive anomaly index, and adjust the task serial numbers of the cable phase log data repair tasks according to the reverse order of the comprehensive anomaly index;

[0030] Calculate the single-weighted offset of the short-term fluctuation and long-term drift anomaly characteristics corresponding to each task serial number, perform secondary sorting in reverse order according to the size of the weighted offset, and sequentially add the corresponding repair tasks to the log data repair queue according to the final sorting result, and control the synchronous execution of the repair tasks through a distributed lock.

[0031] In a preferred embodiment, in S3, based on the short-term fluctuation anomaly characteristics, establish a first-level repair strategy, and use the interpolation algorithm to generate a repair value sequence to reconstruct the short-term fluctuation anomaly data of the phase log, which specifically includes:

[0032] When the execution program of the first-level repair strategy obtains the distributed lock, obtain the sliding log data segment within the log monitoring sliding window corresponding to the first-priority repair task in the log data repair queue;

[0033] Extract the phase fluctuation amplitude in the classification benchmark static characteristics as the upper and lower limits of the abnormal fluctuation range, define the data exceeding the upper and lower limits of the range in the sliding log data segment as short-term fluctuation abnormal data points, and mark their positions;

[0034] Select the interpolation algorithm as the repair basis, and construct a local interpolation with an input dimension matching the abnormal fluctuation range interval according to the time index of the short-term fluctuation abnormal data points and their adjacent non-abnormal data points before and after, and generate the corresponding repair value sequence;

[0035] Replace the short-term fluctuation abnormal data points with the generated repair values in sequence, and gradually perform continuous insertion operations on the reconstructed data in chronological order;

[0036] At the edge of the repaired abnormal data points, recalculate the transition interval value between the repair value and the adjacent non-abnormal data points through weighted moving average, generate a smoothed local data sequence to update the log, and release the distributed lock.

[0037] In a preferred embodiment, in S4, based on the long-term offset anomaly characteristics, establish a second-level repair strategy, and generate trend repair data based on polynomial fitting and piecewise regression to reconstruct the long-term offset anomaly data of the phase log, which specifically includes:

[0038] When the execution program of the second-level repair strategy obtains the distributed lock, obtain the sliding log data segment within the log monitoring sliding window corresponding to the first-priority repair task in the log data repair queue;

[0039] Scan the time series data of the sliding log data segment item by item, and identify the abnormal interval of continuous deviation by calculating the matching degree between the dynamic deviation trend of the data points and the change trend of the sliding mean in the benchmark dynamic characteristics, and generate the corresponding interval identifier;

[0040] Based on the polynomial fitting and piecewise regression methods, a mathematical model is constructed to describe the phase change trend in the abnormal interval, and the trend is described through the change of the key features of the abnormal pattern in the abnormal interval;

[0041] An iterative optimization algorithm is adopted to gradually adjust the parameter set of the fitting model. With the goal of minimizing the fitting relative residual with respect to the classification benchmark features of the long-term offset anomaly, a consistency trend expression of the benchmark features and the key features of the abnormal pattern is constructed;

[0042] Based on the trend fitting model completed by tuning, regression calculations are performed for each time point in the abnormal interval to generate a sequence of repair values covering all time points in the abnormal interval. The abnormal data is sequentially replaced with the data in the repair value sequence, and the distributed lock is released after the replacement is completed.

[0043] In a preferred embodiment, in S5, after the repair is completed, the data is re-identified for short-term fluctuations and long-term drifts to determine whether the phase log data is repaired. If so, the log data repair ends. If not, a phase anomaly alarm signal is issued, which specifically includes:

[0044] After all tasks in the log data repair queue are executed, the repaired data is compared with the classification benchmark again to generate abnormal identification results corresponding to short-term fluctuations and long-term drifts respectively;

[0045] When it is detected that the abnormal identification results of short-term fluctuations and long-term drifts are non-abnormal, the data repair ends. When it is detected that the abnormal identification result of short-term fluctuations or long-term drifts is abnormal, a phase anomaly data alarm signal is sent to the maintenance personnel terminal, and the alarm signal includes the specific time period corresponding to the phase log data where the anomaly occurs and the cable number.

[0046] The technical effects and advantages of a log data processing method based on cable phase measurement according to the present invention:

[0047] By constructing a multi-dimensional feature vector set and establishing a classification benchmark, accurate identification and hierarchical repair of short-term fluctuations and long-term drifts anomalies are achieved. This method sets a log monitoring sliding window to achieve real-time monitoring and anomaly identification, improving the real-time performance and accuracy of data processing. For short-term fluctuation anomalies, a sequence of repair values is generated through an interpolation algorithm to achieve local data reconstruction; for long-term drift anomalies, trend repair data is generated through polynomial fitting and piecewise regression to accurately restore the overall trend of the log data. The hierarchical repair strategy enables this method to flexibly handle different anomaly types, ensuring the efficiency and accuracy of the repair.

[0048] In addition, the present invention adopts an adaptive closed-loop repair mechanism. By re-identifying the repaired data and dynamically evaluating the repair effect, a repair-identification closed-loop is formed. When complete repair is not possible, an alarm signal is sent to ensure the continuous reliability of data quality. This method effectively eliminates the short-term fluctuations and long-term drifts in cable phase log data, significantly improves the integrity and accuracy of the data, provides high-quality data support for the state monitoring and scheduling optimization of the power system, and has a high level of automation and intelligence at the same time. Brief Description of the Drawings

[0049] Figure 1 It is a schematic diagram of a method for processing log data based on cable phase measurement according to the present invention. Detailed Embodiments

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0051] Embodiment 1

[0052] Figure 1 A method for processing log data based on cable phase measurement according to the present invention is given, which includes the following steps:

[0053] S1: Integrate the multi-dimensional features of the cable historical phase log data, construct a set of feature vectors containing multi-dimensional descriptions, and establish a classification benchmark for short-term and long-term feature anomaly patterns of each cable number.

[0054] S2: Set a log monitoring sliding window, monitor the latest log records of each cable in real time, and generate anomaly recognition results corresponding to short-term fluctuations and long-term drifts respectively.

[0055] S3: Establish a first-level repair strategy based on the short-term fluctuation anomaly characteristics, and use the interpolation algorithm to generate a sequence of repair values to reconstruct the short-term fluctuation anomaly data of the phase log.

[0056] S4: Establish a second-level repair strategy based on the long-term offset anomaly characteristics, and generate trend repair data based on polynomial fitting and piecewise regression to reconstruct the long-term offset anomaly data of the phase log.

[0057] S5: Re-identify the short-term fluctuations and long-term drifts of the data after the repair is completed, and judge whether the phase log data is repaired. If so, end the log data repair. If not, send a phase anomaly alarm signal.

[0058] In S1, the multi-dimensional features of the cable historical phase log data are integrated to construct a set of feature vectors containing multi-dimensional descriptions, and the classification benchmarks for short-term and long-term feature anomalies of each cable number are established, specifically including:

[0059] From the cable phase historical log, read the original record set, check the log content item by item, identify and delete the records containing missing items and incomplete formats (such as missing timestamps, missing cable numbers or phase data), and eliminate duplicate records (such as records with the same timestamp) to ensure the integrity and consistency of the data.

[0060] Read the original record set item by item, and based on the upload timestamp and cable number of the data, establish time-series phase historical log data groups corresponding to different cable numbers;

[0061] Independently analyze the time-series phase historical log data group corresponding to each cable number, extract the static features of the data from the historical log, including the mean, extreme values, standard deviation and fluctuation range of the phase data, and construct a set of static feature descriptions. These static features reflect the overall distribution characteristics of the data and help analyze the phase state of the cable.

[0062] For the time-series data of each cable number, extract dynamic features based on the sliding window method, mainly including the change rate of the phase data and the sliding average within the set window size (based on the amount of time-series data and the time interval scale span). By analyzing the change trend of the data over time, dynamic features describing the data change are obtained.

[0063] Extract the frequency-domain eigenvalue of the historical log through frequency-domain analysis, including the high-frequency amplitude and the proportion of high-frequency energy, and construct a set of frequency-domain feature descriptions. The high-frequency energy mainly reflects the noise component in the data, and the frequency-domain features help identify abnormal patterns. Among them, the frequency standard of the high frequency can be freely set based on the recognition accuracy. The fast Fourier transform (FFT) is used to perform frequency-domain conversion on the time-domain data, and the amplitude of the high-frequency component in the frequency domain is calculated. Compare the energy proportion of the low frequency and the high frequency, and use the following formula to calculate the proportion of high-frequency energy:

[0064]

[0065] In the formula, RP is the proportion of high-frequency energy, X(f) is the amplitude of the frequency-domain data, and f h is the set frequency standard of the high frequency. Integrate the static features, dynamic features and frequency-domain features to establish a set of feature vectors containing multi-dimensional descriptions.

[0066] Extract the set of log feature vectors corresponding to the normal operation period of the cable history, and divide the feature vectors into short-term and long-term feature categories through feature clustering. Each category corresponds to a comprehensive feature group;

[0067] Based on the comprehensive feature grouping of the phase history logs corresponding to each cable number for different feature categories, a classification benchmark for the short-term and long-term feature anomaly patterns of the phase history logs of each cable number is established. The matrix representation form corresponding to the classification benchmark is as follows:

[0068]

[0069] where D is the matrix corresponding to the classification benchmark, DS and DL are the classification benchmarks for short-term and long-term feature anomaly patterns respectively, and X max 、X min 、X rate 、X avg are the maximum value, minimum value, change rate, and mean value of the phase data respectively, σ is the standard deviation of the phase data, and F high 、E high are the high-frequency amplitude and high-frequency energy ratio in the frequency domain of the phase data respectively, and W avg is the moving window mean.

[0070] In S2, set a log monitoring sliding window to monitor the latest log records of each cable in real time. The specific anomaly recognition results corresponding to short-term fluctuations and long-term drifts include:

[0071] Use the cable number as the retrieval identifier, set a corresponding log monitoring sliding window for each cable number, and obtain the latest log records of each cable in real time. The size of the monitoring window can be set to a fixed time period (e.g., 1 hour) or a dynamic size triggered by events. Real-time data is obtained through an API or a stream processing framework (such as Kafka).

[0072] Map the log data of each cable number to a feature vector through static, dynamic, and frequency domain feature extraction algorithms. Each feature vector contains multiple dimensions, specifically including the mean value, extreme value, standard deviation, fluctuation amplitude, moving average, etc. of the phase data. The feature vector matrix provides a high-dimensional feature space representation for all the monitoring data of each cable, ensuring that the anomaly patterns in the logs can be captured.

[0073] Use the multi-center iterative distribution method (or K-means algorithm) to cluster the key features in the feature vector matrix. By calculating the Euclidean distance between each feature vector and the classification benchmark, set a threshold to identify the anomaly patterns of short-term fluctuations and long-term drifts, and generate the corresponding anomaly recognition results.

[0074] Set the distance offset anomaly threshold, generate the anomaly recognition results corresponding to short-term fluctuations and long-term drifts respectively through threshold comparison, and establish a repair task for each log data with an abnormal recognition result.

[0075] According to the feature deviation situation and the order of abnormal influence size of the abnormal recognition result, use the weight distribution algorithm to prioritize the selection of the repair path of the phase log data, and generate a priority sequence for the execution of the repair path. Specifically:

[0076] Obtain the abnormal recognition results of the log data corresponding to all cable numbers, and uniformly convert the abnormal recognition results of each type into the expression of the feature offset ratio.

[0077] Preset the influence weight coefficients of short-term fluctuation and long-term drift anomalies (for example, the influence weight of short-term fluctuation is 0.7, and that of long-term drift is 0.3), weight and sum the abnormal feature offset ratios of short-term fluctuation and long-term drift, calculate the comprehensive anomaly index, and adjust the task serial numbers of the cable phase log data repair tasks according to the reverse order of the comprehensive anomaly index.

[0078] Calculate the single-weight offset of the short-term fluctuation and long-term drift abnormal features of the phase log data corresponding to each task serial number, perform secondary sorting in reverse order according to the size of the weighted offset, and add the corresponding repair tasks to the log data repair queue in turn according to the final sorting result.

[0079] Use a distributed lock (such as the sequential lock mechanism based on Zookeeper or Redis) to control the synchronous execution of repair tasks, ensuring that only one repair task is executed at the same time. After each task is completed, the distributed lock will be released, allowing the next task to start execution, preventing data repair thread chaos.

[0080] In S3, based on the short-term fluctuation anomaly characteristics, establish a first-level repair strategy, and use the interpolation algorithm to generate a repair value sequence to reconstruct the short-term fluctuation abnormal data of the phase log. Specifically include:

[0081] In a distributed environment, the execution program first controls the access to the repair tasks in the log data repair queue by obtaining a distributed lock. The tasks in the repair queue are sorted by priority, and the first repair task is extracted. The execution program queries the relevant data segments within the log monitoring sliding window from the data storage system according to the cable number and timestamp corresponding to the task. These data segments are usually composed of historical phase data and contain the corresponding time series data records.

[0082] When the execution program of the first-level repair strategy obtains the distributed lock, obtain the sliding log data segment within the log monitoring sliding window corresponding to the first repair task in the log data repair queue.

[0083] Extract the phase fluctuation amplitude in the classification benchmark static features and use it as the upper and lower limits of the short-term fluctuation anomaly. By comparing each item of the sliding log data segment, identify the data points that exceed the upper and lower limits, mark them as short-term fluctuation abnormal data points, and record their positions.

[0084] Select the interpolation algorithm as the basis for repair. According to the time indices of the short-term fluctuation abnormal data points and their adjacent non-abnormal data points before and after, construct a local interpolation that matches the input dimension and the abnormal fluctuation range interval, generate the corresponding repair value sequence, and construct an interpolation formula based on the adjacent data points before and after the abnormal data points:

[0085]

[0086] In the formula, RE is the repair value, z0 and z1 are the values of adjacent non-abnormal data points before and after, t0 and t1 are the time indices corresponding to the adjacent non-abnormal data points before and after, t is the time index of the current abnormal data point, and □ is the repair tolerance parameter, which is used to prevent the loss of abnormalities caused by over-repair.

[0087] Replace the short-term fluctuation abnormal data points with the generated repair values in sequence, and perform continuous insertion operations on the reconstructed data step by step using the time sequence.

[0088] At the edge of the repaired abnormal data points, recalculate the transition interval values between the repair values and the adjacent non-abnormal data points through weighted moving average, generate a smoothed local data sequence update log, and release the distributed lock.

[0089] In S4, establish a second-level repair strategy based on the long-term offset abnormality characteristics, and reconstruct the long-term offset abnormal data of the phase log based on polynomial fitting and piecewise regression. Specifically, it includes:

[0090] When the execution program of the second-level repair strategy obtains the distributed lock, obtain the sliding log data segment within the sliding log window corresponding to the first-priority repair task in the log data repair queue.

[0091] For each period of data obtained, the execution program analyzes the log data item by item by calculating the matching degree between the dynamic deviation trend of the data points and the change trend of the sliding mean in the reference dynamic characteristics. The dynamic deviation trend refers to the change trend of the data points relative to their adjacent time points before and after within a certain period of time. The reference dynamic characteristics are generated from the dynamic characteristics of the previously established historical phase log and represent the trend change of the data under normal circumstances. When the data points deviate significantly and continuously from the reference dynamic characteristics, they are identified as abnormal data points.

[0092] By calculating the deviation degree of each data point, the system can effectively identify the abnormal intervals of continuous deviation. For example, if multiple consecutive data points deviate from the reference characteristics beyond the set threshold range, then mark this interval as an abnormal interval and generate the corresponding interval identifier. This interval identifier will be recorded and passed to the subsequent processing module to ensure that the subsequent repair algorithm can identify and focus on this abnormal interval.

[0093] Describe the trend through the change of the key features of the abnormal pattern in the abnormal interval, specifically:

[0094] For each data segment in the abnormal interval, we use polynomial fitting to capture the non-linear trend of the data in that interval. Assume that the observed values of the data interval are {(x i , y i ),} where x i is the timestamp and y i is the corresponding phase data. The polynomial function k(x) is expressed as:

[0095] k(x) = a n x n + a n-1 x n-1 + … + a1x 1 + a0;

[0096] In the formula, n is the order of the polynomial, a0, a1, …, a n-1 , a n are the fitting coefficients, and x is the timestamp variable representing {(x i , y i ).}

[0097] Adopt an iterative optimization algorithm to gradually adjust the parameter set of the fitting model, with the goal of minimizing the fitting relative residual (specifically set based on the fitting accuracy requirement, default setting is 15%, to prevent the problem of abnormal loss caused by overfitting) relative to the classification benchmark features of the long-term offset anomaly, and construct a consistency trend expression of the benchmark features and the key features of the abnormal pattern. The deviation calculation method between the data point and the fitting curve is:

[0098]

[0099] In the formula, m is the number of data points, P is the sum of squared errors, and the best fitting coefficients are obtained by minimizing P within the allowable range of the relative residual.

[0100] Based on the trend fitting model completed by tuning, perform regression calculations for each time point in the abnormal interval to generate a sequence of repair values covering all time points in the abnormal interval, replace the abnormal data with the data in the sequence of repair values in sequence, and release the distributed lock after the replacement is completed.

[0101] In S5, re-identify the anomalies of short-term fluctuations and long-term drifts for the data after the repair is completed, and determine whether the phase log data is repaired. If so, end the repair of the log data. If not, send out a phase anomaly alarm signal, specifically including:

[0102] After all tasks in the log data repair queue are executed, the system marks the task status as "completed". The repaired data is compared with the classification benchmark again to generate abnormal identification results corresponding to short-term fluctuations and long-term drifts respectively.

[0103] If both short-term fluctuations and long-term drifts are identified as "non-abnormal", the data repair is ended. If short-term fluctuations or long-term drifts are identified as "abnormal", an alarm signal is generated. The alarm signal includes information such as the abnormal time period, fluctuation amplitude, drift direction, and cable number, and is sent to the maintenance personnel's terminal in real time. The system sends the alarm signal to the maintenance personnel's terminal device through a specified communication protocol (such as MQTT, HTTP, etc.). The signal not only includes the time period and cable number of the abnormal data, but also provides the specific abnormal type (short-term fluctuation or long-term drift) and the degree of abnormal deviation for the maintenance personnel to analyze and use.

[0104] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters and threshold selection in the formulas are set by those skilled in the art according to the actual situation.

[0105] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access, or a data storage device such as a server or data center that contains one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0106] Those of ordinary skill in the art can realize that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0107] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0108] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or modules can be in electrical, mechanical, or other forms.

[0109] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules. They can be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, the functional modules in each embodiment of this application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.

[0111] If the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0112] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0113] Finally: The above is only the preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A log data processing method based on cable phase measurement, characterized in that: The steps include: S1: Integrate the multi-dimensional features of the cable historical phase log data, construct a feature vector set containing multi-dimensional descriptions, and establish a classification benchmark for the short-term and long-term feature anomaly patterns of each cable number; S2: Set the log monitoring sliding window to monitor the latest log records of each cable in real time and generate abnormal identification results corresponding to short-term fluctuations and long-term drifts respectively; S3: Establish the first-level repair strategy based on the short-term fluctuation anomaly characteristics, and use the interpolation algorithm to generate the repair value sequence to reconstruct the short-term fluctuation anomaly data of the phase log; S4: A second-level repair strategy is established based on the long-term offset anomaly characteristics, and trend repair data is generated based on polynomial fitting and piecewise regression to reconstruct the long-term offset anomaly data of the phase log; S5: Re-identify the abnormalities of short-term fluctuations and long-term drifts on the data after the repair is completed, and determine whether the phase log data has been repaired. If so, end the log data repair; if not, issue a phase abnormality alarm signal.

2. A log data processing method based on cable phase measurement according to claim 1, characterized in that: In S1, the multi-dimensional features of the cable historical phase log data are integrated to construct a feature vector set containing multi-dimensional descriptions, and the classification benchmarks of the short-term and long-term feature anomaly patterns of each cable number are established, including: Screen the records in the cable historical phase log data, delete the logs with missing items, incomplete formats and duplicate content, and obtain the original record set with complete data; Read the original record set one by one, and establish the time series phase history log data grouping corresponding to different cable numbers based on the data upload timestamp and cable number; The time series phase historical log data corresponding to each cable number is grouped and analyzed independently. The static features of the data are extracted from the historical logs, including the mean, extreme value, standard deviation and fluctuation amplitude of the phase data, and a static feature description set is constructed. Through time series analysis, the dynamic feature values ​​of historical logs are extracted, including the rate of change of phase data and the sliding window mean of the set window size, and a dynamic feature description set is constructed; Through frequency domain analysis, the frequency domain feature values ​​of historical logs are extracted, including high-frequency amplitude and high-frequency energy ratio, and a frequency domain feature description set is constructed; Integrate static features, dynamic features and frequency domain features to establish a feature vector set containing multi-dimensional descriptions; The extracted log feature vector set corresponding to the normal operation period of the cable history is divided into short-term and long-term feature categories through feature clustering, and each category corresponds to a comprehensive feature grouping; Based on the comprehensive feature grouping of different feature categories corresponding to the phase history log of each cable number, a classification benchmark for the short-term and long-term feature anomaly patterns of the phase history log of each cable number is established. The matrix expression corresponding to the classification benchmark is: Among them, D is the matrix corresponding to the classification benchmark, DS and DL are the classification benchmarks of short-term and long-term feature abnormal patterns, respectively, and X max , X min , X rate , X avg are the maximum, minimum, rate of change and mean of the phase data, σ is the standard deviation of the phase data, F high 、E high are the high-frequency amplitude and high-frequency energy proportion of the phase data frequency domain, W avg is the sliding window mean.

3. A log data processing method based on cable phase measurement according to claim 2, characterized in that: In S2, a log monitoring sliding window is set to monitor the latest log records of each cable in real time, and the abnormal identification results corresponding to short-term fluctuations and long-term drifts are generated, including: Use the cable number as the search identifier to set the cable log monitoring sliding window to obtain the latest log records of each cable in real time; For the log data in each monitoring sliding window, a feature vector matrix based on the feature anomaly classification benchmark is constructed, and the data in the sliding window is mapped to the feature space one by one to extract the key features representing different anomaly patterns; The multi-center iterative distribution method is used to cluster the key features representing different abnormal patterns. The Euclidean distance with the classification benchmark is calculated in the feature vector matrix. The distance offset abnormal threshold is set. The recognition results corresponding to short-term fluctuation and long-term drift anomalies are generated by threshold comparison. A repair task is established for each log data with abnormal recognition results. According to the characteristic deviation of the anomaly recognition results and the order of the anomaly impact, the weight allocation algorithm is used to prioritize the selection of the phase log data repair path and generate a priority sequence for the execution of the repair path, specifically: Obtain the anomaly recognition results of the log data corresponding to all cable numbers, and uniformly convert each type of anomaly recognition result into a feature offset ratio expression; Preset the impact weight coefficients of short-term fluctuations and long-term drift anomalies, perform weighted summation on the abnormal characteristic offset ratios of short-term fluctuations and long-term drifts, calculate the comprehensive anomaly index, and adjust the task sequence number of the cable phase log data repair task according to the reverse order of the comprehensive anomaly index; Calculate the single weighted offset of the short-term fluctuation and long-term drift anomaly characteristics of the phase log data corresponding to each task sequence number, perform secondary sorting in reverse order of the weighted offset size, and add the corresponding repair tasks to the log data repair queue in turn according to the final sorting results, and control the synchronous execution of the repair tasks through distributed locks.

4. A log data processing method based on cable phase measurement according to claim 3, characterized in that: In S3, the first-level repair strategy is established based on the short-term fluctuation anomaly characteristics, and the repair value sequence is generated by the interpolation algorithm to reconstruct the short-term fluctuation anomaly data of the phase log. Specifically, it includes: When the executor of the first-level repair strategy obtains the distributed lock, the sliding log data segment in the log monitoring sliding window corresponding to the first-priority repair task in the log data repair queue is obtained; The phase fluctuation amplitude in the static features of the classification benchmark is extracted as the upper and lower limits of the abnormal fluctuation range. The data exceeding the upper and lower limits in the sliding log data segment are defined as short-term fluctuation abnormal data points, and their positions are marked. The interpolation algorithm is selected as the repair basis. According to the time index of the short-term fluctuation abnormal data point and its adjacent non-abnormal data points before and after it, a local interpolation matching the input dimension with the abnormal fluctuation range is constructed to generate the corresponding repair value sequence. The generated repair values ​​are used to replace the short-term fluctuation abnormal data points in sequence, and the reconstructed data is inserted continuously step by step in time sequence; At the edge of the repaired abnormal data point, the transition interval value between the repaired value and the adjacent non-abnormal data point is recalculated through weighted moving average to generate a smoothed local data sequence update log and release the distributed lock.

5. The method for processing log data based on cable phase measurement according to claim 4, characterized in that: In S4, a second-level repair strategy is established based on the long-term offset anomaly characteristics. The trend repair data is generated based on polynomial fitting and piecewise regression to reconstruct the long-term offset anomaly data of the phase log, which specifically includes: When the executor of the second-level repair strategy obtains the distributed lock, the sliding log data segment in the log monitoring sliding window corresponding to the first-order repair task in the log data repair queue is obtained; Scan the time series data of the sliding log data segment one by one, identify the abnormal interval of continuous deviation by calculating the matching degree between the dynamic deviation trend of the data point and the sliding mean change trend in the benchmark dynamic feature, and generate the corresponding interval mark; Based on polynomial fitting and segmented regression methods, a mathematical model describing the phase change trend of the abnormal interval is constructed, and the trend is described through the changes in the key characteristics of the abnormal pattern of the abnormal interval; The iterative optimization algorithm is used to gradually adjust the parameter set of the fitting model, with the goal of minimizing the relative residual of the fitting relative to the classification benchmark features of the long-term offset anomaly, and constructing the consistent trend expression of the benchmark features and the key features of the anomaly pattern; Based on the tuned trend fitting model, regression calculation is performed on each time point in the abnormal interval to generate a repair value sequence covering all time points in the abnormal interval. The abnormal data is replaced with the repair value sequence data in sequence, and the distributed lock is released after the replacement is completed.

6. A log data processing method based on cable phase measurement according to claim 5, characterized in that: In S5, the data after the repair is completed is re-identified for abnormalities of short-term fluctuations and long-term drifts to determine whether the phase log data is repaired. If so, the log data repair is terminated. If not, a phase abnormality alarm signal is issued. Specifically, the following steps are performed: When all the tasks in the log data repair queue are completed, the repaired data is compared with the classification benchmark again to generate the short-term fluctuation and long-term drift anomaly identification results respectively; When the short-term fluctuation and long-term drift anomaly identification results are detected as non-abnormal, data repair is terminated. When the short-term fluctuation or long-term drift anomaly identification results are detected as abnormal, a phase abnormality data alarm signal is sent to the maintenance personnel terminal. The alarm signal includes the specific time period corresponding to the abnormal phase log data and the cable number.

Images