Data processing method and equipment of cathode protection remote transmission system and storage medium

Abstract

The invention provides a data processing method and device of a cathodic protection remote transmission system and a storage medium. The method comprises the steps of obtaining a plurality of data acquisition samples of the cathodic protection remote transmission system; performing wavelet transform processing on the plurality of data acquisition samples to obtain a plurality of corresponding wavelet energy data; determining a training set according to the plurality of wavelet energy data; the training set comprises a plurality of wavelet energy training data; preprocessing the multiple pieces of wavelet energy training data to obtain multiple pieces of preprocessed training data; determining a plurality of corresponding feature data sets according to the plurality of preprocessed training data; each feature data set comprises model input features and dichotomy tags; according to the plurality of feature data sets and an original prediction model, training to obtain a target prediction model; the target prediction model is used for data acquisition planning of the cathodic protection remote transmission system so as to achieve the technical effects of saving the electric quantity of equipment and prolonging the service life of a power supply.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and storage medium for data processing of a cathodic protection remote transmission system. Background Art

[0002] In the field of oil and gas pipeline safety monitoring and maintenance, given the increasingly stringent requirements placed on pipeline safety by modern industry, traditional monitoring methods are no longer able to meet these high standards. Against this backdrop, cathodic protection remote transmission systems have emerged as an advanced monitoring technology, providing a strong guarantee for the safe operation of oil and gas pipelines. However, the increasing scale and complexity of oil and gas pipelines have placed higher demands on the data processing capabilities of these systems. Therefore, developing more efficient and stable data processing methods for these systems has become a promising area of research.

[0003] In the existing technology, the data processing method of the cathodic protection remote transmission system is to first collect pipeline potential data at regular intervals through intelligent remote transmission test equipment buried in the line (perception layer), then use the wide area network and the Internet of Things (network layer) to transmit the data to the cloud storage platform, and then use the cloud platform's deep learning and analysis model (analysis layer) to perform intelligent analysis of the data, extract abnormal data, and issue early warning prompts in the event of system interruption.

[0004] However, the power supply modules in the existing technology rely on high-energy batteries, but these batteries have limited power and there are a large number of devices on the system. In particular, for pipe sections with severe AC / DC interference, periodic and intensive monitoring is required, which leads to an increase in the collection frequency and adjustments in the face of low-temperature environments, which in turn cause technical problems such as high power consumption and short equipment life. Summary of the Invention

[0005] The present application provides a method, device and storage medium for data processing of a cathodic protection remote transmission system, which are used to achieve the technical effects of saving equipment power and extending the service life of the power supply.

[0006] In a first aspect, the present application provides a method for data processing of a cathodic protection remote transmission system, comprising:

[0007] Obtain multiple data acquisition samples from the cathodic protection remote transmission system;

[0008] Performing wavelet transform processing on multiple data collection samples to obtain corresponding multiple wavelet energy data;

[0009] Determine a training set based on a plurality of wavelet energy data; the training set includes a plurality of wavelet energy training data;

[0010] Preprocessing the plurality of wavelet energy training data to obtain a plurality of preprocessed training data;

[0011] Determine multiple corresponding feature data sets based on multiple pre-processed training data; each feature data set includes model input features and binary classification labels;

[0012] Based on multiple feature data sets and the original prediction model, a target prediction model is trained; the target prediction model is used for data acquisition planning of the cathodic protection remote transmission system.

[0013] In a possible implementation, preprocessing the plurality of wavelet energy training data to obtain a plurality of preprocessed training data includes:

[0014] Deduplication processing is performed on the preprocessed training data according to the wavelet energy value of the preprocessed training data to obtain multiple candidate training data;

[0015] According to the file sources of the candidate training data, the time coverage frequency of the candidate training data is calculated;

[0016] According to the importance values corresponding to the candidate training data, a dynamic programming method is used to construct a 0-1 knapsack problem corresponding to multiple preset time constraints. Based on the solution of the dynamic programming problem, multiple preprocessed training data are screened from the multiple candidate training data; among them, the importance value is positively correlated with the wavelet energy value.

[0017] In a possible implementation, determining corresponding multiple feature data sets based on multiple pre-processed training data includes:

[0018] Add sliding windows to multiple pre-processed training data to extract local features, and add local extreme values to detect signal mutation features to obtain multiple preliminary features;

[0019] Normalizing multiple preliminary features to obtain multiple model input features; the model input features include normalized timestamps, standardized energy values, local energy maximum value marks, and time coverage frequencies;

[0020] Determine the binary classification labels for the model input features;

[0021] Generate a feature dataset based on the model input features and binary classification labels.

[0022] In one possible implementation, determining a binary classification label for a model input feature includes:

[0023] Define the classification rules for binary classification labels;

[0024] According to the classification rules, a state transition model is constructed;

[0025] Based on the state transition model, determine the binary classification labels of the model input features.

[0026] In a possible implementation, wavelet transform processing is performed on multiple data acquisition samples to obtain corresponding multiple wavelet energy data, including:

[0027] Through the cross-validation method, multiple preset wavelet bases are used to verify the optimal decomposition layer number of the data collection samples;

[0028] Under the optimal decomposition level, the target wavelet basis is determined from multiple preset wavelet bases;

[0029] According to the target wavelet basis, wavelet transform processing is performed on multiple data collection samples to obtain corresponding multiple wavelet energy data.

[0030] In a possible implementation, determining a training set based on a plurality of wavelet energy data includes:

[0031] According to the preset ratio, multiple wavelet energy data are split and processed to obtain a training set and a test set, wherein the test set is used to verify the prediction results of the target prediction model.

[0032] In one possible implementation, the prediction model is a gradient boosting tree model.

[0033] In one possible implementation, obtaining multiple data acquisition samples of a cathodic protection remote transmission system includes:

[0034] A plurality of data collection samples from the cathodic protection remote transmission system of a plurality of cities are obtained, wherein the data collection samples are collected by the cathodic protection remote transmission system using a preset period.

[0035] In a second aspect, the present application provides a data processing device for a cathodic protection remote transmission system, comprising:

[0036] An acquisition module is used to acquire multiple data acquisition samples of the cathodic protection remote transmission system;

[0037] A wavelet transform processing module is used to perform wavelet transform processing on multiple data collection samples to obtain corresponding multiple wavelet energy data;

[0038] A first determining module is configured to determine a training set based on a plurality of wavelet energy data; the training set includes a plurality of wavelet energy training data;

[0039] A preprocessing module, used for preprocessing a plurality of wavelet energy training data to obtain a plurality of preprocessed training data;

[0040] A second determination module is used to determine a corresponding plurality of feature data sets based on the plurality of pre-processed training data; each feature data set includes a model input feature and a binary classification label;

[0041] The training module is used to train a target prediction model based on multiple feature data sets and the original prediction model; the target prediction model is used for data acquisition planning of the cathodic protection remote transmission system.

[0042] In one possible implementation, the preprocessing module is further configured to:

[0043] Deduplication processing is performed on the preprocessed training data according to the wavelet energy value of the preprocessed training data to obtain multiple candidate training data;

[0044] According to the file sources of the candidate training data, the time coverage frequency of the candidate training data is calculated;

[0045] According to the importance values corresponding to the candidate training data, a dynamic programming method is used to construct a 0-1 knapsack problem corresponding to multiple preset time constraints. Based on the solution of the dynamic programming problem, multiple preprocessed training data are screened from the multiple candidate training data; among them, the importance value is positively correlated with the wavelet energy value.

[0046] In a possible implementation, the second determining module is further configured to:

[0047] Add sliding windows to multiple pre-processed training data to extract local features, and add local extreme values to detect signal mutation features to obtain multiple preliminary features;

[0048] Normalizing multiple preliminary features to obtain multiple model input features; the model input features include normalized timestamps, standardized energy values, local energy maximum value marks, and time coverage frequencies;

[0049] Determine the binary classification labels for the model input features;

[0050] Generate a feature dataset based on the model input features and binary classification labels.

[0051] In a possible implementation, the second determining module is further configured to:

[0052] Define the classification rules for binary classification labels;

[0053] According to the classification rules, a state transition model is constructed;

[0054] Based on the state transition model, determine the binary classification labels of the model input features.

[0055] In a possible implementation, the wavelet transform processing module is further configured to:

[0056] Through the cross-validation method, multiple preset wavelet bases are used to verify the optimal decomposition layer number of the data collection samples;

[0057] Under the optimal decomposition level, the target wavelet basis is determined from multiple preset wavelet bases;

[0058] According to the target wavelet basis, wavelet transform processing is performed on multiple data collection samples to obtain corresponding multiple wavelet energy data.

[0059] In a possible implementation, the first determining module is further configured to:

[0060] According to the preset ratio, multiple wavelet energy data are split and processed to obtain a training set and a test set, wherein the test set is used to verify the prediction results of the target prediction model.

[0061] In one possible implementation, the training module is further configured to:

[0062] Prediction model, wherein the prediction model is a gradient boosting tree model.

[0063] In a possible implementation, the acquisition module is further configured to:

[0064] A plurality of data collection samples from the cathodic protection remote transmission system of a plurality of cities are obtained, wherein the data collection samples are collected by the cathodic protection remote transmission system using a preset period.

[0065] In a third aspect, the present application provides a device for data processing of a cathodic protection remote transmission system, comprising: a memory, a processor;

[0066] Memory stores computer-executable instructions;

[0067] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0068] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation methods of the first aspect.

[0069] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementations of the first aspect.

[0070] The present application provides a method, device and storage medium for data processing of a cathodic protection telemetry system. First, by widely collecting multiple data acquisition samples from the system, the comprehensiveness and diversity of the data are ensured, providing a solid data foundation for subsequent processing. The collected samples are processed using wavelet transform technology to extract wavelet energy data. This process not only improves the resolution and accuracy of the data, but also helps to reveal the potential laws and characteristics in the data, providing strong support for subsequent analysis. In addition, the training set constructed based on wavelet energy data contains key information of the system in different states, providing rich data resources for model training. By preprocessing the training data, the quality and consistency of the data are improved, creating good conditions for model training. Furthermore, multiple feature data sets are determined based on the preprocessed training data. Each data set contains model input features and corresponding binary classification labels, which provides a clear guiding direction for model training. Finally, using these feature data sets and the original prediction model, the target prediction model is obtained through training. This model can accurately predict the system's data collection needs under different conditions, providing a scientific basis for the data collection planning of the cathodic protection remote transmission system, effectively improving the efficiency and accuracy of data collection, and providing a strong guarantee for the stable operation and performance optimization of the system, thereby achieving the technical effect of saving equipment power and extending the service life of the power supply. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0072] Figure 1 A schematic diagram of an application data processing system architecture provided in an embodiment of the present application;

[0073] Figure 2 Schematic diagram of the data processing method of the cathodic protection remote transmission system provided in the embodiment of the present application Figure 1 ;

[0074] Figure 3 Schematic diagram of the data processing method of the cathodic protection remote transmission system provided in the embodiment of the present application Figure 2 ;

[0075] Figure 4 Schematic diagram of the data processing method of the cathodic protection remote transmission system provided in the embodiment of the present application Figure 3 ;

[0076] Figure 5 Schematic diagram of the data processing method of the cathodic protection remote transmission system provided in the embodiment of the present application Figure 4 ;

[0077] Figure 6 A schematic diagram of the structure of a data processing device for a cathodic protection remote transmission system provided in an embodiment of the present application;

[0078] Figure 7 A schematic diagram of the structure of the data processing equipment of the cathodic protection remote transmission system provided in an embodiment of the present application.

[0079] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0080] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0081] Since the power supply modules in the existing technology rely on high-energy batteries, but these batteries have limited power and there are a large number of devices on the system, especially for pipe sections with severe AC and DC interference, periodic and intensive monitoring is required, which leads to an increase in the collection frequency and adjustments in the face of low-temperature environments, which in turn cause technical problems such as high power consumption and short equipment life.

[0082] In response to the above problems, the present application provides a method, device and storage medium for data processing of a cathodic protection remote transmission system. First, by widely collecting multiple data acquisition samples of the system, the comprehensiveness and diversity of the data are ensured, providing a solid data foundation for subsequent processing. The collected samples are processed using wavelet transform technology to extract wavelet energy data. This process not only improves the resolution and accuracy of the data, but also helps to reveal the potential laws and characteristics in the data, providing strong support for subsequent analysis. In addition, the training set constructed based on wavelet energy data contains key information of the system in different states, providing rich data resources for model training. By preprocessing the training data, the quality and consistency of the data are improved, creating good conditions for model training. Furthermore, multiple feature data sets are determined based on the preprocessed training data. Each data set contains model input features and corresponding binary classification labels, which provides a clear guiding direction for model training. Finally, using these feature data sets and the original prediction model, the target prediction model is obtained through training. This model can accurately predict the system's data collection needs under different conditions, providing a scientific basis for the data collection planning of the cathodic protection remote transmission system, effectively improving the efficiency and accuracy of data collection, and providing a strong guarantee for the stable operation and performance optimization of the system, thereby achieving the technical effect of saving equipment power and extending the service life of the power supply.

[0083] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0084] Figure 1 This is a schematic diagram of an application data processing system architecture provided in an embodiment of the present application. The application data processing system is a computer device. Figure 1 As shown, the above architecture includes at least one of a data acquisition device 101 , a processing device 102 and a display device 103 .

[0085] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the architecture of the application data processing system. In other feasible implementations of this application, the above architecture may include more or fewer components than shown, or combine or split certain components, or arrange the components differently. The specific configuration can be determined based on the actual application scenario and is not limited here. Figure 1 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0086] In a specific implementation process, the data acquisition device 101 may include an input / output interface and may also include a communication interface. The data acquisition device 101 may be connected to the processing device via the input / output interface or the communication interface.

[0087] The processing device 102 can obtain multiple data acquisition samples of the cathodic protection remote transmission system; perform wavelet transform processing on the multiple data acquisition samples to obtain corresponding multiple wavelet energy data; determine a training set based on the multiple wavelet energy data; the training set includes multiple wavelet energy training data; preprocess the multiple wavelet energy training data to obtain multiple preprocessed training data; determine corresponding multiple feature data sets based on the multiple preprocessed training data; each feature data set includes model input features and binary classification labels; based on the multiple feature data sets and the original prediction model, a target prediction model is trained; the target prediction model is used for data acquisition planning of the cathodic protection remote transmission system.

[0088] The display device 103 may also be a touch screen display or a screen of a terminal device, which is used to receive user instructions while displaying the above content to achieve interaction with the user.

[0089] It should be understood that the above-mentioned processing device can be implemented by a processor reading instructions in a memory and executing the instructions, or it can be implemented by a chip circuit.

[0090] In addition, the network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0091] Figure 2 Schematic diagram of the data processing method of the cathodic protection remote transmission system provided in the embodiment of the present application Figure 1 ,like Figure 2 As shown, the data processing method of the cathodic protection remote transmission system provided in this embodiment includes:

[0092] S201, obtaining multiple data acquisition samples of the cathodic protection remote transmission system;

[0093] Utilize a cathodic protection data logger to collect multiple data acquisition samples of the cathodic protection system.

[0094] Specifically, the cathodic protection data recorder used for test piles in different cities collects data using a sampling period of 10 on and 1 off (the power-on period is 10 seconds and the power-off period is 1 second, the power-on potential is collected once per second during the power-on period, and the polarization potential is collected once during the power-off period).

[0095] S202, performing wavelet transform processing on the multiple data collection samples to obtain corresponding multiple wavelet energy data;

[0096] Wavelet transforms are used to decompose and reconstruct the collected raw data samples to extract characteristic information at different frequencies and time scales. Wavelet transforms decompose the signal into a series of wavelet coefficients, which reflect the energy distribution of the data at different frequency components. By calculating the energy of the wavelet coefficients, multiple wavelet energy data can be obtained.

[0097] Specifically, the collected data is subjected to wavelet transform, and the optimal decomposition number of layers is cross-validated among the three wavelet bases, sym6, db4, and coif3. The mean square error (MSE) of the signal decomposition and reconstruction of the three wavelet bases under the optimal decomposition number of layers is compared, and the coif3 wavelet with the smallest error is selected as the data processing tool, thereby obtaining the corresponding multiple wavelet energy data.

[0098] S203, determining a training set based on the plurality of wavelet energy data; the training set includes the plurality of wavelet energy training data;

[0099] After obtaining the wavelet energy data, it is necessary to select a portion of the data as a training set; among them, these training data contain the energy characteristic information of the system in different states. The selection of the training set should be based on the representativeness, diversity and balance of the data to ensure the accuracy and generalization ability of subsequent model training.

[0100] S204, preprocessing the plurality of wavelet energy training data to obtain a plurality of preprocessed training data;

[0101] In order to improve data quality and reduce the influence of noise and outliers, multiple wavelet energy training data are preprocessed according to deduplication processing, time coverage frequency calculation, and 0-1 knapsack problem screening based on dynamic programming to obtain multiple preprocessed training data.

[0102] S205. Determine corresponding multiple feature data sets based on the multiple pre-processed training data; each feature data set includes model input features and binary classification labels;

[0103] Based on the preprocessed training data, features are further extracted to construct a feature dataset.

[0104] S206. A target prediction model is obtained by training based on the multiple feature data sets and the original prediction model; the target prediction model is used for data acquisition planning of the cathodic protection remote transmission system.

[0105] The target prediction model is obtained by using the feature data set and the original prediction model. The target prediction model can learn the potential laws and characteristics in the data, thereby achieving accurate prediction of the data acquisition plan of the cathodic protection remote transmission system.

[0106] In one possible implementation, the prediction model is a gradient boosting tree model.

[0107] The present application provides a method for data processing of a cathodic protection telemetry system. First, by widely collecting multiple data acquisition samples from the system, the comprehensiveness and diversity of the data are ensured, providing a solid data foundation for subsequent processing. The collected samples are processed using wavelet transform technology to extract wavelet energy data. This process not only improves the resolution and accuracy of the data, but also helps to reveal the potential laws and characteristics in the data, providing strong support for subsequent analysis. In addition, the training set constructed based on wavelet energy data contains key information of the system in different states, providing rich data resources for model training. By preprocessing the training data, the quality and consistency of the data are improved, creating good conditions for model training. Furthermore, multiple feature data sets are determined based on the preprocessed training data. Each data set contains model input features and corresponding binary classification labels, which provides a clear guiding direction for model training. Finally, using these feature data sets and the original prediction model, the target prediction model is obtained through training. This model can accurately predict the system's data collection needs under different conditions, providing a scientific basis for the data collection planning of the cathodic protection remote transmission system, effectively improving the efficiency and accuracy of data collection, and providing a strong guarantee for the stable operation and performance optimization of the system, thereby achieving the technical effect of saving equipment power and extending the service life of the power supply.

[0108] Figure 3 Schematic diagram of the data processing method of the cathodic protection remote transmission system provided in the embodiment of the present application Figure 2 ,like Figure 3 As shown, this embodiment, based on the above embodiment, describes in detail the process of obtaining multiple pre-processed training data, including:

[0109] S301, performing deduplication processing on the pre-processed training data according to the wavelet energy value of the pre-processed training data to obtain multiple candidate training data;

[0110] Specifically, the wavelet energy value of each pre-processed training data is first calculated, and then the training data is deduplicated according to the wavelet energy value, and points with larger energy values are selected as candidate training data.

[0111] S302: Calculate the time coverage frequency of the candidate training data based on the file source of the candidate training data;

[0112] According to the file source information of the candidate training data, the frequency of being covered by the candidate training data at each time point is calculated.

[0113] S303. Based on the importance values corresponding to the candidate training data, a dynamic programming solution is used to construct a 0-1 knapsack problem corresponding to multiple preset time constraints, and based on the solution of the dynamic programming problem, multiple candidate training data are screened to obtain multiple preprocessed training data.

[0114] In this embodiment, the importance value is positively correlated with the wavelet energy value.

[0115] Specifically, according to the importance values corresponding to the candidate training data, the dynamic programming method is used to construct a 0-1 knapsack problem with time constraints of 800, 900 and 1000 seconds. The selected points are recorded as 1 and the unselected points are recorded as 0. The energy maximization is used as the objective function. Through iterative solution, a training data set that meets the time constraints and has the largest importance value is obtained. Based on this set, multiple corresponding preprocessed training data are screened from multiple candidate training data.

[0116] The method for data processing of the cathodic protection remote transmission system provided in the embodiment of the present application, first, performs deduplication processing based on the wavelet energy value of the pre-processed training data, effectively reducing data redundancy and ensuring the uniqueness and representativeness of the candidate training data. Secondly, by calculating the time coverage frequency of the candidate training data, the timeliness and comprehensiveness of the data are further improved, providing a more reliable basis for subsequent analysis. Finally, combined with the importance values corresponding to the candidate training data, a dynamic programming solution is used to construct a 0-1 knapsack problem, and a plurality of pre-processed training data are screened accordingly. This process not only takes into account the importance of the data, but also takes into account time constraints, achieving dual optimization of data quality and quantity, and providing a more accurate and efficient solution for data processing of the cathodic protection remote transmission system.

[0117] Figure 4 Schematic diagram of the data processing method of the cathodic protection remote transmission system provided in the embodiment of the present application Figure 3 ,like Figure 4 As shown, this embodiment, based on the above embodiment, describes in detail the process of determining multiple feature data sets, including:

[0118] S401, adding a sliding window to a plurality of pre-processed training data to extract local features, and adding local extreme values to detect signal mutation features to obtain a plurality of preliminary features;

[0119] For the preprocessed training data, the sliding window technology is used to extract local features. Multiple sliding windows of different sizes are set with each time point as the center. Windows of corresponding sizes are selected, and local features such as mean, standard deviation, maximum, minimum, etc. are calculated in each window to capture the local change characteristics of the data. At the same time, in each sliding window, local extreme values are detected to capture the mutation characteristics of the signal, and then multiple corresponding preliminary features are obtained.

[0120] S402, normalizing the multiple preliminary features to obtain multiple model input features; the model input features include normalized timestamps, standardized energy values, local energy maximum value marks, and time coverage frequencies;

[0121] Based on multiple preliminary features, the timestamp feature is normalized to eliminate the influence of the time scale. Then the energy value feature is standardized to make it conform to the standard normal distribution. Then, the gradient boosting tree method is used to combine the normalized timestamp, standardized energy value, local energy maximum mark and time coverage frequency to obtain the corresponding multiple model input features.

[0122] S403: Define classification rules for binary classification labels, and build a state transition model based on the classification rules;

[0123] Based on the time points selected by the dynamic programming algorithm, the binary classification labels are determined. The selected time points are labeled 1 (positive class), and the unselected time points are labeled 0 (negative class). Based on this classification rule, the corresponding state transition model is constructed.

[0124] S404: Determine the binary classification labels of the model input features according to the state transition model;

[0125] According to the constructed state transition model, each model input feature is assigned a corresponding binary classification label.

[0126] S405: Generate a feature dataset based on the model input features and binary classification labels.

[0127] Combine the model input features and the binary classification labels to generate the corresponding feature dataset.

[0128] The method for data processing of the cathodic protection remote transmission system provided in the embodiment of the present application first adds a sliding window and local extreme value detection to the pre-processed training data, effectively extracts local features and signal mutation features, and enriches the preliminary feature set. Subsequently, the preliminary features are normalized to ensure the consistency and comparability of the data, and provide standardized features for the model input. Next, clear binary classification label classification rules are defined, and a state transition model is constructed to achieve effective classification of the model input features. Finally, a feature data set is generated based on the features and classification labels, which lays a solid foundation for subsequent data analysis and model training, and improves the accuracy and efficiency of data processing and feature extraction as a whole.

[0129] Figure 5 Schematic diagram of the data processing method of the cathodic protection remote transmission system provided in the embodiment of the present application Figure 4 ,like Figure 5 As shown, this embodiment, based on the above embodiment, describes in detail the process of determining the training set, including:

[0130] S501, obtaining multiple data collection samples from cathodic protection remote transmission systems in multiple cities;

[0131] In this embodiment, the data collection samples are collected by the cathodic protection remote transmission system using a preset period.

[0132] Cathodic protection data recorders are used to regularly collect multiple data samples from cathodic protection remote transmission systems in multiple cities according to preset cycles.

[0133] Specifically, the cathodic protection data recorder used for test piles in different cities collects data using a sampling period of 10 on and 1 off (the power-on period is 10 seconds and the power-off period is 1 second, the power-on potential is collected once per second during the power-on period, and the polarization potential is collected once during the power-off period).

[0134] It should be noted that the cathodic protection remote transmission system uses a preset cycle to collect samples. It does not need to upload frequently within a very short time interval. The collection cycle can be set according to the content of the generated file to collect the polarization potential value, which saves the power of the equipment to a certain extent and extends the service life of the power supply.

[0135] S502, using a cross-validation method to verify the optimal decomposition level of the data collection sample using multiple preset wavelet bases;

[0136] Specifically, the cross-validation method is used to evaluate the performance of different wavelet bases. A variety of wavelet bases (such as sym6, db4, and coif3) are preset for testing, and the optimal number of decomposition layers for each wavelet base is determined by comparing the mean square error (MSE).

[0137] S503, determining a target wavelet basis from a plurality of preset wavelet bases at the optimal decomposition level;

[0138] At the optimal number of decomposition layers, the wavelet basis with the smallest mean square error is selected as the target wavelet basis, and the performance of each wavelet basis at the optimal number of decomposition layers can be evaluated based on the results of cross-validation.

[0139] S504, performing wavelet transform processing on the multiple data collection samples according to the target wavelet basis to obtain corresponding multiple wavelet energy data;

[0140] Apply the target wavelet basis to the data collection samples for wavelet transform, and according to the following formula:

[0141]

[0142] Calculate the wavelet energy of each point and generate wavelet energy data, where E is the wavelet energy, C i is the i-th target wavelet basis.

[0143] S505 : Split the plurality of wavelet energy data according to a preset ratio to obtain a training set and a test set.

[0144] In this embodiment, the test set is used to verify the prediction results of the target prediction model.

[0145] The wavelet energy data is randomly shuffled and split into training and test sets according to a preset ratio to ensure that the training and test sets are consistent in data distribution.

[0146] It should be noted that according to the above embodiment, after the training set model is trained, the test set can be used to perform predictions and evaluate the model's performance. The final output integrates the dynamic programming results of the training set and the prediction results of the test set. The parameter values under three intervals are compared, and the preset time value is determined as the dynamic interval. The time interval and quantity interval files are generated. This ensures that the risk level of the test point is consistent with the corrosion risk assessment indicators of pipelines subject to dynamic DC stray current interference in the standard SY0087.4, further verifying the feasibility and accuracy of the method.

[0147] The method for data processing of the cathodic protection remote transmission system provided in the embodiment of the present application, on the one hand, widely obtains data collection samples of the cathodic protection remote transmission system from multiple cities, ensuring the diversity and representativeness of the data. On the other hand, the cathodic protection remote transmission system adopts a preset period to collect samples, and does not need to upload frequently in a very short time interval. The collection period can be set according to the content of the generated file to collect the polarization potential value, which saves the power of the equipment to a certain extent and extends the service life of the power supply. Then, the cross-validation method is used to determine the optimal decomposition layer number and target wavelet basis, which effectively improves the accuracy and applicability of the wavelet transform. Then, the data collection samples are subjected to wavelet transform processing based on the target wavelet basis to obtain wavelet energy data, which provides strong support for subsequent data analysis and model training. Finally, the wavelet energy data is split into training set and test set according to the preset ratio, providing a reliable data basis for model training and verification, and overall improving the quality of the training set and the scientific nature of the data processing process, achieving the technical effect of improving the accuracy and reliability of the data processing of the cathodic protection remote transmission system.

[0148] Figure 6 This is a schematic diagram of the structure of the data processing device of the cathodic protection remote transmission system provided in the embodiment of the present application. The device of this embodiment can be in the form of software and / or hardware. Figure 6 As shown, the data processing device 600 of the cathodic protection remote transmission system provided in the embodiment of the present application includes: an acquisition module 601, a wavelet transform processing module 602, a first determination module 603, a preprocessing module 604, a second determination module 605, and a training module 606:

[0149] An acquisition module 601 is used to acquire multiple data acquisition samples of a cathodic protection remote transmission system;

[0150] The wavelet transform processing module 602 is used to perform wavelet transform processing on multiple data collection samples to obtain corresponding multiple wavelet energy data;

[0151] The first determining module 603 is configured to determine a training set based on a plurality of wavelet energy data; the training set includes a plurality of wavelet energy training data;

[0152] A preprocessing module 604 is used to preprocess the plurality of wavelet energy training data to obtain a plurality of preprocessed training data;

[0153] The second determination module 605 is used to determine a corresponding plurality of feature data sets based on the plurality of pre-processed training data; each feature data set includes a model input feature and a binary classification label;

[0154] The training module 606 is used to train a target prediction model based on multiple feature data sets and the original prediction model; the target prediction model is used for data acquisition planning of the cathodic protection remote transmission system.

[0155] In one possible implementation, the pre-processing module 604 is further configured to:

[0156] Deduplication processing is performed on the preprocessed training data according to the wavelet energy value of the preprocessed training data to obtain multiple candidate training data;

[0157] According to the file sources of the candidate training data, the time coverage frequency of the candidate training data is calculated;

[0158] According to the importance values corresponding to the candidate training data, a dynamic programming method is used to construct a 0-1 knapsack problem corresponding to multiple preset time constraints. Based on the solution of the dynamic programming problem, multiple preprocessed training data are screened from the multiple candidate training data; among them, the importance value is positively correlated with the wavelet energy value.

[0159] In a possible implementation, the second determining module 605 is further configured to:

[0160] Add sliding windows to multiple pre-processed training data to extract local features, and add local extreme values to detect signal mutation features to obtain multiple preliminary features;

[0161] Normalizing multiple preliminary features to obtain multiple model input features; the model input features include normalized timestamps, standardized energy values, local energy maximum value marks, and time coverage frequencies;

[0162] Determine the binary classification labels for the model input features;

[0163] Generate a feature dataset based on the model input features and binary classification labels.

[0164] In a possible implementation, the second determining module 605 is further configured to:

[0165] Define the classification rules for binary classification labels;

[0166] According to the classification rules, a state transition model is constructed;

[0167] Based on the state transition model, determine the binary classification labels of the model input features.

[0168] In a possible implementation, the wavelet transform processing module 602 is further configured to:

[0169] Through the cross-validation method, multiple preset wavelet bases are used to verify the optimal decomposition layer number of the data collection samples;

[0170] Under the optimal decomposition level, the target wavelet basis is determined from multiple preset wavelet bases;

[0171] According to the target wavelet basis, wavelet transform processing is performed on multiple data collection samples to obtain corresponding multiple wavelet energy data.

[0172] In a possible implementation, the first determining module 603 is further configured to:

[0173] According to the preset ratio, multiple wavelet energy data are split and processed to obtain a training set and a test set, wherein the test set is used to verify the prediction results of the target prediction model.

[0174] In one possible implementation, the training module 606 is further configured to:

[0175] Prediction model, wherein the prediction model is a gradient boosting tree model.

[0176] In a possible implementation, the acquisition module 601 is further configured to:

[0177] A plurality of data collection samples from the cathodic protection remote transmission system of a plurality of cities are obtained, wherein the data collection samples are collected by the cathodic protection remote transmission system using a preset period.

[0178] The data processing device of the cathodic protection remote transmission system provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and are not described in detail in this embodiment.

[0179] Figure 7 This is a schematic diagram of the structure of the data processing equipment of the cathodic protection remote transmission system provided in the embodiment of the present application. Figure 7 As shown, the data processing device 700 of the cathodic protection remote transmission system provided in this embodiment includes: at least one processor 701 and a memory 702. Optionally, the device 700 also includes a communication component 703. The processor 701, the memory 702, and the communication component 703 are connected via a bus.

[0180] During the specific implementation process, at least one processor 701 executes the computer-executable instructions stored in the memory 702, so that the at least one processor 701 performs the above method.

[0181] The specific implementation process of the processor 701 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0182] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0183] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0184] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0185] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0186] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned method is implemented.

[0187] The readable storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0188] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0189] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, either through an interface, electrical, mechanical, or other means.

[0190] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0191] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0192] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0193] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0194] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. A method for data processing of a cathodic protection remote transmission system, characterized in that: include: Obtain multiple data acquisition samples from the cathodic protection remote transmission system; Performing wavelet transform processing on the plurality of data acquisition samples to obtain corresponding plurality of wavelet energy data; Determining a training set based on the plurality of wavelet energy data; The training set includes a plurality of wavelet energy training data; Preprocessing the plurality of wavelet energy training data to obtain a plurality of preprocessed training data; Determining corresponding multiple feature data sets based on the multiple pre-processed training data; Each feature dataset includes model input features and binary classification labels; A target prediction model is obtained by training based on the multiple feature data sets and the original prediction model; the target prediction model is used for data acquisition planning of the cathodic protection remote transmission system.

2. The method according to claim 1, characterized in that The preprocessing of the plurality of wavelet energy training data to obtain a plurality of preprocessed training data includes: performing deduplication processing on the preprocessed training data according to the wavelet energy value of the preprocessed training data to obtain a plurality of candidate training data; Calculating the time coverage frequency of the candidate training data according to the file source of the candidate training data; According to the importance values corresponding to the candidate training data, a dynamic programming solution is used to construct a 0-1 knapsack problem corresponding to multiple preset time constraints, and according to the solution of the dynamic programming problem, multiple preprocessed training data are screened from the multiple candidate training data; wherein the importance value is positively correlated with the wavelet energy value.

3. The method according to claim 2, characterized in that Determining corresponding multiple feature data sets based on the multiple pre-processed training data includes: Adding a sliding window to the plurality of preprocessed training data to extract local features, and adding local extreme values to detect signal mutation features to obtain a plurality of preliminary features; Normalizing the multiple preliminary features to obtain multiple model input features; the model input features include a normalized timestamp, a normalized energy value, a local energy maximum value mark, and a time coverage frequency; Determining binary classification labels for the model input features; A feature dataset is generated based on the model input features and the binary classification labels.

4. The method according to claim 3, characterized in that Determining the binary classification labels of the model input features includes: Defining classification rules for the binary classification labels; constructing a state transition model according to the classification rules; According to the state transition model, a binary classification label of the model input feature is determined.

5. The method according to any one of claims 1 to 4, characterized in that The performing wavelet transform processing on the plurality of data acquisition samples to obtain corresponding plurality of wavelet energy data includes: By using a cross-validation method, multiple preset wavelet bases are used to verify the optimal decomposition layer number of the data collection sample; Under the optimal decomposition level, determining a target wavelet basis from the plurality of preset wavelet bases; According to the target wavelet basis, wavelet transform processing is performed on the multiple data collection samples to obtain corresponding multiple wavelet energy data.

6. The method according to any one of claims 1 to 4, characterized in that Determining a training set according to the plurality of wavelet energy data includes: The plurality of wavelet energy data are split according to a preset ratio to obtain a training set and a test set, wherein the test set is used to verify the prediction result of the target prediction model.

7. The method according to any one of claims 1 to 4, characterized in that The prediction model is a gradient boosting tree model.

8. The method according to any one of claims 1 to 4, characterized in that The method of obtaining multiple data acquisition samples of the cathodic protection remote transmission system includes: A plurality of data collection samples from the cathodic protection remote transmission system of a plurality of cities are obtained, wherein the data collection samples are collected by the cathodic protection remote transmission system using a preset period.

9. A data processing device for a cathodic protection remote transmission system, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

Images